# disallow spiders index fs and user controllers User-agent: * Request-rate: 1/5 # maximum rate is one page every 5 seconds Visit-time: 1800-2400 # only visit between 18:00 and 24:00 UTC (GMT)