Blocked another misbehaving Chinese bot. Looks like they’ve been spamming the access logs for constantine.su since about March 2011. They generate an average of 10 requests a day (226 for the month of October, similarly for multiple months prior to that), to a static single-page site (“Last-Modified: Fri, 02 Jan 2009 03:11:18 GMT”, e.g. like two and a half years ago). And unlike just about any other bot, none of the requests are for robots.txt, so, they would effectively be ignoring it even if it was present.

For 2011-11-01/10, compare that Googlebot does only 8 requests in total (even all Google services, e.g. including Feedfetcher and AppEngine, do only 16 requests), whereas YodaoBot does 65.

# fgrep Googlebot constantine.su.access.log | wc -l
8
# fgrep Google constantine.su.access.log | wc -l
16
# fgrep YodaoBot constantine.su.access.log | wc -l
65

# fgrep YodaoBot constantine.su.access.log | fgrep robots | wc -l
0
# fgrep Google constantine.su.access.log | fgrep robots | wc -l
4

61.135.248.202 - - [10/Nov/2011:13:19:13 -0500] “GET / HTTP/1.1” 200 3132 “-” “Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; )”

deny tcp from { 61.135.248.173/25 or 61.135.249.0/24 } // YodaoBot .cn

This block looks nicely next to my Baidu blocks. Baidu also generates too many excessive requests for single-page static sites, spamming out the access logs.