access.log spam from baidu
So tired of stupid baidu spider spamming my access logs for cnst.su and some other domains. Seriously, the site only has front page, and it hasn’t been changed in years! The Last-Modified date that’s reported by the server is correct — 2007-12-30, oh my! Please, stop wasting my time and traffic, baidu, I’m tired of blacklisting your servers on the IP level with ipfw.
220.181.108.172 - - [02/Oct/2011:08:40:24 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 123.125.71.101 - - [02/Oct/2011:08:41:48 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 193.242.108.63 - - [02/Oct/2011:10:57:03 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 220.181.108.168 - - [02/Oct/2011:12:56:10 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 220.181.108.174 - - [02/Oct/2011:13:39:49 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 220.181.108.185 - - [02/Oct/2011:14:38:41 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 123.125.71.102 - - [02/Oct/2011:14:40:49 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 220.181.108.181 - - [02/Oct/2011:15:39:11 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 123.125.71.98 - - [02/Oct/2011:15:40:46 -0400] "GET / HTTP/1.1" 200 2115 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
-
cnst posted this