User Agents > sogou spider
sogou spider
Rating: +1
Mike - 4 Apr 2007 9:15 PM
Mike - 4 Apr 2007 9:15 PMThis is a web crawler that collects documents for a Chinese search engine sogou.com. They claim that their bot supports robots.txt, however when it requests robots.txt file from my web sites, IIS returns HTTP 406 error code. Sogou Spider does not support HTTP compression.
I've seen it coming from two IP addresses 220.181.19.172 and 220.181.19.179.
I've seen it coming from two IP addresses 220.181.19.172 and 220.181.19.179.
Reply
Rating: +1 dan - 14 Jun 2007 7:52 PM
dan - 14 Jun 2007 7:52 PMI've seen it coming from 220.181.19.176 and it uses a great deal of bandwidth...
Rating: 0 Connie - 4 Aug 2007 5:36 PM
Connie - 4 Aug 2007 5:36 PMHere are two more 220.181.19.153 and 220.181.19.94. When it's hit my site it was looking for non existent files.
Rating: 0 t - 7 Aug 2007 5:14 PM
t - 7 Aug 2007 5:14 PM220.181.19.176 - - [17/Jul/2007:09:56:32 +0000] "GET /robots.txt HTTP/1.1" 200 117 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [17/Jul/2007:10:40:45 +0000] "GET / HTTP/1.1" 200 2194 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [31/Jul/2007:13:10:33 +0000] "GET /robots.txt HTTP/1.1" 200 117 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [31/Jul/2007:13:10:43 +0000] "GET / HTTP/1.1" 200 2194 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [07/Aug/2007:14:59:55 +0000] "GET /robots.txt HTTP/1.1" 200 117 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [07/Aug/2007:15:00:08 +0000] "GET / HTTP/1.1" 200 2194 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
GG, so much for robots.txt compliance.
220.181.19.176 - - [17/Jul/2007:10:40:45 +0000] "GET / HTTP/1.1" 200 2194 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [31/Jul/2007:13:10:33 +0000] "GET /robots.txt HTTP/1.1" 200 117 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [31/Jul/2007:13:10:43 +0000] "GET / HTTP/1.1" 200 2194 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [07/Aug/2007:14:59:55 +0000] "GET /robots.txt HTTP/1.1" 200 117 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.19.176 - - [07/Aug/2007:15:00:08 +0000] "GET / HTTP/1.1" 200 2194 "-" "Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
GG, so much for robots.txt compliance.