October 2005
I have almost all my webpages on some servers here at home. I have (or had?) a 3Mbit/s xDSL connection, and it have worked fine for a long time.
But as the nuber of customers in my area has increased alot, the connection quality has decreased. So, while I'm waiting for a reconnection to a different route, I'm close to the good old dial-up connection speed.
Yay!
I can't help it. I like, or disslike crawlers. I have written one, I have tracked them, and I have traped them. It's fun. It's challanging.
What's so fun about them? There are good spiders, there are bad spiders, there are spamming spiders, there are email harversters, and there are probably alot of useless spiders, like mine.
The challange is: what to allow, what to block?
I have started a project that aims to gather information about the behaviour of webspiders. Good and bad.
Oh.. and here is the link: SpiderWatch.
Have fun!
