Here’s an interesting presentation entitled “Challenges in Running a Commercial Search Engine” by Amit Singhal, Google Senior Research Scientist. It’s a 3.2MB PDF. It’s a fascinating synopsis of the current state of the disgusting spectacle that is search engine optimization.
I delete about 50 comment spams at a time on this site and I check it several times a day so as to rob them of any value they’re seeking. Nothing can stop them; I’ve given up blocking their IP addresses. If Google can solve this problem, then the incentive for comment (and other forms of) spam disappears.
Keep up the good work, Google!