I’ve been looking at new ways to monitor online debates. My technical requirements are:
- Return examples where people have been talking about keywords X, Y or Z in blog posts, forums, comments, Tweets, news and ideally videos/podcasts/general web page content too
- Filter out duplicates
- Filter out spam blogs
- Filter out stuff that I/we put out there ourselves on our own site(s)
- Sort the combined list in date order, and return an RSS feed I can use in my reader or in a Netvibes widget
The heart of my system is Yahoo Pipes (which I’ll describe here later) which handles the merging and filtering of RSS feeds beautifully. But relevancy is still a problem.
Backtype helps you track comments across the web. You can use it to generate a neat RSS feed of all the comments you leave on other blogs, and perhaps pipe this into a sidebar widget on your own (I currently do this with a manually-bookmarked tag via Delicious but may switch over to Backtype in due course). You can also use it to search more generally for mentions of keywords and individuals, and it looks to be fast and pretty wide-ranging, and looks lovely too.