TechCrunch Feed Filter

This is a Python script run as a cronjob to read the TechCrunch article feed, and decide which articles to include in its own feed.

The Problem

TechCrunch was a great blog about innovation and entrepreneurship. As it grew, it published more articles than I cared to read. Like many savvy blog readers, I used a feed reader to present the latest articles to me, but TechCrunch's feed was simply too profuse.

The Solution

I created a service that'd visit TechCrunch's feed, and make note of who made which articles, what the articles were about, how many comments each article had, and how many Diggs, Facebook likes and Facebook shares each article had.

With that data, the service would determine the median, mean, standard deviation, and create a minimum threshold for whether the article merited being seen by me.

I go into deeper detail in this blog post about it.

Here's the status page it generates.

To Do

• Maybe use Reddit upvotes

Pre-Git History

This was originally archived in a Subversion repo. I'd forgotten about the version control and had gotten into the habit of just modifying the production site.

• 2010-09-03: Original
• 2010-09-03: Save off the disqus identifier for use later.
• 2011-02-04: Algorithm changes (tags and author checked), new chart drawing, spaces used instead of tabs.
• 2011-02-04: Update to the chart drawing algorithm.
• 2013-08-04: Miscellaneous changes to techcrunch.py
• 2015-11-23: Resync svn with production site.
• 2015-11-27: Remove obsolete disqus and retweet code, and refactor style to be more PEP-8ish.

Is it any good?

Yes.

Licence

This software uses the MIT license.