Download:
rawdog-2.13.tar.gz
(99K,
MD5sum 140a2572bc66809e5f12c1616b519e87,
GPG signature)
Available through Darcs
(Atom feed of changes):
darcs get http://offog.org/darcs/rawdog/
About rawdog
rawdog is an RSS Aggregator Without Delusions Of Grandeur. It is a "river of news"-style aggregator: it uses feedparser to download feeds in RSS, Atom and a variety of other formats, and (by default) produces static HTML pages containing the newest articles in date order. For example, rawdog's default configuration produces output that looks like this:
rawdog is designed to be invoked periodically by cron (or a similar task-scheduling mechanism). It stores data in flat files, so no database is required. It can use threads to download multiple feeds in parallel, and uses ETags, Last-Modified, and gzip compression to minimise network bandwidth usage.
Written in Python, rawdog is highly customisable and extendable. Its output is generated using templates, so it's easy to integrate the output of rawdog into your own web pages. You can extend rawdog's functionality using plugin modules written in Python -- for example, you can provide output in different formats, or filter or modify incoming data from feeds.
Full documentation is included in the source package to get you started with rawdog. If you're using a binary package that doesn't include the documentation, you can view the latest README, config and style.css files here.
Dependencies
rawdog requires Python 2.2 or later, with some features only available on Python 2.4 or later. The most recent version of Python that has been tested is 2.6, but I expect it to work with later 2.x versions of Python too, if there are any. rawdog will definitely not work with Python 3.x, which is really a completely different (and, I'd argue, broken) programming language; stick with 2.x instead.
rawdog will work quite happily without any additional Python modules being installed, but it can make use of the mx.Tidy module to generate cleaner output HTML if it's available.
rawdog is designed to run on POSIX-compliant free operating systems such as Linux and FreeBSD, and requires Unix-like filesystem semantics. rawdog is not supported on proprietary operating systems.
Mailing list
There is a mailing list for rawdog users and developers, upon which new versions of rawdog are announced and discussed.
Federico Sevilla III has set up a searchable archive of the rawdog mailing list.
Packaged versions of rawdog
rawdog packages are available for some systems. These are usually more convenient for end-users to install than the source tarball above, but they might not contain the very latest version of rawdog.
On FreeBSD, you can install the
news/rawdog
port maintained by Tim Bishop.
SuSE users can use Tristan Miller's rawdog RPMs.
Xavier Santolaria maintains a
www/rawdog
port for OpenBSD.
Decklin Foster packages rawdog for Debian.
Plugins
Available through Darcs
(Atom feed of changes):
darcs get http://offog.org/darcs/rawdog-plugins/
Here are the plugins that are currently available; if you've written rawdog plugins that you'd like to be listed here, please let me know. If you can't see a plugin to do what you want, you can probably write it yourself without too much effort. See the PLUGINS file in the source package for more details of the plugin API, or look at the existing plugins provided. If you're not sure where to start, please contact me.
To install a plugin, make sure that you have plugindirs
plugins in your config file, and drop the plugin into your
~/.rawdog/plugins directory.
| Name | Maintainer | Purpose |
|---|---|---|
| archive | Adam Sampson | Write incoming articles in Atom format to a local archive (needs my atomwriter module) |
| article-filter | Adam Sampson | Filter articles on a per-feed basis using regular expressions |
| article-stats | Adam Sampson | Print counts of articles added, updated, expired and stored when rawdog exits |
| author-no-link | Stephan Manske | Provide author with HTML removed for templates |
| backwards | Adam Sampson | Example plugin: reverse the article sort order |
| dated-output | Adam Sampson | Split output into dated files |
| digest-auth | Adam Sampson | Add HTTP digest authentication support |
| enclosure | Virgil Bucoci | Display links to article enclosures |
| feedgrep | Steve Atwell | Filter articles by regular expression |
| feedgroup | Zephaniah E. Hull | Index articles by user-defined groups |
| feedwise | Virgil Bucoci | Group articles by feed rather than by date |
| feed-execute | Adam Sampson | Execute commands before or after fetching feeds |
| imgstrip | Virgil Bucoci | Replace images with links |
| inline_link | Brian Jaress | Provide data: links
to long article descriptions |
| links | TheCrypto | Add static links to rawdog's output |
| paged-output | Adam Sampson | Split output into several smaller files (needs rawdog 2.5rc1 or later) |
| printnew | Ted T | Send new articles by email |
| rss | Jonathan Riddell | Export RSS, FOAF and OPML automatically for all feeds |
| select-feeds | Adam Sampson | Select feeds to include in the output |
| sidebarfeedwise | Gutemberg A. Vieira | Feed-grouped display with a sidebar |
| slashdot | Virgil Bucoci | Display Slashdot's department lines in articles |
| status-log | Adam Sampson | Keep and display a log of errors when fetching feeds |
| tagcat | Decklin Foster | Organise feeds using user-defined tags |
| truncate | Adam Sampson | Truncate and/or strip HTML markup from articles |
| vellum-templates | Adam Sampson | Replace rawdog's templating system with the one from Vellum |
| xml_archiver | BAM | Write rawdog's article database out as XML for external processing |
Styles
The appearance of rawdog's output can be customised using CSS stylesheets and HTML templates. rawdog includes a default style (shown in the screenshot above), but here are some more that rawdog users have contributed. If you've come up with a new stylesheet or template that others might like, please let me know.
| Name | Author |
|---|---|
| dec23 | Adam Sampson |
| shym | Samuel Hym |
Related work
Wari has written an xscreensaver fortune program that retrieves random articles from rawdog 1.x.
Nimrod's Dogwalker is a modified version of rawdog 1.x that supports FTP uploading and RTL languages like Hebrew.
Steve Pomeroy's rawdog to OPML exporter converts rawdog config files to OPML, with categories.
Conversely, Tero Karvinen's opml-export-feeds script can turn an OPML list into a rawdog config fragment.
Austin Bingham's rded provides a web-based interface for editing your rawdog config.
Brian M. Clapper's curn is very much like rawdog, but written in Java. Neither of us were aware of the other's project!
Keith Fieldhouse has written an article about using rawdog to keep track of software development.
Planet KDE uses rawdog, with some customisations by Jonathan Riddell; the code and configuration are in KDE's repository.
If you've done something cool related to rawdog, let me know so I can add a link here to your project.