Download:
rawdog-2.23.tar.gz
(68K, released 2018-12-26)
Signature
(about);
SHA1 171520c909fae81ac8ec0e063e7ca36593aae372
Available through Git
(Atom feed of changes):
git clone http://offog.org/git/rawdog.git
About rawdog
rawdog is an RSS Aggregator Without Delusions Of Grandeur. It is a "river of news"-style aggregator: it uses feedparser to download feeds in RSS, Atom and a variety of other formats, and (by default) produces static HTML pages containing the newest articles in date order. For example, rawdog's default configuration produces output that looks like this:
rawdog is designed to be invoked periodically by cron (or a similar task-scheduling mechanism). It stores data in flat files, so no database is required. It can use threads to download multiple feeds in parallel, and uses ETags, Last-Modified, and gzip compression to minimise network bandwidth usage.
Written in Python, rawdog is highly customisable and extendable. Its output is generated using templates, so it's easy to integrate the output of rawdog into your own web pages. You can extend rawdog's functionality using plugin modules written in Python — for example, you can provide output in different formats, or filter or modify incoming data from feeds.
Full documentation is included in the source package to get you started with rawdog. If you're using a binary package that doesn't include the documentation, you can view the latest README, config and style.css files here.
Dependencies
rawdog requires Python 2.7 or later, and feedparser 5.1.2 or later. (rawdog won't work with Python 3.x; Python 3 is a different language from Python 2.)
I'd also strongly recommend installing PyTidyLib, which rawdog can use to generate cleaner output HTML.
rawdog is designed to run on POSIX-compliant free operating systems such as Linux and FreeBSD, and requires Unix-like filesystem semantics. rawdog is not supported on proprietary operating systems.
Mailing list
There is a mailing list for rawdog users and developers, upon which new versions of rawdog are announced and discussed.
Packaged versions of rawdog
rawdog packages are available for some systems. These are usually more convenient for end-users to install than the source tarball above, but they might not contain the very latest version of rawdog.
For Debian,
the rawdog package was
originally developed by Decklin Foster, and is now maintained by me.
(The debian directory for the rawdog package is
available in git.)
For Fedora, Andrew Grover maintains a rawdog package.
For FreeBSD, you can install the
news/rawdog
port, maintained by Tim Bishop.
For Gentoo, rawdog is available as
net-news/rawdog.
For OpenBSD, Xavier Santolaria maintains the
www/rawdog
port.
Plugins
Available through Git
(Atom feed of changes):
git clone http://offog.org/git/rawdog-plugins.git
Here are the plugins that are currently available; if you've written rawdog plugins that you'd like to be listed here, please let me know. If you can't see a plugin to do what you want, you can probably write it yourself without too much effort. See the PLUGINS file in the source package for more details of the plugin API, or look at the existing plugins provided. If you're not sure where to start, please contact me.
To install a plugin, make sure that you have plugindirs
plugins in your config file, and drop the plugin into your
~/.rawdog/plugins directory.
| Name | Maintainer | Purpose |
|---|---|---|
| archive | Adam Sampson | Write incoming articles in Atom format to a local archive (needs my atomwriter module) |
| article-filter | Adam Sampson | Filter articles on a per-feed basis using regular expressions |
| article-stats | Adam Sampson | Print counts of articles added, updated, expired and stored when rawdog exits |
| author-no-link | Stephan Manske | Provide author with HTML removed for templates |
| backwards | Adam Sampson | Example plugin: reverse the article sort order |
| dated-output | Adam Sampson | Split output into dated files |
| detwit | Adam Sampson | Remove prefixes from Twitter messages |
| digest-auth | Adam Sampson | Add HTTP digest authentication support |
| download-articles | Adam Sampson | Download copies of articles automatically |
| enclosure | Virgil Bucoci | Display links to article enclosures |
| feed-execute | Adam Sampson | Execute commands before or after fetching feeds |
| feedgrep | Steve Atwell | Filter articles by regular expression |
| feedgroup | Zephaniah E. Hull | Index articles by user-defined groups |
| feedwise | Virgil Bucoci | Group articles by feed rather than by date |
| imgstrip | Virgil Bucoci | Replace images with links |
| inline_link | Brian Jaress | Provide data: links
to long article descriptions |
| links | TheCrypto | Add static links to rawdog's output |
| ljkludge | Adam Sampson | Work around a bug in LiveJournal digest authentication |
| paged-output | Adam Sampson | Split output into several smaller files (needs rawdog 2.5rc1 or later) |
| printnew | Ted T | Send new articles by email |
| rss | Jonathan Riddell | Export RSS, FOAF and OPML automatically for all feeds |
| select-feeds | Adam Sampson | Select feeds to include in the output |
| sidebarfeedwise | Gutemberg A. Vieira | Feed-grouped display with a sidebar |
| since-last | Adam Sampson | When writing, only include articles that haven't previously been written |
| slashdot | Virgil Bucoci | Display Slashdot's department lines in articles |
| status-log | Adam Sampson | Keep and display a log of errors when fetching feeds |
| tagcat | Decklin Foster | Organise feeds using user-defined tags |
| truncate | Adam Sampson | Truncate and/or strip HTML markup from articles |
| vellum-templates | Adam Sampson | Replace rawdog's templating system with the one from Vellum |
| xml_archiver | BAM | Write rawdog's article database out as XML for external processing |
Styles
The appearance of rawdog's output can be customised using CSS stylesheets and HTML templates. rawdog includes a default style (shown in the screenshot above), but here are some more that rawdog users have contributed. If you've come up with a new stylesheet or template that others might like, please let me know.
| Name | Author |
|---|---|
| dec23 | Adam Sampson |
| shym | Samuel Hym |
Related work
Wari has written an xscreensaver fortune program that retrieves random articles from rawdog 1.x.
Nimrod's Dogwalker is a modified version of rawdog 1.x that supports FTP uploading and RTL languages like Hebrew.
Steve Pomeroy's rawdog to OPML exporter converts rawdog config files to OPML, with categories.
Conversely, Tero Karvinen's opml-export-feeds script can turn an OPML list into a rawdog config fragment.
Austin Bingham's rded provides a web-based interface for editing your rawdog config.
Brian M. Clapper's curn is very much like rawdog, but written in Java. Neither of us were aware of the other's project!
Planet KDE uses rawdog, with some customisations by Jonathan Riddell; the code and configuration are in KDE's repository.
If you've done something cool related to rawdog, let me know so I can add a link here to your project.