I'm just using a realtime json parser in Ruby written as a native C extension (http://github.com/brianmario/yajl-ruby/tree/master) It's really simple to use and well documented.
I'm just storing everything in a Postgres database, and then using other scripts to query it. Note: using gardenhose at least you get a LOT of data fast. In just a few days, I have a 4GB+ database now or so On Jun 11, 5:10 pm, "M. Edward (Ed) Borasky" <[email protected]> wrote: > Right now, I'm collecting spritzer data with a simple shell script > "curl <magic incantations> | bzip2 -c > <yyyymmddhhmmss>.bz2". A cron > job checks every minute and restarts the script if it crashes. The > rest is simple ETL. :)
