I've not used Splunk, but I find myself using Papertrail more and more lately, and I love it - especially combined with ActiveSupport::Notifications and the lograge gem. Something I could perhaps ramble about further next Melbourne Ruby meet (ping Justin, Dave).
Papertrail also has a Heroku add-on, which can make it super easy to set up if that's where you're hosting, plus their support is excellent. </spruik> -- Pat On 26/10/2012, at 9:49 AM, Chris Aitchison wrote: > If you don't have those sort of $$ and are happy to trust your logs to a > cloud service, I've had a great experience with Papertrail. > > It aggregates your logs, makes them easily searchable via a web interface, > and even lets you set up notification hooks to alert you if logs that match a > particular regex occur. You can archive your logs to S3. Oh, and you can > just sit and watch the tail of your aggregated logs via a web interface if > that sort of thing interests you (I swear it helps you grok an app, lol). It > is free for less than 100MB of logs/month. I've never got above $7 for 1Gb > logs/month, but it does get more expensive quickly if you exceed a few Gb of > logs per month. > > Before I tried Papertrail, I tried Loggly and found it to be nowhere near as > useful. > > If anyone is interested, I have Chef recipes to set up Papertrail that I'd be > happy to share with folks. Not that it is hard, quite the opposite actually. > > Cheers, > > Chris > @cmaitchison > > On 26/10/2012, at 8:58 AM, Julian Doherty <[email protected]> wrote: > > Splunk is awesome. We're using it at Envato, and I used it a bit at Lonely > Planet. > > Super expensive (five figure+ $$$ for most installs), but does pretty much > anything you want it to do in terms of aggregating + searching logs. Is > invaluable for debugging and diagnosing production issues in realtime that > would have been impossible before. > > You need to do a bit of setup and learning to get the most value out of it. > Main thing is setting it up so it understands the log format(s) of your > app(s) and can parse info out of it. It does a lot of things out of the box, > and there are plugins for Rails logging etc. > > Cheers > Julian > @madlep > > On 25/10/2012, at 2:00 PM, Craig Read wrote: > >> I'm currently parsing some large log files and populating a rails db with >> 'key' pieces of information from those logs via ActiveResource. >> They're not 'web logs', and each line can have totally different data >> (including how the data is structured), so most of the tools I see around >> aren't applicable to my particular use case. Also, only about 0.05% of the >> data is actually relevant, so I'm looking for a 'better way'. >> >> I did see mention of 'treating logs as data' (and Splunk in particular) on >> the latest TW Tech Radar. >> Is anybody using Splunk with (or without) the ruby-splunk gem? >> If so, do you recommend it, or is there a better approach to doing this? >> >> Cheers, >> >> -- >> Craig Read >> >> @Catharz >> https://github.com/Catharz >> http://stackoverflow.com/users/158893/catharz >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ruby or Rails Oceania" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/rails-oceania?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. -- You received this message because you are subscribed to the Google Groups "Ruby or Rails Oceania" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/rails-oceania?hl=en.
