Well, you could use something like (the soon to be released) Precis ( https://github.com/precis-logging/precis) project. Built on top of a centralized event message bus (by default it will use MongoDB capped collections) and already being used for 100,000+ messages a second.
The event bus used by Precis can be switched out to fit your needs so you could use RabbitMQ, Redis, etc instead of Mongo's capped collections. I'm busy right now removing all of our custom code and open sourcing the entire thing. Goal is to have the first GA in the wild by end of March to middle of April. Disadvantage to this scale of logging is your talking about a full SOA with many services to maintain. Advantage is that scaling just requires deploying new processing nodes. Honestly Logger looks pretty snazzy and I've already added it to my list of projects to look at as a plugin in place of or along side FluentD in Precis. Of course there is always ELK (ick), Splunk, Octopussy, Graylog, or you could just build it yourself if you don't like the features of what is already available. - Jeremy On Tue, Mar 10, 2015 at 9:47 PM, Aria Stewart <[email protected]> wrote: > > > On Mar 10, 2015, at 7:36 PM, Tim Dickinson <[email protected]> > wrote: > > > > You don't. It was designed to collect hundreds if not thousands of log > lines per second. The UDP server is made to scale. So to prevent packet > lose just scale up the UDP server. > > > Aw, sad. It's easier to scale without things if you have a protocol that > handles retries like TCP. This way you just have to overprovision like 10x > and hope you still don't miss any due to bursting. :( > > -- > Job board: http://jobs.nodejs.org/ > New group rules: > https://gist.github.com/othiym23/9886289#file-moderation-policy-md > Old group rules: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/nodejs/B39400AF-9039-4339-9093-60C9A828BA8E%40nbtsc.org > . > For more options, visit https://groups.google.com/d/optout. > -- Job board: http://jobs.nodejs.org/ New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAAhs7Ej%3Daj47hbN7hL_dAETx5HU7xAY0Ysq%2BrzsaS68O6pENjQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
