They are in low 100's in the best case scenario, and could be in 1000 in the worst case scenario.
I believe this aspect can be pretty much shielded from application if the underlying platform has the right set of responsibilities. -- Regards, Praveen Ramachandra ________________________________ From: Ralph Goers <ralph.go...@dslextreme.com> To: flume-user@incubator.apache.org Sent: Monday, January 9, 2012 6:53 PM Subject: Re: Flume-NG Channels On Jan 9, 2012, at 2:28 AM, Praveen Ramachandra wrote: Hi, > > >We were trying to design a multi-tenanted system using flume-ng, where each >logically independent data set is modelled through a channel going through the >system of collectors, aggregators and delivery agents (to end destination). >Each channel will carry data that logically belong together. The requirement >is that we should be able to bring up and tear down a channel with ease. > > > > >When we completed the exercise, it turned out that we have to run a separate >Source/Sink, at a designated host/port combination for each channel. The issue >with this is that, it is an operational overhead that we have work with >net-ops to punch holes in the firewall to let tcp traffic flow on non-standard >ports. I would imagine that it would be the case in many organizations as well. > > >Two questions. > > >1. Did I get the modeling right with flume-ng >2. Is there a better way to do it at a platform level > 2.1 I know if I can write a bunch of custom sinks/sources and >embed a notion of channel to which each events belong to in the message, I can >effectively mux and demux the events at either ends. > 2.2 Which means the default support for channel is also not of >much use What is your target destination(s) for the tenants? Can they all flow through a single channel in Flume and then be delivered to the correct destination by a smarter sink at the end? Ralph
