Try taking a look at HDFSEventSink. The relevant code is in the process()
method:
transaction.begin();
Event event = null;
for (int txnEventCount = 0; txnEventCount < txnEventMax;
txnEventCount++) {
...where the # of events grabbed from the channel by this sink is dependent
on the txnEventMax var, which corresponds to the hdfs.txnEventMax property
that you specify in the config file.
Cheers,
Will
On Tue, May 8, 2012 at 11:42 AM, S Ahmed <[email protected]> wrote:
> Can someone point me to where the actual sink takes place? The api seems
> on the channels that are implemented seem to only support pulling 1 item
> off of the collection at a time.
>
> Starting to understand the design.....agian the comments are wonderful.
>
> On Tue, May 8, 2012 at 12:41 PM, Arvind Prabhakar <[email protected]>
> wrote:
>
> > Hi,
> >
> > On Tue, May 8, 2012 at 9:35 AM, S Ahmed <[email protected]> wrote:
> >
> > > Great thanks, that cleared things up.
> > >
> > > It seems from the api exposed that when things are being sinked,
> > > items/messages are being pulled of the collection 1 by 1 as oppose to
> > > grabbing the entire collection and replacing it when an empty one
> right?
> > >
> >
> > You could potentially pull multiple items within the same transaction to
> > make it a unit of work for the channel. How such a multi-take() request
> > gets optimally implemented is within the implementation of the channel.
> >
> > Thanks,
> > Arvind
> >
>
