May I suggest, before making any code changes, you measure the difference in MB/s between large writes and small writes? I do recall there was some advantage to using large entries in the past, more than 1k, but I don't remember what it was, and it may not still be true. A lot of code has changed since then.
In theory, anything greater than the MTU shouldn't give too much of a boost. -Ivan On Mon, Sep 14, 2015 at 3:35 PM Flavio Junqueira <[email protected]> wrote: > Hi Enrico, > > What's the size of each entry? If they are small say just a few bytes, > then you're indeed better off grouping them. If they are 1k or more, then > the benefit of grouping shouldn't be much. > > About extending the API, the only disadvantage I can see of grouping > writes into an entry rather than writing a batch of entries is that a read > request will have to read them all. I personally don't like so much the > idea of a batch call because it makes the code a bit messier. You need to > start a batch, add a bunch of stuff, flush the batch, start a new batch, > add a bunch of stuff, and so on. With addEntry, you just invoke it every > time you have a new message. > > -Flavio > > On 14 Sep 2015, at 05:02, Enrico Olivelli - Diennea < > [email protected]> wrote: > > Hi, > What is the fastest way to write to BookKeeper a batch of entries ? > > I’m using a sequence of asyncAddEntry, some thing like the code below: > > List<Long> res= new ArrayList<>(); // holds entry sequence numbers > CountDownLatch latch = new CountDownLatch(size); > for (int i = 0; i < size; i++) { > ….. > this.out.asyncAddEntry(entry, new AsyncCallback.AddCallback() { > > public void addComplete(int rc, LedgerHandle lh, long entryId, Object > i) { > int index = (Integer) i; > if (rc != BKException.Code.OK) { > BKException error = BKException.create(rc); > exception.value = error; > res.set(index, null); > for (int j = 0; j < size; j++) { > // early exit > latch.countDown(); > } > } else { > res.set(index, entryId); > latch.countDown(); > } > } > }, i); > } > latch.await(); > > Would it be faster to group all the entries in one “large” entry ? This > may alter application semantics but if it would be faster I will do the > refactor > > Can I file an issue in order to implement a “batchAddEntries” which > implements the write of a batch of entries within the native Bookkeeper > client ? > > > > > > *Enrico Olivelli* > Software Development Manager @Diennea > Tel.: (+39) 0546 066100 - Int. 925 > Viale G.Marconi 30/14 - 48018 Faenza (RA) > > MagNews - E-mail Marketing Solutions > http://www.magnews.it > Diennea - Digital Marketing Solutions > http://www.diennea.com > > > ------------------------------ > Iscriviti alla nostra newsletter per rimanere aggiornato su digital ed > email marketing! http://www.magnews.it/newsletter/ > > >
