On Wed, Jun 20, 2018 at 12:45 PM Enrico Olivelli <eolive...@gmail.com> wrote:
> > > Il mer 20 giu 2018, 16:52 Sijie Guo <guosi...@gmail.com> ha scritto: > >> On Wed, Jun 20, 2018 at 4:09 AM Enrico Olivelli <eolive...@gmail.com> >> wrote: >> >>> Hi, >>> I am experimenting DBLedgerStorage with some project. >>> With my workloads I don't see much benefit, or sometimes a performance >>> degradation, >>> >> >> Do you have a performance comparison? Or can you describe what have you >> observed? >> > > No real 'sharable' numbers, just run a bunch of benchmarks of one > application. My data has no real meaning. > > > > >> >>> just by keeping the same bookie and application configuration (just >>> switching the storage manager classe name in the bookie configuration). >>> >>> Which is the expected workload for DBLedgerStorage ? >>> (Usually I am using the default SortedLedgerStorage) >>> >> >> Ideally it should be working out better when you have large number of >> ledgers per bookie (let's say more than 10k ledgers). >> > > My test created only a few ledger ( less then 10) so maybe this is why I > see no benefit. > As far as I understand RocksDB is used for indexes and maybe with a few > ledgers the overhead is greater than the gain in term of resource saving. > I have to tune my test with more concurrency. > > In this project I will have concurrent readers and writers, but readers > will access data at any point of ledgers, so no usual 'tailing reads'. I > will be using ledgers more likes files than like streams. > Are you fetching data by entry ids or would you expect to fetch entries by their *bytes* offset? In other words, does this https://github.com/apache/bookkeeper/issues/1376 meet your requirement if you are looking for an interface to fetch entries more about their "bytes" offset? > So having rocks db to store offsets seems a very good chance for me. > In real systems I will have thousands of active ledgers per bookie. > I have to use a more complex suite of benchmarks for the use case. > > I will be back with more interesting comparisons > > Thank you > Enrico > > > > >> >>> >>> Cheers >>> Enrico >>> >>> >>> >>> -- > > > -- Enrico Olivelli >
