On Mon, Jul 3, 2017 at 1:35 AM, Enrico Olivelli <eolive...@gmail.com> wrote:
> 2017-07-03 7:00 GMT+02:00 Sijie Guo <guosi...@gmail.com>: > > Hi all, > > > > It has been almost 6-7 years since Apache BookKeeper was born. Apache > > BookKeeper has already grown beyond a WAL system. Both Twitter and Yahoo > > have used it as their storage foundation for their messaging systems, > > Salesforce is using it for storage service. We also started talking > Apache > > BookKeeper as a storage service since 2016 ([1][2]). > > > > I am thinking of changing the description of Apache BookKeeper from a WAL > > system to "a High Performance and Low Latency Storage Service (that > > optimized for immutable/append-only data)" in the new website that we are > > building for BP-11 > > <https://cwiki.apache.org/confluence/pages/viewpage. > action?pageId=71012301>. > > This will help us to bring more use cases/adoptions to the project and > help > > grow the community. > > > > Any thoughts? > > My two cents > > Honestly when I found BookKeeper I was very happy because I found an > "original" building block to build replicated state machines. > I think that the main soul of BK is exactly to be a WAL and this is > really "original". > > From my point of view the "key features" of BookKeeper are "Fencing" > and "Last-Add-Confirmed protocol" > > BookKeeper is really good at storing data, but IMHO it is because it > has been designed and implemented by very skilled engineers, > BookKeeper needs to be "fast", because in order to provide a fast WAL > you have to give an ultra-fast storage, because the essence of a WAL > is "durability" and usually "durable" comes together with 'sync' and > so with 'slow' . > > I am not a "marketing expert" but IMHO we should stress on the > distinctive features of BK in respect to other softwares. > > I am not against the proposed change but as an user I wanted to point > that I happy with BK because it is the most powerful distributed WAL > (and maybe it is the unique in the opensource/free world) > > I would like to write in the website that BookKeeper is the real > answer to whom who are looking for a distributed WAL. Agree, we should make a clear case for distributed WAL. It is worth just putting down all the use cases that BookKeeper has supported. - WAL (e.g. HDFS NameNode) - Message Store (e.g. Apache Pulsar, Twitter Pub/Sub via DistributedLog) - Offset/Cursor Store (e.g. Apache Pulsar stores cursors in ledgers) - Object/Blob Store (e.g. in replicated state machine, storing state machine snapshots in ledgers. We used this pattern at distributedlog based replicated state machines.) - ... They are not all typical WAL use cases. But the common thing on all these use cases - they are using bookkeeper as an append-only/immutable store. - Sijie > > > -- Enrico > > > > > > > [1] > > https://www.slideshare.net/hustlmsp/apache-bookkeeper-a- > high-performance-and-low-latency-storage-service > > [2] https://www.slideshare.net/jujjuri/apache-con2016final >
