[ANNOUNCE] Apache BookKeeper 4.3.1 released

The Apache BookKeeper team is proud to announce Apache BookKeeper version 4.3.1. This is the second release of Apache BookKeeper as an Apache Top Level Project. The BookKeeper project is made up of a distributed logging service called BookKeeper and a distributed publish/subscribe system build on

Re: Cannot find bookkeeper-4.3.1 binary

Hmm, it is in the dist repo: https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.3.1/ I guess there seems to be one step missing in the release procedure, which the build isn't pushed to mirror. Let me check with Ivan. - Sijie On Wed, Jun 10, 2015 at 5:21 AM, Flavio Junqueira wrote:

Re: Low write bandwidth

Just out of curiosity, does your system only allow one outstanding request? or just one outstanding request a ledger? without overlapping network transporting and request processing, it is hard to fully utilize the resources (either network or disk). - Sijie On Wed, Jun 10, 2015 at 7:21 AM, Macie

Re: Cannot find bookkeeper-4.3.1 binary

I just moved the release from dev to release. https://dist.apache.org/repos/dist/release/bookkeeper/bookkeeper-4.3.1/ It might be a bit time to push to mirror. Sorry for the inconvenience. Thanks Youngjoon for pointing out. - Sijie On Wed, Jun 10, 2015 at 10:11 AM, Sijie Guo wrote: > Hmm,

Re: Journal file never deleted

The journal files are only deleted when its corresponding data in ledger storage is flushed. What is your rate on adding entries? How many .txn files you have? On Wed, Jul 8, 2015 at 3:27 AM, Enrico Olivelli - Diennea < enrico.olive...@diennea.com> wrote: > I’m running all the default values, ex

Re: Journal file never deleted

turn; // we didn't create this checkpoint, so dont do > anything with it > > } > > > > Actually my test bookie gets a > org.apache.bookkeeper.bookie.CheckpointSource$Checkpoint$1@98aabc2 class > instead of a LogMarkCheclpoint, when the > > checkpoin

Messaging User Group Meetup on 9/21

Hello bookkeepers, We are organizing a cross-community meetup around messaging technologies on 9/21. Come and join us if you happen to be in Bay Area. https://www.eventbrite.com/e/messaging-user-group-meetup-tickets-18190307700 - Sijie

Re: Messaging User Group Meetup on 9/21

We just fixed the registration form now. Come and join us on 9/21. - Sijie On Tue, Aug 25, 2015 at 10:46 AM, Sijie Guo wrote: > Hello bookkeepers, > > We are organizing a cross-community meetup around messaging technologies > on 9/21. Come and join us if you happen to be

Re: Apache Bookkeeper 4.0 release date

Hello Enrico: We could port this change to 4.3 branch and cut a bug fix release 4.3.1. Are u running 4.3.x or 4.2.x branch? - Sijie On Wed, Sep 9, 2015 at 4:02 AM, Enrico Olivelli - Diennea < enrico.olive...@diennea.com> wrote: > (sorry for cross-posting) > > > > Due to that bug in our producti

Re: Apache Bookkeeper 4.0 release date

I think a bug fix release is much safer for you rather than cutting 4.4.0, since there is much more newer features in 4.4.0. On Wed, Sep 9, 2015 at 10:59 AM, Sijie Guo wrote: > Hello Enrico: > > We could port this change to 4.3 branch and cut a bug fix release 4.3.1. > Are u runn

PoweredBy page

Hello bookkeepers, I just added Twitter's use cases in the PoweredBy page. If you are using BookKeeper in your system, please add your use case in the PoweredBy page too :) - Sijie

Re: Apache Bookkeeper 4.0 release date

> > > > commons-beanutils-core > > commons-beanutils > > > > > > commons-cli > > commons-cli > > > > > > jna > > net.java.dev.jna > > >

Re: Fastest way to write to bookkeeper

Hmm. This calculation is a bit misleading. Throughput is a matter of the bandwidth, it is dominated by your disk or network. Latency is a matter of bookie group commit interval, how fast that the bookie flushes (syncs) data to disk. High latency doesn't necessary mean low throughput when your reque

Re: Messaging User Group Meetup on 9/21

Just a reminder, the meetup is on 9/21. Join us for an evening of food, drinks and discussions. - Sijie On Tue, Aug 25, 2015 at 10:46 AM, Sijie Guo wrote: > Hello bookkeepers, > > We are organizing a cross-community meetup around messaging technologies > on 9/21. Come and join us i

DistributedLog, a high-performance replicated log service built over BookKeeper

Hello bookkeepers, FYI. We published a blog post on DistributedLog, a replicated log service built on Apache BookKeeper. The replicated log service has been used in production for years, powering tier-0 services from transaction log for distributed database, search ingestion pipeline to self-serve

Re: DistributedLog, a high-performance replicated log service built over BookKeeper

Thanks. Yup, we will record it, and also try periscope it. - Sijie On Wed, Sep 16, 2015 at 1:26 AM, Ivan Kelly wrote: > This is excellent work Sijie :D It's a pity I can't make the meetup. Will > it be recorded? > > -Ivan > > On Wed, Sep 16, 2015 at 4:51 AM

Re: DistributedLog, a high-performance replicated log service built over BookKeeper

://www.periscope.tv/w/1YpKkXOadpXGj (second part) - Sijie On Wed, Sep 16, 2015 at 9:33 AM, Sijie Guo wrote: > Thanks. Yup, we will record it, and also try periscope it. > > - Sijie > > On Wed, Sep 16, 2015 at 1:26 AM, Ivan Kelly wrote: > >> This is excellent work Sijie :D It&#

Re: Ability to force BookieID

What is a more stable identifier for your bookie host? hostname or ip-address. If your ip address doesn't change, we'd encourage you to set useHostNameAsBookieID. Right now, it only supports hostname or ip-address. and those ids are used in ledger's metadata. So it doesn't support the cases where

Re: DistributedLog, a high-performance replicated log service built over BookKeeper

://www.slideshare.net/MessagingMeetup/cloud-messaging-service-technical-overview - Video: https://www.youtube.com/watch?v=QW1OEQxcjZc - Sijie On Tue, Sep 22, 2015 at 12:51 AM, Sijie Guo wrote: > We did periscope during the event. We also have recorded a high-quality > version, will publish the

Re: News on 4.3.2

Sorry for being late on this release. We should be ready for 4.3.2 soon. - Sijie On Mon, Oct 12, 2015 at 2:11 AM, Enrico Olivelli - Diennea < enrico.olive...@diennea.com> wrote: > Hi, > > Any news on the release of 4.3.2 ? > > > > Thank you very much > > > > *Enrico Olivelli* > Software Developm

Re: switching from FlatLedgerManager to HierarchicalLedgerManager

It isn't doable in 4.3.* version. since the id generation path isn't configurable in HierarchicalLedgerManager. It is doable after we have https://issues.apache.org/jira/browse/BOOKKEEPER-438 in master. created a jira about verifying and documenting the steps: https://issues.apache.org/jira/browse

Re: switching from FlatLedgerManager to HierarchicalLedgerManager

machines by creating 100.000 ledgers and I will try to upgrade otherwise I > do not known how to recover my production system (with 800.000 ledgers) > > > > Thank you very much > > > > *Da:* Sijie Guo [mailto:si...@apache.org] > *Inviato:* giovedì 5 novembr

Re: newbie question

Hello Mauricio: Are you looking for a mechanism to erase data within a ledger? - Sijie On Fri, Nov 6, 2015 at 8:36 AM, Maurício Santiago de Castro < mauricioscas...@hotmail.com> wrote: > Hi, if I want to erase historical data in the log, say older than 5 days > for example, is there any ledger

Re: newbie question

vel above the lmdb jni client. > > Thanks. > > > > > On 11/06/2015 04:23 PM, Sijie Guo wrote: > > Hello Mauricio: > > Are you looking for a mechanism to erase data within a ledger? > > - Sijie > > On Fri, Nov 6, 2015 at 8:36 AM, Maurício Santiago de Cast

Re: newbie question

the id of the ledger, for > instance in zookeeper, but if that reference gets lost there is now way to > know how old a ledger is. > > -- Enrico > > Il giorno 20:57 Ven 06/Nov/2015 Sijie Guo ha scritto: > >> How you manage the ledgers? Do you keep a list of ledgers? Cou

Re: switching from FlatLedgerManager to HierarchicalLedgerManager

oon as it will be testable > > Thank you again > -- Enrico > > Il giorno 19:21 Ven 06/Nov/2015 Sijie Guo ha scritto: > >> Yup. I would backport BOOKKEEPER-438 to 4.3.2. >> >> - Sijie >> >> >> On Fri, Nov 6, 2015 at 12:23 AM, Enrico Olivelli - Di

Re: Cannot compile 4.3.2 tag

Try run : I think Matteo missed one file on committing BOOKKEEPER-438 :( Matteo, could you add the missing file? - Sijie On Tue, Nov 17, 2015 at 1:32 AM, Enrico Olivelli - Diennea < enrico.olive...@diennea.com> wrote: > Hi, > > I’m trying to compile at 4.3.2 tag from > https://github.com/apach

Re: Cannot compile 4.3.2 tag

gt; Sorry about that, fixing and releasing rc1 > > Matteo > > > Sent from Yahoo Mail for iPhone <https://yho.com/footer0> > > > > > On Tuesday, November 17, 2015, 11:04 AM, Sijie Guo > wrote: > > Try run : > > I think Matteo missed one file on

Re: R: Cannot compile 4.3.2 tag

*Da:* Matteo Merli [mailto:mme...@yahoo-inc.com] > *Inviato:* martedì 17 novembre 2015 23:41 > *A:* user@bookkeeper.apache.org; Sijie Guo > *Cc:* Enrico Olivelli - Diennea > *Oggetto:* Re: Cannot compile 4.3.2 tag > > Sijie / Enrico, > > I've updated the tag and the r

Re: hint about ledger removal

for my understanding, this seems to be a coordination case, you are coordinating the bookkeeper clients to delete ledgers after meeting a certain condition. zookeeper is a good candidate to do such coordination than bookkeeper. as bookkeeper is more focus on replication and consistency on replicat

Re: [ANNOUNCE] Apache BookKeeper 4.3.2 released

Great job! Matteo! thank you so much! - Sijie On Tue, Dec 1, 2015 at 2:48 PM, Matteo Merli wrote: > The Apache BookKeeper team is proud to announce Apache BookKeeper version > 4.3.2. > > This is the second release of Apache BookKeeper as an Apache Top Level > Project. > > The BookKeeper project

Re: Large Ledger Entry Writes

Lucas, I think there is a hard limitation on entry size, which is 1MB. Did you successfully write entries that are large than 1MB? In bookkeeper, the entry is unit of durability. It potentially has latency impacts, as it has to fsync all 1MB to disk before acknowledge. If your traffic is comprise

Re: BOOKKEEPER-878

Hello Enrico: Sorry for the delays. Trying to get to it today. - Sijie On Thu, Dec 17, 2015 at 3:54 AM, Enrico Olivelli - Diennea < enrico.olive...@diennea.com> wrote: > Hi, > > Any news on the switch from FlagLedgerManager to HierarchicalLedgerManager > ? > > https://issues.apache.org/jira/bro

Re: BOOKKEEPER-878

ed that our systems are growing > and bookkeeper is one of the fundamental pieces or our infrastructure > Good to hear that. That's a very impression action from your description. Let me work with you to complete the migration from FlatLedgerManager to HierarchicalLedgerManager

Re: Distributed key valute store using bookkeeper

Twitter Manhattan is using distributedlog (built upon bookkeeper) for strong consistent operations. https://blog.twitter.com/2014/manhattan-our-real-time-multi-tenant-distributed-database-for-twitter-scale https://blog.twitter.com/2015/building-distributedlog-twitter-s-high-performance-replicated-

Re: Running Bookkeeper from trunk

The bookkeeper 4.4 includes the changes from Twitter and Yahoo, which that the code/improvements has been running on production for a long time. We are working towards cutting a 4.4. release by end of Q1. - Sijie On Thu, Feb 18, 2016 at 5:08 AM, Enrico Olivelli wrote: > Hi, > Bookkeeper 4.4 con

Re: [ANNOUNCE] Apache BookKeeper 4.4.0 released

Great job! Matteo and the community! Sijie On Monday, May 16, 2016, Matteo Merli wrote: > The Apache BookKeeper team is proud to announce Apache BookKeeper version > 4.4.0. > > This is the first feature release of Apache BookKeeper as an Apache Top > Level > Project. > > This release contains a

Re: How to speed up reads from bookie

What is your observation? How slow is it? And how do you do the scan? - Sijie On Fri, May 20, 2016 at 6:45 AM, Enrico Olivelli - Diennea < enrico.olive...@diennea.com> wrote: > Hi, > I'm wondering how to speed up reads from BookKeeper. > "Tail" reads are very fast because they hit Bookie cache,

[ANNOUNCE] Matteo Merli joins the Apache BookKeeper PMC

On behalf of the Apache BookKeeper PMC I am pleased to announce that Matteo Merli has accepted our invitation to become a PMC member on the Apache BookKeeper project. Matteo has been an active contributor in many areas, including recently release 4.3.2 and 4.4.0 as release manager. Please join me i

Re: Announcing BookKeeper Meetup - Tuesday, June 28, 2016 5:30 PM to 8:30 PM

Big Data Analytics. > > *Panel*: > > *JV Jujjuri* (Venkateswararao Jujjuri), is a cloud storage architect at > Salesforce leading an effort to build cloud storage services and a Apache > Bookkeeper committer. Prior to this JV worked at IBM’s Linux Technology > Center and worke

Re: [DISCUSS] Retire BKJM from trunk?

+ Rakesh and Uma Rakesh and Uma might have a better idea on this. I think Huawei was using it when Rakesh and Uma worked there. - Sijie On Wed, Jul 27, 2016 at 12:06 PM, Chris Nauroth wrote: > I recommend including the BookKeeper community in this discussion. I’ve > added their user@ and dev@

Re: Bookkeeper Backup and Disaster Recovery

Enrico, At twitter, we didn't do cross-cluster replication for bookkeeper. We did this at higher level. For zookeeper, we periodically dump the zookeeper files (data and txn log files) to HDFS for backup. - Sijie On Wed, Aug 3, 2016 at 7:13 AM, Enrico Olivelli - Diennea < enrico.olive...@dienne

Re: BookKeeper as low-latency distributed data storage

In general, reading an entry from bookie is pretty fast as most of the index pages will be cached in memory, so it typically only needs one disk access. You might see evaluated latency when index pages begin swapping in and out. So ideally I would suggest using small index page size and large memo

Re: How to implement a custom Auth Provider

Ah, good catch, Enrico. It is really bad for this behavior. I think it should be fine to break the code backward compatibility for this interface. - Sijie On Wed, Oct 19, 2016 at 8:02 AM, Enrico Olivelli wrote: > Hi, > I'm going to implement a custom AuthProvider for BookKeeper, I have > come

Re: How to implement a custom Auth Provider

I think we also need to think a solution on how to prevent this kind of shade behavior. It is not good to expose shade classes to any public interface. - Sijie On Wed, Oct 19, 2016 at 9:40 AM, Sijie Guo wrote: > Ah, good catch, Enrico. > > It is really bad for this behavior. I think

BookKeeper Meetup at Yahoo - Tuesday, Nov 15, 2016

http://www.meetup.com/Apache-BookKeeper/events/235098921/ Community participation is greatly appreciated. Please RSVP. Agenda • 6:00 - 6-30 - Socialize over food and beers • 6:30 - 7:00 - Reliable Data Replication using Apache DistributedLog/BookKeeper (Twitter) • 7:00 - 7:30 - Sherpa - Distri

Re: BookKeeper Meetup at Yahoo - Tuesday, Nov 15, 2016

(Matteo is on vacation. I sent this announcement out for him) - Sijie On Tue, Nov 1, 2016 at 10:12 PM, Sijie Guo wrote: > http://www.meetup.com/Apache-BookKeeper/events/235098921/ > > Community participation is greatly appreciated. Please RSVP. > > Agenda > > • 6:00 -

[ANNOUNCE] New BookKeeper Committer: Siddharth Sunil Boobna

The Apache BookKeeper PMC recently extended committer karma to Siddharth Sunil Boobna and he has accepted. Siddharth has worked with Matteo at Yahoo, contributing bookkeeper changes to the community. He is active in code contributions, discussion and code reviews. We are looking forward to more con

Re: Bookkeeper Recovery Issues

Sebastian, What version are you running? What is the ensemble size, write quorum size for your ledger? - Sijie On Fri, Nov 18, 2016 at 12:00 PM, Sebastián Schepens < sebastian.schep...@mercadolibre.com> wrote: > Hi guys, > I'm running into some issues while trying to recover a decomissioned nod

Re: Bookkeeper Recovery Issues

I think what Sebastian said is that manual recovery didn't even work. This seems to a bit strange to me. The autorecovery will check if the bookie is available or not. After that, it should rereplicate the data from other nodes in the ensemble. This seems to indicate something is broken. Sebastian,

Re: Bookkeeper Recovery Issues

> > > The cluster currently has 6 nodes, and as I said before we're using > ensemble size 3, write quorum 3 and ack quorum 2. > > Thanks, > Sebastian > > On Tue, Nov 22, 2016 at 2:10 PM Sijie Guo wrote: > > I think what Sebastian said is that manual recovery didn&

Re: Bookkeeper Recovery Issues

nds and thousands of these logs while >> some data movement seems to be occurring, but it's really weird that it's >> constantly trying to connect to the failed node. >> Couldn't it realize it's down because it's not shown as available on >> zookeeper? &g

Re: Bookkeeper Recovery Issues

ly distribuiting ledgers across > all nodes. > > > Agian, thank you very much for all the help. > > Cheers, > > Sebastian > > On Mon, Nov 28, 2016 at 5:52 PM Sijie Guo wrote: > >> Sebastian, >> >> Sorry for late response. (I didn't have netwo

Re: Understanding LAC

On Fri, Dec 2, 2016 at 8:19 AM, Enrico Olivelli wrote: > Hi, > I'm doing some benchs in order to use BookKeeper as low-latency data > storage, but I'm missing some piece of the LAC protocol. > > 1) From client1 I create a ledger and then perform an addEntry and > wait for the ACK. My LedgerHandle

Re: Understanding LAC

tually I'm going to store BLOBs) > Is it possible to store the BLOBs as separated ledgers? That might be easier to manage. As far as I will have a working proof-of-concept I will share by code on > GitHub, I think this will be another awesome example of usage of BookKeeper > &g

Re: Ledgers failing to replicate

On Wed, Jan 11, 2017 at 11:15 AM, Sebastián Schepens < sebastian.schep...@mercadolibre.com> wrote: > Hi guys, > I'm doing some tests and turned off 2 bookies almost simultaneously hoping > that all the ledgers would still be able to replicate since we have > ensemble and quorum size of 3. > Almost

Re: Ledgers failing to replicate

ebastian On Wed, Jan 11, 2017 at 5:04 PM Sijie Guo wrote: > On Wed, Jan 11, 2017 at 11:15 AM, Sebastián Schepens mercadolibre.com> wrote: > > Hi guys, > I'm doing some tests and turned off 2 bookies almost simultaneously hoping > that all the ledgers would still be able to

Re: Ledgers failing to replicate

lity that I can think of - there are ledgers created after the auditor of auto recovery audits all the existing ledgers. What is your auditorPeriodicBookieCheckInterval ? - Sijie > > Thanks for the tip on the quorums! > > Sebastian > > On Thu, Jan 12, 2017 at 1:31 PM Sijie Guo w

Re: Ledgers failing to replicate

On Thu, Jan 12, 2017 at 10:33 AM, Sebastián Schepens < sebastian.schep...@mercadolibre.com> wrote: > On Thu, Jan 12, 2017 at 3:20 PM Sijie Guo wrote: > >> On Thu, Jan 12, 2017 at 9:12 AM, Sebastián Schepens > mercadolibre.com> wrote: >> >> Pulsar is creating

Re: HerdDB: a Distributed 'embeddable' Database built on Apache BookKeeper

Congrats Enrico! It would be great if you can share more details about embedding bookkeeper as a jvm-local library. I think that is a very interesting topic. (I missed this email during holidays) - Sijie On Fri, Dec 30, 2016 at 8:02 AM, Enrico Olivelli wrote: > Hi all, > I'm glad to announce t

Re: Memory constraints on Bookie

On Tue, Feb 28, 2017 at 11:54 PM, Enrico Olivelli wrote: > Hi, > I would like to put a limit on the amount of memory used by a Bookie. > The reason is that I am running the Bookie inside a JVM together with > other services and so it is quite important to be able to constraint the > usage of memo

Re: Write guarantees and the LAC Protocol

On Wed, Mar 15, 2017 at 9:17 AM, Enrico Olivelli wrote: > Hi BookKeepers, > I'm trying to understand some basic aspect of BookKeeper > > I have a strange case, and I would like to have some opinion from any > "expert". > > I am using BookKeeper as a transaction log for a database, this is an

Apache BookKeeper Talk at #lfvault

FYI. This is the slides of the bookkeeper talk JV and me gave at #lfvault. You can check it out if you are interested in. https://vault2017.sched.com/event/A4wM/apache-bookkeeper-a-high-performance-and-low-latency-cloud-storage-service-sijie-guo-twitter http://www.slideshare.net/hustlmsp/apache

Re: New BookKeeper Proposal "Allow Reads outside the LAC Protocol"

+ dev@ and moved user@ to cc On Thu, Mar 30, 2017 at 8:26 AM, Enrico Olivelli wrote: > I would like to share with the community this BookKeeper proposal > > https://cwiki.apache.org/confluence/display/BOOKKEEPER/ > BP-5+Allow+reads+outside+the+LAC+Protocol > > I will submit a Pull Request to exp

Re: New BookKeeper Proposal "Allow Reads outside the LAC Protocol"

) or use readUncommittedEntries (to read potentially *uncommitted* entries. So the correctness is managed by application itself (out side of the logic). - Sijie On Thu, Mar 30, 2017 at 8:55 AM, Sijie Guo wrote: > + dev@ and moved user@ to cc > > On Thu, Mar 30, 2017 at 8:26 AM, Enrico

Re: New BookKeeper Proposal "Allow Reads outside the LAC Protocol"

til it reaches a > > NoEntry error. > > > > Thank you Matteo > That's exacly what I meant. But I would like to port it to the standard > client (non-admin) > I will refer to this work in the BP > > > > > > > > > > On Fri, Mar 31, 2017 a

High Performance Logging with Apache BookKeeper

FYI. https://www.linux.com/news/event/apachecon/2017/4/high-performance-logging-apache-bookkeeper This is the follow article for Apache BookKeeper talk at #vault. - Sijie

[ANNOUNCE] New BookKeeper Committer: Enrico Olivelli & Charan Reddy

The Apache BookKeeper PMC recently extended committer karma to Enrico Olivelli & Charan Reddy and they have accepted. Charan Reddy and Enrico Olivelli have been contributing bookkeeper changes to the community for a while. They are active in code contributions, discussion and code reviews. We are

Re: BookKeeper and CheckStyle

Hi, >>>> >>>> Yes, I created a PR for the first part of the change. (All modules >>>> except >>>> bookkeeper-server) >>>> I started to do the second part (bookkeeper-server), but It is a huge >>>> change. It contains almost 5000 issue

[ANNOUNCE] JV Jujjuri joins the Apache BookKeeper PMC

On behalf of the Apache BookKeeper PMC I am pleased to announce that JV Jujjuri has accepted our invitation to become a PMC member on the Apache BookKeeper project. JV has been an active contributor in many areas, including driving community meetings, reviewing patches, helping with triaging bugs

Re: BookKeeper and CheckStyle

per-server), but It is a huge >>> change. It contains almost 5000 issues. >>> I'm thinking about how to slice it up to small steps. >>> >> >> Thank you David, >> Yep, the idea is to create a single patch per package if possible >> >&g

Re: BookKeeper#openLedgerNoRecovery hangs

On Wed, Jul 19, 2017 at 4:04 PM, Enrico Olivelli wrote: > Hi, > in some internal benchmarks we are experiencing openLedgerNoRecovery calls > which remain hung. > I see that basically that function calls ZookKeeper#getData. > > Does anyone have an idea of how it can happen ? > What version are y

Re: InterleavedLedgerStorage vs SortedLedgerStorage

Currently we don't have such documentation about these two ledger storages. We are working on improving the documentation as part of BP-11. - Sijie On Wed, Jul 19, 2017 at 3:34 PM, Enrico Olivelli wrote: > Hi, > is there any doc about the differences and usecases for > InterleavedLedgerStorage

Re: InterleavedLedgerStorage vs SortedLedgerStorage

, better I/O isolation. But if you only have a few ledgers or you rarely read from the ledgers, the memtables in Sorted might bring in overhead for you. - Sijie > Enrico > > Il mer 19 lug 2017, 11:12 Sijie Guo ha scritto: > >> Currently we don't have such documentation about t

[ANNOUNCE] Four DistributedLog PPMC members joins the Apache BookKeeper PMC

As DistributedLog graduated as a subproject of Apache BookKeeper, I am pleased to announce that following DistributedLog PPMC members have accepted our invitations to join Apache BookKeeper PMC on behalf of the Apache BookKeeper PMC. - Dave Rusek (drusek) - Franck Cuny (fcuny) - Henry Saputra (hsa

Re: BookKeeper docker image is avaliable under apache now

Awesome, Francesco and Jia! Nice work! - Sijie On Fri, Aug 11, 2017 at 3:53 AM, Enrico Olivelli wrote: > Great! > Thank you all, expecially Jia and Francesco ! > > Enrico > > Il ven 11 ago 2017, 10:48 Jia Zhai ha scritto: > >> FYI. BookKeeper docker image is avaliable under apache now: >> htt

Re: [ANNOUNCE] Apache BookKeeper 4.5.0 released

Awesome! Well done! This is a great milestone to the bookkeeper community! Thanks JV for doing the release! Thanks everyone for making this happen. Sijie On Aug 11, 2017 9:56 AM, "JV Jujjuri" wrote: > The Apache BookKeeper team is proud to announce Apache BookKeeper version > 4.5.0. > > This i

Re: BookKeeper docker image is avaliable under apache now

>> On Fri, Aug 11, 2017 at 6:04 AM, Francesco Caliumi - Diennea < >> francesco.cali...@diennea.com> wrote: >> >> > Good, happy to finally see it in docker hub! Thank you all! >> > >> > On Fri, 2017-08-11 at 03:59 -0700, Sijie Guo wrote: >> >

Re: HTTP / REST API?

Hi Istvan, This feature is coming in the next `4.6.0` release (it should be available in 2 weeks or so). We have an issue to document this feature. At the same time, you can checkout this BP for details - https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP-17%3A+Define+BookKeeper+public+ht

Re: HTTP / REST API?

And yes, as Enrico said, it only supports admin related API. On Tue, Nov 7, 2017 at 3:02 AM, Enrico Olivelli wrote: > Hi Istvan, > > 2017-11-07 11:17 GMT+01:00 Istvan Soos : > >> Hi, >> >> Is there a HTTP / REST-like API for bookkeeper? I've seen that there >> is this sub-project, but even if it

Re: log compaction of entries

On Tue, Nov 7, 2017 at 10:47 AM, Istvan Soos wrote: > On the website [0] I gather that data compaction is mostly about > cleaning up after we delete a ledger. Is there a feature or plan to > implement entry-level compaction, e.g. to have an ID that uniquely > identifies an entity, and if there ar

Re: HTTP / REST API?

+ Jia I think when Jia did BP-17 initially, he actually had the apis for ledgers for writing and reading. I think he was asked for removing the API during reviews (it might be me). We can see if it is still there in his branches. Then we can discuss bringing this features back in. - Sijie On Tu

Re: HTTP / REST API?

On Tue, Nov 7, 2017 at 7:22 AM, Enrico Olivelli wrote: > > > 2017-11-07 16:16 GMT+01:00 Ivan Kelly : > >> > We have an issue to document this feature. At the same time, you can >> > checkout this BP for details - >> > https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP- >> 17%3A+Define+Boo

Re: log compaction of entries

On Tue, Nov 7, 2017 at 12:18 PM, Istvan Soos wrote: > On Tue, Nov 7, 2017 at 8:09 PM, Sijie Guo wrote: > > But I would to learn more about your use case and to see how we can > support > > you. > > It is a nice feature in Kafka, and I've seen a complex app using i

Re: HTTP / REST API?

ms need more thinking. > > Thanks. > -Jia > > On Wed, Nov 8, 2017 at 3:18 AM, Sijie Guo wrote: > >> + Jia >> >> I think when Jia did BP-17 initially, he actually had the apis for >> ledgers for writing and reading. I think he was asked for removing the AP

Re: [ANNOUNCE] Apache BookKeeper 4.5.1 released

Awesome! Well done, Enrico. Thanks Enrico for doing this release and making this happen. Thanks everyone in the community for making this happen. - Sijie On Wed, Nov 22, 2017 at 12:53 PM, Enrico Olivelli wrote: > The Apache BookKeeper team is proud to announce Apache BookKeeper version > 4.5.1

[ANNOUNCE] New BookKeeper Committer: Yiming Zang

The Apache BookKeeper PMC recently extended committer karma to Yiming Zang and he has accepted. Yiming Zang has been contributing bookkeeper & distributedlog changes to the community for a while. He is active in code contributions, discussion and code reviews. We are looking forward to more contri

Re: [ANNOUNCE] Apache BookKeeper 4.6.0 released

Jia, congrats for you first release! Thanks everyone for making contributions to 4.6.0 release. - Sijie On Mon, Dec 25, 2017 at 11:23 PM, Jia Zhai wrote: > The Apache BookKeeper team is proud to announce Apache BookKeeper version > 4.6.0. > > Apache BookKeeper is a scalable, fault-tolerant, an

Re: [ANNOUNCE] Apache BookKeeper 4.6.1 released

Congrats Enrico! Thank you so much for driving 4.6.1 release. Thanks everyone for making contributions to bookkeeper and making this happen! - Sijie On Wed, Jan 31, 2018 at 12:04 AM, Enrico Olivelli wrote: > The Apache BookKeeper team is proud to announce Apache BookKeeper version > 4.6.1. > >

[ANNOUNCE] New BookKeeper Committer: Sam Just

The Apache BookKeeper PMC recently extended committer karma to Sam Just and he has accepted. Sam Just has been contributing bookkeeper changes to the community for a while. He is the main person who is leading the efforts of pushing salesforce changes to open source. He has been active in code con

[ANNOUNCE] New BookKeeper Committer: Andrey Yegorov

The Apache BookKeeper PMC recently extended committer karma to Andrey Yegorov and he has accepted. Andrey Yegorov has been making contributions to bookkeeper community since a few years ago. He is active in sending pull requests, attending the discussions in the community meetings. He made a coupl

[ANNOUNCE] Enrico Olivelli and Jia Zhai join the Apache BookKeeper PMC

On behalf of the Apache BookKeeper PMC I am pleased to announce that Enrico Olivelli and Jia Zhai have accepted our invitation to become a PMC member on the Apache BookKeeper project. Please join me in thanking Enrico and Jia for their contributions to date and anticipation of many more contributi

Re: Help with bad errors on 4.6.1

On Thu, Mar 8, 2018 at 7:42 AM, Enrico Olivelli wrote: > > > 2018-03-08 14:50 GMT+01:00 Ivan Kelly : > >> It just occurred to me that this could be a problem with the recycler. >> If we recycle a buffer too early, but then keep using it, another user >> could pick it up, and between them they cou

Re: Help with bad errors on 4.6.1

idea of solution for that is to run PCBC's errorOut > asynchronously / submit it back to orderedSafeExecutor so addOp can > increment counts properly. > Oh PendingAddOp should be executed at safe executor. Otherwise it is not just recycle problem, it might be a correctness issue. >

Re: Help with bad errors on 4.6.1

wrote: > > > Il gio 8 mar 2018, 20:52 Sijie Guo ha scritto: > >> On Thu, Mar 8, 2018 at 7:42 AM, Enrico Olivelli >> wrote: >> >>> >>> >>> 2018-03-08 14:50 GMT+01:00 Ivan Kelly : >>> >>>> It just occurred to me that th

Re: Help with bad errors on 4.6.1

So the problem here is: - a corrupted request failed the V3 request decoder, so bookie switched to use v2 request decoder. Once the switch happen, the bookie will always use v2 request decoder decoding v3 request. then all your v3 requests will be failing with unknown op and trigger the bytebuf do

Re: Help with bad errors on 4.6.1

Sent out a PR for the issues that I observed: https://github.com/apache/bookkeeper/pull/1240 On Thu, Mar 8, 2018 at 10:47 PM, Sijie Guo wrote: > So the problem here is: > > - a corrupted request failed the V3 request decoder, so bookie switched to > use v2 request decoder. Onc

[ANNOUNCE] Apache BookKeeper 4.7.0 released

The Apache BookKeeper team is proud to announce Apache BookKeeper version 4.7.0. Apache BookKeeper is a scalable, fault-tolerant, and low-latency storage service optimized for real-time workloads. It has been used for a fundamental service to build reliable services. It is also the log segment sto

Re: Community Meeting

Thank you Enrico, Yiming. 👍 On Thu, Jun 14, 2018 at 8:23 AM Enrico Olivelli wrote: > > > Il giorno gio 14 giu 2018 alle ore 15:40 Venkateswara Rao Jujjuri < > jujj...@gmail.com> ha scritto: > >> Sorry I can’t make it today. But I will get to the BP 14 review today. >> Please publish minutes. Th

Re: Expected workload for DBLedgerStorage ?

On Wed, Jun 20, 2018 at 4:09 AM Enrico Olivelli wrote: > Hi, > I am experimenting DBLedgerStorage with some project. > With my workloads I don't see much benefit, or sometimes a performance > degradation, > Do you have a performance comparison? Or can you describe what have you observed? > jus

  1   2   >