I have just created a PR to upgrade RocksDB and resolve the license issue. https://github.com/apache/incubator-pulsar/pull/587
We don't need to wait on BK release, since we can force which RocksDB version we want to include. -- Matteo Merli <mme...@apache.org> On Tue, Jul 18, 2017 at 2:11 PM, Matteo Merli <mme...@apache.org> wrote: > Dave, > I think RocksDB is cutting a new release 5.5.4 with Apache License right > now : > > https://github.com/facebook/rocksdb/commits/5.5.fb > > Thankfully they're moving quickly to solve the issue! > > Matteo > > -- > Matteo Merli > <mme...@apache.org> > > On Tue, Jul 18, 2017 at 9:17 AM, Dave Fisher <dave2w...@comcast.net> > wrote: > >> Hi - >> >> Current guidance is to wait for the relicensed version of RocksDB before >> it can be allowed. Projects currently have until August 31 to resolve the >> issues in Apache releases. Pulsar does not yet have an Apache release. >> >> It becomes a question of timing within RocksDB, Bookkeeper and this >> podling. If the podling gets to the point of releasing before the proper >> licensed version is available then following Matteo’s suggestion is >> probably needed. That wouldn’t stop the project from publishing benchmark >> data based on. RocksDB. >> >> Regards, >> Dave >> >> > On Jul 16, 2017, at 12:02 PM, Dave Fisher <dave2w...@comcast.net> >> wrote: >> > >> > Hi - >> > >> > Please see https://issues.apache.org/jira/plugins/servlet/mobile#issue/ >> LEGAL-303/comment/16088730 >> > >> > It looks like the RocksDB community is resolving the issue! >> > >> > Regards, >> > Dave >> > >> > >> > Sent from my iPhone >> > >> >> On Jul 14, 2017, at 3:26 PM, Matteo Merli <mme...@apache.org> wrote: >> >> >> >> As we have already talked about, the RocksDb library has been put in >> the >> >> Category-X (Forbidden) by the ASF, due to licensing problems. >> >> >> >> With my very limited understanding of the issue, it seems the problem >> >> resides in the patent clauses that's added the license. >> >> >> >> https://www.apache.org/legal/resolved.html#category-x >> >> ------------------------------------------------------------ >> ------------ >> >> Rocks DB License >> >> >> >> The Rocks DB license includes a specification of a PATENTS file that >> passes >> >> along risk to downstream consumers of our software imbalanced in favor >> of >> >> the licensor, not the licensee, thereby violating our Apache legal >> policy >> >> of being a universal donor <https://s.apache.org/4Uzg>. The terms of >> >> ROCKSDB license are not a subset of those found in the ALv2, and they >> >> cannot be sublicensed as ALv2. >> >> ------------------------------------------------------------ >> ------------ >> >> >> >> Pulsar is using RocksDb since that is being pulled in from the Yahoo >> branch >> >> of BookKeeper, which includes an alternative LedgerStorage >> implementation >> >> that stores the indexes for the entries in a RocksDB databases instead >> of >> >> using individual index files as it's default for BookKeeper. That is >> >> important to store many tens of thousands of ledgers per single Bookie >> node. >> >> >> >> To be able to release "Apache Pulsar" we need to fix the dependency >> >> problem. These are the options that I have been thinking of (feel free >> to >> >> propose alternatives! ): >> >> >> >> 1. Configure out of the box Pulsar to use InterleavedLedgerStorage and >> >> explicitly exclude the RocksDb dependency from BookKeeper (yahoo >> branch) >> >> >> >> 2. Provide an additional key-value storage implementation for the >> >> DbLedgerStorage (eg: LevelDb) and make that the default >> >> >> >> 3. Wait until there's more clarity on the subject (that would mean to >> >> delay the Pulsar release) >> >> >> >> As I see it, I prefer option #1 for now, because I don't see any >> comparable >> >> replacement for RocksDB. LevelDb works fine for small workloads but >> show >> >> several problems at higher load, and if we include it, we would need to >> >> support it. >> >> >> >> The main concern I have is related to the out-of-the-box performance. >> We >> >> need to document how to enable the RocksDb configuration and we should >> make >> >> clear that the published results were accomplished with that config. >> >> >> >> Additionally, since the ultimate goal for the DbLedgerStorage was to be >> >> merged into BookKeeper, it would be good to understand what are the >> options >> >> on that side. >> >> For example, will it be fine to have RocksDb as an optional dependency >> >> (needed to compile from source, but not used or included in binary >> >> distribution) ? >> >> That way we could include the KeyValueRocksDb java class that uses the >> API >> >> and compile it. Then, since the dependency is optional, it would >> require >> >> the user to explicitly include RocksDB to have it working. >> >> >> >> >> >> What do you think? >> >> Matteo >> >> >> >> >> >> -- >> >> Matteo Merli >> >> <mme...@apache.org> >> >> >
