2018-01-11 22:36:03 UTC - Tiago Caxias: @Tiago Caxias has joined the channel
----
2018-01-12 00:19:20 UTC - Jaebin Yoon: I'm trying to bring up pulsar in my
company's environment where I created a wrapper for pulsar broker and bookie.
Basically the wrapper sets up company environment and starts the broker or
bookie server. In the runtime, I'm getting protobuf-java jar library issue
since the wrapper requires 3.2.0 while pulsar is using 2.4.1 (I'm using
pulsar-1.21.0).
The following is the error message I'm getting with protobuf-java-3.2.0.jar in
runtime.
```java.lang.NoSuchMethodError:
com.google.protobuf.TextFormat.printToString(Lcom/google/protobuf/Message;)Ljava/lang/String;
at org.apache.bookkeeper.bookie.Cookie.toString(Cookie.java:121)
at org.apache.bookkeeper.bookie.Cookie.writeToDirectory(Cookie.java:175)
at org.apache.bookkeeper.bookie.Bookie.checkEnvironment(Bookie.java:359)
at org.apache.bookkeeper.bookie.Bookie.<init>(Bookie.java:481)```
Is there any reason that pulsar broker and bookeeper need to stay with
protobuf-java-2.4.1 ? Any issue with moving to 3.x ?
----
2018-01-12 02:36:38 UTC - Jaebin Yoon: I just got the yahoo forked Bookkeeper
and regenerated code from protobuf definitions with protoc (v3.5) and built the
bookkeeper-server jar. It seems it works with that. I noticed that apache
bookkeeper has already moved to protobuf 3.4.0
----
2018-01-12 04:16:24 UTC - Jaebin Yoon: I tried to do the same thing for pulsar
broker but it seems the pulsar broker is using the modified version of protobuf
code generator. Hmm not sure the patch would work with v3.5.0
----
2018-01-12 06:44:07 UTC - Sijie Guo: @Jaebin Yoon can you try
pulsar-broker-shaded?
----
2018-01-12 07:14:01 UTC - Jaebin Yoon: oh what is it? it seems it's just a
packaging. What I'm doing is that I have my own java main to hook up in-house
environment and have dependency to pulsar-broker and instantiate PulsarService
object from that main.
----
2018-01-12 07:20:01 UTC - Jaebin Yoon: The crux of the problem is that pulsar
is using protobuf 2.x and our environment is using protobuf 3.x. I'm trying to
apply the pulsar protobuf patch to 3.x and generate code. That seems to be the
only way if I want to keep my environment and pulsar in the same JVM.
----
2018-01-12 08:49:18 UTC - Matteo Merli: Hi @Jaebin Yoon, I think the problem
would be there even using protobuf 3.x, because the code generated is typically
not compatible with the runtime lib of a different minor version.
----
2018-01-12 08:50:12 UTC - Matteo Merli: The only option is indeed to shade
protobuf (and other potentially conflicting dependencies) when embedding the
broker into an existing Java process
----
2018-01-12 08:51:44 UTC - Matteo Merli: initially we were just shading the
client lib, though recently @Nicolas Ha added support to have the shaded broker
artifact, but that will be included in 1.22 release
----
2018-01-12 08:53:51 UTC - Matteo Merli: if you want to have the “shaded broker
jar” without waiting for 1.22 (or using master), you could also create a module
in your project that depends on pulsar broker and then applies the shading. You
just need a `pom.xml` like in
<https://github.com/apache/incubator-pulsar/tree/master/pulsar-broker-shaded>
----
2018-01-12 08:57:58 UTC - Matteo Merli: > Is there any reason that pulsar
broker and bookeeper need to stay with protobuf-java-2.4.1 ? Any issue with
moving to 3.x ?
In any case, yes, we’ll move to 3.x soon, after we do the switch from
BookKeeper-4.3-Yahoo to BookKeeper-4.7. The patch itself is very easy to port
to the new protobuf version (The code is just adding hooks so that we can avoid
all allocation when serializing/deserializing protobuf objects)
----
2018-01-12 09:04:02 UTC - Matteo Merli: @Jaebin Yoon I actually stand
corrected: `pulsar-broker-shaded` was already included in 1.21 release.
<https://mvnrepository.com/artifact/org.apache.pulsar/pulsar-broker-shaded>
----
2018-01-12 09:53:31 UTC - Nicolas Ha: Yes - I had the exact same problem
:smile: Using the published version of `pulsar-broker-shaded` works for me with
my current protobuf dependency (same JVM as well).
Note: I would still welcome `SNAPSHOT` versions at some point - this would make
it easier to experiment with not shading Netty for instance.
----
2018-01-12 17:15:27 UTC - Jaebin Yoon: @Matteo Merli @Sijie Guo oh this is
nice. I didn't know about maven-shade-plugin. I'll try that. Thanks a lot!!
----
