2018-03-09 15:25:15 UTC - Nicolas Ha: I am trying to set up a CI environment 
using CircleCI 2, the command I use to run tests fails with this:

```
Caused by: java.io.IOException: Failed to load RocksDB JNI library
 at 
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.<init> 
(KeyValueStorageRocksDB.java:81)
    
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.<init> 
(KeyValueStorageRocksDB.java:73)
    
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB$1.newKeyValueStorage
 (KeyValueStorageRocksDB.java:44)
    org.apache.bookkeeper.bookie.storage.ldb.LedgerMetadataIndex.<init> 
(LedgerMetadataIndex.java:51)
    org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize 
(DbLedgerStorage.java:142)
    org.apache.bookkeeper.bookie.Bookie.<init> (Bookie.java:516)
    org.apache.bookkeeper.proto.BookieServer.newBookie (BookieServer.java:313)
    org.apache.bookkeeper.proto.BookieServer.<init> 
(BookieServer.java:116)
    org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.runBookies 
(LocalBookkeeperEnsemble.java:179)
    org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.startStandalone 
(LocalBookkeeperEnsemble.java:231)
    sun.reflect.NativeMethodAccessorImpl.invoke0 
(NativeMethodAccessorImpl.java:-2)
    sun.reflect.NativeMethodAccessorImpl.invoke 
(NativeMethodAccessorImpl.java:62)
    sun.reflect.DelegatingMethodAccessorImpl.invoke 
(DelegatingMethodAccessorImpl.java:43)
    java.lang.reflect.Method.invoke (Method.java:498)
    clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:93)
    clojure.lang.Reflector.invokeNoArgInstanceMember (Reflector.java:313)
    backend.components.pulsar_broker.PulsarBroker/fn (pulsar_broker.clj:109)
    backend.components.pulsar_broker.PulsarBroker.start (pulsar_broker.clj:101)
```

Is this a know issue with CI providers? Do I have to package a special 
dependency for some environments?
----
2018-03-09 17:25:14 UTC - Matteo Merli: @Nicolas Ha What kind of environment 
the CI build is running on?
----
2018-03-09 17:27:09 UTC - Nicolas Ha: not sure about the details - I was trying 
out circle CI 2.0 <https://circleci.com/docs/2.0/>
I know 1.0 works fine, but as far as I know 2.0 is nearly a rewrite to support 
a container/docker-based workflow
----
2018-03-09 17:27:23 UTC - Nicolas Ha: I can SSH into the box though, so I will 
try learning more
----
2018-03-09 17:27:54 UTC - Matteo Merli: Ok, also is there any other logged 
exception that explains why it fails to load the library?
----
2018-03-09 17:28:33 UTC - Matteo Merli: @david-jin You can tune down the memory 
a lot more. In any case, if you have a single VM for all the components, you 
should consider using the pulsar standalone, rather than the individual services
----
2018-03-09 17:29:14 UTC - Matteo Merli: With standalone, you have 1 single JVM 
process that contains everything (broker, bookie, zookeeper).
----
2018-03-09 17:30:11 UTC - Matteo Merli: I’ve seen it failing on Windows… but it 
should work in virtually every linux/mac distribution, since the jni library is 
statically compiled
----
2018-03-09 17:30:28 UTC - Nicolas Ha: Oh there is actually! I missed that:
```
Caused by: java.lang.UnsatisfiedLinkError: 
/tmp/librocksdbjni4356631003914935156.so: Error loading shared library 
ld-linux-x86-64.so.2: No such file or directory (needed by 
/tmp/librocksdbjni4356631003914935156.so)
 at java.lang.ClassLoader$NativeLibrary.load (ClassLoader.java:-2)
    java.lang.ClassLoader.loadLibrary0 (ClassLoader.java:1941)
    java.lang.ClassLoader.loadLibrary (ClassLoader.java:1824)
    java.lang.Runtime.load0 (Runtime.java:809)
    java.lang.System.load (System.java:1086)
    org.rocksdb.NativeLibraryLoader.loadLibraryFromJar 
(NativeLibraryLoader.java:78)
    org.rocksdb.NativeLibraryLoader.loadLibrary (NativeLibraryLoader.java:56)
    org.rocksdb.RocksDB.loadLibrary (RocksDB.java:64)
    org.rocksdb.RocksDB.&lt;clinit&gt; (RocksDB.java:35)
    
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.&lt;init&gt; 
(KeyValueStorageRocksDB.java:79)
    
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.&lt;init&gt; 
(KeyValueStorageRocksDB.java:73)
    
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB$1.newKeyValueStorage
 (KeyValueStorageRocksDB.java:44)
    org.apache.bookkeeper.bookie.storage.ldb.LedgerMetadataIndex.&lt;init&gt; 
(LedgerMetadataIndex.java:51)
    org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize 
(DbLedgerStorage.java:142)
    org.apache.bookkeeper.bookie.Bookie.&lt;init&gt; (Bookie.java:516)
    org.apache.bookkeeper.proto.BookieServer.newBookie (BookieServer.java:313)
    org.apache.bookkeeper.proto.BookieServer.&lt;init&gt; 
(BookieServer.java:116)
    org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.runBookies 
(LocalBookkeeperEnsemble.java:179)
    org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.startStandalone 
(LocalBookkeeperEnsemble.java:231)
```
----
2018-03-09 17:31:26 UTC - Nicolas Ha: (using 
`org.apache.pulsar/pulsar-broker-shaded "1.21.0-incubating"`)
----
2018-03-09 17:31:40 UTC - Matteo Merli: Ok, is that based on Alpine linux ?
----
2018-03-09 17:32:27 UTC - Nicolas Ha: It is based on 
`clojure:boot-2.7.2-alpine` (so I guess yes)
----
2018-03-09 17:33:01 UTC - Matteo Merli: ok, that’s the issue, Alpine doesn’t 
have the regular linux glibc
----
2018-03-09 17:33:27 UTC - Matteo Merli: can you switch the image to one based 
on debian/redhat ?
----
2018-03-09 17:33:51 UTC - Nicolas Ha: I sure can :slightly_smiling_face: which 
one is it? (not super familiar with the current state of docker)
----
2018-03-09 17:34:17 UTC - Nicolas Ha: I think all I have to do is edit this one 
<https://github.com/Quantisan/docker-clojure/blob/master/alpine/boot/Dockerfile>
 and change the “FROM”
----
2018-03-09 17:34:21 UTC - Matteo Merli: just try to remove the “alpine” tag: 
`clojure:boot-2.7.2`
----
2018-03-09 17:34:53 UTC - Nicolas Ha: Oh right! :smile:
----
2018-03-09 17:39:59 UTC - Nicolas Ha: it worked! onto the next error 
:slightly_smiling_face: Thanks!
----
2018-03-09 17:42:21 UTC - Matteo Merli: :slightly_smiling_face:
----
2018-03-10 01:11:31 UTC - David Kjerrumgaard: @David Kjerrumgaard has joined 
the channel
----
2018-03-10 01:36:46 UTC - david-jin: but, i want to experiment with the 
geo-replication.  if i just use standalone mode, can i deploy a instance? and 
geo-replication?
----
2018-03-10 02:03:07 UTC - david-jin: Is there any advice to deploy an instance 
using standalone mode ?
----
2018-03-10 02:07:05 UTC - david-jin: So, if I want to deploy an instance. I 
should first deploy 3 global zookeeper in 3 machines, then deploy 3 standalone 
pulsar?  Is that right?
----
2018-03-10 02:08:25 UTC - david-jin: thus, every machine have 2 JVMs?
----
2018-03-10 02:10:15 UTC - Matteo Merli: For this kind of scenario, with just 
one machine in each region, I would suggest to not bother deploying the global 
zookeeper.  Just do the standalone in each region and set the configuration 
manually in each of them
----
2018-03-10 02:11:31 UTC - Matteo Merli: For example you have to use the CLI 
tool to create the "cluster" and pass a name and the URL for that cluster (the 
IP of the standalone service...)
----
2018-03-10 02:12:13 UTC - Matteo Merli: You have to do that for each region so 
that everyone knows about the others and where they sre
----
2018-03-10 02:12:21 UTC - david-jin: but, how to do the geo-replication 
manually?
----
2018-03-10 02:13:01 UTC - Matteo Merli: After that, you can create a global 
namespace and set it to do replication between regions a, b and c
----
2018-03-10 02:13:36 UTC - Matteo Merli: Since there is no global zookeeper, you 
have to repeat the operation for each region 
----
2018-03-10 02:13:57 UTC - Matteo Merli: At that point everything is done
----
2018-03-10 02:14:35 UTC - david-jin: In my opion, pulsar's advantage over other 
MQ, the geo-replication is important.
----
2018-03-10 02:15:39 UTC - Matteo Merli: Yes, the point is that you get 
geo-replication of data without need for global zookeeper 
----
2018-03-10 02:16:03 UTC - Matteo Merli: That is just to keep the configuration 
consistent across region
----
2018-03-10 02:16:25 UTC - Matteo Merli: Which is necessary if you have many 
namespaces and tenants
----
2018-03-10 02:16:59 UTC - Matteo Merli: But otherwise you can get away without 
it -- but still have geo replicationw
----
2018-03-10 02:17:53 UTC - Matteo Merli: Let me try to get a simple example for 
replicating between 2 standalone servers
----
2018-03-10 02:30:06 UTC - david-jin: yes, could you give me an example? thank 
you very much.
----
2018-03-10 02:44:36 UTC - Matteo Merli: @david-jin Here’s the quick example to 
setup the replication between Pulsar standalone processes: 
<https://gist.github.com/merlimat/bfdd3c66bfa7387fe6e3bfa3bb263051>
----
2018-03-10 02:44:45 UTC - Matteo Merli: @Matteo Merli shared a file: 
<https://apache-pulsar.slack.com/files/U680ZCXA5/F9M31L1PS/standalonereplication.md|StandaloneReplication.md>
----
2018-03-10 03:17:36 UTC - david-jin: thanks very much
----
2018-03-10 03:49:23 UTC - Matteo Merli: no problem, let me know if it works for 
you
----

Reply via email to