In our application we are fetching data from kafka, doing some modifications in data and then persist that data into elastic search.
As we are using elasticsearch verion 2.2.3, first we did upgrade of elastic search client API in malhar contrib from 1.1.2 to 2.3.3 [ here is pull request : https://github.com/apache/apex-malhar/pull/325 ] With new elastic search api we able to run the application in local mode using junite test case which worked very well and able to see the data in elastic search. Now we are trying to run our application using APEX cli. First we get the exception about the googles guava library: *java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor; at org.elasticsearch.threadpool.ThreadPool.<clinit>(ThreadPool.java:190) at org.elasticsearch.client.transport.TransportClient$Builder.build(TransportClient.java:131) at com.datatorrent.contrib.elasticsearch.ElasticSearchConnectable.connect(ElasticSearchConnectable.java:123)* The reason behind above exception is : elastic search java client requires guava 18.0 and the Hadoop 2.2 comes with guava 11.0.2. In our application package (apa file) we have guava 18.0 jar. But i think while launching application apex gives preference to the guava 11.0.2. So we updated the apex-engine.jar and changed guava version to 18.0 manually which is not good practice :) After this we are able to see that application is running in hadoop cluster but in log there is an exception : *2016-06-28 15:54:28,721 ERROR com.datatorrent.netlet.AbstractClient: Exception in event loop {id=ProcessWideEventLoop, head=1, tail=1, capacity=1024} java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at com.datatorrent.netlet.DefaultEventLoop.handleSelectedKey(DefaultEventLoop.java:371) at com.datatorrent.netlet.OptimizedEventLoop$SelectedSelectionKeySet.forEach(OptimizedEventLoop.java:59) at com.datatorrent.netlet.OptimizedEventLoop.runEventLoop(OptimizedEventLoop.java:192)* *We want to know about how to override the default apex dependencies with dependency that we are using in our application.* Regards, Akshay S. Harale Software Developer @ Synerzip Skype – akshayharale -- This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.
