Thanks for logging the JIRA case, Kunwar. I'll add some comments there.
On Fri, Sep 29, 2017 at 9:33 AM, Kunwar Deep Singh Toor <[email protected]> wrote: > HI, > > I have logged the issue with JIRA. ID is CALCITE-1994. > > Best, > Kunwar Singh > > On Fri, Sep 29, 2017 at 12:19 PM, Kunwar Deep Singh Toor < > [email protected]> wrote: > >> Hello, >> >> Sorry for the late reply. I have been on travel for the past 2 days. >> >> I have never really used JIRA but I will try my best to report the issue >> there. >> >> In the meantime can you tell me the exact way to solve this problem. I am >> new to calcite and I have been trying to figure out the classpath.txt in >> the ElasticSearch5 module. >> >> Thanks, >> Kunwar Singh >> >> On Thu, Sep 28, 2017 at 1:28 PM, Christian Beikov < >> [email protected]> wrote: >> >>> Maybe I am wrong, but I was assuming the command "mvn >>> dependency:build-classpath" will include the dependencies of all modules >>> regardless of whether we include the elasticsearch2 target/classes or not. >>> >>> >>> >>> Mit freundlichen Grüßen, >>> ------------------------------------------------------------------------ >>> *Christian Beikov* >>> Am 28.09.2017 um 19:16 schrieb Julian Hyde: >>> >>>> I understand that. But if we remove calcite-elasticsearch2 from >>>> sqlline’s classpath, sqlline will still get calcite-elasticsearch5 and the >>>> elastic search 5.x client driver libraries that it depends upon. >>>> >>>> And the elastic search 2.x client driver libraries will not be on >>>> sqlline's class path, because calcite-elasticsearch2 is the only module >>>> that depends on them. So there will be no clash. >>>> >>>> Julian >>>> >>>> >>>> On Sep 28, 2017, at 9:53 AM, Christian Beikov < >>>>> [email protected]> wrote: >>>>> >>>>> The elasticsearch2 module depens on the elasticsearch client driver >>>>> libraries with version 2.x and the elasticsearch5 module depends on the >>>>> elasticsearch client driver libraries with version 5.x. That's the version >>>>> conflict Kunwar was running into. >>>>> >>>>> >>>>> Mit freundlichen Grüßen, >>>>> ------------------------------------------------------------ >>>>> ------------ >>>>> *Christian Beikov* >>>>> Am 28.09.2017 um 18:35 schrieb Julian Hyde: >>>>> >>>>>> Christian, >>>>>> >>>>>> I don’t understand. The elasticsearch5 module does not depend on the >>>>>> elasticsearch2 module, only on core and linq4j. Therefore if it (and its >>>>>> dependencies) are added to sqlline’s class path, it should have >>>>>> everything >>>>>> it needs. What am I missing? >>>>>> >>>>>> Julian >>>>>> >>>>>> >>>>>> On Sep 27, 2017, at 11:02 PM, Christian Beikov < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>> Well it's not so simple I suppose. You also need to exclude the >>>>>>> elasticsearch2 maven module from the pom.xml since that one is actually >>>>>>> contributing the elasticsearch client to the classpath. For the sake of >>>>>>> simplicity, we could introduce profiles that only have a specific ES >>>>>>> version, but that doesn't scale obviously regarding possible other >>>>>>> version >>>>>>> combinations. >>>>>>> >>>>>>> I can introduce a profile if you want, but a real fix as I said needs >>>>>>> classloader magic like what e.g. JBoss Modules does. >>>>>>> >>>>>>> >>>>>>> Mit freundlichen Grüßen, >>>>>>> ------------------------------------------------------------ >>>>>>> ------------ >>>>>>> *Christian Beikov* >>>>>>> Am 27.09.2017 um 20:28 schrieb Julian Hyde: >>>>>>> >>>>>>>> I agree with Christian that it’s just a problem with sqlline’s class >>>>>>>> path. The simplest solution is to remove elasticsearch2 from the line >>>>>>>> that >>>>>>>> builds the class path: >>>>>>>> >>>>>>>> for module in core cassandra druid elasticsearch2 elasticsearch5 >>>>>>>> file mongodb spark splunk example/csv example/function; do >>>>>>>> CP=${CP}${module}/target/classes: >>>>>>>> CP=${CP}${module}/target/test-classes: >>>>>>>> done >>>>>>>> >>>>>>>> Then sqlline will work out of the box for ES5. People who want to >>>>>>>> run ES2 will have to edit the script. >>>>>>>> >>>>>>>> Kunwar, Can you please log a JIRA case? >>>>>>>> >>>>>>>> Christian, Can you you fix it? >>>>>>>> >>>>>>>> It’s conceivable that someone would want to combine data from an ES2 >>>>>>>> and ES5 instance, but we don’t have an answer for them, whether they >>>>>>>> invoke >>>>>>>> Calcite via sqlline or any other way. The incompatible libraries will >>>>>>>> clash, unless there is some class-loader magic keeping them in >>>>>>>> different >>>>>>>> namespaces. The right answer is probably something like Jboss modules, >>>>>>>> but >>>>>>>> it’s low priority. >>>>>>>> >>>>>>>> Julian >>>>>>>> >>>>>>>> >>>>>>>> On Sep 27, 2017, at 9:29 AM, Christian Beikov < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>> Depends if you want to be able to connect to an ES2 and ES5 server >>>>>>>>> at the >>>>>>>>> same time through sqlline. >>>>>>>>> >>>>>>>>> Am 27.09.2017 17:53 schrieb "Josh Elser" <[email protected]>: >>>>>>>>> >>>>>>>>> A quick solution is to add some documentation around how to give a >>>>>>>>>> specific version of ElasticSearch at build-time. This would help >>>>>>>>>> folks >>>>>>>>>> build Calcite with the exact version that they want (assuming that >>>>>>>>>> the API >>>>>>>>>> hasn't changed and the adapter actually works with that version of >>>>>>>>>> ES) >>>>>>>>>> >>>>>>>>>> We don't need to have multiple versions of ElasticSearch on the >>>>>>>>>> classpath >>>>>>>>>> at once, right? >>>>>>>>>> >>>>>>>>>> On 9/27/17 2:13 AM, Christian Beikov wrote: >>>>>>>>>> >>>>>>>>>> Sqlline is problematic as it will have both elastic search >>>>>>>>>>> versions on >>>>>>>>>>> the classpath. I haven't thought about that before. In the >>>>>>>>>>> meantime you >>>>>>>>>>> could remove old versions of libraries from the >>>>>>>>>>> "target/fullclasspath.txt" >>>>>>>>>>> file to workaround the classpath problem. >>>>>>>>>>> >>>>>>>>>>> @Julian Hyde: How should we deal with something like this? We'd >>>>>>>>>>> need >>>>>>>>>>> isolated "modules" instead of a flat class path for sqlline. >>>>>>>>>>> Maybe we could >>>>>>>>>>> build sqlline on JBoss Modules(https://github.com/jbo >>>>>>>>>>> ss-modules/jboss-modules) to achieve that? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Mit freundlichen Grüßen, >>>>>>>>>>> ------------------------------------------------------------ >>>>>>>>>>> ------------ >>>>>>>>>>> *Christian Beikov* >>>>>>>>>>> Am 27.09.2017 um 01:30 schrieb Kunwar Deep Singh Toor: >>>>>>>>>>> >>>>>>>>>>> ---------- Forwarded message ---------- >>>>>>>>>>>> From: Kunwar Deep Singh Toor <[email protected]> >>>>>>>>>>>> Date: Mon, Sep 25, 2017 at 8:16 PM >>>>>>>>>>>> Subject: ElasticSearch5 not working >>>>>>>>>>>> To: [email protected] >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I have been trying to connect to the ElasticSearch suing ES >>>>>>>>>>>> adapter >>>>>>>>>>>> through sqlline. I was able to connect to ES 2.1.4 using the ES2 >>>>>>>>>>>> adapter >>>>>>>>>>>> but when I try connecting to ES 5.1.1 using the ES5 adapter the >>>>>>>>>>>> following >>>>>>>>>>>> error pops up, I checked the logs for ElasticSearch and it does >>>>>>>>>>>> not >>>>>>>>>>>> show a >>>>>>>>>>>> connection attempt being made. >>>>>>>>>>>> >>>>>>>>>>>> The following is the error: >>>>>>>>>>>> java.lang.NoSuchMethodError: org.elasticsearch.transport.cl >>>>>>>>>>>> ient.PreBuiltTransportClient.addPlugins(Ljava/util/Collectio >>>>>>>>>>>> n;Ljava/util/Collection;)Ljava/util/Collection; >>>>>>>>>>>> at org.elasticsearch.transport.cl >>>>>>>>>>>> ient.PreBuiltTransportClient.< >>>>>>>>>>>> init>(PreBuiltTransportClient.java:130) >>>>>>>>>>>> at org.elasticsearch.transport.cl >>>>>>>>>>>> ient.PreBuiltTransportClient.< >>>>>>>>>>>> init>(PreBuiltTransportClient.java:116) >>>>>>>>>>>> at org.elasticsearch.transport.cl >>>>>>>>>>>> ient.PreBuiltTransportClient.< >>>>>>>>>>>> init>(PreBuiltTransportClient.java:106) >>>>>>>>>>>> at org.apache.calcite.adapter.ela >>>>>>>>>>>> sticsearch5.Elasticsearch5Sche >>>>>>>>>>>> ma.open(Elasticsearch5Schema.java:119) >>>>>>>>>>>> at org.apache.calcite.adapter.ela >>>>>>>>>>>> sticsearch5.Elasticsearch5Sche >>>>>>>>>>>> ma.<init>(Elasticsearch5Schema.java:74) >>>>>>>>>>>> at org.apache.calcite.adapter.ela >>>>>>>>>>>> sticsearch5.Elasticsearch5Sche >>>>>>>>>>>> maFactory.create(Elasticsearch5SchemaFactory.java:56) >>>>>>>>>>>> at org.apache.calcite.model.Model >>>>>>>>>>>> Handler.visit(ModelHandler. >>>>>>>>>>>> java:269) >>>>>>>>>>>> at org.apache.calcite.model.JsonC >>>>>>>>>>>> ustomSchema.accept(JsonCustomS >>>>>>>>>>>> chema.java:45) >>>>>>>>>>>> at org.apache.calcite.model.Model >>>>>>>>>>>> Handler.visit(ModelHandler. >>>>>>>>>>>> java:195) >>>>>>>>>>>> at org.apache.calcite.model.Model >>>>>>>>>>>> Handler.<init>(ModelHandler. >>>>>>>>>>>> java:87) >>>>>>>>>>>> at org.apache.calcite.jdbc.Driver >>>>>>>>>>>> $1.onConnectionInit(Driver. >>>>>>>>>>>> java:104) >>>>>>>>>>>> at org.apache.calcite.avatica.Unr >>>>>>>>>>>> egisteredDriver.connect(Unregi >>>>>>>>>>>> steredDriver.java:139) >>>>>>>>>>>> at sqlline.DatabaseConnection.con >>>>>>>>>>>> nect(DatabaseConnection.java:156) >>>>>>>>>>>> at sqlline.DatabaseConnection.get >>>>>>>>>>>> Connection(DatabaseConnection. >>>>>>>>>>>> java:204) >>>>>>>>>>>> at sqlline.Commands.connect(Commands.java:1095) >>>>>>>>>>>> at sqlline.Commands.connect(Commands.java:1001) >>>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>>>>>>>>> Method) >>>>>>>>>>>> at sun.reflect.NativeMethodAccess >>>>>>>>>>>> orImpl.invoke(NativeMethodAcce >>>>>>>>>>>> ssorImpl.java:62) >>>>>>>>>>>> at sun.reflect.DelegatingMethodAc >>>>>>>>>>>> cessorImpl.invoke(DelegatingMe >>>>>>>>>>>> thodAccessorImpl.java:43) >>>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>>>>>>>>>> at sqlline.ReflectiveCommandHandl >>>>>>>>>>>> er.execute(ReflectiveCommandHa >>>>>>>>>>>> ndler.java:38) >>>>>>>>>>>> at sqlline.SqlLine.dispatch(SqlLine.java:791) >>>>>>>>>>>> at sqlline.SqlLine.begin(SqlLine.java:668) >>>>>>>>>>>> at sqlline.SqlLine.start(SqlLine.java:373) >>>>>>>>>>>> at sqlline.SqlLine.main(SqlLine.java:265) >>>>>>>>>>>> >>>>>>>>>>>> I have recently started working on calcite and any help would be >>>>>>>>>>>> much >>>>>>>>>>>> appreciated. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Kunwar Singh >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>> >>
