Thanks for the changes, I have checked and it passed on my env. I think it should be also checked with other JDK versions besides 8.
Kind regards, Volodymyr Vysotskyi пн, 25 черв. 2018 о 07:05 Andrei Sereda <[email protected]> пише: > Can you please check the branch in my fork ? > > fork: https://github.com/asereda-gs/calcite > branch: javadoc-fixes > > I've tried to remove most of the warnings and javadoc, javadoc-test and > site all work for me. > > ~/I/c/elasticsearch5 ❯❯❯ mvn --version > javadoc-fixes ✭ > Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; > 2018-06-17T14:33:14-04:00) > Maven home: /usr/local/Cellar/maven/3.5.4/libexec > Java version: 1.8.0_151, vendor: Oracle Corporation, runtime: > /Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home/jre > Default locale: en_US, platform encoding: UTF-8 > OS name: "mac os x", version: "10.13.2", arch: "x86_64", family: "mac" > > > > On Sun, Jun 24, 2018 at 12:40 PM Volodymyr Vysotskyi <[email protected] > > > wrote: > > > I have also checked "mvn javadoc:javadoc javadoc:test-javadoc" on my > > machine (Ubuntu, JDK 8, Maven 3.3.9) and it fails on elasticsearch2 > module > > (didn't check this problem for the next modules). > > > > "mvn clean -DskipTests site" fails for MongoDB module. > > > > Kind regards, > > Volodymyr Vysotskyi > > > > > > сб, 23 черв. 2018 о 03:12 Julian Hyde <[email protected]> пише: > > > > > Same failure for me on JDK 8/macOS. The most likely other variable is > > > maven: I have 3.5.2 on macOS, 3.5.3 on ubuntu. > > > > > > I’m mystified why no one else is seeing javadoc failures on macOS. > > > > > > Julian > > > > > > > > > > On Jun 22, 2018, at 4:35 PM, Julian Hyde <[email protected]> wrote: > > > > > > > > On JDK 8/ubuntu, “mvn clean -DskipTests site” gives the following > > > failure: > > > > > > > > [ERROR] Failed to execute goal > > > org.apache.maven.plugins:maven-site-plugin:3.7:site (default-site) on > > > project calcite-mongodb: Error generating > > > maven-javadoc-plugin:3.0.1:test-javadoc report: > > > > [ERROR] Exit code: 1 - > > > > > > /home/jhyde/open1/calcite.4/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoAdapterTest.java:22: > > > error: cannot find symbol > > > > [ERROR] import org.apache.calcite.test.CalciteAssert; > > > > [ERROR] ^ > > > > [ERROR] symbol: class CalciteAssert > > > > [ERROR] location: package org.apache.calcite.test > > > > [ERROR] > > > > > > /home/jhyde/open1/calcite.4/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoAdapterTest.java:137: > > > error: package CalciteAssert does not exist > > > > [ERROR] private CalciteAssert.AssertThat assertModel(String model) > { > > > > [ERROR] ^ > > > > [ERROR] > > > > > > /home/jhyde/open1/calcite.4/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoAdapterTest.java:145: > > > error: package CalciteAssert does not exist > > > > [ERROR] private CalciteAssert.AssertThat assertModel(URL url) { > > > > [ERROR] ^ > > > > [ERROR] > > > > > > /home/jhyde/open1/calcite.4/mongodb/src/test/java/org/apache/calcite/test/MongoAssertions.java:43: > > > error: reference not found > > > > [ERROR] /** Similar to {@link CalciteAssert#checkResultUnordered}, > > but > > > filters strings > > > > [ERROR] ^ > > > > > > > > > > > > > > > >> On Jun 22, 2018, at 3:46 PM, Andrei Sereda <[email protected]> > wrote: > > > >> > > > >> Hmm. Both master and c12cb4b0de work for me. Perhaps the issue is > JDK > > 8 > > > vs > > > >> 9 ? > > > >> > > > >> # JDK8 (oracle) / macOS > > > >> $ mvn clean -DskipTests site > > > >> > > > >> On Fri, Jun 22, 2018 at 5:56 PM Julian Hyde <[email protected]> > wrote: > > > >> > > > >>> I just tried it again. The following fails for me (jdk9, ubuntu): > > > >>> > > > >>> $ git checkout c12cb4b0de > > > >>> $ mvn clean -DskipTests site > > > >>> > > > >>> Constructing Javadoc information... > > > >>> 1 error > > > >>> [INFO] > > > >>> > > > > ------------------------------------------------------------------------ > > > >>> [INFO] Reactor Summary: > > > >>> [INFO] > > > >>> [INFO] Calcite 1.17.0-SNAPSHOT ............................ FAILURE > > > [02:29 > > > >>> min] > > > >>> [INFO] Calcite Linq4j ..................................... SKIPPED > > > >>> [INFO] Calcite Core ....................................... SKIPPED > > > >>> [INFO] Calcite Cassandra .................................. SKIPPED > > > >>> [INFO] Calcite Druid ...................................... SKIPPED > > > >>> [INFO] Calcite Elasticsearch .............................. SKIPPED > > > >>> [INFO] Calcite Elasticsearch5 ............................. SKIPPED > > > >>> [INFO] Calcite Examples ................................... SKIPPED > > > >>> [INFO] Calcite Example CSV ................................ SKIPPED > > > >>> [INFO] Calcite Example Function ........................... SKIPPED > > > >>> [INFO] Calcite File ....................................... SKIPPED > > > >>> [INFO] Calcite Geode ...................................... SKIPPED > > > >>> [INFO] Calcite MongoDB .................................... SKIPPED > > > >>> [INFO] Calcite Pig ........................................ SKIPPED > > > >>> [INFO] Calcite Piglet ..................................... SKIPPED > > > >>> [INFO] Calcite Plus ....................................... SKIPPED > > > >>> [INFO] Calcite Server ..................................... SKIPPED > > > >>> [INFO] Calcite Spark ...................................... SKIPPED > > > >>> [INFO] Calcite Splunk ..................................... SKIPPED > > > >>> [INFO] Calcite Ubenchmark 1.17.0-SNAPSHOT ................. SKIPPED > > > >>> [INFO] > > > >>> > > > > ------------------------------------------------------------------------ > > > >>> [INFO] BUILD FAILURE > > > >>> [INFO] > > > >>> > > > > ------------------------------------------------------------------------ > > > >>> [INFO] Total time: 02:29 min > > > >>> [INFO] Finished at: 2018-06-22T14:53:13-07:00 > > > >>> [INFO] > > > >>> > > > > ------------------------------------------------------------------------ > > > >>> [ERROR] Failed to execute goal > > > >>> org.apache.maven.plugins:maven-site-plugin:3.7:site (default-site) > on > > > >>> project calcite: Error generating > > > maven-javadoc-plugin:3.0.1:test-aggregate > > > >>> report: > > > >>> [ERROR] Exit code: 1 - > > > >>> > > > > > > /home/jhyde/open1/calcite.4/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/EmbeddedElasticNode.java:29: > > > >>> error: package org.elasticsearch.node.internal does not exist > > > >>> [ERROR] import > > > org.elasticsearch.node.internal.InternalSettingsPreparer; > > > >>> [ERROR] ^ > > > >>> [ERROR] > > > >>> [ERROR] Command line was: /usr/lib/jvm/jdk9/bin/javadoc @options > > > @packages > > > >>> @argfile > > > >>> [ERROR] > > > >>> [ERROR] Refer to the generated Javadoc files in > > > >>> '/home/jhyde/open1/calcite.4/target/site/testapidocs' dir. > > > >>> [ERROR] -> [Help 1] > > > >>> [ERROR] > > > >>> [ERROR] To see the full stack trace of the errors, re-run Maven > with > > > the > > > >>> -e switch. > > > >>> [ERROR] Re-run Maven using the -X switch to enable full debug > > logging. > > > >>> [ERROR] > > > >>> [ERROR] For more information about the errors and possible > solutions, > > > >>> please read the following articles: > > > >>> [ERROR] [Help 1] > > > >>> > > > > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > > > >>> > > > >>> > > > >>>> On Jun 22, 2018, at 2:41 PM, Michael Mior <[email protected]> > wrote: > > > >>>> > > > >>>> Odd, mvn clean site still works fine for me. > > > >>>> -- > > > >>>> Michael Mior > > > >>>> [email protected] > > > >>>> > > > >>>> > > > >>>> > > > >>>> Le ven. 22 juin 2018 à 17:12, Julian Hyde <[email protected]> a > > écrit > > > : > > > >>>> > > > >>>>> Looks like this change broke “mvn site” (perhaps also “mvn > > > >>>>> javadoc:test-javadoc”). > > > >>>>> > > > >>>>> [ERROR] Failed to execute goal > > > >>>>> org.apache.maven.plugins:maven-site-plugin:3.7:site > (default-site) > > on > > > >>>>> project calcite: Error generating > > > >>> maven-javadoc-plugin:3.0.1:test-aggregate > > > >>>>> report: > > > >>>>> [ERROR] Exit code: 1 - > > > >>>>> > > > >>> > > > > > > /home/jhyde/regress/calcite/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/EmbeddedElasticNode.java:29: > > > >>>>> error: package org.elasticsearch.node.internal does not exist > > > >>>>> [ERROR] import > > > org.elasticsearch.node.internal.InternalSettingsPreparer; > > > >>>>> [ERROR] ^ > > > >>>>> [ERROR] > > > >>>>> > > > >>>>>> On Jun 21, 2018, at 3:39 AM, [email protected] wrote: > > > >>>>>> > > > >>>>>> [CALCITE-2347] running ElasticSearch in embedded mode for unit > > > tests of > > > >>>>> ES adapter (Andrei Sereda) > > > >>>>>> > > > >>>>>> After discussion on dev-list Integration tests (for ES) have > been > > > >>>>> removed. They're now > > > >>>>>> superseded by unit tests (which execute queries against a real > > > elastic > > > >>>>> instance) > > > >>>>>> > > > >>>>>> Added local file (zips-mini.json) which contains a small subset > of > > > >>>>> original zips.json > > > >>>>>> (allows to bootstrap tests faster) > > > >>>>>> > > > >>>>>> Created separate ES JUnit rule which can be re-used across > > different > > > >>>>> tests. > > > >>>>>> > > > >>>>>> Both v2 and v5 of ES adapters are supported. > > > >>>>>> > > > >>>>>> Close apache/calcite#716 > > > >>>>>> > > > >>>>>> > > > >>>>>> Project: http://git-wip-us.apache.org/repos/asf/calcite/repo > > > >>>>>> Commit: > > > http://git-wip-us.apache.org/repos/asf/calcite/commit/c12cb4b0 > > > >>>>>> Tree: > > http://git-wip-us.apache.org/repos/asf/calcite/tree/c12cb4b0 > > > >>>>>> Diff: > > http://git-wip-us.apache.org/repos/asf/calcite/diff/c12cb4b0 > > > >>>>>> > > > >>>>>> Branch: refs/heads/master > > > >>>>>> Commit: c12cb4b0de1baa3f7cbb9952ee350fdd1701662d > > > >>>>>> Parents: 37944bb > > > >>>>>> Author: Andrei Sereda <[email protected]> > > > >>>>>> Authored: Thu May 31 18:19:10 2018 -0400 > > > >>>>>> Committer: Michael Mior <[email protected]> > > > >>>>>> Committed: Thu Jun 21 06:38:50 2018 -0400 > > > >>>>>> > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> .../AbstractElasticsearchTable.java | 12 + > > > >>>>>> .../elasticsearch/ElasticsearchProject.java | 61 ++- > > > >>>>>> elasticsearch2/pom.xml | 6 + > > > >>>>>> .../Elasticsearch2Enumerator.java | 12 +- > > > >>>>>> .../elasticsearch2/Elasticsearch2Schema.java | 16 +- > > > >>>>>> .../elasticsearch2/Elasticsearch2Table.java | 9 +- > > > >>>>>> .../ElasticSearch2AdapterTest.java | 395 > > > >>> ++++++++++++++++++ > > > >>>>>> .../elasticsearch2/EmbeddedElasticNode.java | 147 +++++++ > > > >>>>>> .../elasticsearch2/EmbeddedElasticRule.java | 97 +++++ > > > >>>>>> .../org/apache/calcite/test/ElasticChecker.java | 49 +++ > > > >>>>>> .../calcite/test/Elasticsearch2AdapterIT.java | 270 > > ------------- > > > >>>>>> .../resources/elasticsearch-zips-model.json | 50 --- > > > >>>>>> .../src/test/resources/zips-mini.json | 149 +++++++ > > > >>>>>> elasticsearch5/pom.xml | 31 ++ > > > >>>>>> .../elasticsearch5/Elasticsearch5Schema.java | 17 +- > > > >>>>>> .../elasticsearch5/Elasticsearch5Table.java | 11 +- > > > >>>>>> .../ElasticSearch5AdapterTest.java | 399 > > > >>> +++++++++++++++++++ > > > >>>>>> .../elasticsearch5/EmbeddedElasticNode.java | 153 +++++++ > > > >>>>>> .../elasticsearch5/EmbeddedElasticRule.java | 98 +++++ > > > >>>>>> .../org/apache/calcite/test/ElasticChecker.java | 49 +++ > > > >>>>>> .../calcite/test/Elasticsearch5AdapterIT.java | 270 > > ------------- > > > >>>>>> .../resources/elasticsearch-zips-model.json | 50 --- > > > >>>>>> elasticsearch5/src/test/resources/log4j2.xml | 16 + > > > >>>>>> .../src/test/resources/zips-mini.json | 149 +++++++ > > > >>>>>> pom.xml | 20 +- > > > >>>>>> 25 files changed, 1866 insertions(+), 670 deletions(-) > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>> > > > > > > http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> diff --git > > > >>>>> > > > >>> > > > > > > a/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java > > > >>>>> > > > >>> > > > > > > b/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java > > > >>>>>> index 0980469..8cc5933 100644 > > > >>>>>> --- > > > >>>>> > > > >>> > > > > > > a/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java > > > >>>>>> +++ > > > >>>>> > > > >>> > > > > > > b/core/src/main/java/org/apache/calcite/adapter/elasticsearch/AbstractElasticsearchTable.java > > > >>>>>> @@ -75,6 +75,18 @@ public abstract class > > AbstractElasticsearchTable > > > >>>>> extends AbstractQueryableTable > > > >>>>>> relOptTable, this, null); > > > >>>>>> } > > > >>>>>> > > > >>>>>> + /** > > > >>>>>> + * In ES 5.x scripted fields start with {@code > > > params._source.foo} > > > >>>>> while in ES2.x > > > >>>>>> + * {@code _source.foo}. Helper method to build correct query > > > based > > > >>> on > > > >>>>> runtime version of elastic. > > > >>>>>> + * > > > >>>>>> + * @see <a href=" > > > >>>>> https://github.com/elastic/elasticsearch/issues/20068";>_source > > > >>>>> variable</a> > > > >>>>>> + * @see <a href=" > > > >>>>> > > > >>> > > > > > > https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-fields.html > > > >>> ">Scripted > > > >>>>> Fields</a> > > > >>>>>> + */ > > > >>>>>> + protected String scriptedFieldPrefix() { > > > >>>>>> + // this is default pattern starting 5.x > > > >>>>>> + return "params._source"; > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> /** Executes a "find" operation on the underlying type. > > > >>>>>> * > > > >>>>>> * <p>For example, > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>> > > > > > > http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> diff --git > > > >>>>> > > > >>> > > > > > > a/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java > > > >>>>> > > > >>> > > > > > > b/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java > > > >>>>>> index b42abd7..961c8b0 100644 > > > >>>>>> --- > > > >>>>> > > > >>> > > > > > > a/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java > > > >>>>>> +++ > > > >>>>> > > > >>> > > > > > > b/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java > > > >>>>>> @@ -27,11 +27,15 @@ import > > > >>>>> org.apache.calcite.rel.metadata.RelMetadataQuery; > > > >>>>>> import org.apache.calcite.rel.type.RelDataType; > > > >>>>>> import org.apache.calcite.rex.RexNode; > > > >>>>>> import org.apache.calcite.util.Pair; > > > >>>>>> -import org.apache.calcite.util.Util; > > > >>>>>> + > > > >>>>>> +import com.google.common.base.Function; > > > >>>>>> +import com.google.common.collect.Lists; > > > >>>>>> > > > >>>>>> import java.util.ArrayList; > > > >>>>>> import java.util.List; > > > >>>>>> > > > >>>>>> +import javax.annotation.Nullable; > > > >>>>>> + > > > >>>>>> /** > > > >>>>>> * Implementation of {@link org.apache.calcite.rel.core.Project} > > > >>>>>> * relational expression in Elasticsearch. > > > >>>>>> @@ -57,41 +61,60 @@ public class ElasticsearchProject extends > > > Project > > > >>>>> implements ElasticsearchRel { > > > >>>>>> implementor.visitChild(0, getInput()); > > > >>>>>> > > > >>>>>> final List<String> inFields = > > > >>>>>> - > > > >>>>> > > ElasticsearchRules.elasticsearchFieldNames(getInput().getRowType()); > > > >>>>>> + > > > >>>>> > > ElasticsearchRules.elasticsearchFieldNames(getInput().getRowType()); > > > >>>>>> final ElasticsearchRules.RexToElasticsearchTranslator > translator > > = > > > >>>>>> - new ElasticsearchRules.RexToElasticsearchTranslator( > > > >>>>>> - (JavaTypeFactory) getCluster().getTypeFactory(), > > > >>> inFields); > > > >>>>>> + new > ElasticsearchRules.RexToElasticsearchTranslator( > > > >>>>>> + (JavaTypeFactory) > > > getCluster().getTypeFactory(), > > > >>>>> inFields); > > > >>>>>> > > > >>>>>> - final List<String> findItems = new ArrayList<>(); > > > >>>>>> - final List<String> scriptFieldItems = new ArrayList<>(); > > > >>>>>> + final List<String> fields = new ArrayList<>(); > > > >>>>>> + final List<String> scriptFields = new ArrayList<>(); > > > >>>>>> for (Pair<RexNode, String> pair: getNamedProjects()) { > > > >>>>>> final String name = pair.right; > > > >>>>>> final String expr = pair.left.accept(translator); > > > >>>>>> > > > >>>>>> if (expr.equals("\"" + name + "\"")) { > > > >>>>>> - findItems.add(ElasticsearchRules.quote(name)); > > > >>>>>> + fields.add(name); > > > >>>>>> } else if (expr.matches("\"literal\":.+")) { > > > >>>>>> - scriptFieldItems.add(ElasticsearchRules.quote(name) > > > >>>>>> - + ":{\"script\": " > > > >>>>>> - + expr.split(":")[1] + "}"); > > > >>>>>> + scriptFields.add(ElasticsearchRules.quote(name) > > > >>>>>> + + ":{\"script\": " > > > >>>>>> + + expr.split(":")[1] + "}"); > > > >>>>>> } else { > > > >>>>>> - scriptFieldItems.add(ElasticsearchRules.quote(name) > > > >>>>>> - + ":{\"script\":\"params._source." > > > >>>>>> - + expr.replaceAll("\"", "") + "\"}"); > > > >>>>>> + scriptFields.add(ElasticsearchRules.quote(name) > > > >>>>>> + + ":{\"script\":" > > > >>>>>> + // _source (ES2) vs params._source (ES5) > > > >>>>>> + + "\"" + > > > >>>>> implementor.elasticsearchTable.scriptedFieldPrefix() + "." > > > >>>>>> + + expr.replaceAll("\"", "") + "\"}"); > > > >>>>>> + } > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + StringBuilder query = new StringBuilder(); > > > >>>>>> + if (scriptFields.isEmpty()) { > > > >>>>>> + List<String> newList = Lists.transform(fields, new > > > >>>>> Function<String, String>() { > > > >>>>>> + @Nullable > > > >>>>>> + @Override public String apply(@Nullable String input) { > > > >>>>>> + return ElasticsearchRules.quote(input); > > > >>>>>> + } > > > >>>>>> + }); > > > >>>>>> + > > > >>>>>> + final String findString = String.join(", ", newList); > > > >>>>>> + query.append("\"_source\" : > > > [").append(findString).append("]"); > > > >>>>>> + } else { > > > >>>>>> + // if scripted fields are present, ES ignores _source > > > attribute > > > >>>>>> + for (String field: fields) { > > > >>>>>> + scriptFields.add(ElasticsearchRules.quote(field) + > > > >>>>> ":{\"script\": " > > > >>>>>> + // _source (ES2) vs params._source (ES5) > > > >>>>>> + + "\"" + > > > >>>>> implementor.elasticsearchTable.scriptedFieldPrefix() + "." > > > >>>>>> + + field + "\"}"); > > > >>>>>> } > > > >>>>>> + query.append("\"script_fields\": {" + String.join(", ", > > > >>>>> scriptFields) + "}"); > > > >>>>>> } > > > >>>>>> - final String findString = Util.toString(findItems, "", ", > ", > > > ""); > > > >>>>>> - final String scriptFieldString = "\"script_fields\": {" > > > >>>>>> - + Util.toString(scriptFieldItems, "", ", ", "") + "}"; > > > >>>>>> - final String fieldString = "\"_source\" : [" + findString + > > "]" > > > >>>>>> - + ", " + scriptFieldString; > > > >>>>>> > > > >>>>>> for (String opfield : implementor.list) { > > > >>>>>> if (opfield.startsWith("\"_source\"")) { > > > >>>>>> implementor.list.remove(opfield); > > > >>>>>> } > > > >>>>>> } > > > >>>>>> - implementor.add(fieldString); > > > >>>>>> + implementor.add(query.toString()); > > > >>>>>> } > > > >>>>>> } > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>> > > > > > > http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/pom.xml > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> diff --git a/elasticsearch2/pom.xml b/elasticsearch2/pom.xml > > > >>>>>> index f24622c..6fbee03 100644 > > > >>>>>> --- a/elasticsearch2/pom.xml > > > >>>>>> +++ b/elasticsearch2/pom.xml > > > >>>>>> @@ -73,6 +73,12 @@ limitations under the License. > > > >>>>>> <version>${elasticsearch-java-driver.version}</version> > > > >>>>>> </dependency> > > > >>>>>> <dependency> > > > >>>>>> + <!-- Lang groovy dependency is needed for testing with > > > embedded > > > >>>>> ES (scripted fields like loc[0]) --> > > > >>>>>> + <groupId>org.elasticsearch.module</groupId> > > > >>>>>> + <artifactId>lang-groovy</artifactId> > > > >>>>>> + <scope>test</scope> > > > >>>>>> + </dependency> > > > >>>>>> + <dependency> > > > >>>>>> <groupId>com.carrotsearch</groupId> > > > >>>>>> <artifactId>hppc</artifactId> > > > >>>>>> <version>${hppc.version}</version> > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>> > > > > > > http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> diff --git > > > >>>>> > > > >>> > > > > > > a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java > > > >>>>> > > > >>> > > > > > > b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java > > > >>>>>> index 84370ab..c3d2ac0 100644 > > > >>>>>> --- > > > >>>>> > > > >>> > > > > > > a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java > > > >>>>>> +++ > > > >>>>> > > > >>> > > > > > > b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Enumerator.java > > > >>>>>> @@ -26,6 +26,7 @@ import org.elasticsearch.search.SearchHit; > > > >>>>>> import java.util.Date; > > > >>>>>> import java.util.Iterator; > > > >>>>>> import java.util.List; > > > >>>>>> +import java.util.Locale; > > > >>>>>> import java.util.Map; > > > >>>>>> > > > >>>>>> /** > > > >>>>>> @@ -101,15 +102,18 @@ public class Elasticsearch2Enumerator > > > implements > > > >>>>> Enumerator<Object> { > > > >>>>>> private static Function1<SearchHit, Object[]> listGetter( > > > >>>>>> final List<Map.Entry<String, Class>> fields) { > > > >>>>>> return new Function1<SearchHit, Object[]>() { > > > >>>>>> - public Object[] apply(SearchHit searchHitFields) { > > > >>>>>> + public Object[] apply(SearchHit hit) { > > > >>>>>> Object[] objects = new Object[fields.size()]; > > > >>>>>> for (int i = 0; i < fields.size(); i++) { > > > >>>>>> final Map.Entry<String, Class> field = fields.get(i); > > > >>>>>> final String name = field.getKey(); > > > >>>>>> - if (searchHitFields.fields().isEmpty()) { > > > >>>>>> - objects[i] = > > > >>> convert(searchHitFields.getSource().get(name), > > > >>>>> field.getValue()); > > > >>>>>> + if (hit.fields().isEmpty()) { > > > >>>>>> + objects[i] = convert(hit.getSource().get(name), > > > >>>>> field.getValue()); > > > >>>>>> + } else if (hit.fields().containsKey(name)) { > > > >>>>>> + objects[i] = convert(hit.field(name).getValue(), > > > >>>>> field.getValue()); > > > >>>>>> } else { > > > >>>>>> - objects[i] = > > > >>>>> convert(searchHitFields.field(name).getValue(), > field.getValue()); > > > >>>>>> + throw new IllegalStateException( > > > >>>>>> + String.format(Locale.getDefault(), "No > result > > > for > > > >>>>> %s", field)); > > > >>>>>> } > > > >>>>>> } > > > >>>>>> return objects; > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>> > > > > > > http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> diff --git > > > >>>>> > > > >>> > > > > > > a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java > > > >>>>> > > > >>> > > > > > > b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java > > > >>>>>> index 668402b..46e3fc5 100644 > > > >>>>>> --- > > > >>>>> > > > >>> > > > > > > a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java > > > >>>>>> +++ > > > >>>>> > > > >>> > > > > > > b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Schema.java > > > >>>>>> @@ -22,6 +22,9 @@ import > > > org.apache.calcite.schema.impl.AbstractSchema; > > > >>>>>> > > > >>>>>> import com.carrotsearch.hppc.cursors.ObjectObjectCursor; > > > >>>>>> > > > >>>>>> +import com.google.common.annotations.VisibleForTesting; > > > >>>>>> +import com.google.common.base.Preconditions; > > > >>>>>> + > > > >>>>>> import com.google.common.collect.ImmutableList; > > > >>>>>> import com.google.common.collect.ImmutableMap; > > > >>>>>> > > > >>>>>> @@ -86,6 +89,16 @@ public class Elasticsearch2Schema extends > > > >>>>> AbstractSchema > > > >>>>>> } > > > >>>>>> } > > > >>>>>> > > > >>>>>> + /** > > > >>>>>> + * Allows schema to be instantiated from existing elastic > > search > > > >>>>> client. > > > >>>>>> + * This constructor is used in tests. > > > >>>>>> + */ > > > >>>>>> + @VisibleForTesting > > > >>>>>> + Elasticsearch2Schema(Client client, String index) { > > > >>>>>> + this.client = Preconditions.checkNotNull(client, "client"); > > > >>>>>> + this.index = Preconditions.checkNotNull(index, "index"); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> @Override protected Map<String, Table> getTableMap() { > > > >>>>>> final ImmutableMap.Builder<String, Table> builder = > > > >>>>> ImmutableMap.builder(); > > > >>>>>> > > > >>>>>> @@ -120,7 +133,8 @@ public class Elasticsearch2Schema extends > > > >>>>> AbstractSchema > > > >>>>>> > > > >>>>>> final List<DiscoveryNode> nodes = > > > >>>>> ImmutableList.copyOf(transportClient.connectedNodes()); > > > >>>>>> if (nodes.isEmpty()) { > > > >>>>>> - throw new RuntimeException("Cannot connect to any > > > elasticsearch > > > >>>>> nodes"); > > > >>>>>> + throw new IllegalStateException("Cannot connect to any > > > >>>>> elasticsearch node: " > > > >>>>>> + + transportNodes); > > > >>>>>> } > > > >>>>>> > > > >>>>>> client = transportClient; > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>> > > > > > > http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> diff --git > > > >>>>> > > > >>> > > > > > > a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java > > > >>>>> > > > >>> > > > > > > b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java > > > >>>>>> index 636aa5f..2928835 100644 > > > >>>>>> --- > > > >>>>> > > > >>> > > > > > > a/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java > > > >>>>>> +++ > > > >>>>> > > > >>> > > > > > > b/elasticsearch2/src/main/java/org/apache/calcite/adapter/elasticsearch2/Elasticsearch2Table.java > > > >>>>>> @@ -45,8 +45,15 @@ public class Elasticsearch2Table extends > > > >>>>> AbstractElasticsearchTable { > > > >>>>>> this.client = client; > > > >>>>>> } > > > >>>>>> > > > >>>>>> + /** > > > >>>>>> + * ES version 2.x. To access document attributes ES2 uses > > {@code > > > >>>>> _source.foo} syntax. > > > >>>>>> + */ > > > >>>>>> + @Override protected String scriptedFieldPrefix() { > > > >>>>>> + return "_source"; > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> @Override protected Enumerable<Object> find(String index, > > > List<String> > > > >>>>> ops, > > > >>>>>> - List<Map.Entry<String, Class>> fields) { > > > >>>>>> + > > > List<Map.Entry<String, > > > >>>>> Class>> fields) { > > > >>>>>> final String dbName = index; > > > >>>>>> > > > >>>>>> final String queryString = "{" + Util.toString(ops, "", ", ", > > "") + > > > >>>>> "}"; > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>> > > > > > > http://git-wip-us.apache.org/repos/asf/calcite/blob/c12cb4b0/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/ElasticSearch2AdapterTest.java > > > >>>>>> > > > ---------------------------------------------------------------------- > > > >>>>>> diff --git > > > >>>>> > > > >>> > > > > > > a/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/ElasticSearch2AdapterTest.java > > > >>>>> > > > >>> > > > > > > b/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/ElasticSearch2AdapterTest.java > > > >>>>>> new file mode 100644 > > > >>>>>> index 0000000..287e094 > > > >>>>>> --- /dev/null > > > >>>>>> +++ > > > >>>>> > > > >>> > > > > > > b/elasticsearch2/src/test/java/org/apache/calcite/adapter/elasticsearch2/ElasticSearch2AdapterTest.java > > > >>>>>> @@ -0,0 +1,395 @@ > > > >>>>>> +/* > > > >>>>>> + * Licensed to the Apache Software Foundation (ASF) under one > or > > > more > > > >>>>>> + * contributor license agreements. See the NOTICE file > > distributed > > > >>> with > > > >>>>>> + * this work for additional information regarding copyright > > > ownership. > > > >>>>>> + * The ASF licenses this file to you under the Apache License, > > > Version > > > >>>>> 2.0 > > > >>>>>> + * (the "License"); you may not use this file except in > > compliance > > > >>> with > > > >>>>>> + * the License. You may obtain a copy of the License at > > > >>>>>> + * > > > >>>>>> + * http://www.apache.org/licenses/LICENSE-2.0 > > > >>>>>> + * > > > >>>>>> + * Unless required by applicable law or agreed to in writing, > > > software > > > >>>>>> + * distributed under the License is distributed on an "AS IS" > > > BASIS, > > > >>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express > > or > > > >>>>> implied. > > > >>>>>> + * See the License for the specific language governing > > permissions > > > and > > > >>>>>> + * limitations under the License. > > > >>>>>> + */ > > > >>>>>> +package org.apache.calcite.adapter.elasticsearch2; > > > >>>>>> + > > > >>>>>> +import org.apache.calcite.jdbc.CalciteConnection; > > > >>>>>> +import org.apache.calcite.schema.SchemaPlus; > > > >>>>>> +import org.apache.calcite.schema.impl.ViewTable; > > > >>>>>> +import org.apache.calcite.schema.impl.ViewTableMacro; > > > >>>>>> +import org.apache.calcite.test.CalciteAssert; > > > >>>>>> +import org.apache.calcite.test.ElasticChecker; > > > >>>>>> + > > > >>>>>> +import com.google.common.io.LineProcessor; > > > >>>>>> +import com.google.common.io.Resources; > > > >>>>>> + > > > >>>>>> +import org.elasticsearch.action.bulk.BulkItemResponse; > > > >>>>>> +import org.elasticsearch.action.bulk.BulkRequestBuilder; > > > >>>>>> +import org.elasticsearch.action.bulk.BulkResponse; > > > >>>>>> +import org.elasticsearch.common.xcontent.XContentBuilder; > > > >>>>>> +import org.elasticsearch.common.xcontent.XContentFactory; > > > >>>>>> + > > > >>>>>> +import org.junit.BeforeClass; > > > >>>>>> +import org.junit.ClassRule; > > > >>>>>> +import org.junit.Test; > > > >>>>>> + > > > >>>>>> +import java.io.IOException; > > > >>>>>> +import java.nio.charset.StandardCharsets; > > > >>>>>> +import java.sql.Connection; > > > >>>>>> +import java.sql.DriverManager; > > > >>>>>> +import java.sql.SQLException; > > > >>>>>> +import java.util.Arrays; > > > >>>>>> +import java.util.Collections; > > > >>>>>> +import java.util.Locale; > > > >>>>>> + > > > >>>>>> +/** > > > >>>>>> + * Set of tests for ES adapter. Uses real instance via {@link > > > >>>>> EmbeddedElasticRule}. Document > > > >>>>>> + * source is local {@code zips-mini.json} file (located in the > > > >>>>> classpath). > > > >>>>>> + */ > > > >>>>>> +public class ElasticSearch2AdapterTest { > > > >>>>>> + > > > >>>>>> + @ClassRule //init once for all tests > > > >>>>>> + public static final EmbeddedElasticRule NODE = > > > >>>>> EmbeddedElasticRule.create(); > > > >>>>>> + > > > >>>>>> + private static final String ZIPS = "zips"; > > > >>>>>> + > > > >>>>>> + /** > > > >>>>>> + * Used to create {@code zips} index and insert some data > > > >>>>>> + */ > > > >>>>>> + @BeforeClass > > > >>>>>> + public static void setupInstance() throws Exception { > > > >>>>>> + // define mapping so fields are searchable (term query) > > > >>>>>> + XContentBuilder mapping = > > > >>>>> XContentFactory.jsonBuilder().startObject() > > > >>>>>> + .startObject("properties") > > > >>>>>> + .startObject("city").field("type", "string") > > > >>>>>> + .field("index", "not_analyzed").endObject() > > > >>>>>> + .startObject("state").field("type", "string") > > > >>>>>> + .field("index", "not_analyzed").endObject() > > > >>>>>> + .startObject("pop").field("type", > "long").endObject() > > > >>>>>> + .endObject() > > > >>>>>> + .endObject(); > > > >>>>>> + > > > >>>>>> + // create index > > > >>>>>> + NODE.client().admin().indices() > > > >>>>>> + .prepareCreate(ZIPS) > > > >>>>>> + .addMapping(ZIPS, mapping) > > > >>>>>> + .get(); > > > >>>>>> + > > > >>>>>> + BulkRequestBuilder bulk = > > > >>>>> NODE.client().prepareBulk().setRefresh(true); > > > >>>>>> + > > > >>>>>> + // load records from file > > > >>>>>> + > > > >>>>> > > > >>> > > > > > > Resources.readLines(ElasticSearch2AdapterTest.class.getResource("/zips-mini.json"), > > > >>>>>> + StandardCharsets.UTF_8, new LineProcessor<Void>() { > > > >>>>>> + @Override public boolean processLine(String line) > > > throws > > > >>>>> IOException { > > > >>>>>> + line = line.replaceAll("_id", "id"); // _id is > a > > > >>>>> reserved attribute in ES > > > >>>>>> + bulk.add(NODE.client().prepareIndex(ZIPS, > > > >>>>> ZIPS).setSource(line)); > > > >>>>>> + return true; > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Override public Void getResult() { > > > >>>>>> + return null; > > > >>>>>> + } > > > >>>>>> + }); > > > >>>>>> + > > > >>>>>> + if (bulk.numberOfActions() == 0) { > > > >>>>>> + throw new IllegalStateException("No records to be > > indexed"); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + BulkResponse response = bulk.execute().get(); > > > >>>>>> + > > > >>>>>> + if (response.hasFailures()) { > > > >>>>>> + throw new IllegalStateException( > > > >>>>>> + String.format(Locale.getDefault(), "Failed to > > > populate > > > >>>>> %s:\n%s", NODE.httpAddress(), > > > >>>>>> + > > > >>>>> > > Arrays.stream(response.getItems()).filter(BulkItemResponse::isFailed) > > > >>>>>> + > > > >>>>> > > > >>> > > > > > > .map(BulkItemResponse::getFailureMessage).findFirst().orElse("<unknown>"))); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + private CalciteAssert.ConnectionFactory > newConnectionFactory() > > { > > > >>>>>> + return new CalciteAssert.ConnectionFactory() { > > > >>>>>> + @Override public Connection createConnection() throws > > > >>>>> SQLException { > > > >>>>>> + final Connection connection = > > > >>>>> DriverManager.getConnection("jdbc:calcite:"); > > > >>>>>> + final SchemaPlus root = > > > >>>>> connection.unwrap(CalciteConnection.class).getRootSchema(); > > > >>>>>> + > > > >>>>>> + root.add("elastic", new > > Elasticsearch2Schema(NODE.client(), > > > >>>>> ZIPS)); > > > >>>>>> + > > > >>>>>> + // add calcite view programmatically > > > >>>>>> + final String viewSql = "select cast(_MAP['city'] AS > > > >>>>> varchar(20)) AS \"city\", " > > > >>>>>> + + " cast(_MAP['loc'][0] AS float) AS > > > \"longitude\",\n" > > > >>>>>> + + " cast(_MAP['loc'][1] AS float) AS > > > \"latitude\",\n" > > > >>>>>> + + " cast(_MAP['pop'] AS integer) AS \"pop\", " > > > >>>>>> + + " cast(_MAP['state'] AS varchar(2)) AS > > \"state\", > > > " > > > >>>>>> + + " cast(_MAP['id'] AS varchar(5)) AS \"id\" " > > > >>>>>> + + "from \"elastic\".\"zips\""; > > > >>>>>> + > > > >>>>>> + ViewTableMacro macro = ViewTable.viewMacro(root, > viewSql, > > > >>>>>> + Collections.singletonList("elastic"), > > > >>>>> Arrays.asList("elastic", "view"), false); > > > >>>>>> + root.add("ZIPS", macro); > > > >>>>>> + > > > >>>>>> + return connection; > > > >>>>>> + } > > > >>>>>> + }; > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + private CalciteAssert.AssertThat calciteAssert() { > > > >>>>>> + return CalciteAssert.that() > > > >>>>>> + .with(newConnectionFactory()); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + /** > > > >>>>>> + * Tests using calcite view > > > >>>>>> + */ > > > >>>>>> + @Test > > > >>>>>> + public void view() throws Exception { > > > >>>>>> + calciteAssert() > > > >>>>>> + .query("select * from zips where \"city\" = > > 'BROOKLYN'") > > > >>>>>> + .returns("city=BROOKLYN; longitude=-73.956985; > > > >>>>> latitude=40.646694; " > > > >>>>>> + + "pop=111396; state=NY; id=11226\n") > > > >>>>>> + .returnsCount(1); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test > > > >>>>>> + public void emptyResult() { > > > >>>>>> + CalciteAssert.that() > > > >>>>>> + .with(newConnectionFactory()) > > > >>>>>> + .query("select * from zips limit 0") > > > >>>>>> + .returnsCount(0); > > > >>>>>> + > > > >>>>>> + CalciteAssert.that() > > > >>>>>> + .with(newConnectionFactory()) > > > >>>>>> + .query("select * from \"elastic\".\"zips\" where > > > >>>>> _MAP['Foo'] = '_MISSING_'") > > > >>>>>> + .returnsCount(0); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test > > > >>>>>> + public void basic() throws Exception { > > > >>>>>> + CalciteAssert.that() > > > >>>>>> + .with(newConnectionFactory()) > > > >>>>>> + .query("select * from \"elastic\".\"zips\" where > > > >>>>> _MAP['city'] = 'BROOKLYN'") > > > >>>>>> + .returnsCount(1); > > > >>>>>> + > > > >>>>>> + CalciteAssert.that() > > > >>>>>> + .with(newConnectionFactory()) > > > >>>>>> + .query("select * from \"elastic\".\"zips\" where" > > > >>>>>> + + " _MAP['city'] in ('BROOKLYN', > > > 'WASHINGTON')") > > > >>>>>> + .returnsCount(2); > > > >>>>>> + > > > >>>>>> + // lower-case > > > >>>>>> + CalciteAssert.that() > > > >>>>>> + .with(newConnectionFactory()) > > > >>>>>> + .query("select * from \"elastic\".\"zips\" where " > > > >>>>>> + + "_MAP['city'] in ('brooklyn', 'Brooklyn', > > > >>>>> 'BROOK') ") > > > >>>>>> + .returnsCount(0); > > > >>>>>> + > > > >>>>>> + // missing field > > > >>>>>> + CalciteAssert.that() > > > >>>>>> + .with(newConnectionFactory()) > > > >>>>>> + .query("select * from \"elastic\".\"zips\" where > > > >>>>> _MAP['CITY'] = 'BROOKLYN'") > > > >>>>>> + .returnsCount(0); > > > >>>>>> + > > > >>>>>> + // limit works > > > >>>>>> + CalciteAssert.that() > > > >>>>>> + .with(newConnectionFactory()) > > > >>>>>> + .query("select * from \"elastic\".\"zips\" limit > 42") > > > >>>>>> + .returnsCount(42); > > > >>>>>> + > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testSort() { > > > >>>>>> + final String explain = > > > "PLAN=ElasticsearchToEnumerableConverter\n" > > > >>>>>> + + " ElasticsearchSort(sort0=[$4], dir0=[ASC])\n" > > > >>>>>> + + " ElasticsearchProject(city=[CAST(ITEM($0, > > > >>>>> 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE > > > >>>>> \"ISO-8859-1$en_US$primary\"], longitude=[CAST(ITEM(ITEM($0, > > 'loc'), > > > >>>>> 0)):FLOAT], latitude=[CAST(ITEM(ITEM($0, 'loc'), 1)):FLOAT], > > > >>>>> pop=[CAST(ITEM($0, 'pop')):INTEGER], state=[CAST(ITEM($0, > > > >>>>> 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE > > > >>>>> \"ISO-8859-1$en_US$primary\"], id=[CAST(ITEM($0, > 'id')):VARCHAR(5) > > > >>>>> CHARACTER SET \"ISO-8859-1\" COLLATE > > > \"ISO-8859-1$en_US$primary\"])\n" > > > >>>>>> + + " ElasticsearchTableScan(table=[[elastic, > > > zips]])"; > > > >>>>>> + > > > >>>>>> + calciteAssert() > > > >>>>>> + .query("select * from zips order by \"state\"") > > > >>>>>> + .returnsCount(10) > > > >>>>>> + .explainContains(explain); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testSortLimit() { > > > >>>>>> + final String sql = "select \"state\", \"id\" from zips\n" > > > >>>>>> + + "order by \"state\", \"id\" offset 2 rows fetch > > next > > > 3 > > > >>>>> rows only"; > > > >>>>>> + calciteAssert() > > > >>>>>> + .query(sql) > > > >>>>>> + .returnsUnordered("state=AK; id=99801", > > > >>>>>> + "state=AL; id=35215", > > > >>>>>> + "state=AL; id=35401") > > > >>>>>> + .queryContains( > > > >>>>>> + ElasticChecker.elasticsearchChecker( > > > >>>>>> + "\"_source\" : [\"state\", > \"id\"]", > > > >>>>>> + "\"sort\": [ {\"state\": \"asc\"}, > > > >>> {\"id\": > > > >>>>> \"asc\"}]", > > > >>>>>> + "\"from\": 2", > > > >>>>>> + "\"size\": 3")); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + > > > >>>>>> + > > > >>>>>> + @Test public void testOffsetLimit() { > > > >>>>>> + final String sql = "select \"state\", \"id\" from zips\n" > > > >>>>>> + + "offset 2 fetch next 3 rows only"; > > > >>>>>> + calciteAssert() > > > >>>>>> + .query(sql) > > > >>>>>> + .runs() > > > >>>>>> + .queryContains( > > > >>>>>> + ElasticChecker.elasticsearchChecker( > > > >>>>>> + "\"from\": 2", > > > >>>>>> + "\"size\": 3", > > > >>>>>> + "\"_source\" : [\"state\", > > \"id\"]")); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testLimit() { > > > >>>>>> + final String sql = "select \"state\", \"id\" from zips\n" > > > >>>>>> + + "fetch next 3 rows only"; > > > >>>>>> + > > > >>>>>> + calciteAssert() > > > >>>>>> + .query(sql) > > > >>>>>> + .runs() > > > >>>>>> + .queryContains( > > > >>>>>> + ElasticChecker.elasticsearchChecker( > > > >>>>>> + "\"size\": 3", > > > >>>>>> + "\"_source\" : [\"state\", > > \"id\"]")); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testFilterSort() { > > > >>>>>> + final String sql = "select * from zips\n" > > > >>>>>> + + "where \"state\" = 'CA' and \"id\" >= '70000'\n" > > > >>>>>> + + "order by \"state\", \"id\""; > > > >>>>>> + final String explain = > > > "PLAN=ElasticsearchToEnumerableConverter\n" > > > >>>>>> + + " ElasticsearchSort(sort0=[$4], sort1=[$5], > > > dir0=[ASC], > > > >>>>> dir1=[ASC])\n" > > > >>>>>> + + " ElasticsearchProject(city=[CAST(ITEM($0, > > > >>>>> 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE > > > >>>>> \"ISO-8859-1$en_US$primary\"], longitude=[CAST(ITEM(ITEM($0, > > 'loc'), > > > >>>>> 0)):FLOAT], latitude=[CAST(ITEM(ITEM($0, 'loc'), 1)):FLOAT], > > > >>>>> pop=[CAST(ITEM($0, 'pop')):INTEGER], state=[CAST(ITEM($0, > > > >>>>> 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE > > > >>>>> \"ISO-8859-1$en_US$primary\"], id=[CAST(ITEM($0, > 'id')):VARCHAR(5) > > > >>>>> CHARACTER SET \"ISO-8859-1\" COLLATE > > > \"ISO-8859-1$en_US$primary\"])\n" > > > >>>>>> + + " > > > >>> ElasticsearchFilter(condition=[AND(=(CAST(ITEM($0, > > > >>>>> 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE > > > >>>>> \"ISO-8859-1$en_US$primary\", 'CA'), >=(CAST(ITEM($0, > > > 'id')):VARCHAR(5) > > > >>>>> CHARACTER SET \"ISO-8859-1\" COLLATE > \"ISO-8859-1$en_US$primary\", > > > >>>>> '70000'))])\n" > > > >>>>>> + + " ElasticsearchTableScan(table=[[elastic, > > > >>> zips]])"; > > > >>>>>> + calciteAssert() > > > >>>>>> + .query(sql) > > > >>>>>> + .returnsOrdered("city=LOS ANGELES; > > > longitude=-118.258189; > > > >>>>> latitude=34.007856; " > > > >>>>>> + + "pop=96074; state=CA; id=90011", > > > >>>>>> + "city=BELL GARDENS; longitude=-118.17205; > > > >>>>> latitude=33.969177; " > > > >>>>>> + + "pop=99568; state=CA; id=90201", > > > >>>>>> + "city=NORWALK; longitude=-118.081767; > > > >>>>> latitude=33.90564; " > > > >>>>>> + + "pop=94188; state=CA; id=90650") > > > >>>>>> + .queryContains( > > > >>>>>> + > > ElasticChecker.elasticsearchChecker("\"query\" > > > : " > > > >>>>>> + + > > > >>>>> "{\"constant_score\":{\"filter\":{\"bool\":" > > > >>>>>> + + > > > >>>>> "{\"must\":[{\"term\":{\"state\":\"CA\"}}," > > > >>>>>> + + > > > >>>>> "{\"range\":{\"id\":{\"gte\":\"70000\"}}}]}}}}", > > > >>>>>> + "\"script_fields\": > > > >>>>> {\"longitude\":{\"script\":\"_source.loc[0]\"}, " > > > >>>>>> + + > > > >>>>> "\"latitude\":{\"script\":\"_source.loc[1]\"}, " > > > >>>>>> + + "\"city\":{\"script\": > > > >>>>> \"_source.city\"}, " > > > >>>>>> + + "\"pop\":{\"script\": > > > >>>>> \"_source.pop\"}, " > > > >>>>>> + + "\"state\":{\"script\": > > > >>>>> \"_source.state\"}, " > > > >>>>>> + + "\"id\":{\"script\": \"_ > > > >>> source.id > > > >>>>> \"}}", > > > >>>>>> + "\"sort\": [ {\"state\": \"asc\"}, > > > >>> {\"id\": > > > >>>>> \"asc\"}]")) > > > >>>>>> + .explainContains(explain); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testFilterSortDesc() { > > > >>>>>> + final String sql = "select * from zips\n" > > > >>>>>> + + "where \"pop\" BETWEEN 95000 AND 100000\n" > > > >>>>>> + + "order by \"state\" desc, \"pop\""; > > > >>>>>> + calciteAssert() > > > >>>>>> + .query(sql) > > > >>>>>> + .limit(4) > > > >>>>>> + .returnsOrdered( > > > >>>>>> + "city=LOS ANGELES; longitude=-118.258189; > > > >>>>> latitude=34.007856; pop=96074; state=CA; id=90011", > > > >>>>>> + "city=BELL GARDENS; longitude=-118.17205; > > > >>>>> latitude=33.969177; pop=99568; state=CA; id=90201"); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testFilterRedundant() { > > > >>>>>> + final String sql = "select * from zips\n" > > > >>>>>> + + "where \"state\" > 'CA' and \"state\" < 'AZ' and > > > >>>>> \"state\" = 'OK'"; > > > >>>>>> + calciteAssert() > > > >>>>>> + .query(sql) > > > >>>>>> + .runs() > > > >>>>>> + .queryContains( > > > >>>>>> + ElasticChecker.elasticsearchChecker("" > > > >>>>>> + + "\"query\" : > > > >>>>> {\"constant_score\":{\"filter\":{\"bool\":" > > > >>>>>> + + > > > >>>>> "{\"must\":[{\"term\":{\"state\":\"OK\"}}]}}}}", > > > >>>>>> + "\"script_fields\": > > > >>>>> {\"longitude\":{\"script\":\"_source.loc[0]\"}, " > > > >>>>>> + + > > > >>>>> "\"latitude\":{\"script\":\"_source.loc[1]\"}, " > > > >>>>>> + + "\"city\":{\"script\": > > > >>>>> \"_source.city\"}, " > > > >>>>>> + + "\"pop\":{\"script\": > > > >>>>> \"_source.pop\"}, \"state\":{\"script\": \"_source.state\"}, " > > > >>>>>> + + > > > "\"id\":{\"script\": > > > >>>>> \"_source.id\"}}" > > > >>>>>> + )); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testInPlan() { > > > >>>>>> + final String[] searches = { > > > >>>>>> + "\"query\" : > > > >>>>> {\"constant_score\":{\"filter\":{\"bool\":{\"should\":" > > > >>>>>> + + > > > >>>>> > > > >>> > > > > > > "[{\"bool\":{\"must\":[{\"term\":{\"pop\":96074}}]}},{\"bool\":{\"must\":[{\"term\":" > > > >>>>>> + + "{\"pop\":99568}}]}}]}}}}", > > > >>>>>> + "\"script_fields\": > > > >>>>> {\"longitude\":{\"script\":\"_source.loc[0]\"}, " > > > >>>>>> + + > > "\"latitude\":{\"script\":\"_source.loc[1]\"}, > > > " > > > >>>>>> + + "\"city\":{\"script\": \"_source.city\"}, " > > > >>>>>> + + "\"pop\":{\"script\": \"_source.pop\"}, " > > > >>>>>> + + "\"state\":{\"script\": > \"_source.state\"}, " > > > >>>>>> + + "\"id\":{\"script\": \"_source.id\"}}" > > > >>>>>> + }; > > > >>>>>> + > > > >>>>>> + calciteAssert() > > > >>>>>> + .query("select * from zips where \"pop\" in (96074, > > > >>> 99568)") > > > >>>>>> + .returnsUnordered( > > > >>>>>> + "city=BELL GARDENS; longitude=-118.17205; > > > >>>>> latitude=33.969177; pop=99568; state=CA; id=90201", > > > >>>>>> + "city=LOS ANGELES; longitude=-118.258189; > > > >>>>> latitude=34.007856; pop=96074; state=CA; id=90011" > > > >>>>>> + ) > > > >>>>>> + > > > >>>>> .queryContains(ElasticChecker.elasticsearchChecker(searches)); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testZips() { > > > >>>>>> + calciteAssert() > > > >>>>>> + .query("select \"state\", \"city\" from zips") > > > >>>>>> + .returnsCount(10); > > > >>>>>> + } > > > >>>>>> + > > > >>>>>> + @Test public void testProject() { > > > >>>>>> + final String sql = "select \"state\", \"city\", 0 as > > > \"zero\"\n" > > > >>>>>> + + "from zips\n" > > > >>>>>> + + "order by \"state\", \"city\""; > > > >>>>>> + > > > >>>>>> + calciteAssert() > > > >>>>>> + .query(sql) > > > >>>>>> + .limit(2) > > > >>>>>> + .returnsUnordered("state=AK; city=ANCHORAGE; > zero=0", > > > >>>>>> + "state=AK; city=FAIRBANKS; zero=0") > > > >>>>>> + .queryContains( > > > >>>>>> + > > > >>>>> ElasticChecker.elasticsearchChecker("\"script_fields\": " > > > >>>>>> + + "{\"zero\":{\"script\": > > > \"0\"}, > > > >>> " > > > >>>>>> + + "\"state\":{\"script\": > > > >>>>> \"_source.state\"}, " > > > >>>>>> + + "\"city\":{\"script\": > > > >>>>> \"_source.city\"}}", > > > >>>>>> + "\"sort
