date:20180528

[jira] [Created] (CALCITE-2333) Stop releasing ZIPs

2018-05-28 Thread Francis Chuang (JIRA)

Francis Chuang created CALCITE-2333:
---

 Summary: Stop releasing ZIPs
 Key: CALCITE-2333
 URL: https://issues.apache.org/jira/browse/CALCITE-2333
 Project: Calcite
  Issue Type: Task
  Components: avatica, avatica-go, core
Reporter: Francis Chuang
Assignee: Francis Chuang


There was some discussion on the mailing list regarding releasing our source in 
just 1 format.

See [https://mail-archives.apache.org/mod_mbox/calcite-dev/201804.mbox/browser]

There is some consensus around just releasing a tar.gz. I think this is a good 
idea and is something we should aim for, for the Calcite 1.17 and Avatica 1.17 
releases.

The following changes will be needed:
 * Update build script for avatica-go.
 * Update maven config for avatica.
 * Update calcite config for calcite.

We will need to update the website to deal the zip files not existing. My 
proposal is that we edit the old release posts to include a `has_zip: true` 
property. For the old releases, the website should render a link to the zip. 
For the new releases, we do not need to do anything.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Releases

2018-05-28 Thread Francis Chuang


Hey Julian,

I would like to be the release manager for Avatica 1.12. I think this 
release would be a good opportunity to see if we can resolve some of the 
staler bugs in JIRA.


Sergey: Thanks for offering! Would love to have you help out with this 
release!


Francis

On 29/05/2018 12:52 PM, Julian Hyde wrote:

Sergey,

Thanks for the offer of help. Since you are not a committer you cannot
be release manager* but you can help out. When we find a release
manager he/she will let you know what you can do to help.

Julian

* Strictly, only PMC members can be release manager, but in the last
avatica-go release Francis (then a committer) did practically
everything except generate and sign the artifacts.

On Mon, May 28, 2018 at 6:22 AM, Sergey Nuyanzin  wrote:

Hello

If it is possible I would like to be a volunteer here.

On Sun, May 27, 2018 at 3:06 AM, Julian Hyde  wrote:


Thanks, Volodymyr. It would be great if you could be Calcite RM.

Does anyone volunteer to be release manager for Avatica? It is an easier
job than Calcite, and as Volodymyr
says, a great way to gain experience.

Does anyone have any time constraints for Calcite or Avatica releases?

We’ve been releasing Calcite every 2-3 months, so if we released Calcite
on June 19 that would be exactly 3 months. If we need to make an Avatica
release before that, timing will be tight.

Julian



On May 26, 2018, at 2:13 AM, Volodymyr Vysotskyi 

wrote:

Hello,

I volunteer to be the release manager for Calcite if there are no
objections. It will be a great experience for me.

Calcite 1.16 was released more than two months ago (March 19), but since

it

depends on the Avatica release, Avatica should be released asap.

I suppose it is needed two weeks for Avatica release, so the estimated
target date starting Calcite release process is the middle of the Jun.

Kind regards,
Volodymyr Vysotskyi


сб, 26 трав. 2018 о 08:26 Julian Hyde  пише:


When should we release Calcite 1.17? When should we release Avatica

1.12?

Anyone volunteer to be release manager?

Because of https://issues.apache.org/jira/browse/CALCITE-2303 <
https://issues.apache.org/jira/browse/CALCITE-2303>, we need to release
Avatica before Calcite.

I have logged https://issues.apache.org/jira/browse/CALCITE-2330 <
https://issues.apache.org/jira/browse/CALCITE-2330> to track Avatica

1.12.

Julian






--
Best regards,
Sergey

using "fakes" in unit tests

2018-05-28 Thread Andrei Sereda

Hello,

Current IT tests require manual startup and seeding of data-sources (Mongo,
ES, Geode etc.) from separate git project (see calcite-test-dataset
 ). While useful they force
developer to bootstrap additional environment on his own and don't work by
default (unless properly initialized).

Would you consider using "fake" (in memory) data-sources for unit tests
like:
- Fongo   for Mongo
- TestNode

for ElasticSearch

I'm not proposing to stop using IT but rather having a better coverage for
unit tests (with fakes)

Interesting to hear your opinion.

Regards,
Andrei.

[jira] [Created] (CALCITE-2332) It's wrong to simplify FLOOR(CEIL(x)) to FLOOR(x)

2018-05-28 Thread Julian Hyde (JIRA)

Julian Hyde created CALCITE-2332:


 Summary: It's wrong to simplify FLOOR(CEIL(x)) to FLOOR(x)
 Key: CALCITE-2332
 URL: https://issues.apache.org/jira/browse/CALCITE-2332
 Project: Calcite
  Issue Type: Bug
Reporter: Julian Hyde
Assignee: Julian Hyde


It's wrong to simplify {{FLOOR(CEIL(x))}} to {{FLOOR(x)}}.

Specifically, it is not valid to simplify {{FLOOR(CEIL(x TO YEAR) TO YEAR)}} to 
{{FLOOR(x TO YEAR)}} when {{x}} is {{TIMESTAMP '2010-10-10 00:00:00'}}. I 
believe the correct simplification is {{CEIL(x TO YEAR)}}.

I have disabled {{RexImplicationCheckerTest.testSimplifyFloor}} due to this 
issue. Please re-enable it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?

2018-05-28 Thread liu xiaorui

I also encountered this problem .   After I debug the code I found the
cause of the problem .  for example  condition is A AND (B OR C )   .

 orNode is AND (A,OR (B,C))  and  then execute method translateAdd  ,
condition A can  be translated successfully , and exception will be throw
out when translating  condition  OR(B,C) .  so I think we can think
about recursively.










On 29 May 2018 at 09:56, Julian Hyde  wrote:

> Looks like a bug. I don't know that code well, so I can't speculate
> what might be the cause.
>
> Can you log a JIRA case please? If you can provide a unit test (say, a
> method to add to MongoAdapterIT) even better.
>
> On Mon, May 28, 2018 at 2:47 PM, Andrei Sereda  wrote:
> > Hello,
> >
> > Please correct me if my expectations are inaccurate about query
> predicates
> > in Calcite.
> >
> > I think there is an issue in how expression translators work when AST is
> > more complex (has more depth). Same code / logic is used across different
> > adapters (Mongo / Geode / ES) so multiple data-sources might be affected.
> >
> > Examples
> >
> > 1. Works
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where _MAP['foo'] in ('1') and true;
> >
> > 2. Works
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where _MAP['foo'] ='1' and true;
> >
> > 3. Works
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where _MAP['foo'] in ('1', '2');
> >
> >
> > 4. Fails
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where _MAP['foo'] in ('1', '2') and true;
> >
> >
> > 5. Fails
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where true and _MAP['foo'] in ('1', '2');
> >
> >
> > 6. Fails
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where (_MAP['foo'] ='1' or _MAP['foo'] = '2') and true;
> >
> >
> >
> > Exception Stack (Elastic Adapter)
> >
> >
> > java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
> > =(ITEM($0, 'foo'), '2'))
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.translateMatch2(ElasticsearchFilter.java:234)
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.translateAnd(ElasticsearchFilter.java:158)
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.translateOr(ElasticsearchFilter.java:115)
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.translateMatch(ElasticsearchFilter.java:101)
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.access$000(ElasticsearchFilter.java:87)
> >
> > at org.apache.calcite.adapter.elasticsearch.
> > ElasticsearchFilter.implement(ElasticsearchFilter.java:80)
> >
> > at org.apache.calcite.adapter.elasticsearch.
> > ElasticsearchRel$Implementor.visitChild(ElasticsearchRel.java:53)
> >
> > at org.apache.calcite.adapter.elasticsearch.
> > ElasticsearchToEnumerableConverter.implement(
> ElasticsearchToEnumerableConve
> > rter.java:71)
> >
> > at org.apache.calcite.adapter.enumerable.
> EnumerableRelImplementor.
> > visitChild(EnumerableRelImplementor.java:98)
> >
> > at org.apache.calcite.adapter.enumerable.
> > EnumerableAggregate.implement(EnumerableAggregate.java:106)
> >
> > Exception Stack (Mongo Adapter)
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select * from
> "mongo"."col"
> > where _MAP['foo'] in ('1', '2') and true;
> >
> > java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
> > =(ITEM($0, 'foo'), '2'))
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.translateMatch2(MongoFilter.java:181)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.translateAnd(MongoFilter.java:116)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.translateOr(MongoFilter.java:98)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.translateMatch(MongoFilter.java:91)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.access$000(MongoFilter.java:77)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter.implement(
> > MongoFilter.java:72)
> >
> > at org.apache.calcite.adapter.mongodb.MongoRel$Implementor.
> > visitChild(MongoRel.java:51)
> >
> > at org.apache.calcite.adapter.mongodb.
> MongoToEnumerableConverter.
> > implement(MongoToEnumerableConverter.java:84)
> >
> > at org.apache.calcite.adapter.enumerable.
> EnumerableRelImplementor.
> > implementRoot(EnumerableRelImplementor.java:103)
> >
> > at org.apache.calcite.adapter.enumerable.
> EnumerableInterpretable.
> > toBindable(EnumerableInterpretable.java:92)
> >
> >
> >
> >

Re: Releases

2018-05-28 Thread Julian Hyde

Sergey,

Thanks for the offer of help. Since you are not a committer you cannot
be release manager* but you can help out. When we find a release
manager he/she will let you know what you can do to help.

Julian

* Strictly, only PMC members can be release manager, but in the last
avatica-go release Francis (then a committer) did practically
everything except generate and sign the artifacts.

On Mon, May 28, 2018 at 6:22 AM, Sergey Nuyanzin  wrote:
> Hello
>
> If it is possible I would like to be a volunteer here.
>
> On Sun, May 27, 2018 at 3:06 AM, Julian Hyde  wrote:
>
>> Thanks, Volodymyr. It would be great if you could be Calcite RM.
>>
>> Does anyone volunteer to be release manager for Avatica? It is an easier
>> job than Calcite, and as Volodymyr
>> says, a great way to gain experience.
>>
>> Does anyone have any time constraints for Calcite or Avatica releases?
>>
>> We’ve been releasing Calcite every 2-3 months, so if we released Calcite
>> on June 19 that would be exactly 3 months. If we need to make an Avatica
>> release before that, timing will be tight.
>>
>> Julian
>>
>>
>> > On May 26, 2018, at 2:13 AM, Volodymyr Vysotskyi 
>> wrote:
>> >
>> > Hello,
>> >
>> > I volunteer to be the release manager for Calcite if there are no
>> > objections. It will be a great experience for me.
>> >
>> > Calcite 1.16 was released more than two months ago (March 19), but since
>> it
>> > depends on the Avatica release, Avatica should be released asap.
>> >
>> > I suppose it is needed two weeks for Avatica release, so the estimated
>> > target date starting Calcite release process is the middle of the Jun.
>> >
>> > Kind regards,
>> > Volodymyr Vysotskyi
>> >
>> >
>> > сб, 26 трав. 2018 о 08:26 Julian Hyde  пише:
>> >
>> >> When should we release Calcite 1.17? When should we release Avatica
>> 1.12?
>> >> Anyone volunteer to be release manager?
>> >>
>> >> Because of https://issues.apache.org/jira/browse/CALCITE-2303 <
>> >> https://issues.apache.org/jira/browse/CALCITE-2303>, we need to release
>> >> Avatica before Calcite.
>> >>
>> >> I have logged https://issues.apache.org/jira/browse/CALCITE-2330 <
>> >> https://issues.apache.org/jira/browse/CALCITE-2330> to track Avatica
>> 1.12.
>> >>
>> >> Julian
>> >>
>> >>
>>
>>
>
>
> --
> Best regards,
> Sergey

Re: RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?

2018-05-28 Thread Andrei Sereda

https://issues.apache.org/jira/browse/CALCITE-2331


On Mon, May 28, 2018 at 9:56 PM, Julian Hyde  wrote:

> Looks like a bug. I don't know that code well, so I can't speculate
> what might be the cause.
>
> Can you log a JIRA case please? If you can provide a unit test (say, a
> method to add to MongoAdapterIT) even better.
>
> On Mon, May 28, 2018 at 2:47 PM, Andrei Sereda  wrote:
> > Hello,
> >
> > Please correct me if my expectations are inaccurate about query
> predicates
> > in Calcite.
> >
> > I think there is an issue in how expression translators work when AST is
> > more complex (has more depth). Same code / logic is used across different
> > adapters (Mongo / Geode / ES) so multiple data-sources might be affected.
> >
> > Examples
> >
> > 1. Works
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where _MAP['foo'] in ('1') and true;
> >
> > 2. Works
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where _MAP['foo'] ='1' and true;
> >
> > 3. Works
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where _MAP['foo'] in ('1', '2');
> >
> >
> > 4. Fails
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where _MAP['foo'] in ('1', '2') and true;
> >
> >
> > 5. Fails
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where true and _MAP['foo'] in ('1', '2');
> >
> >
> > 6. Fails
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> > "elastic" where (_MAP['foo'] ='1' or _MAP['foo'] = '2') and true;
> >
> >
> >
> > Exception Stack (Elastic Adapter)
> >
> >
> > java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
> > =(ITEM($0, 'foo'), '2'))
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.translateMatch2(ElasticsearchFilter.java:234)
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.translateAnd(ElasticsearchFilter.java:158)
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.translateOr(ElasticsearchFilter.java:115)
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.translateMatch(ElasticsearchFilter.java:101)
> >
> > at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> > Translator.access$000(ElasticsearchFilter.java:87)
> >
> > at org.apache.calcite.adapter.elasticsearch.
> > ElasticsearchFilter.implement(ElasticsearchFilter.java:80)
> >
> > at org.apache.calcite.adapter.elasticsearch.
> > ElasticsearchRel$Implementor.visitChild(ElasticsearchRel.java:53)
> >
> > at org.apache.calcite.adapter.elasticsearch.
> > ElasticsearchToEnumerableConverter.implement(
> ElasticsearchToEnumerableConve
> > rter.java:71)
> >
> > at org.apache.calcite.adapter.enumerable.
> EnumerableRelImplementor.
> > visitChild(EnumerableRelImplementor.java:98)
> >
> > at org.apache.calcite.adapter.enumerable.
> > EnumerableAggregate.implement(EnumerableAggregate.java:106)
> >
> > Exception Stack (Mongo Adapter)
> >
> > 0: jdbc:calcite:model=target/test-classes/mod> select * from
> "mongo"."col"
> > where _MAP['foo'] in ('1', '2') and true;
> >
> > java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
> > =(ITEM($0, 'foo'), '2'))
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.translateMatch2(MongoFilter.java:181)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.translateAnd(MongoFilter.java:116)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.translateOr(MongoFilter.java:98)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.translateMatch(MongoFilter.java:91)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter$
> > Translator.access$000(MongoFilter.java:77)
> >
> > at org.apache.calcite.adapter.mongodb.MongoFilter.implement(
> > MongoFilter.java:72)
> >
> > at org.apache.calcite.adapter.mongodb.MongoRel$Implementor.
> > visitChild(MongoRel.java:51)
> >
> > at org.apache.calcite.adapter.mongodb.
> MongoToEnumerableConverter.
> > implement(MongoToEnumerableConverter.java:84)
> >
> > at org.apache.calcite.adapter.enumerable.
> EnumerableRelImplementor.
> > implementRoot(EnumerableRelImplementor.java:103)
> >
> > at org.apache.calcite.adapter.enumerable.
> EnumerableInterpretable.
> > toBindable(EnumerableInterpretable.java:92)
> >
> >
> >
> > Anything I'm doing wrong ?
> >
> > Thanks,
> > Andrei.
>

[jira] [Created] (CALCITE-2331) invalid filter evaluation in adapters

2018-05-28 Thread Andrei Sereda (JIRA)

Andrei Sereda created CALCITE-2331:
--

 Summary: invalid filter evaluation in adapters
 Key: CALCITE-2331
 URL: https://issues.apache.org/jira/browse/CALCITE-2331
 Project: Calcite
  Issue Type: Bug
  Components: elasticsearch-adapter, geode, mongodb
Reporter: Andrei Sereda
Assignee: Julian Hyde


h2. Working
{code}
select count(*) from "elastic" where _MAP['foo'] in ('1') and true;
select count(*) from "elastic" where _MAP['foo'] ='1' and true;
select count(*) from "elastic" where _MAP['foo'] in ('1', '2');
{code}

h2. Fails
{code}
select count(*) from "elastic" where _MAP['foo'] in ('1', '2') and true;
select count(*) from "elastic" where true and _MAP['foo'] in ('1', '2');
select count(*) from "elastic" where (_MAP['foo'] ='1' or _MAP['foo'] = '2') 
and true;
{code}

h2. Elastic Adapter Exception
{quote}
java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
=(ITEM($0, 'foo'), '2'))
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateMatch2(ElasticsearchFilter.java:234)
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateAnd(ElasticsearchFilter.java:158)
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateOr(ElasticsearchFilter.java:115)
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateMatch(ElasticsearchFilter.java:101)
at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.access$000(ElasticsearchFilter.java:87)
at org.apache.calcite.adapter.elasticsearch.
ElasticsearchFilter.implement(ElasticsearchFilter.java:80)
at org.apache.calcite.adapter.elasticsearch.
ElasticsearchRel$Implementor.visitChild(ElasticsearchRel.java:53)
at org.apache.calcite.adapter.elasticsearch.
ElasticsearchToEnumerableConverter.implement(ElasticsearchToEnumerableConve
rter.java:71)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.
visitChild(EnumerableRelImplementor.java:98)
at org.apache.calcite.adapter.enumerable.
EnumerableAggregate.implement(EnumerableAggregate.java:106)
{quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

skipping JDBC API (or ResultSet metadata / pagination)

2018-05-28 Thread Andrei Sereda

Hello,

Is there a way to get Enumerator directly from RelNode bypassing JDBC
calls like createStatement() / executeQuery()  ?

I'm looking for something like this:

RelBuilder builder  = .
RelNode filter = builder.scan("mySchema", "myTable").filter().build();

Enumerable result = something.execute(filter);

Follow-up question (or workaround). Some datasources (like Elastic
)
return total number of documents matching the criteria in addition to
current result (which can be smaller than total number):

hits: {
   total: 200,
   hits: [{...}, {...}] // can be smaller than 200 (batch)
}

Is there a way to expose that information to the user in ResultSet (useful
during pagination) ? Otherwise one has to perform two queries:

select count(*) from foo where $filter
select * from foo where $filter LIMIT $batch

Regards,
Andrei.

Re: RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?

2018-05-28 Thread Julian Hyde

Looks like a bug. I don't know that code well, so I can't speculate
what might be the cause.

Can you log a JIRA case please? If you can provide a unit test (say, a
method to add to MongoAdapterIT) even better.

On Mon, May 28, 2018 at 2:47 PM, Andrei Sereda  wrote:
> Hello,
>
> Please correct me if my expectations are inaccurate about query predicates
> in Calcite.
>
> I think there is an issue in how expression translators work when AST is
> more complex (has more depth). Same code / logic is used across different
> adapters (Mongo / Geode / ES) so multiple data-sources might be affected.
>
> Examples
>
> 1. Works
>
> 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> "elastic" where _MAP['foo'] in ('1') and true;
>
> 2. Works
> 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> "elastic" where _MAP['foo'] ='1' and true;
>
> 3. Works
> 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> "elastic" where _MAP['foo'] in ('1', '2');
>
>
> 4. Fails
>
> 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> "elastic" where _MAP['foo'] in ('1', '2') and true;
>
>
> 5. Fails
>
> 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> "elastic" where true and _MAP['foo'] in ('1', '2');
>
>
> 6. Fails
>
> 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> "elastic" where (_MAP['foo'] ='1' or _MAP['foo'] = '2') and true;
>
>
>
> Exception Stack (Elastic Adapter)
>
>
> java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
> =(ITEM($0, 'foo'), '2'))
>
> at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> Translator.translateMatch2(ElasticsearchFilter.java:234)
>
> at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> Translator.translateAnd(ElasticsearchFilter.java:158)
>
> at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> Translator.translateOr(ElasticsearchFilter.java:115)
>
> at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> Translator.translateMatch(ElasticsearchFilter.java:101)
>
> at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
> Translator.access$000(ElasticsearchFilter.java:87)
>
> at org.apache.calcite.adapter.elasticsearch.
> ElasticsearchFilter.implement(ElasticsearchFilter.java:80)
>
> at org.apache.calcite.adapter.elasticsearch.
> ElasticsearchRel$Implementor.visitChild(ElasticsearchRel.java:53)
>
> at org.apache.calcite.adapter.elasticsearch.
> ElasticsearchToEnumerableConverter.implement(ElasticsearchToEnumerableConve
> rter.java:71)
>
> at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.
> visitChild(EnumerableRelImplementor.java:98)
>
> at org.apache.calcite.adapter.enumerable.
> EnumerableAggregate.implement(EnumerableAggregate.java:106)
>
> Exception Stack (Mongo Adapter)
>
> 0: jdbc:calcite:model=target/test-classes/mod> select * from "mongo"."col"
> where _MAP['foo'] in ('1', '2') and true;
>
> java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
> =(ITEM($0, 'foo'), '2'))
>
> at org.apache.calcite.adapter.mongodb.MongoFilter$
> Translator.translateMatch2(MongoFilter.java:181)
>
> at org.apache.calcite.adapter.mongodb.MongoFilter$
> Translator.translateAnd(MongoFilter.java:116)
>
> at org.apache.calcite.adapter.mongodb.MongoFilter$
> Translator.translateOr(MongoFilter.java:98)
>
> at org.apache.calcite.adapter.mongodb.MongoFilter$
> Translator.translateMatch(MongoFilter.java:91)
>
> at org.apache.calcite.adapter.mongodb.MongoFilter$
> Translator.access$000(MongoFilter.java:77)
>
> at org.apache.calcite.adapter.mongodb.MongoFilter.implement(
> MongoFilter.java:72)
>
> at org.apache.calcite.adapter.mongodb.MongoRel$Implementor.
> visitChild(MongoRel.java:51)
>
> at org.apache.calcite.adapter.mongodb.MongoToEnumerableConverter.
> implement(MongoToEnumerableConverter.java:84)
>
> at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.
> implementRoot(EnumerableRelImplementor.java:103)
>
> at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.
> toBindable(EnumerableInterpretable.java:92)
>
>
>
> Anything I'm doing wrong ?
>
> Thanks,
> Andrei.

Re: Datetime_Plus and TimestampAdd operators

2018-05-28 Thread Julian Hyde

I think we should use the datetime_plus operator. It is standard, and
sufficiently general. Its second argument needs to be an interval
value, not necessarily an interval literal.

On Mon, May 28, 2018 at 2:35 PM, James Duong  wrote:
> There are essentially two ways to add an interval to a datetime vaue in
> Calcite
>
> Call the timestampadd() function:
> select {fn timestampadd(year, 1, hire_date)}...
>
> Use datetime_plus interval arithmetic:
> select hire_date + interval '1' year
>
>
> Note that timestampadd's second argument does not need to be a literal.
> Often it is a column expression. For datetime_plus literals are usually
> used.
>
> I propose we create a new SqlOperator that can canonicalize both of these
> inputs into one node. This lets us apply any transformations on this
> canonical type regardless of what the original query was.
>
> It takes in the following arguments:
> 1. A date/time/timestamp input
> 2. an interval input as an integer
> 3. a time unit for the input
> 4. a synthetic argument indicating the source form of the function call
> (either datetime_plus or timestampadd).
>
>
> The idea is that this canonical form is easy to get into for both types,
> and provides methods to easily convert to either type. This would help with
> unparsing in SqlDialects (you do not need to implement pushdown for both
> types of inputs).

Re: Does Calcite hold all records output from a node before passing them to a higher node ?

2018-05-28 Thread Muhammad Gelbana

I'm also hitting the following exception when selecting all records from a
huge table.

java.lang.OutOfMemoryError: Java heap space
at java.util.stream.IntPipeline$4.opWrapSink(IntPipeline.java:247)
at java.util.stream.AbstractPipeline.wrapSink(AbstractPipeline.java:517)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545)
at
java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at
java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:438)
at
java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:444)
at
com.sqlinterface.data.schema.ResultsEnumerator.current(ResultsEnumerator.java:96)
// Removed the rest of stacktrace

Thanks,
Gelbana

On Tue, May 29, 2018 at 1:05 AM, Muhammad Gelbana 
wrote:

> I'm not sure if I phrased my question correctly so let me explain more.
>
> I'm running a (SELECT * FROM TABLE) query against a 50 million records
> table (Following the BINDABLE convention, so it sends it's rows through a
> "sink"). Since the extracted rows aren't processed in any way, I was
> expecting that the output JDBC resultset would be able to enumerate through
> all the results in a matter of seconds, but instead, my machine didn't
> print anything. What exactly happens is that (PreparedStatement.executeQuery)
> doesn't return a resultset promptly even after a few minutes have passed.
>
> I tried a table with hundreds of rows and my testing code printed those
> results right away so it's not something I missed there, but probably a
> configuration I didn't set ? Or may be that's just how it is ? Does anyone
> else believe that the behaviour I expected is reasonable ? It would also
> lower the amount of memory consumed to hold the complete results before
> bursting them to their final destination, if that's the case in the first
> place.
>
>
> Thanks,
> Gelbana
>

Datetime_Plus and TimestampAdd operators

2018-05-28 Thread James Duong

There are essentially two ways to add an interval to a datetime vaue in
Calcite

Call the timestampadd() function:
select {fn timestampadd(year, 1, hire_date)}...

Use datetime_plus interval arithmetic:
select hire_date + interval '1' year


Note that timestampadd's second argument does not need to be a literal.
Often it is a column expression. For datetime_plus literals are usually
used.

I propose we create a new SqlOperator that can canonicalize both of these
inputs into one node. This lets us apply any transformations on this
canonical type regardless of what the original query was.

It takes in the following arguments:
1. A date/time/timestamp input
2. an interval input as an integer
3. a time unit for the input
4. a synthetic argument indicating the source form of the function call
(either datetime_plus or timestampadd).


The idea is that this canonical form is easy to get into for both types,
and provides methods to easily convert to either type. This would help with
unparsing in SqlDialects (you do not need to implement pushdown for both
types of inputs).

Does Calcite hold all records output from a node before passing them to a higher node ?

2018-05-28 Thread Muhammad Gelbana

I'm not sure if I phrased my question correctly so let me explain more.

I'm running a (SELECT * FROM TABLE) query against a 50 million records
table (Following the BINDABLE convention, so it sends it's rows through a
"sink"). Since the extracted rows aren't processed in any way, I was
expecting that the output JDBC resultset would be able to enumerate through
all the results in a matter of seconds, but instead, my machine didn't
print anything. What exactly happens is that
(PreparedStatement.executeQuery) doesn't return a resultset promptly even
after a few minutes have passed.

I tried a table with hundreds of rows and my testing code printed those
results right away so it's not something I missed there, but probably a
configuration I didn't set ? Or may be that's just how it is ? Does anyone
else believe that the behaviour I expected is reasonable ? It would also
lower the amount of memory consumed to hold the complete results before
bursting them to their final destination, if that's the case in the first
place.


Thanks,
Gelbana

RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?

2018-05-28 Thread Andrei Sereda

Hello,

Please correct me if my expectations are inaccurate about query predicates
in Calcite.

I think there is an issue in how expression translators work when AST is
more complex (has more depth). Same code / logic is used across different
adapters (Mongo / Geode / ES) so multiple data-sources might be affected.

Examples

1. Works

0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
"elastic" where _MAP['foo'] in ('1') and true;

2. Works
0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
"elastic" where _MAP['foo'] ='1' and true;

3. Works
0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
"elastic" where _MAP['foo'] in ('1', '2');


4. Fails

0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
"elastic" where _MAP['foo'] in ('1', '2') and true;


5. Fails

0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
"elastic" where true and _MAP['foo'] in ('1', '2');


6. Fails

0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
"elastic" where (_MAP['foo'] ='1' or _MAP['foo'] = '2') and true;



Exception Stack (Elastic Adapter)


java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
=(ITEM($0, 'foo'), '2'))

at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateMatch2(ElasticsearchFilter.java:234)

at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateAnd(ElasticsearchFilter.java:158)

at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateOr(ElasticsearchFilter.java:115)

at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.translateMatch(ElasticsearchFilter.java:101)

at org.apache.calcite.adapter.elasticsearch.ElasticsearchFilter$
Translator.access$000(ElasticsearchFilter.java:87)

at org.apache.calcite.adapter.elasticsearch.
ElasticsearchFilter.implement(ElasticsearchFilter.java:80)

at org.apache.calcite.adapter.elasticsearch.
ElasticsearchRel$Implementor.visitChild(ElasticsearchRel.java:53)

at org.apache.calcite.adapter.elasticsearch.
ElasticsearchToEnumerableConverter.implement(ElasticsearchToEnumerableConve
rter.java:71)

at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.
visitChild(EnumerableRelImplementor.java:98)

at org.apache.calcite.adapter.enumerable.
EnumerableAggregate.implement(EnumerableAggregate.java:106)

Exception Stack (Mongo Adapter)

0: jdbc:calcite:model=target/test-classes/mod> select * from "mongo"."col"
where _MAP['foo'] in ('1', '2') and true;

java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
=(ITEM($0, 'foo'), '2'))

at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.translateMatch2(MongoFilter.java:181)

at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.translateAnd(MongoFilter.java:116)

at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.translateOr(MongoFilter.java:98)

at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.translateMatch(MongoFilter.java:91)

at org.apache.calcite.adapter.mongodb.MongoFilter$
Translator.access$000(MongoFilter.java:77)

at org.apache.calcite.adapter.mongodb.MongoFilter.implement(
MongoFilter.java:72)

at org.apache.calcite.adapter.mongodb.MongoRel$Implementor.
visitChild(MongoRel.java:51)

at org.apache.calcite.adapter.mongodb.MongoToEnumerableConverter.
implement(MongoToEnumerableConverter.java:84)

at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.
implementRoot(EnumerableRelImplementor.java:103)

at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.
toBindable(EnumerableInterpretable.java:92)



Anything I'm doing wrong ?

Thanks,
Andrei.

Re: Releases

2018-05-28 Thread Sergey Nuyanzin

Hello

If it is possible I would like to be a volunteer here.

On Sun, May 27, 2018 at 3:06 AM, Julian Hyde  wrote:

> Thanks, Volodymyr. It would be great if you could be Calcite RM.
>
> Does anyone volunteer to be release manager for Avatica? It is an easier
> job than Calcite, and as Volodymyr
> says, a great way to gain experience.
>
> Does anyone have any time constraints for Calcite or Avatica releases?
>
> We’ve been releasing Calcite every 2-3 months, so if we released Calcite
> on June 19 that would be exactly 3 months. If we need to make an Avatica
> release before that, timing will be tight.
>
> Julian
>
>
> > On May 26, 2018, at 2:13 AM, Volodymyr Vysotskyi 
> wrote:
> >
> > Hello,
> >
> > I volunteer to be the release manager for Calcite if there are no
> > objections. It will be a great experience for me.
> >
> > Calcite 1.16 was released more than two months ago (March 19), but since
> it
> > depends on the Avatica release, Avatica should be released asap.
> >
> > I suppose it is needed two weeks for Avatica release, so the estimated
> > target date starting Calcite release process is the middle of the Jun.
> >
> > Kind regards,
> > Volodymyr Vysotskyi
> >
> >
> > сб, 26 трав. 2018 о 08:26 Julian Hyde  пише:
> >
> >> When should we release Calcite 1.17? When should we release Avatica
> 1.12?
> >> Anyone volunteer to be release manager?
> >>
> >> Because of https://issues.apache.org/jira/browse/CALCITE-2303 <
> >> https://issues.apache.org/jira/browse/CALCITE-2303>, we need to release
> >> Avatica before Calcite.
> >>
> >> I have logged https://issues.apache.org/jira/browse/CALCITE-2330 <
> >> https://issues.apache.org/jira/browse/CALCITE-2330> to track Avatica
> 1.12.
> >>
> >> Julian
> >>
> >>
>
>


-- 
Best regards,
Sergey

[jira] [Created] (CALCITE-2333) Stop releasing ZIPs

Re: Releases

using "fakes" in unit tests

[jira] [Created] (CALCITE-2332) It's wrong to simplify FLOOR(CEIL(x)) to FLOOR(x)

Re: RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?

Re: Releases

Re: RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?

[jira] [Created] (CALCITE-2331) invalid filter evaluation in adapters

skipping JDBC API (or ResultSet metadata / pagination)

Re: RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?

Re: Datetime_Plus and TimestampAdd operators

Re: Does Calcite hold all records output from a node before passing them to a higher node ?

Datetime_Plus and TimestampAdd operators

Does Calcite hold all records output from a node before passing them to a higher node ?

RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?

Re: Releases

16 matches

Site Navigation

Mail list logo

Footer information