Re: Avatica CI test failures

2018-11-20 Thread Francis Chuang
This is excellent! This also means that we do not have to manually update the dockerfile during a release. I think we have 2 options here: - Keep the dockerfile in dockerfile.template and during release, replace placeholders inside the file and replace the dockerfile file in the repository

Re: Avatica CI test failures

2018-11-20 Thread Vladimir Sitnikov
Francis, can you just generate the Dockerfile **during** the build? I guess you can generate the file with proper version, then "docker-check" would be obsolete. Am I missing something? Vladimir

[jira] [Created] (CALCITE-2689) ES Adapter. Grouping on date / number fields fails

2018-11-20 Thread Andrei Sereda (JIRA)
Andrei Sereda created CALCITE-2689: -- Summary: ES Adapter. Grouping on date / number fields fails Key: CALCITE-2689 URL: https://issues.apache.org/jira/browse/CALCITE-2689 Project: Calcite

Re: Avatica CI test failures

2018-11-20 Thread Francis Chuang
I think the original proposed approach of using `-DskipDockerCheck` is not possible to implement in maven. My original implementation is to implement profile activation using a property for the docker-check profile:         !skipDockerCheck       Unfortunately,

Re: Avatica CI test failures

2018-11-20 Thread Josh Elser
Thanks. The first red build I saw was after your change had landed. I may be wrong, but I think that's how we got here :) On 11/20/18 4:37 PM, Francis Chuang wrote: Oh dear! I was under the impression that the builds were breaking before the fix for CALCITE-2385 was committed, but I was wrong!

Re: Avatica CI test failures

2018-11-20 Thread Francis Chuang
Oh dear! I was under the impression that the builds were breaking before the fix for CALCITE-2385 was committed, but I was wrong! I'll try to get this fixed today. On 21/11/2018 3:57 am, Josh Elser wrote: Wait, back up a second? Francis -- did you break every other build just to try to fix

Re: Contribution of IN operator handling

2018-11-20 Thread Julian Hyde
PS Here’s why we should focus on the SQL-to-JDBC example: it is concrete. If get it right, I’m sure the other adapters will benefit. We need to be sure not to over-fit the internal representation to SQL semantics (e.g. SQL’s IN allows composite values, e.g. “(x, y) IN ((1, 2), (3, 4))”, has a

Re: Contribution of IN operator handling

2018-11-20 Thread Julian Hyde
Yes, the other adapters need to write their own converters, specific to their target language. DB languages tend to be divergent in how they represent “column in (constant1, …, constantN)”[1][2], so this makes sense. Of course they can borrow/share code with the JDBC adapter. Julian [1]

Re: Contribution of IN operator handling

2018-11-20 Thread Andrei Sereda
Hi Julian, Thanks for your reply. I have a question … where t is a table based on a JDBC data source … Do we agree on that goal? Does that mean that each of adapters (Elastic, Geode, Mongo etc.) have to implement their own version of converters (similar to RexToSqlConverter) ? If it is a flat

Re: Stream SQL Workload

2018-11-20 Thread Anton Kedin
Yes, Beam SQL implements few Nexmark queries (not everything is currently easily implementable in Beam SQL): https://beam.apache.org/documentation/sdks/java/testing/nexmark/#nexmark-dashboards . Code can be found here:

Re: Contribution of IN operator handling

2018-11-20 Thread Andrei Sereda
Convert SqlInOperator to In-Expression : https://issues.apache.org/jira/browse/CALCITE-2630 Related. full table scans and subQueryThreshold. https://lists.apache.org/thread.html/1a25c956262633f8ef0d224ed76400761f6797c494a21796579eb4f2@%3Cdev.calcite.apache.org%3E On Tue, Nov 20, 2018 at 12:08

Re: Stream SQL Workload

2018-11-20 Thread Julian Hyde
I believe that Beam and Flink use Nexmark http://datalab.cs.pdx.edu/niagara/pstream/nexmark.pdf https://issues.apache.org/jira/browse/BEAM-3147 Julian > On Nov 20, 2018, at 9:43 AM,

Re: Contribution of IN operator handling

2018-11-20 Thread Julian Hyde
I recall contributing to some other conversations about large IN lists over the past few months. Before we jump into a discussion, can you locate those threads? Also, if there is not a JIRA case, can you please create one? Julian > On Nov 20, 2018, at 8:23 AM, Mykola Zerniuk > wrote: > >

Re: Avatica CI test failures

2018-11-20 Thread Josh Elser
Wait, back up a second? Francis -- did you break every other build just to try to fix CALCITE-2385? We should not have had to be making changes to CI just to make this work... On 11/19/18 4:04 PM, Francis Chuang wrote: Thanks for organizing that, Michael! Kevin: Currently, all builds will

Re: Avatica CI test failures

2018-11-20 Thread Josh Elser
I think a detail that was dropped here is that this only affects a `deploy`, not a normal dev build. Re-reading the context on CALCITE-2385 reminded me about what exactly Francis was trying to do. This shouldn't being affecting the average person, just those deploying SNAPSHOTs by hand. There

Contribution of IN operator handling

2018-11-20 Thread Mykola Zerniuk
Dear Calcite Administrators, my name is Mykola, software engineer from Ukraine. I had an issue with Calcite IN operator handling. Here is my previous email to you:

[jira] [Created] (CALCITE-2688) Improve diagnosability when return type could not be inferred.

2018-11-20 Thread Zoltan Haindrich (JIRA)
Zoltan Haindrich created CALCITE-2688: - Summary: Improve diagnosability when return type could not be inferred. Key: CALCITE-2688 URL: https://issues.apache.org/jira/browse/CALCITE-2688 Project:

[jira] [Created] (CALCITE-2687) Is distinct from could lead to Exceptions in ReduceExpressionRule

2018-11-20 Thread Zoltan Haindrich (JIRA)
Zoltan Haindrich created CALCITE-2687: - Summary: Is distinct from could lead to Exceptions in ReduceExpressionRule Key: CALCITE-2687 URL: https://issues.apache.org/jira/browse/CALCITE-2687

[jira] [Created] (CALCITE-2686) Project fail to build

2018-11-20 Thread Brandon Jiang (JIRA)
Brandon Jiang created CALCITE-2686: -- Summary: Project fail to build Key: CALCITE-2686 URL: https://issues.apache.org/jira/browse/CALCITE-2686 Project: Calcite Issue Type: Bug Affects

Re: Automatic validation for commit messages

2018-11-20 Thread Francis Chuang
On 20/11/2018 6:24 pm, Julian Hyde wrote: Before we embark on this, are we all agreed that once this hook is in place, no one will say “Hey, why are you complaining about my commit message? It passed the validation.” I am +1 for this. Validation rules are always going to have holes and edge

Re: JoinPushThroughJoinRule and JoinAssociateRule in calcite

2018-11-20 Thread Anton Haidai
According to my testing, some join configurations are unreachable by default rules, here is a bug about it with a possible cause suggestion: https://issues.apache.org/jira/browse/CALCITE-2666 On Tue, Nov 20, 2018 at 8:07 AM Wang, Ken wrote: > Hi, Julian > > I’m reading Apache Calcite source