Re: Pull requests to review
A TimestampString has no timezone. It does not represent a moment in time. It represents a position of the hands on a clock. A Timestamp does not have a timezone either, but it represents a moment in time. (Internally represented by milliseconds since the UTC epoch.) Therefore, to go from a TimestampString to a Timestamp, or vice versa, you need a time zone. I’m not sure where that should come from in your code. Julian > On Nov 5, 2018, at 1:19 PM, ptr.bo...@gmail.com wrote: > > Julian, so.. is it correct to translate TimestampString to > java.sql.Timestamp on RexToLixTranslator.convert(...) level as in pull > request https://github.com/apache/calcite/pull/900/commits ? > > > I hope so, then I could revoke pull request > https://github.com/apache/calcite/pull/878 > > > On Mon, Nov 5, 2018 at 9:34 PM Julian Hyde wrote: > >> Yeah, TimestampString is for SqlNode and RexNode only. Not JDBC code. >> Likewise DateString, TimeString, NlsString. Sorry the doc didn’t make that >> clear. Although frankly it’s impractical to document all of the places >> something is NOT used. >> >>> On Nov 5, 2018, at 5:49 AM, ptr.bo...@gmail.com wrote: >>> >>> Vladimir, >>> >>> I thought that TimestampString IS the appropriate type :D (knowledge >> based >>> on Rexbuilder code, where TimestampString is being created as >> RexLiteral). >>> >>> My problem is that I don't see what is the scope of TimestampString, >>> DateString, etc in Calcite. Does it span to Rel/Rex tree? Or it should >> not? >>> This is why I've created two different patches :( >>> >>> Any help with the responsibility of TimestampString appreciated. Without >> it >>> - bug is still there and I could create lots of other mishit patches. >> >> > > -- > Piotr Bojko > http://about.me/ptr.bojko
Re: [DISCUSS] State of the project 2018
One more thing I forgot. There were proposals for a new Calcite logo a couple of months ago and some really cool ones were proposed. The discussion died down, but I hope it's still something we can move forward on. On 6/11/2018 8:27 AM, Francis Chuang wrote: Thanks for starting this discussion, Michael! The Calcite project and it's Avatica sub project has been doing really well in the past year under your stewardship. I have seen a lot of new contributors who are highly motivated and we also added new committers. In terms of what we are doing well, I feel that the community is something we've managed really well. Newcomers and people with questions generally have questions answered quickly and I have seen quite a few threads with in-depth discussion on new and interesting use-cases for Calcite. Our git repositories are also quite active (Avatica and Avatica-Go less so) and I think this is a testament to our community and the flatness of the learning curve for new contributors to contribute to the project. In terms of what we need to do better, I think the load on Julian is still quite high (although much better than the previous years). I also hope more contributors and committers can help us review PRs (you don't need to be a committer to review!). I also feel that we should maintain a release cadence for the project, otherwise we might end up in a position where there hasn't been a new release for a while (Avatica had a 10 month gap between 1.10.0 and 1.11.0). I know Julian has been asking for release managers for a release of Calcite + Avatica in late October and I think we should try to see if we can get a release out for both projects soon. There has also been some discussion on the list regarding the appropriate tone to use in the community. All I want to say regarding this is to be kind and understanding and try not to be too stressed out if a proposal is not perfect. Francis On 6/11/2018 1:21 AM, Michael Mior wrote: It's that time of year again where we consider where Calcite has gone over the past year and where it's going in the future. I've been happy to see our first academic publication on Calcite finally get out this year (thanks to Edmon, Dan, Jesús, and Julian!) It's also been really positive to see more committers join the project over the past year with many contributions in new areas. The quality of our tests has improved and the feature base continues to expand. There's a few big things that come to mind. Calcite now has some support for geometry and JSON functions and also new parsing capabilities via the babel and server modules. There are probably other important advancements that I missed. The project continues to have some challenges in keeping up with reviewing JIRA cases and PRs. Suggestions on streamlining this process are always welcome. Of course, many contributors to Calcite are doing this in their spare time. Maintaining the health of the community is another important goal. There have been a few challenges with friction among contributors but I think we're working towards resolving this. Lots going on with Avatica as well and I thank Francis has done a fantastic job of taking care of that aspect of the project. (Far better than I have done overall as PMC chair :) It's also the time to find a new chair for the coming year. If he's willing, I think Francis would be a great choice. To repeat questions from previous years: 1) What else are we doing well in the project? 2) What areas do we need to do better? Please take some time to share your thoughts! -- Michael Mior mm...@apache.org
Re: [DISCUSS] Towards Calcite 1.18
I think we should consider releasing new versions of Calcite and Avatica soon, especially with the Thanksgiving holiday in North America and Christmas + New Year later in December/January coming up. Am I correct to assume that we need to release Avatica 1.13.0 before Calcite 1.18.0? If so, I am able to be release manager for Avatica if no one else is interested. On 21/10/2018 6:13 PM, Michael Mior wrote: Thanks for continuing to push releases forward! Unfortunately I won't be able to volunteer to be release manager this time around, but I'll try to set aside some time to go through some PRs. On Sat, Oct 20, 2018, 02:21 Julian Hyde wrote: OK, it's now exactly 3 months since 1.17. I think it's time. I think we should aim for a first RC a week from today (Friday 26th October). Can we have a volunteer to be release manager? Also, there are lots of PRs to review and merge. Please help out with that task, committers. Julian On Thu, Sep 20, 2018 at 11:00 AM Julian Hyde wrote: We’ll be sure to get PRs into the release. If you like, you can make the JIRA case depend on those 3 cases - that will remind us. (Other contributors: You can do that also, but only if you have a PR that you believe is ready to submit.) On Sep 20, 2018, at 8:18 AM, Andrew Pilloud wrote: Beam has a few JIRAs we'd like to see make the next release (which will enable us to replace 11k lines of code with calls to Calcite). They all have open PRs. * https://issues.apache.org/jira/browse/CALCITE-2404 Accessing structured-types is not implemented by the runtime * https://issues.apache.org/jira/browse/CALCITE-2529 linq4j should promote integer to floating point when generating function calls * https://issues.apache.org/jira/browse/CALCITE-2571 TRIM does not match the behavior of most SQL implementations Andrew On Wed, Sep 19, 2018 at 6:38 PM Kevin Risden wrote: 9/25 should be the GA release of JDK 11 according to [1]. Would be good to ensure that the next release is compatible with it. We should be in good shape but would be a good release note too. 1. http://openjdk.java.net/projects/jdk/11/ Kevin Risden On Wed, Sep 19, 2018, 20:56 Francis Chuang wrote: For avatica, there are currently 6 open PRs. A few of them appear to be blocked due to incompatibilities with JDK or something else. Perhaps the situation has now changed and we can get those in for 1.13. On 20/09/2018 9:26 AM, Julian Hyde wrote: I have logged * https://issues.apache.org/jira/browse/CALCITE-2576 < https://issues.apache.org/jira/browse/CALCITE-2576> Release Avatica 1.13 * https://issues.apache.org/jira/browse/CALCITE-2575 < https://issues.apache.org/jira/browse/CALCITE-2575> Release Calcite 1.18.0 On Sep 19, 2018, at 4:19 PM, Julian Hyde wrote: Calcite 1.17 was released on 20th July (2 months ago) and since then there have been almost 100 commits. Does anyone have constraints on when 1.18 should be released? (My opinion: we should aim for 3 months after 1.17, and therefore we will need an RC before 10th October.) Does anyone have any JIRA cases that they would like to get into 1.18 but they have not yet finished? Who would like to be release manager? Do we need a new Avatica release before we make this Calcite release? https://issues.apache.org/jira/browse/CALCITE-2467 < https://issues.apache.org/jira/browse/CALCITE-2467> describes one reason to do so: jetty versions. Also, it would be nice if both projects used the new parent POM (see https://issues.apache.org/jira/browse/CALCITE-2486 < https://issues.apache.org/jira/browse/CALCITE-2486>) but it’s not essential. Julian
Re: [DISCUSS] Towards Calcite 1.18
I think we should consider releasing new versions of Calcite and Avatica soon, especially with the Thanksgiving holiday in North America and Christmas + New Year later in December/January coming up. Am I correct to assume that we need to release Avatica 1.13.0 before Calcite 1.17.0? If so, I am able to be release manager for Avatica if no one else is interested. On 21/10/2018 6:13 PM, Michael Mior wrote: Thanks for continuing to push releases forward! Unfortunately I won't be able to volunteer to be release manager this time around, but I'll try to set aside some time to go through some PRs. On Sat, Oct 20, 2018, 02:21 Julian Hyde wrote: OK, it's now exactly 3 months since 1.17. I think it's time. I think we should aim for a first RC a week from today (Friday 26th October). Can we have a volunteer to be release manager? Also, there are lots of PRs to review and merge. Please help out with that task, committers. Julian On Thu, Sep 20, 2018 at 11:00 AM Julian Hyde wrote: We’ll be sure to get PRs into the release. If you like, you can make the JIRA case depend on those 3 cases - that will remind us. (Other contributors: You can do that also, but only if you have a PR that you believe is ready to submit.) On Sep 20, 2018, at 8:18 AM, Andrew Pilloud wrote: Beam has a few JIRAs we'd like to see make the next release (which will enable us to replace 11k lines of code with calls to Calcite). They all have open PRs. * https://issues.apache.org/jira/browse/CALCITE-2404 Accessing structured-types is not implemented by the runtime * https://issues.apache.org/jira/browse/CALCITE-2529 linq4j should promote integer to floating point when generating function calls * https://issues.apache.org/jira/browse/CALCITE-2571 TRIM does not match the behavior of most SQL implementations Andrew On Wed, Sep 19, 2018 at 6:38 PM Kevin Risden wrote: 9/25 should be the GA release of JDK 11 according to [1]. Would be good to ensure that the next release is compatible with it. We should be in good shape but would be a good release note too. 1. http://openjdk.java.net/projects/jdk/11/ Kevin Risden On Wed, Sep 19, 2018, 20:56 Francis Chuang wrote: For avatica, there are currently 6 open PRs. A few of them appear to be blocked due to incompatibilities with JDK or something else. Perhaps the situation has now changed and we can get those in for 1.13. On 20/09/2018 9:26 AM, Julian Hyde wrote: I have logged * https://issues.apache.org/jira/browse/CALCITE-2576 < https://issues.apache.org/jira/browse/CALCITE-2576> Release Avatica 1.13 * https://issues.apache.org/jira/browse/CALCITE-2575 < https://issues.apache.org/jira/browse/CALCITE-2575> Release Calcite 1.18.0 On Sep 19, 2018, at 4:19 PM, Julian Hyde wrote: Calcite 1.17 was released on 20th July (2 months ago) and since then there have been almost 100 commits. Does anyone have constraints on when 1.18 should be released? (My opinion: we should aim for 3 months after 1.17, and therefore we will need an RC before 10th October.) Does anyone have any JIRA cases that they would like to get into 1.18 but they have not yet finished? Who would like to be release manager? Do we need a new Avatica release before we make this Calcite release? https://issues.apache.org/jira/browse/CALCITE-2467 < https://issues.apache.org/jira/browse/CALCITE-2467> describes one reason to do so: jetty versions. Also, it would be nice if both projects used the new parent POM (see https://issues.apache.org/jira/browse/CALCITE-2486 < https://issues.apache.org/jira/browse/CALCITE-2486>) but it’s not essential. Julian
Re: [DISCUSS] State of the project 2018
Thanks for starting this discussion, Michael! The Calcite project and it's Avatica sub project has been doing really well in the past year under your stewardship. I have seen a lot of new contributors who are highly motivated and we also added new committers. In terms of what we are doing well, I feel that the community is something we've managed really well. Newcomers and people with questions generally have questions answered quickly and I have seen quite a few threads with in-depth discussion on new and interesting use-cases for Calcite. Our git repositories are also quite active (Avatica and Avatica-Go less so) and I think this is a testament to our community and the flatness of the learning curve for new contributors to contribute to the project. In terms of what we need to do better, I think the load on Julian is still quite high (although much better than the previous years). I also hope more contributors and committers can help us review PRs (you don't need to be a committer to review!). I also feel that we should maintain a release cadence for the project, otherwise we might end up in a position where there hasn't been a new release for a while (Avatica had a 10 month gap between 1.10.0 and 1.11.0). I know Julian has been asking for release managers for a release of Calcite + Avatica in late October and I think we should try to see if we can get a release out for both projects soon. There has also been some discussion on the list regarding the appropriate tone to use in the community. All I want to say regarding this is to be kind and understanding and try not to be too stressed out if a proposal is not perfect. Francis On 6/11/2018 1:21 AM, Michael Mior wrote: It's that time of year again where we consider where Calcite has gone over the past year and where it's going in the future. I've been happy to see our first academic publication on Calcite finally get out this year (thanks to Edmon, Dan, Jesús, and Julian!) It's also been really positive to see more committers join the project over the past year with many contributions in new areas. The quality of our tests has improved and the feature base continues to expand. There's a few big things that come to mind. Calcite now has some support for geometry and JSON functions and also new parsing capabilities via the babel and server modules. There are probably other important advancements that I missed. The project continues to have some challenges in keeping up with reviewing JIRA cases and PRs. Suggestions on streamlining this process are always welcome. Of course, many contributors to Calcite are doing this in their spare time. Maintaining the health of the community is another important goal. There have been a few challenges with friction among contributors but I think we're working towards resolving this. Lots going on with Avatica as well and I thank Francis has done a fantastic job of taking care of that aspect of the project. (Far better than I have done overall as PMC chair :) It's also the time to find a new chair for the coming year. If he's willing, I think Francis would be a great choice. To repeat questions from previous years: 1) What else are we doing well in the project? 2) What areas do we need to do better? Please take some time to share your thoughts! -- Michael Mior mm...@apache.org
Re: Pull requests to review
Julian, so.. is it correct to translate TimestampString to java.sql.Timestamp on RexToLixTranslator.convert(...) level as in pull request https://github.com/apache/calcite/pull/900/commits ? I hope so, then I could revoke pull request https://github.com/apache/calcite/pull/878 On Mon, Nov 5, 2018 at 9:34 PM Julian Hyde wrote: > Yeah, TimestampString is for SqlNode and RexNode only. Not JDBC code. > Likewise DateString, TimeString, NlsString. Sorry the doc didn’t make that > clear. Although frankly it’s impractical to document all of the places > something is NOT used. > > > On Nov 5, 2018, at 5:49 AM, ptr.bo...@gmail.com wrote: > > > > Vladimir, > > > > I thought that TimestampString IS the appropriate type :D (knowledge > based > > on Rexbuilder code, where TimestampString is being created as > RexLiteral). > > > > My problem is that I don't see what is the scope of TimestampString, > > DateString, etc in Calcite. Does it span to Rel/Rex tree? Or it should > not? > > This is why I've created two different patches :( > > > > Any help with the responsibility of TimestampString appreciated. Without > it > > - bug is still there and I could create lots of other mishit patches. > > -- Piotr Bojko http://about.me/ptr.bojko
Re: Pull requests to review
Yeah, TimestampString is for SqlNode and RexNode only. Not JDBC code. Likewise DateString, TimeString, NlsString. Sorry the doc didn’t make that clear. Although frankly it’s impractical to document all of the places something is NOT used. > On Nov 5, 2018, at 5:49 AM, ptr.bo...@gmail.com wrote: > > Vladimir, > > I thought that TimestampString IS the appropriate type :D (knowledge based > on Rexbuilder code, where TimestampString is being created as RexLiteral). > > My problem is that I don't see what is the scope of TimestampString, > DateString, etc in Calcite. Does it span to Rel/Rex tree? Or it should not? > This is why I've created two different patches :( > > Any help with the responsibility of TimestampString appreciated. Without it > - bug is still there and I could create lots of other mishit patches.
[DISCUSS] State of the project 2018
It's that time of year again where we consider where Calcite has gone over the past year and where it's going in the future. I've been happy to see our first academic publication on Calcite finally get out this year (thanks to Edmon, Dan, Jesús, and Julian!) It's also been really positive to see more committers join the project over the past year with many contributions in new areas. The quality of our tests has improved and the feature base continues to expand. There's a few big things that come to mind. Calcite now has some support for geometry and JSON functions and also new parsing capabilities via the babel and server modules. There are probably other important advancements that I missed. The project continues to have some challenges in keeping up with reviewing JIRA cases and PRs. Suggestions on streamlining this process are always welcome. Of course, many contributors to Calcite are doing this in their spare time. Maintaining the health of the community is another important goal. There have been a few challenges with friction among contributors but I think we're working towards resolving this. Lots going on with Avatica as well and I thank Francis has done a fantastic job of taking care of that aspect of the project. (Far better than I have done overall as PMC chair :) It's also the time to find a new chair for the coming year. If he's willing, I think Francis would be a great choice. To repeat questions from previous years: 1) What else are we doing well in the project? 2) What areas do we need to do better? Please take some time to share your thoughts! -- Michael Mior mm...@apache.org
Re: Pull requests to review
Vladimir, I thought that TimestampString IS the appropriate type :D (knowledge based on Rexbuilder code, where TimestampString is being created as RexLiteral). My problem is that I don't see what is the scope of TimestampString, DateString, etc in Calcite. Does it span to Rel/Rex tree? Or it should not? This is why I've created two different patches :( Any help with the responsibility of TimestampString appreciated. Without it - bug is still there and I could create lots of other mishit patches.
Re: Pull requests to review
Piotr, Have you tried to use SqlToRelConverter to convert argument from SqlNode format to the appropriate one? Vladimir
Pull requests to review
Hello, Anyone willing to review my pull requests? https://github.com/apache/calcite/pull/878 - this is already patched after reviews. It fixes CALCITE-2609. https://github.com/apache/calcite/pull/878 https://github.com/apache/calcite/pull/900 Those pulls are alternative fixes to CALCITE-2641 . See discussion in the JIRA issue. I've created alternative approaches because here, maybe one of them will be merged :) Thank you in advance for help Regards. -- Piotr Bojko http://about.me/ptr.bojko
Re: Looking for help on query optimization statistics
Hi Tian, To inject statistics, you need to implement your own RelMetadataProvider and register it into your runtime environment. I'm not able to find a valid example in Calcite, but there is a good example you can find in Drill https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java#L405 . On Mon, Nov 5, 2018 at 11:54 AM Tian Ye wrote: > Hi, > > I'm working on comparing the query plan generated by Calcite with that > from a DBMS I'm developing. > > In particular, I can generate some TPC-H data and a SQL query, and I want > to let Calcite generate as good query plan as it can. So far, I haven't > passed any statistics to Calcite, so the query plan it generates is just a > baseline. > > Now I wonder how I could pass statistics to Calcite. What I don't > understand is that since Calcite does not load the data until query plan is > actually executed, how could I pass statistics to it? > I fail to find any document talking about how to do that in detail. The > last paragraphs of http://calcite.apache.org/docs/adapter.html do > mention it but are not so clear. > > > > > Would you please help me? > > Thanks a lot. > > > > > > Tian Ye > y...@pku.edu.cn
