Re: [VOTE] Release Apache Calcite 1.37.0 (release candidate 4)
+1 (binding) checked signatures and hash, compiled and ran tests. Thanks Sergey! -- Michael Mior mm...@apache.org On Mon, Apr 29, 2024 at 10:56 AM Sergey Nuyanzin wrote: > Hi all, > > The issue CALCITE-6390 with failing of Arrow Adapter tests on Windows while > building from source is fixed. > Thanks a lot to Caican Can for highlighting the issue > Stamatis for quick PR > and Ruben and Alessandro for the review! > > I have created a build for Apache Calcite 1.37.0, release > candidate 4. > > Thanks to everyone who has contributed to this release. > > You can read the release notes here: > > https://github.com/apache/calcite/blob/calcite-1.37.0-rc4/site/_docs/history.md > > The commit to be voted upon: > > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=044db3d72ee53c1a82ce68b0c2a9b4f0bed81f18 > > Its hash is 044db3d72ee53c1a82ce68b0c2a9b4f0bed81f18 > > Tag: > https://github.com/apache/calcite/tree/calcite-1.37.0-rc4 > > The artifacts to be voted on are located here: > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.37.0-rc4 > (revision 68857) > > The hashes of the artifacts are as follows: > > a6ae7ea1ebbab3e5b723122c9517412d74b6bccba9fb41d360f7c2262ed7f26e5a4cc861a711d4813d27d72a0ffd086ae3a9175abc3c874b079137c84d13f83a > *apache-calcite-1.37.0-src.tar.gz > > A staged Maven repository is available for review at: > > https://repository.apache.org/content/repositories/orgapachecalcite-1230/org/apache/calcite/ > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/snuyanzin.asc > https://www.apache.org/dist/calcite/KEYS > > To create the jars and test Apache Calcite: "gradle build" > (requires an appropriate Gradle/JDK installation) > > Please vote on releasing this package as Apache Calcite 1.37.0. > > The vote is open for the next 72 hours and passes if a majority of at > least three +1 PMC votes are cast. > > [ ] +1 Release this package as Apache Calcite 1.37.0 > [ ] 0 I don't feel strongly about it, but I'm okay with the release > [ ] -1 Do not release this package because... > > Here is my vote: > > +1 (binding) > > > -- > Best regards, > Sergey >
Re: Contributor Rights for Calcite JIRA
Done! Thanks for contributing to Calcite. -- Michael Mior mm...@apache.org On Wed, Apr 10, 2024 at 11:27 AM Norman Jordan wrote: > Hello, > > Can I get contributor rights for the Calcite JIRA? I would like to be able > to create tickets, assign tickets to myself and edit tickets. > > My JIRA username is "njordan". I have already worked on CALCITE-6315. > > Thank you. >
Re: [VOTE] Release Apache Calcite Avatica 1.25.0 (release candidate 0)
Francis, +1 (checked signature and hash, ran tests) I can't recall if this is expected, but the gradle wrapper is missing from the release. This means that running tests via docker compose doesn't work. -- Michael Mior mm...@apache.org On Mon, Apr 1, 2024 at 11:40 PM Francis Chuang wrote: > Hi all, > > I have created a build for Apache Calcite Avatica 1.25.0, release > candidate 0. > > Thanks to everyone who has contributed to this release. > > You can read the release notes here: > > https://github.com/apache/calcite-avatica/blob/avatica-1.25.0-rc0/site/_docs/history.md > > The commit to be voted upon: > > https://gitbox.apache.org/repos/asf?p=calcite-avatica.git;a=commit;h=62b0fdd3f4e0173d8492eb0e203056f0938dd6f5 > > Its hash is 62b0fdd3f4e0173d8492eb0e203056f0938dd6f5 > > Tag: > https://github.com/apache/calcite-avatica/tree/avatica-1.25.0-rc0 > > The artifacts to be voted on are located here: > > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-avatica-1.25.0-rc0 > > RAT report: > > https://gitbox.apache.org/repos/asf/calcite-avatica-site-preview.git/avatica/rat/rat-report.txt > > Site preview is here: > > https://gitbox.apache.org/repos/asf/calcite-avatica-site-preview.git/avatica/ > > JavaDoc API preview is here: > > https://gitbox.apache.org/repos/asf/calcite-avatica-site-preview.git/avatica/api > > The hashes of the artifacts are as follows: > > 0daa1da33ca0712cb3a4918260081e707c336e06eaacee87962a8ff914f377e8ad21d5fd24490612dbca7d56b83e9d062c39b5cd51701564cf8cba7840ed4806 > *apache-calcite-avatica-1.25.0-src.tar.gz > > A staged Maven repository is available for review at: > > https://repository.apache.org/content/repositories/orgapachecalcite-1223/org/apache/calcite/ > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/francischuang.asc > https://www.apache.org/dist/calcite/KEYS > > To create the jars and test Apache Calcite Avatica: "gradle build > -Prelease -PskipSign". > > If you do not have a Java/Gradle environment available, you can run the > tests using docker. To do so, install docker and docker-compose, then > run "docker compose run test" from the root of the directory. > > Please vote on releasing this package as Apache Calcite Avatica 1.25.0. > > The vote is open for the next 72 hours and passes if a majority of at > least three +1 PMC votes are cast. > > [ ] +1 Release this package as Apache Calcite Avatica 1.25.0 > [ ] 0 I don't feel strongly about it, but I'm okay with the release > [ ] -1 Do not release this package because... > > > Here is my vote: > > +1 (binding) > > Francis >
Re: Draft: board report for 2024 Q1
+1 Thanks Benchao! -- Michael Mior mm...@apache.org On Wed, Apr 3, 2024 at 6:11 AM Benchao Li wrote: > Hello, > > Below you can find a draft of this quarter's board report. I plan to > submit the final version next Tuesday (Apr 9, 2024). > > Please let me know if you have any additions or corrections. > > -- > > Best, > Benchao Li > > > ## Description: > Apache Calcite is a highly customizable framework for parsing and planning > queries on data in a wide variety of formats. It allows database-like > access, > and in particular a SQL interface and advanced query optimization, for data > not residing in a traditional database. > > Avatica is a sub-project within Calcite and provides a framework for > building > local and remote JDBC and ODBC database drivers. Avatica has an independent > release schedule and its own repository. > > ## Issues: > There are no issues requiring board attention. > > ## Membership Data: > Apache Calcite was founded 2015-10-22 (8 years ago) > There are currently 74 committers and 28 PMC members in this project. > The Committer-to-PMC ratio is roughly 5:2. > > Community changes, past quarter: > - Sergey Nuyanzin was added to the PMC on 2024-03-05 > - No new committers. Last addition was Hongyu Guo on 2023-11-03. > > It's worth mentioning that a few people in Calcite PMC have been > invited as ASF members this year: > - Benchao Li > - Francis Chuang > - Ruben Quesada Lopez > - Sergey Nuyanzin > > ## Project Activity: > There is no release rolled out in Q1 yet, but we have planned to > release avatica 1.25.0 and calcite 1.37.0 lately. For now, avatica > 1.25.0 RC0 is in voting stage, and calcite 1.37.0 will be kicked off > just after avatica 1.25.0 is released since we have a few co-related > issues for these two versions. > > ## Community Health: > The community maintains a super healthy status, due to the new > invitation of PMC member, previously it's healthy. > > Most of the statistics are steady compared to last quarter, except a > slight decrease for the last three weeks. I assume the reason would be > that we are in the process of calcite 1.37.0 release.It's a little bit > longer than usual, the reason is that we have a few co-related issues > for calcite and avatica, and we need to release avatica before > calcite. > > The number of non-committer (contributor) commits per month: > +--+---+-+ > | year | month | contributor_commits | > +--+---+-+ > | 2024 | 1 | 20 | > | 2024 | 2 | 20 | > | 2024 | 3 | 14 | > +--+---+-+ > > The number of active reviewers per month: > +--+---+--+ > | year | month | active_reviewers | > +--+---+--+ > | 2024 | 1 |6 | > | 2024 | 2 |7 | > | 2024 | 3 |6 | > +--+---+--+ > > Top reviewers in the last 3 months: > +---+-+ > | committer | reviews | > +---+-+ > | hongyu guo| 15 | > | Mihai Budiu | 13 | > | Tanner Clary | 10 | > +---+-+ >
Re: [ANNOUNCE] Sergey Nuyanzin joins Calcite PMC
Congratulations and welcome Sergey! -- Michael Mior mm...@apache.org On Tue, Mar 5, 2024 at 7:14 AM Benchao Li wrote: > I am pleased to announce that Sergey has accepted an invitation to > join the Calcite PMC. Sergey has been a consistent and helpful figure > in the Calcite community for which we are very grateful. We look > forward to the continued contributions and support. > > Please join me in congratulating Sergey! > > Benchao (on behalf of the Calcite PMC) >
Re: Create adapter for Apache Arrow
Thanks for picking this up. One of my students and myself wrote a lot of the original code. For a while this was blocked on necessary changes to Apache Arrow, but this has since been resolved. I'd love to see this get merged. It would help to rebase into a single commit. I'd also appreciate one more set of eyes before this gets wrapped up. Thanks again Hongyu! -- Michael Mior mm...@apache.org On Fri, Feb 2, 2024 at 6:40 AM Hongyu Guo wrote: > Hi devs, > > Recently I found a [CALCITE-2040] "Create adapter for Apache Arrow"[1] that > is almost finished in PR#2810[2], but no one continues to push it forward, > so I want to take it. I made some changes to the existing PR, including > - Upgrade arrow to 15.0 > - Prohibit arrow adapter CI on Windows (for arrow system compatibility[3]) > - Add JAVA_OPTIONS for JDK19 CI --add-opens=java.base/java.nio=ALL-UNNAMED > (for arrow java compatibility[4]) > > This is my PR link: https://github.com/apache/calcite/pull/3666 > > I haven't made any feature changes because I'm not familiar enough with > Apache Arrow. I found that Jenkins CI will fail due to the JAVA_OPTIONS, > and I will fix it later. > > Best, > Hongyu > > [1] https://issues.apache.org/jira/browse/CALCITE-2040 > [2] https://github.com/apache/calcite/pull/2810/commits > [3] https://arrow.apache.org/docs/java/install.html#system-compatibility > [4] https://arrow.apache.org/docs/java/install.html#java-compatibility >
Re: [DISCUSS] Does web site use Google analytics?
Worth noting that using Google Analytics is possible in compliance with the
GDPR if configured appropriately. That said, although we could fix any
configuration issues or find an alternative, I think it would also be fine
to simply remove any analytics from the site altogether.
--
Michael Mior
mm...@apache.org
On Wed, Jan 17, 2024 at 1:05 PM Julian Hyde wrote:
> Does Calcite’s web site use Google analytics?
>
> If so, we should fix it. Google analytics collects information that is not
> allowed by GDPR. Infra is considering blocking Google analytics for all *.
> apache.org <http://apache.org/> sites, and if they do that it may make
> our site unusable.
>
> Whimsy has a check on all project sites [1] and it finds one ‘external
> resource’:
>
> > Found 1 external resources: {"fonts.googleapis.com"=>1}
>
> Julian
>
> [1] https://whimsy.apache.org/site/
>
>
Re: squashing commit already in upstream
Since this would require a force push, I would suggest fetching from the remote first and then making sure you rebase if necessary while squashing then use --force-with-lease instead of --force to make sure any changes that may have been pushed are incorporated. If you do end up seeing that someone else has pushed before you squash, I don't think that means a force push is out of the question, but in that case, it would probably be good to send a quick note to the dev list to make sure everyone is aware. -- Michael Mior mm...@apache.org On Sun, Dec 10, 2023 at 4:21 PM Mihai Budiu wrote: > Hello, > > By mistake I merged a PR which contained two commits instead of the > customary one. What is the policy (and method) for fixing this for the > Calcite repository? > > > https://github.com/apache/calcite/commit/5b441c639de1ec428891061b931bd69d9996efe4 > > Should have been squashed into > > > https://github.com/apache/calcite/commit/1b36f07ce98f0e6d620b5ea1e1c773b376c3e81e > > Sorry for the complications, I will double check next time. > > Thank you, > Mihai >
Re: Access to Calcite plan execution
Keep in mind that for general queries, this is a very non-trivial problem and an active area of research. However, doing this for the case of a filter applied to a single table should be much more straightforward. One way to do this would be to create your own subclass of EnumerableFilter with its own convention and then choose your rules carefully (along with a copy of EnumerableFilterRule for your new convention). You would then implement whatever logging logic you need inside EnumerableFilter. It's still a nontrivial amount of code, but definitely doable. Others might have suggestions on an easier way to implement this. -- Michael Mior mm...@apache.org On Tue, Sep 26, 2023 at 2:07 PM Luis Brassara wrote: > Hi, all. > > I'm using Apache Calcite with SQL. > > I'm trying to execute an SQL query like > > SELECT * FROM table WHERE some_field = 1 AND other_field = 5) > > where table is: > >some_field | other field > > row 1: 1 | 2 > row 2: 1 | 3 > > > This query will return empty results. > > I want to tell the user why the query returned empty results, row by row. > Then, I want to be able to access the query plan and transverse it to the > node that produced the empty results (other_field = 5) and I want to have > in context the current value of other_field (2) so I can print messages > like: > > Row 1 failed because `other_field` was `2` and `5` was expected > Row 2 failed because `other_field` was `3` and `5` was expected > > I'm not sure which Calcite classes should I use to achieve that, any advice > / example? > > Thanks in advance! >
Re: A link in the code has expired
I updated the Javadoc link. However, I also noticed there are a bunch of references to the Eigenbase JIRA in Bug.java that are also broken with no obvious alternative. -- Michael Mior mm...@apache.org On Tue, Aug 22, 2023 at 4:33 PM Julian Hyde wrote: > Yes, when eigenbase.org <http://eigenbase.org/> shut down I copied the > contents of the wiki onto my server, http://www.hydromatic.net/wiki. > http://www.hydromatic.net/wiki/Eigenbase_Introduction is a good place to > start reading. > > Maybe there’s a better place to store those files for posterity. I have > just checked the files into git; see my fork of LucidDB: > https://github.com/julianhyde/luciddb/commits/master. I’m trying to > figure out how to make those files appear via GitHub pages. > > Julian > > > > On Aug 16, 2023, at 6:54 AM, Michael Mior wrote: > > > > Thanks for pointing this out! It looks like Julian might have a mirror of > > that page at the link below that could be swapped out. Unfortunately, > with > > the original page unavailable, I can't confirm. > > > > http://www.hydromatic.net/wiki/RelationalExpressionMetadata > > > > -- > > Michael Mior > > mm...@apache.org > > > > > > On Wed, Aug 16, 2023 at 9:30 AM Hongyu Guo > wrote: > > > >> Hello, > >> > >> I found a link in RelMetadataProvider.java has expired. > >> > >> The link is http://wiki.eigenbase.org/RelationalExpressionMetadata > >> > >> Found in > >> > >> > https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/metadata/RelMetadataProvider.java > >> > >> > >> > >> Best, > >> > >> Hongyu Guo > >> > >
Re: A link in the code has expired
Thanks for pointing this out! It looks like Julian might have a mirror of that page at the link below that could be swapped out. Unfortunately, with the original page unavailable, I can't confirm. http://www.hydromatic.net/wiki/RelationalExpressionMetadata -- Michael Mior mm...@apache.org On Wed, Aug 16, 2023 at 9:30 AM Hongyu Guo wrote: > Hello, > > I found a link in RelMetadataProvider.java has expired. > > The link is http://wiki.eigenbase.org/RelationalExpressionMetadata > > Found in > > https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/metadata/RelMetadataProvider.java > > > > Best, > > Hongyu Guo >
Re: Force push to calcite main
I would also be +1 for keeping it in place. It certainly doesn't solve all the problems, but using --force-with-lease instead of --force is generally a good idea. It just makes sure no one else has pushed anything since your last update. If they have, it will block the force push. At that point, if I still wanted to force push, I would rebase to incorporate their changes if possible, and make sure I notify whoever pushed. -- Michael Mior mm...@apache.org On Fri, Jul 7, 2023 at 2:19 PM Julian Hyde wrote: > > I prefer keeping force push in place > > +1. I use force push about a dozen times a year and hopefully no one > notices. It helps the commit history clean. > > I know that force pushes can confuse CI systems and make things confusing > to other users, so I try to be judicious. I trust other committers to do > the same. (I suspect that this was a learning experience for Tanner.) > > Julian > > > > On Jul 7, 2023, at 4:12 AM, Stamatis Zampetakis > wrote: > > > > @Tanner No worries we all did this at some point in time, thanks a lot > > for following up! > > > > @Stanilovsky: I prefer keeping force push in place and avoiding messy > > reverts that are usually necessary in various situations where we make > > a mistake. All commits are archived so there is nothing that we can't > > fix (I think). > > > > On Thu, Jul 6, 2023 at 3:45 PM Tanner Clary > > wrote: > >> > >> Hello, > >> > >> This was my mistake, my apologies. I will update the hashes. Sorry for > any > >> inconvenience. > >> > >> Tanner > >> > >> On Thu, Jul 6, 2023 at 3:34 AM stanilovsky evgeny < > >> estanilovs...@gridgain.com> wrote: > >> > >>> I already told that community need to vote for prohibit force push. > >>> > >>>> Hello, > >>>> > >>>> It appears that there was a force push to main yesterday [1] rewriting > >>>> the history for a bunch of commits. I don't know if it was intentional > >>>> or not but it seems that now resolved JIRAs (after CALCITE-5810 I > >>>> think) are pointing to non-existent commits. > >>>> > >>>> Can someone please update the JIRA tickets with the correct commit > >>>> hashes and also ensure that we didn't lose anything after the rebase? > >>>> > >>>> Best, > >>>> Stamatis > >>>> > >>>> [1] https://lists.apache.org/thread/7jjnbkkh9tv49sjcc5kg2tm7c54tj861 > >>> > >
Re: Virtual key signing party
Signed key attached. -- Michael Mior mm...@apache.org On Tue, Jun 27, 2023 at 3:32 AM Ruben Q L wrote: > Hello, > > I'll try to attend too. > Best, > Ruben > > > pub rsa4096 2020-09-26 [SC] > > 464F A4A3 D7E4 2112 E4CB 68F2 DF92 5FEB A08B 032B > > uid[ absoluta ] Ruben Quesada Lopez > > sub rsa4096 2020-09-26 [E] > > > > > On Tue, Jun 27, 2023 at 8:29 AM Stamatis Zampetakis > wrote: > > > Great let's do one on June 29, 2023, 14:30 UTC+2 [1, 2]. For those who > > can't make it in this slot feel free to propose additional ones. > > > > Best, > > Stamatis > > > > [1] https://s.apache.org/cn5xq > > [2] https://meet.google.com/vwx-mxxz-ibk > > > > > > On Tue, Jun 27, 2023 at 4:22 AM xiong duan wrote: > > > > > > Hi Stamatis, > > > Thanks for organizing this party. I am in UTC+8. How about June 29, > > > 2023, 20:30 (UTC+8) So the UTC+2 time should be June 29, 2023, 14:30? > > > Here is my fingerprint: > > > > > > pub rsa4096 2023-06-27 [SC] > > > > > > EC78 877D 98B8 CDD4 9613 3DF2 B989 C696 D102 A552 > > > > > > uid [ultimate] Xiong Duan (CODE SIGNING KEY) < > xi...@apache.org > > > > > > > > > sub rsa4096 2023-06-27 [E] > > > > > > Michael Mior 于2023年6月27日周二 02:19写道： > > > > > > > I'll attend if possible. My key needs to be updated. (Not > compromised, > > but > > > > I lost access to my previous signing key.) > > > > > > > > pub rsa4096 2022-06-17 [SC] [expires: 2038-06-13] > > > > EAC5 89C4 44F4 68FE 7E11 3D2D D8D7 2C13 CF2D 8DDB > > > > uid [ultimate] Michael Mior > > > > uid [ultimate] Michael Mior > > > > uid [ultimate] Michael Mior > > > > sub rsa4096 2022-06-17 [E] [expires: 2038-06-13] > > > > > > > > -- > > > > Michael Mior > > > > mm...@apache.org > > > > > > > > > > > > On Mon, Jun 26, 2023 at 3:27 AM Stamatis Zampetakis < > zabe...@gmail.com > > > > > > > wrote: > > > > > > > > > Hello, > > > > > > > > > > The last virtual key signing party [1] was about a year ago [2]. In > > > > > the upcoming releases, we have some people that are serving as > > release > > > > > managers for the first time, thus it may be a good time to hold > > > > > another party. > > > > > > > > > > Since Duan Xiong is the RM for the next release, I would suggest > > > > > holding a meeting in a timezone that is convenient for him. > > > > > > > > > > @Duan: Can you please propose a time slot that works for you? > > > > > > > > > > I am in UTC+2 and will try to attend if the slot doesn't fall in > the > > > > > middle of the night (23:00 - 06:00). > > > > > > > > > > For those that would like to attend, please reply to this thread > with > > > > > your public > > > > > keys' fingerprint (gpg --fingerprint) before the meeting. > > > > > > > > > > pub rsa4096 2019-03-15 [SC] [expires: 2027-03-15] > > > > > 0474 9577 FD93 4674 B9CD 45C5 D77C 3383 F192 7570 > > > > > uid [ultimate] Stamatis Zampetakis > > > > > uid [ultimate] Stamatis Zampetakis > > > > > sub rsa4096 2019-03-15 [E] [expires: 2027-03-15] > > > > > > > > > > To verify your identity, please bring with you a government issued > ID > > > > > (preferably passport). > > > > > > > > > > As a reminder of the procedure, have a look at the notes [1] taken > by > > > > > Francis > > > > > during a previous party! > > > > > > > > > > Anybody can participate (not need to be a committer to the > project). > > > > > > > > > > Best, > > > > > Stamatis > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > http://www.cryptnet.net/fdp/crypto/keysigning_party/en/keysigning_party.html > > > > > [2] > https://lists.apache.org/thread/4lvch7sfwszbylpzjt11jc9tokm3d0l4 > > > > > [3] https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84 > > > > > > > > > > > >
Re: Virtual key signing party
Signed key attached. -- Michael Mior mm...@apache.org On Mon, Jun 26, 2023 at 10:22 PM xiong duan wrote: > Hi Stamatis, > Thanks for organizing this party. I am in UTC+8. How about June 29, > 2023, 20:30 (UTC+8) So the UTC+2 time should be June 29, 2023, 14:30? > Here is my fingerprint: > > pub rsa4096 2023-06-27 [SC] > > EC78 877D 98B8 CDD4 9613 3DF2 B989 C696 D102 A552 > > uid [ultimate] Xiong Duan (CODE SIGNING KEY) > > sub rsa4096 2023-06-27 [E] > > Michael Mior 于2023年6月27日周二 02:19写道： > > > I'll attend if possible. My key needs to be updated. (Not compromised, > but > > I lost access to my previous signing key.) > > > > pub rsa4096 2022-06-17 [SC] [expires: 2038-06-13] > > EAC5 89C4 44F4 68FE 7E11 3D2D D8D7 2C13 CF2D 8DDB > > uid [ultimate] Michael Mior > > uid [ultimate] Michael Mior > > uid [ultimate] Michael Mior > > sub rsa4096 2022-06-17 [E] [expires: 2038-06-13] > > > > -- > > Michael Mior > > mm...@apache.org > > > > > > On Mon, Jun 26, 2023 at 3:27 AM Stamatis Zampetakis > > wrote: > > > > > Hello, > > > > > > The last virtual key signing party [1] was about a year ago [2]. In > > > the upcoming releases, we have some people that are serving as release > > > managers for the first time, thus it may be a good time to hold > > > another party. > > > > > > Since Duan Xiong is the RM for the next release, I would suggest > > > holding a meeting in a timezone that is convenient for him. > > > > > > @Duan: Can you please propose a time slot that works for you? > > > > > > I am in UTC+2 and will try to attend if the slot doesn't fall in the > > > middle of the night (23:00 - 06:00). > > > > > > For those that would like to attend, please reply to this thread with > > > your public > > > keys' fingerprint (gpg --fingerprint) before the meeting. > > > > > > pub rsa4096 2019-03-15 [SC] [expires: 2027-03-15] > > > 0474 9577 FD93 4674 B9CD 45C5 D77C 3383 F192 7570 > > > uid [ultimate] Stamatis Zampetakis > > > uid [ultimate] Stamatis Zampetakis > > > sub rsa4096 2019-03-15 [E] [expires: 2027-03-15] > > > > > > To verify your identity, please bring with you a government issued ID > > > (preferably passport). > > > > > > As a reminder of the procedure, have a look at the notes [1] taken by > > > Francis > > > during a previous party! > > > > > > Anybody can participate (not need to be a committer to the project). > > > > > > Best, > > > Stamatis > > > > > > [1] > > > > > > > > > http://www.cryptnet.net/fdp/crypto/keysigning_party/en/keysigning_party.html > > > [2] https://lists.apache.org/thread/4lvch7sfwszbylpzjt11jc9tokm3d0l4 > > > [3] https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84 > > > > > >
Re: [ANNOUNCE] New committer: Zhe Hu
Congratulations and welcome Zhe Hu! -- Michael Mior mm...@apache.org On Wed, Jun 28, 2023 at 7:04 AM Stamatis Zampetakis wrote: > Apache Calcite's Project Management Committee (PMC) has invited Zhe Hu to > become a committer, and we are pleased to announce that they have accepted. > > Zhe Hu has been contributing to the project for a while now. They > improved the stability of the ElasticSearch adapter, worked on > supporting new java versions, and landed various enhancements around > CONCAT and CONVERT functions. > > Zhe Hu, welcome, thank you for your contributions, and we look forward to > your > further interactions with the community! If you wish, please feel free to > tell > us more about yourself and what you are working on. > > As your first commit, please add yourself to the contributors list [1] > and the community page will re-generate [2]. > > Stamatis (on behalf of the Apache Calcite PMC) > > [1] > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > [2] https://calcite.apache.org/community/#project-members >
Re: [ANNOUNCE] New committer: Jacky Lau
Congratulations and welcome Jacky! -- Michael Mior mm...@apache.org On Wed, Jun 28, 2023 at 6:47 AM Stamatis Zampetakis wrote: > Apache Calcite's Project Management Committee (PMC) has invited Jacky Lau > to > become a committer, and we are pleased to announce that they have accepted. > > Jacky has started contributing to the project very recently and in a > very short time they landed many notable improvements around ARRAY and > MAP functions with a particular focus on the Spark dialect. > > Jacky, welcome, thank you for your contributions, and we look forward to > your > further interactions with the community! If you wish, please feel free to > tell > us more about yourself and what you are working on. > > As your first commit, please add yourself to the contributors list [1] > and the community page will re-generate [2]. > > Stamatis (on behalf of the Apache Calcite PMC) > > [1] > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > [2] https://calcite.apache.org/community/#project-members >
Re: Virtual key signing party
I'll attend if possible. My key needs to be updated. (Not compromised, but I lost access to my previous signing key.) pub rsa4096 2022-06-17 [SC] [expires: 2038-06-13] EAC5 89C4 44F4 68FE 7E11 3D2D D8D7 2C13 CF2D 8DDB uid [ultimate] Michael Mior uid [ultimate] Michael Mior uid [ultimate] Michael Mior sub rsa4096 2022-06-17 [E] [expires: 2038-06-13] -- Michael Mior mm...@apache.org On Mon, Jun 26, 2023 at 3:27 AM Stamatis Zampetakis wrote: > Hello, > > The last virtual key signing party [1] was about a year ago [2]. In > the upcoming releases, we have some people that are serving as release > managers for the first time, thus it may be a good time to hold > another party. > > Since Duan Xiong is the RM for the next release, I would suggest > holding a meeting in a timezone that is convenient for him. > > @Duan: Can you please propose a time slot that works for you? > > I am in UTC+2 and will try to attend if the slot doesn't fall in the > middle of the night (23:00 - 06:00). > > For those that would like to attend, please reply to this thread with > your public > keys' fingerprint (gpg --fingerprint) before the meeting. > > pub rsa4096 2019-03-15 [SC] [expires: 2027-03-15] > 0474 9577 FD93 4674 B9CD 45C5 D77C 3383 F192 7570 > uid [ultimate] Stamatis Zampetakis > uid [ultimate] Stamatis Zampetakis > sub rsa4096 2019-03-15 [E] [expires: 2027-03-15] > > To verify your identity, please bring with you a government issued ID > (preferably passport). > > As a reminder of the procedure, have a look at the notes [1] taken by > Francis > during a previous party! > > Anybody can participate (not need to be a committer to the project). > > Best, > Stamatis > > [1] > > http://www.cryptnet.net/fdp/crypto/keysigning_party/en/keysigning_party.html > [2] https://lists.apache.org/thread/4lvch7sfwszbylpzjt11jc9tokm3d0l4 > [3] https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84 >
Re: OSS-Fuzz
Thanks for sharing Julian! Do we *need* to respond to security issues that are uncovered? I certainly agree that we *should* if at all possible. But by choosing not to participate, we would be choosing not to respond to *all* security issues that might only be uncovered via fuzzing. It seems reasonable to me (assuming any discovered vulnerabilities can be kept private), that we should be free to ignore issues that are uncovered. -- Michael Mior mm...@apache.org On Fri, Jun 16, 2023 at 2:31 PM Julian Hyde wrote: > Someone from Google logged a case offering to add Calcite to the > OSS-Fuzz program. (I work for Google but was not aware that we were > being considered.) > > https://issues.apache.org/jira/browse/CALCITE-5781 > > How do people feel about participating in this program? > > I think that it could improve our security significantly, but it will > take work. The fuzzer might generate a lot of false negatives. It > might also generate quite a few genuine security issues that we will > need to respond to appropriately. As an all-volunteer project it might > put a strain on us. > > Julian >
Re: [ANNOUNCE] New committer: Oliver Lee
Congratulations and welcome Oliver! On Tue, Jun 13, 2023, 06:45 Stamatis Zampetakis wrote: > Apache Calcite's Project Management Committee (PMC) has invited Oliver > Lee to become a committer, and we are pleased to announce that they > have accepted. > > Oliver started working with us around November 2022 and since then > they contributed multiple SQL functions to the Calcite repository > bringing lots of improvements to the BigQuery dialect. They improved > the extensibility of the SQL validator and pushed various fixes in > RelNode serialization to JSON. > > Oliver, welcome, thank you for your contributions, and we look forward > to your further interactions with the community! If you wish, please > feel free to tell us more about yourself and what you are working on. > > As your first commit, please add yourself to the contributors list [1] > and the community page will re-generate [2]. > > Stamatis (on behalf of the Apache Calcite PMC) > > [1] > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > [2] https://calcite.apache.org/community/#project-members >
Re: [ANNOUNCE] New committer: Tanner Clary
Congratulations and welcome Tanner! -- Michael Mior mm...@apache.org On Fri, May 26, 2023 at 5:05 AM Stamatis Zampetakis wrote: > Apache Calcite's Project Management Committee (PMC) has invited Tanner > Clary to become a committer, and we are pleased to announce that they > have accepted. > > In less than 5 months, Tanner has introduced, enabled, and tested over > 30 SQL functions in Calcite. They have been a driving force in > improving the BigQuery dialect and by now an expert in library and > parser changes. > > Tanner, welcome, thank you for your contributions, and we look forward > to your further interactions with the community! If you wish, please > feel free to tell us more about yourself and what you are working on. > > As your first commit, please add yourself to the contributors list [1] > and the community page will re-generate [2]. > > Stamatis (on behalf of the Apache Calcite PMC) > > [1] > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > > [2] https://calcite.apache.org/community/#project-members >
Re: [DISCUSS] Disable JIRA worklog for GitHub PRs
+1 from me as well On Wed, Apr 19, 2023, 04:19 Stamatis Zampetakis wrote: > Hello, > > Everything that happens in a GitHub PR creates a worklog entry under > the respective JIRA ticket. > For every worklog entry we receive a notification from j...@apache.org > when we are watching an issue. The worklog entry and email > notification usually appear messy. > > Moreover, if we are watching the GitHub PR we are going to get a > notification from notificati...@github.com which has the same content > with the JIRA worklog entry and is much more readable. > > Finally, the PR notification is also going to > comm...@calcite.apache.org so those who are subscribed to that list > will get the same notification three times. > > Personally, I never read the JIRA worklog notifications and I largely > prefer those from notificati...@github.com. > > How do you feel about disabling the worklog entries in JIRA coming > from GitHub PRs? > > For archiving purposes, the notifications already go to commits@ so we > don't lose anything from disabling the worklog entries. On the > contrary, I find that this would reduce the noise and redundancy on > our inboxes. > > Concretely this is what I have in mind in terms of change: > https://github.com/apache/calcite/pull/3166 > > Best, > Stamatis >
Re: [DISCUSS] Running Sql Logic Tests for Calcite
Very cool! One approach could be to add set these tests to run periodically (daily/weekly) as opposed to being part of the CI pipeline. That way we still have a mechanism to keep tabs on bugs but the whole build isn't slow/broken until this is fixed. On Fri, Apr 14, 2023, 15:20 Mihai Budiu wrote: > Hello all, > > I have submitted a PR for Calcite with a standalone executable that runs > the Sql Logic Test suite of 7+ million tests from sqlite. > > This is the JIRA case: https://issues.apache.org/jira/browse/CALCITE-5615 > And this is the PR: https://github.com/apache/calcite/pull/3145 > > As Stamatis pointed out, the PR isn't really specific to Calcite, it is a > general framework in Java to run these tests on any JDBC compliant > executor. So a question is whether this belongs to the Calcite project, or > some place else. sqlite is a C project, I didn't see any Java in their > source tree. > > Please note that SQLite is in the public domain, so their licensing terms > are not an obstacle to using the test scripts. > > The submitted code runs Calcite in its default configuration, but the > intent is for other projects that build Calcite-based compilers to be able > to test them by subclassing the "TestExecutors". In our own project ( > https://github.com/vmware/sql-to-dbsp-compiler) we have done exactly that, > and we are not using the JDBC API. > > The testsuite does find bugs in Calcite, both crashes and incorrect > results. So I think it's usefulness is not debated. > > The second question is about the packaging of this program; right now it > has a main() entry point and it prints the results to stderr for human > consumption and triage. It is not clear to me how it should be inserted in > a CI infrastructure, since running all 7 million tests could take a long > time. One possible extension would be to have the program generate a > regression test for Calcite for each bug it finds, but I haven't > implemented this feature yet (and many failures could be due to the same > bug). But even that mode would not naturally integrate in a CI > infrastructure. > > A simple possibility is for me to just publish the code as an independent > project on github with an MIT license (the code is derived from our > MIT-licensed project) and just advertise it to the Calcite community. > > I would very much appreciate guidance. > > Mihai Budiu >
Calcite development video
Hi all, I happened to come across the Gource visualization tool today. It takes a log of repository history and generates a timelapse history of development. I decided to do one for Calcite. It's fairly entertaining to watch. Thanks to all contributors! https://www.youtube.com/watch?v=qSpokrc0Ofs -- Michael Mior mm...@apache.org
Re: A new logo for avatica?
> I don't want people to see it as a standalone component. Can I assume you meant that you *do* want it to be seen as a standalone component? -- Michael Mior mm...@apache.org On Mon, Mar 6, 2023 at 8:50 PM Julian Hyde wrote: > A logo would be great. > > I don't think the logo should be a variation of Calcite's logo. > Although Avatica is developed by the Calcite community, I don't want > people to see it as a standalone component. > > By the way, the name Avatica is inspired by the scientific name of the > Barn spider, Araneus cavaticus. So, if people are looking for an > inspiration for a logo, a spider might be a good place to start. > > Julian > > On Mon, Mar 6, 2023 at 12:33 AM Francis Chuang > wrote: > > > > Calcite had a new logo designed in 2018/2019 [1] [2]. > > > > I was wondering if it would be possible to get one made for Avatica. As > > Avatica is a subproject, perhaps its logo should be a variation of the > > Calcite logo to indicate that? > > > > Francis > > > > [1] https://lists.apache.org/thread/px48o0o16h6t3l22gvxkoxpnozcwocfn > > [2] https://lists.apache.org/thread/6pv73tdrox1lpf8ohz46gj78j4t3ko7s >
Re: Schema
You should take a look at Apache Drill. Drill makes use of Calcite, but its primary focus is querying data without a schema. https://drill.apache.org/ -- Michael Mior mm...@apache.org On Wed, Feb 1, 2023 at 6:36 PM Matt Youill wrote: > Hi, quick question... does Calcite work with schema-less data? As an > example, would it be possible to say assemble an algebra that references > a table/field/whatever without first defining those in a schema? > > Thanks, Matt > > >
Re: [ANNOUNCE] Benchao Li joins Calcite PMC
Congratulations and welcome Benchao! -- Michael Mior mm...@apache.org On Fri, Jan 27, 2023 at 4:51 AM Stamatis Zampetakis wrote: > I am pleased to announce that Benchao has accepted an invitation to join > the Calcite PMC. Benchao has been a consistent and helpful figure in > the Calcite community for which we are very grateful. We look forward to > the continued contributions and support. > > Please join me in congratulating Benchao! > > Stamatis (on behalf of the Calcite PMC) >
Re: [ANNOUNCE] New committer: Istvan Toth
Welcome Istvan! -- Michael Mior mm...@apache.org On Tue, Jan 24, 2023 at 4:58 PM Stamatis Zampetakis wrote: > Apache Calcite's Project Management Committee (PMC) has invited Istvan > Toth to become a committer, and we are pleased to announce that he > has accepted. > > Istvan began contributing to Avatica in 2019 and has since made notable > contributions, > particularly in the areas of authentication and connection handling. His > efforts have > helped improve the project's stability and security. > > Istvan, welcome, thank you for your contributions, and we look forward > to your further interactions with the community! If you wish, feel free to > tell us more about yourself and what you are working on. > > As your first commit, please add yourself to the contributors list [1] > and the community page will re-generate [2]. > > Stamatis (on behalf of the Apache Calcite PMC) > > [1] > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > > [2] https://calcite.apache.org/community/#project-members >
Re: [ANNOUNCE] New Calcite PMC chair: Stamatis Zampetakis
Congratulations Stamatis and thanks Reuben! -- Michael Mior mm...@apache.org On Thu, Jan 19, 2023 at 12:04 PM Ruben Q L wrote: > Calcite community members, > > I am pleased to announce that we have a new PMC chair and VP as per our > tradition of rotating the chair once a year. I have resigned, and Stamatis > Zampetakis > was duly elected by the PMC and approved unanimously by the Board. > > Please join me in congratulating Stamatis! > > Best regards, > Ruben >
Re: [ANNOUNCE] New committer: Alex Plehanov
Welcome Alex! -- Michael Mior mm...@apache.org On Fri, Jan 6, 2023 at 6:04 AM Ruben Q L wrote: > Apache Calcite's Project Management Committee (PMC) has invited Alex > Plehanov to become a committer, and we are pleased to announce that he has > accepted the invitation. > > Alex is an Apache Ignite PMC member and the main author of the > ignite-calcite module. He has done significant contributions on Calcite, > especially fixing bugs detected by Ignite. > > Alex, welcome, thank you for your contributions, and we look forward to > your further interactions with the community! If you wish, please feel free > to tell us a bit more about yourself and what you are currently working on. > As your first commit, you can add yourself to the contributors list [1] and > the community page will re-generate [2]. > > Ruben (on behalf of the Apache Calcite PMC) > > [1] > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > [2] https://calcite.apache.org/community/#project-members >
Re: [DISCUSS] Code quality/coverage with SonarCloud & JaCoCo
Thanks Stamatis! I haven't used SonarCloud before, but in general I think such tools can be quite useful. -- Michael Mior mm...@apache.org On Sat, Dec 24, 2022 at 4:01 PM Stamatis Zampetakis wrote: > Since there were no objections, I just logged INFRA-24038 [1] and plan to > move this forward. > > Let me know if you have questions or concerns regarding the adoption of > SonarCloud. > > Best, > Stamatis > > [1] https://issues.apache.org/jira/browse/INFRA-24038 > > On Sat, Dec 10, 2022 at 11:32 AM Benchao Li wrote: > > > Thanks Stamatis for bringing this up. > > > > I haven't used Sonar yet, but thanks for the demo[1] you provided, it > looks > > really interesting. I think it's worth a try for Calcite. > > > > [1] https://github.com/zabetak/calcite/pull/9 > > > > Alessandro Solimando 于2022年12月10日周六 > > 02:54写道： > > > > > Hi all, > > > thanks Stamatis for the proposal and the work, I am a huge fan of Sonar > > and > > > I really miss it on Calcite, so a big +1 from me on this. > > > > > > In Hive so far we have received good feedback, so I hope it will be > > > welcomed here too. > > > > > > Best regards, > > > Alessandro > > > > > > On Fri, 9 Dec 2022 at 19:02, Stamatis Zampetakis > > > wrote: > > > > > > > Hi all, > > > > > > > > I just logged CALCITE-5427 [1] for introducing code quality & > coverage > > > > metrics in Calcite CI. > > > > > > > > I added some motivation and examples under the ticket so please have > a > > > look > > > > and let me know what you think. > > > > > > > > If there are no objections, I will follow-up with INFRA to set things > > up > > > > for the official Calcite repo. > > > > > > > > The integration with SonarCloud has been inspired by HIVE-26196 [2] > > that > > > > Alessandro put in place for Hive and has been very helpful so far. > > > > > > > > Best, > > > > Stamatis > > > > > > > > [1] https://issues.apache.org/jira/browse/CALCITE-5427 > > > > [2] https://issues.apache.org/jira/browse/HIVE-26196 > > > > > > > > > > > > > -- > > > > Best, > > Benchao Li > > >
Re: [ANNOUNCE] New committer: Dmitry Sysolyatin
Congratulations and welcome Dmitry! -- Michael Mior mm...@apache.org On Tue, Nov 8, 2022 at 12:44 PM Ruben Q L wrote: > Apache Calcite's Project Management Committee (PMC) has invited Dmitry > Sysolyatin to become a committer, and we are pleased to announce he has > accepted the invitation. > > During the last months Dmitry has pushed a lot of high quality patches, > fixing and improving code around several places in Calcite core. Apart from > code contributions, he has been regularly involved in discussions regarding > multiple topics in the mailing list, always with a hardworking and > courteous spirit. > > Dmitry, welcome, thank you for your contributions, and we look forward to > your further interactions with the community! If you wish, please feel free > to tell us a bit more about yourself and what you are currently working on. > As your first commit, you can add yourself to the contributors list [1] and > the community page will re-generate [2]. > > Ruben (on behalf of the Apache Calcite PMC) > > [1] > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > [2] https://calcite.apache.org/community/#project-members >
Re: Migrating away from Travis-CI
Thanks Francis! On Wed, Oct 26, 2022, 05:17 Francis Chuang wrote: > CALCITE-5306 has been merged for both Calcite and Avatica. I have lodged > a ticket with Infra to remove AppVeyor [1] from the Calcite repo, so > that it won't fail on each PR / push. > > [1] https://issues.apache.org/jira/browse/INFRA-23827 > > On 25/10/2022 8:28 pm, Alessandro Solimando wrote: > > +1, and as discussed under CALCITE-5306 > > < > https://issues.apache.org/jira/browse/CALCITE-5306?focusedCommentId=17613606=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17613606 > >, > > it would be probably a good idea to retire AppVeyor too as ORC did with > > success (possibly in a separate ticket). > > > > I have seen frequent failures on AppVeyor while retrieving dependencies, > > and I have often found myself waiting for it to kick in for quite some > time > > while GitHub actions and Travis were done already. > > > > On Tue, 25 Oct 2022 at 09:38, Ruben Q L wrote: > > > >> +1 > >> > >> > >> On Tue, Oct 25, 2022 at 2:31 AM Benchao Li > wrote: > >> > >>> +1, thanks Francis for driving this. > >>> > >>> Julian Hyde 于2022年10月25日周二 07:57写道： > >>> > +1 > > > On Oct 24, 2022, at 3:49 PM, Francis Chuang < > >> francischu...@apache.org> > wrote: > > > > Travis will no longer be available for ASF projects at the end of > >> 2022. > > > > For Calcite, only the calcite and calcite-avatica repos use travis > >> and > those projects are also using Github Actions already. Perhaps in our > >> case > we will just need to remove the Travis configs from those repos. > > > > Francis > > > > Forwarded Message > > Subject: Migrating away from Travis-CI > > Date: Mon, 24 Oct 2022 15:46:16 -0500 > > From: Drew Foulks > > Reply-To: us...@infra.apache.org > > To: annou...@infra.apache.org > > > > > > > > Greetings PMC Members! > > > > Infrastructure is moving away from Travis-CI at the end of 2022. > > > > If your project is not currently using Travis-CI, this email is not > actionable. > > > > Projects using Travis-CI: > > > > Please take a look at > https://cwiki.apache.org/confluence/display/INFRA/Travis+Migrations < > https://cwiki.apache.org/confluence/display/INFRA/Travis+Migrations> > > > > and join the builds meetings and mailing list for project updates. > > > > > > -- > > Cheers, > > > > Drew Foulks > > ASF Infra > > > > > > > >>> > >>> -- > >>> > >>> Best, > >>> Benchao Li > >>> > >> > > >
Re: [ANNOUNCE] New committer: Bertil Chapuis
Thank you and welcome Bertil! -- Michael Mior mm...@apache.org On Sun, Oct 16, 2022 at 5:29 PM Julian Hyde wrote: > Apache Calcite's Project Management Committee (PMC) has invited Bertil > Chapuis to become a committer, and we are pleased to announce that he > has accepted. > > Over a few months, Bertil has revitalized Calcite's geospatial > support, with commits that switched our geospatial library from ESRI > to the more modern JTS and proj4j libraries, and broadened our support > for the industry-standard geospatial functions that are in PostGIS and > Open Geospatial Consortium (OGC) Simple Features, including adding > geospatial aggregate functions. He has been easy to work with, and his > contributions are of a uniformly high quality. > > Incidentally, Bertil is one of the initial committers of the Baremaps > project, which has just entered the Apache Incubator. > > Bertil, welcome, thank you for your contributions, and we look forward > to your further interactions with the community! If you wish, please > feel free to tell us more about yourself and what you are working on. > > As your first commit, please add yourself to the contributors list [1] > and the community page will re-generate [2]. > > Julian (on behalf of the Apache Calcite PMC) > > [1] > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > > [2] https://calcite.apache.org/community/#project-members >
Re: mocking service for Database
Thanks Julian for correcting which package should be used. This is not a
circular dependency. There should be no problem with a dependency both
being required directly by your project and transitively via another
dependency. A circular dependency would be if the file adapter required the
core code *and* the core code required the file adapter.
--
Michael Mior
mm...@apache.org
On Mon, Sep 26, 2022 at 12:03 PM Kartik Kudada
wrote:
> Thanks for the quick response.
>
> org.apache.calcite.adapter.csv is in the file module.
> I tried to add example and file module in core's build.gradle.kts but that
> results in circular dependency as file's build.gradle.kts also has core
> dependency.
>
> Core's build.gradle.kts
>
> api(project(":example"))
> api(project(":file"))
>
>
> Regards,
>
> Kartik
>
>
>
> On Mon, Sep 26, 2022 at 9:18 PM Julian Hyde
> wrote:
>
> > Or, better, the file adapter. It also handles CSV files and is not “toy”
> > code.
> >
> > Julian
> >
> > > On Sep 26, 2022, at 08:20, Michael Mior wrote:
> > >
> > > Do you have the calcite-csv package as a dependency of your project?
> > This
> > > must be added in addition to calcite-core.
> > > --
> > > Michael Mior
> > > mm...@apache.org
> > >
> > >
> > >> On Mon, Sep 26, 2022 at 10:37 AM Kartik Kudada
> > >> wrote:
> > >>
> > >> Hi Calcite Developers,
> > >>
> > >> I am working on a requirement where user queries to RDMS database and
> > >> under the hood, Calcite will send back the data from JSON, not from
> > RDMS.
> > >>
> > >> For this, I have added below code snippet in CalciteStatement execute
> > >> method to add CSV schema runtime.
> > >>
> > >> sample code :
> > >>
> > >> final Schema schema =
> > >>CsvSchemaFactory.INSTANCE
> > >>.create(connection.getRootSchema(), null,
> > >>ImmutableMap.of("directory",
> > >>"EMPS.json", "flavor", "scannable"));
> > >>
> > >> connection.getRootSchema().add("mycsv", schema);
> > >>
> > >> So, when the user query "SELECT * FROM .EMPS" converts to
> > >> "SELECT * FROM \"mycsv\".EMPS"
> > >>
> > >> The above code says *package org.apache.calcite.adapter.csv does not
> > >> exist,*
> > >>
> > >> I am trying to fix it for 5 hours.
> > >> How to do this?
> > >>
> > >> Regards,
> > >> Kartik
> > >>
> >
>
Re: mocking service for Database
Do you have the calcite-csv package as a dependency of your project? This
must be added in addition to calcite-core.
--
Michael Mior
mm...@apache.org
On Mon, Sep 26, 2022 at 10:37 AM Kartik Kudada
wrote:
> Hi Calcite Developers,
>
> I am working on a requirement where user queries to RDMS database and
> under the hood, Calcite will send back the data from JSON, not from RDMS.
>
> For this, I have added below code snippet in CalciteStatement execute
> method to add CSV schema runtime.
>
> sample code :
>
> final Schema schema =
> CsvSchemaFactory.INSTANCE
> .create(connection.getRootSchema(), null,
> ImmutableMap.of("directory",
> "EMPS.json", "flavor", "scannable"));
>
> connection.getRootSchema().add("mycsv", schema);
>
> So, when the user query "SELECT * FROM .EMPS" converts to
> "SELECT * FROM \"mycsv\".EMPS"
>
> The above code says *package org.apache.calcite.adapter.csv does not
> exist,*
>
> I am trying to fix it for 5 hours.
> How to do this?
>
> Regards,
> Kartik
>
Re: [ANNOUNCE] Andrei Sereda joins Calcite PMC
Congratulations and welcome Andrei! -- Michael Mior mm...@apache.org On Fri, Aug 12, 2022 at 1:48 PM Ruben Q L wrote: > I am pleased to announce that Andrei has accepted an invitation to join > the Calcite PMC. Andrei has been a consistent and helpful figure in > the Calcite community for which we are very grateful. We look forward to > the continued contributions and support. > > Please join me in congratulating Andrei! > > Ruben (on behalf of the Calcite PMC) >
Re: Calcite Release 1.31 started
Thanks for all your work Andrei! -- Michael Mior mm...@apache.org On Wed, Aug 3, 2022 at 12:44 PM Andrei Sereda wrote: > 1.31 has been released. > > You can resume dev work on the main branch. > > Thanks for your patience. > > > On Wed, Jul 20, 2022 at 2:52 AM Andrei Sereda wrote: > > > Hello, > > > > I'm starting the Calcite 1.31 release. Main branch will be frozen until > > completion. > > > > Please avoid committing anything to main until further notice. > > > > Andrei. > > >
Re: Draft: board report for 2022 Q2
+1 Thanks Ruben! -- Michael Mior mm...@apache.org Le lun. 4 juil. 2022 à 07:52, Ruben Q L a écrit : > Hello, > > Below these lines you can find a draft of this quarter's board report. I > plan to submit it > at the end of this week. > Please let me know if you have any additions or corrections. Note that our > last report got a question from the board, for which I include a proposed > answer in this report (cf "Issues" section). > Best, > Ruben > > > > ## Description: > Apache Calcite is a highly customizable framework for parsing and planning > queries on data in a wide variety of formats. It allows database-like > access, > and in particular a SQL interface and advanced query optimization, for data > not residing in a traditional database. > > Avatica is a sub-project within Calcite and provides a framework for > building > local and remote JDBC and ODBC database drivers. Avatica has an independent > release schedule and its own repository. > > > ## Issues: > There are no issues requiring board attention. > > The last report got one comment from the board: > "It's certainly up to you all, but please remember that these reports are > published, publicly. Are you sure you want to have your names and email > addresses as part of this (public) report?". > > There is no problem from our side. This report has been pre-approved by the > Calcite community and the section with names / emails has already been part > of > our reports in the "community health" section for a while. We decided to > add it some time ago to highlight the contributions from non-committers and > the top reviewers, in order to encourage the members of the community. > > > ## Membership Data: > Apache Calcite was founded 2015-10-22 (7 years ago) > There are currently 56 committers and 25 PMC members in this project. > The Committer-to-PMC ratio is roughly 7:4. > > Community changes, past quarter: > - Chunwei Lei was added to the PMC on 2022-05-24 > - Vladimir Ozerov was added to the PMC on 2022-05-24 > - No new committers. Last addition was Alessandro Solimando on 2021-12-17. > > > ## Project Activity: > Avatica 1.21.0 was released on 2022-05-08, it was a maintenance release > with > dependency upgrades and added support for Oracle JDK 16 to 18. Of > particular > note is Log4j2 being upgrade to 2.16.0 and subsequently 2.17.0 and 2.17.1 > to > address CVE-2021-44228, CVE-2021-45105 and CVE-2021-44832. > > There has not been any Calcite release in this quarter. Calcite 1.31.0 was > planned for mid-June, but it has suffered a small delay. It should be > released > during July. > > > ## Community Health: > The overall status of the community is healthy and stable. We are keeping > the > good momentum in terms of contributions and opened discussions to improve > the > project. This is reflected also on the statistics of Jira & GitHub > activity: > -1% issues opened, +9% issues closed, -9% commits, +18% code contributors, > -11% PRs opened, +21% PRs closed. > > As usual, we struggle a bit on the PRs reviews, with some of them being > opened > and blocked for a while due a lack of active reviewers. This fact has > triggered an interesting discussion in both the Apache Calcite and Apache > Flink communities about the pros and cons of forking Calcite by the latter. > Many voices from the Apache Calcite community advised against this idea, > due > to the elevated maintenance costs in the long term. Instead, we are > discussing > several ideas about how to improve our PR review situation, among them the > possibility of increasing the number of reviewers by granting Calcite > committership per request to people who are already ASF committers (in > other > projects) and have a proven record of working with Calcite (on a case by > case > basis to be approved by the Calcite PMC). > > The number of non-committer (contributor) commits per month: > +-+-+-+ > |year |month| contributor_commits | > +-+-+-+ > | 2022| 4 | 15 | > | 2022| 5 | 13 | > | 2022| 6 | 19 | > +-+-+-+ > > The number of active reviewers per month: > +-+-+-+ > |year |month| active_reviewers | > +-+-
Re: [DISCUSS] How we choose a PMC chair
+1 from me as well. -- Michael Mior mm...@apache.org Le dim. 3 juil. 2022 à 19:46, Julian Hyde a écrit : > As you know, Calcite has a tradition of choosing a new PMC chair (VP) > each year, around the anniversary of the project's graduation[1][2]. I > think this is a great tradition, but I'd like to discuss an > improvement to that process. > > (I'm starting the conversation now - several months after the previous > vote, and several months before the next - so that it's clear that I > am not criticizing the process or the outcome or previous votes.) > > I've noticed that the outgoing chair sends an email on dev@ saying > words to the following effect: > > I think Xyz would be a great person to succeed me. > What do you all think? > > (I fear that I may have started this tradition when, at the end of my > tenure as first chair, I approached Jesus and asked him whether he'd > be prepared to do the job[3]. Mea culpa.) > > After such an outright endorsement, especially on a public list, it > would be churlish for someone to reply "Actually, I think Abc would be > better." As a result, it's rather difficult to have an open debate, > and the candidate selected by the outgoing chair tends to win > unopposed. > > I suggest that the outgoing chair says something like > > It's time to change the PMC chair. > Please send nominations to private@ and the PMC will discuss and vote. > > That would allow for several nominations, allow people to give reasons > why they prefer a candidate (without disparaging other candidates), > and lead to a more informed outcome. > > What do you think? Are there any other aspects of the election process > we should change? > > Julian > > [1] https://lists.apache.org/thread/rmj9qm9wlol3nb7z4phddoljbgvypkrt > [2] https://lists.apache.org/thread/5tzb8w655pj2vo9omz20th5jnbn9zww7 > [3] https://lists.apache.org/thread/y4wjdj5h1y3sypnlmhpoz9r6bkk3cv6o >
Re: Question about babel
This has nothing to do with Javascript. Babel is a SQL parser in Calcite that is designed to be as permissive as possible. While the default Calcite parser attempts to adhere to SQL standards, the goal of the Babel parser is to accept as much as possible across varying dialects of SQL. -- Michael Mior mm...@apache.org Le sam. 11 juin 2022 à 03:13, junww2...@gmail.com a écrit : > Hi, > > In Calcite source code, there is a package called SQL.parser.babel. What is > the package for? Is it for this parser? > https://babeljs.io/docs/en/babel-parser > > Thanks, > > Jack >
Re: Apply contributor permission
I have added you as a contributor. Welcome to Calcite! Le ven. 27 mai 2022 à 21:27, 4wei a écrit : > Hi all, > > I want to contribute to calcite, could anyone please give me the > contributor permission? > > My apache Jira account info: > Account name: 4wei > Fullname: Mou Wu > Email: wumou.4...@outlook.com
Re: [ANNOUNCE] Vladimir Ozerov joins Calcite PMC
Congratulations Vladimir! -- Michael Mior mm...@apache.org Le mar. 24 mai 2022 à 16:47, Ruben Q L a écrit : > I am pleased to announce that Vladimir has accepted an invitation to join > the Calcite PMC. Vladimir has been a consistent and helpful figure in the > Calcite community for which we are very grateful. We look forward to the > continued contributions and support. > > Please join me in congratulating Vladimir! > > - Ruben (on behalf of the Calcite PMC) >
Re: [ANNOUNCE] Chunwei Lei joins Calcite PMC
Congratulations and thank you Chunwei! -- Michael Mior mm...@apache.org Le mar. 24 mai 2022 à 16:47, Ruben Q L a écrit : > I am pleased to announce that Chunwei has accepted an invitation to join > the > Calcite PMC. Chunwei has been a consistent and helpful figure in the > Calcite community for which we are very grateful. We look forward to the > continued contributions and support. > > Please join me in congratulating Chunwei! > > - Ruben (on behalf of the Calcite PMC) >
Re: [ANNOUNCE] Master branch renamed to main
Thanks for checking Francis! Hopefully that means we won't have any issues. -- Michael Mior mm...@apache.org Le mar. 26 avr. 2022 à 22:23, Francis Chuang a écrit : > Quick update. I just did a dry-run release for Avatica and it built fine > without needing to set the branch in vlsi-release-plugin's configuration. > > The plugin only pushes tags to git and I don't think it ever commits any > files, so I don't think it needs to know about us renaming our default > branch to main. I also ran `git branch -a` in my local copy of Avatica > and I don't see any branch or references to "master" after running a > dry-run release. > > On 27/04/2022 9:03 am, Michael Mior wrote: > > One outstanding issue is that I was unable to set the branch to main for > > stage-vote-release-plugin. I believe it may default to "master", so I'm > not > > sure if we'll run into issues on the next release. > > -- > > Michael Mior > > mm...@apache.org > > > > > > Le mar. 26 avr. 2022 à 16:30, Julian Hyde a écrit : > > > >> Thanks, Michael. It all looks good from my end. > >> > >> I had opened a pull request yesterday (when the branch was named > >> 'master') and it merged smoothly today (after the rename). > >> > >> Julian > >> > >> On Tue, Apr 26, 2022 at 7:54 AM Michael Mior wrote: > >>> > >>> The master branch has now been renamed to main. If you have an existing > >>> checkout, please follow the steps below (thanks to Francis!) to ensure > >> your > >>> local master branch is also correctly renamed. Note that if you do not > >> have > >>> any local changes, you can also simply clone a new copy of the > repository > >>> and the default branch will now be main. > >>> > >>> 1. Ensure you are in the repository's folder in your command line. > >>> 2. Stash any uncommitted changes: > >>>- git stash > >>> 3. Check out your local master branch: > >>>- git checkout master > >>> 4. Rename your local master branch to main: > >>>- git branch -m master main > >>> 5. Fetch latest commits and branches from remote: > >>>- git fetch > >>> 6. Remove the tracking branch: > >>>- git branch --unset-upstream > >>> 7. Create a new tracking branch (assuming the remote is called > >> origin): > >>>- git branch -u origin/main > >>> 8. Restore your uncommitted changes: > >>>- git stash pop > >>> > >>> -- > >>> Michael Mior > >>> mm...@apache.org > >> > > >
Re: [ANNOUNCE] Master branch renamed to main
One outstanding issue is that I was unable to set the branch to main for stage-vote-release-plugin. I believe it may default to "master", so I'm not sure if we'll run into issues on the next release. -- Michael Mior mm...@apache.org Le mar. 26 avr. 2022 à 16:30, Julian Hyde a écrit : > Thanks, Michael. It all looks good from my end. > > I had opened a pull request yesterday (when the branch was named > 'master') and it merged smoothly today (after the rename). > > Julian > > On Tue, Apr 26, 2022 at 7:54 AM Michael Mior wrote: > > > > The master branch has now been renamed to main. If you have an existing > > checkout, please follow the steps below (thanks to Francis!) to ensure > your > > local master branch is also correctly renamed. Note that if you do not > have > > any local changes, you can also simply clone a new copy of the repository > > and the default branch will now be main. > > > >1. Ensure you are in the repository's folder in your command line. > >2. Stash any uncommitted changes: > > - git stash > >3. Check out your local master branch: > > - git checkout master > >4. Rename your local master branch to main: > > - git branch -m master main > >5. Fetch latest commits and branches from remote: > > - git fetch > >6. Remove the tracking branch: > > - git branch --unset-upstream > >7. Create a new tracking branch (assuming the remote is called > origin): > > - git branch -u origin/main > >8. Restore your uncommitted changes: > > - git stash pop > > > > -- > > Michael Mior > > mm...@apache.org >
[ANNOUNCE] Master branch renamed to main
The master branch has now been renamed to main. If you have an existing checkout, please follow the steps below (thanks to Francis!) to ensure your local master branch is also correctly renamed. Note that if you do not have any local changes, you can also simply clone a new copy of the repository and the default branch will now be main. 1. Ensure you are in the repository's folder in your command line. 2. Stash any uncommitted changes: - git stash 3. Check out your local master branch: - git checkout master 4. Rename your local master branch to main: - git branch -m master main 5. Fetch latest commits and branches from remote: - git fetch 6. Remove the tracking branch: - git branch --unset-upstream 7. Create a new tracking branch (assuming the remote is called origin): - git branch -u origin/main 8. Restore your uncommitted changes: - git stash pop -- Michael Mior mm...@apache.org
Re: History for unreleased versions being published to site
+1 Commented release notes seem to be an easy solution. -- Michael Mior mm...@apache.org Le mer. 20 avr. 2022 à 10:00, Julian Hyde a écrit : > Usually we don’t add release notes to history.md until the RC vote, so > this isn’t a problem. > > But occasionally we want to start writing the rekease notes early. An > example is my recent change to upgrade Avatica’s gradle and Java. For such > changes we could add the release notes for the upcoming release commented > out, and the release manager could uncommon them when preparing the RC. > > What you think? > > Julian > > > On Apr 20, 2022, at 5:33 AM, Michael Mior wrote: > > > > I don't necessarily see this as a problem, although instead of > 2022-MM-DD > > for the release date, it might be nice to clearly indicate this is > > unreleased. > > -- > > Michael Mior > > mm...@apache.org > > > > > >> Le mer. 20 avr. 2022 à 08:00, Francis Chuang > a > >> écrit : > >> > >> I noticed that some changes to Calcite's changelog was deployed to the > >> website automatically: https://calcite.apache.org/docs/history.html > >> > >> The reason is that in the Github Actions workflow > >> ( > >> > https://github.com/apache/calcite/blob/master/.github/workflows/publish-non-release-website-updates.yml#L24 > ), > >> > >> we deploy the site immediately when we update the history.md file. This > >> is because after a release, there's a possibility that we might need to > >> go back to update the release date or to make some small tweaks to the > >> changelog. > >> > >> I was wondering if we can change history.md, so that the changelog for a > >> release is only visible if there's a release announcement in the _posts > >> folder, similar to how the downloads page is generated. > >> > >> Would anyone familiar with jekyll like to take a stab at this? > >> >
Re: History for unreleased versions being published to site
I don't necessarily see this as a problem, although instead of 2022-MM-DD for the release date, it might be nice to clearly indicate this is unreleased. -- Michael Mior mm...@apache.org Le mer. 20 avr. 2022 à 08:00, Francis Chuang a écrit : > I noticed that some changes to Calcite's changelog was deployed to the > website automatically: https://calcite.apache.org/docs/history.html > > The reason is that in the Github Actions workflow > ( > https://github.com/apache/calcite/blob/master/.github/workflows/publish-non-release-website-updates.yml#L24), > > we deploy the site immediately when we update the history.md file. This > is because after a release, there's a possibility that we might need to > go back to update the release date or to make some small tweaks to the > changelog. > > I was wondering if we can change history.md, so that the changelog for a > release is only visible if there's a release announcement in the _posts > folder, similar to how the downloads page is generated. > > Would anyone familiar with jekyll like to take a stab at this? >
Re: Website build failure
I created the PR below which I think should resolve the issue. If Francis or someone else could have a look, that would be great. I think it's safe to merge and we should know when GitHub next runs the action if the issue is solved. https://github.com/apache/calcite-avatica/pull/177 -- Michael Mior mm...@apache.org Le mar. 19 avr. 2022 à 15:46, Michael Mior a écrit : > I took a brief look and it seems this was caused by the Ruby bundler not > being able to find a specific version of a gem. Looking further, I think > this is because the Docker image being used for Jekyll upgraded from Ruby > 2.x to 3.x and this specific version of the gem is not available for Ruby > 3.x. It should be a matter of either using an older version of the Docker > image or (probably better) updating the Ruby dependencies. > > -- > Michael Mior > mm...@apache.org > > Le mar. 19 avr. 2022 à 15:27, Julian Hyde a écrit : > >> Francis, >> >> The job to build the Avatica website failed after my commit. I don't >> think it's anything I did wrong. Can you please take a look? >> >> >> https://github.com/apache/calcite-avatica/runs/6083671773?check_suite_focus=true >> >> Julian >> >
[jira] [Created] (CALCITE-5102) Avatica website fails to build
Michael Mior created CALCITE-5102:
-
Summary: Avatica website fails to build
Key: CALCITE-5102
URL: https://issues.apache.org/jira/browse/CALCITE-5102
Project: Calcite
Issue Type: Bug
Components: avatica
Reporter: Michael Mior
{{The error below occurs when trying to build the repository via GitHub
Actions.}}
{{ }}
Creating network "site_default" with the default driver
{{Creating volume "site_gradle-cache" with default driver }}
{{Pulling build-site (jekyll/jekyll:4)... }}
{{4: Pulling from jekyll/jekyll }}
{{Digest:
sha256:5776c8eed572003d9ec021767d725b6aa37226bebf4a5219049c063ff8b698ef }}
{{Status: Downloaded newer image for jekyll/jekyll:4 }}
{{Creating site_build-site_run ... }}
{{Creating site_build-site_run ... done }}
{{Cloning into '/tmp/calcite-avatica-go'... }}
{{Updated 0 paths from 74ec7ce }}
{{Bundler 2.3.11 is running, but your lockfile was generated with 2.3.10.
Installing Bundler 2.3.10 and restarting using that version. }}
{{Fetching gem metadata from [https://rubygems.org/.|https://rubygems.org/] }}
{{Fetching bundler 2.3.10 }}
{{Installing bundler 2.3.10 }}
{{Fetching gem metadata from [https://rubygems.org/.] }}
{{Your bundle is locked to listen (3.1.5) from rubygems repository }}
{{[https://rubygems.org/] or installed locally, but that version can no longer
be }}
{{found in that source. That means the author of listen (3.1.5) has removed it.
}}
{{You'll need to update your bundle to a version other than listen (3.1.5) that
}}
{{hasn't been removed in order to install. }}
7
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
Re: Website build failure
I took a brief look and it seems this was caused by the Ruby bundler not being able to find a specific version of a gem. Looking further, I think this is because the Docker image being used for Jekyll upgraded from Ruby 2.x to 3.x and this specific version of the gem is not available for Ruby 3.x. It should be a matter of either using an older version of the Docker image or (probably better) updating the Ruby dependencies. -- Michael Mior mm...@apache.org Le mar. 19 avr. 2022 à 15:27, Julian Hyde a écrit : > Francis, > > The job to build the Avatica website failed after my commit. I don't > think it's anything I did wrong. Can you please take a look? > > > https://github.com/apache/calcite-avatica/runs/6083671773?check_suite_focus=true > > Julian >
Re: [DISCUSS] Rename 'master' branch to 'main' (part 2)
+1 Thanks for resurfacing this Julian. Since main is becoming the default pretty much everywhere and the tooling has improved, I think several of the criticisms of this are less relevant. Yes, it still does require work and some things may break. But this is a one time change and only affects contributors, not any consumers of Calcite artifacts. -- Michael Mior mm...@apache.org Le mar. 12 avr. 2022 à 15:10, Julian Hyde a écrit : > In August 2020 Michael started a discussion[1] about renaming the > 'master' branch in Git to 'main'. A majority of the community seemed > to be in favor, although some people did not agree with the change. > > If I recall correctly, we did not move forward with the rename at that > time in part because of tooling: GitHub tooling was not mature enough > to guarantee that the change would be smooth. I now believe the > tooling is mature. I have migrated several personal projects, and > GitHub makes it very easy. > > I propose that we move forward with renaming the 'master' branch in > the calcite, calcite-avatica and calcite-avatica-go repositories. I > propose that we do calcite-avatica first, because it is less used than > calcite. > > Julian > > [1] https://lists.apache.org/thread/7f2kpkf8lrq0mmt46lb2orsh39kq7dbs >
Re: [DISCUSS] Github PR link to JIRA issue
Autolinks are listed as an "upcoming feature." I don't believe they can be currently controlled via .asf.yaml. -- Michael Mior mm...@apache.org Le sam. 2 avr. 2022 à 01:19, Julian Hyde a écrit : > The present hyperlinking was enabled by Stamatis: > https://issues.apache.org/jira/browse/CALCITE-4104 < > https://issues.apache.org/jira/browse/CALCITE-4104> > > I’m not exactly sure how the .asf.yaml file accomplishes this. Maybe it > instructs some ASF robot to modify the GitHub project settings on our > behalf. > > More info on the .asf.yaml file: > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features > < > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features> > > > Julian > > > > On Apr 1, 2022, at 9:18 PM, Haisheng Yuan wrote: > > > > Yes, that is what I meant. > > > > I was expecting to see the link to JIRA in PR's commit page: > > https://github.com/apache/calcite/pull/2752/commits > > > > I am lazy to navigate to JIRA manually. :) > > > > Haisheng > > > > On 2022/04/01 01:04:19 Julian Hyde wrote: > >> Ah, I see. In the page for a commit (e.g. [1]) the text > “[CALCITE-5064]” appears as a blue hyperlink whereas the page for the > corresponding pull request (e.g. [2]) the “[CALCITE-5064]” text is not a > hyperlink. > >> > >> Is that what you meant, Haisheng? > >> > >> Julian > >> > >> > >> [1] > https://github.com/apache/calcite/commit/d85b2a602a547290bd5be0bba68092b702400731 > < > https://github.com/apache/calcite/commit/d85b2a602a547290bd5be0bba68092b702400731 > > > >> > >> [2] https://github.com/apache/calcite/pull/2752 < > https://github.com/apache/calcite/pull/2752> > >> > >>> On Mar 31, 2022, at 11:05 AM, Michael Mior wrote: > >>> > >>> Stamatis, > >>> > >>> Unless I'm misunderstanding, Haisheng was referring to links in the > >>> opposite direction. That is, linking to JIRA issues from GitHub. The > >>> setting you reference is for creating a link from JIRA to GitHub. > >>> > >>> -- > >>> Michael Mior > >>> mm...@apache.org > >>> > >>> > >>> Le jeu. 31 mars 2022 à 13:53, Stamatis Zampetakis > a > >>> écrit : > >>> > >>>> It's already done via .asf.yaml [1]. > >>>> > >>>> Best, > >>>> Stamatis > >>>> > >>>> [1] > >>>> > >>>> > https://github.com/apache/calcite/blob/88cc385f98c551c1aca7ffab101934f1c34fdffd/.asf.yaml#L35 > >>>> > >>>> On Thu, Mar 31, 2022 at 7:19 PM Michael Mior > wrote: > >>>> > >>>>> Not sure what might have changed, but here's the GitHub > documentation on > >>>>> the feature. If this isn't working as expected, I would contact > INFRA to > >>>>> make sure things are correctly configured. (Apparently in the future, > >>>> this > >>>>> may be done via .asf.yaml) > >>>>> > >>>>> > >>>>> > >>>> > https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/configuring-autolinks-to-reference-external-resources > >>>>> > >>>>> > >>>>> -- > >>>>> Michael Mior > >>>>> mm...@apache.org > >>>>> > >>>>> > >>>>> Le mer. 30 mars 2022 à 13:46, Haisheng Yuan a > écrit : > >>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> Previously, the JIRA issue e.g. [CALCITE-6789] in Calcite github PR > and > >>>>>> commit message will be automatically linked to the JIRA site. Now > there > >>>>> is > >>>>>> no link anymore. > >>>>>> > >>>>>> Does anyone know what happened? What can we do to add the link back? > >>>>>> > >>>>>> Thanks, > >>>>>> Haisheng Yuan > >>>>>> > >>>>> > >>>> > >> > >> > >
Re: [DISCUSS] Github PR link to JIRA issue
Stamatis, Unless I'm misunderstanding, Haisheng was referring to links in the opposite direction. That is, linking to JIRA issues from GitHub. The setting you reference is for creating a link from JIRA to GitHub. -- Michael Mior mm...@apache.org Le jeu. 31 mars 2022 à 13:53, Stamatis Zampetakis a écrit : > It's already done via .asf.yaml [1]. > > Best, > Stamatis > > [1] > > https://github.com/apache/calcite/blob/88cc385f98c551c1aca7ffab101934f1c34fdffd/.asf.yaml#L35 > > On Thu, Mar 31, 2022 at 7:19 PM Michael Mior wrote: > > > Not sure what might have changed, but here's the GitHub documentation on > > the feature. If this isn't working as expected, I would contact INFRA to > > make sure things are correctly configured. (Apparently in the future, > this > > may be done via .asf.yaml) > > > > > > > https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/configuring-autolinks-to-reference-external-resources > > > > > > -- > > Michael Mior > > mm...@apache.org > > > > > > Le mer. 30 mars 2022 à 13:46, Haisheng Yuan a écrit : > > > > > Hi all, > > > > > > Previously, the JIRA issue e.g. [CALCITE-6789] in Calcite github PR and > > > commit message will be automatically linked to the JIRA site. Now there > > is > > > no link anymore. > > > > > > Does anyone know what happened? What can we do to add the link back? > > > > > > Thanks, > > > Haisheng Yuan > > > > > >
Re: [DISCUSS] Github PR link to JIRA issue
Not sure what might have changed, but here's the GitHub documentation on the feature. If this isn't working as expected, I would contact INFRA to make sure things are correctly configured. (Apparently in the future, this may be done via .asf.yaml) https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/configuring-autolinks-to-reference-external-resources -- Michael Mior mm...@apache.org Le mer. 30 mars 2022 à 13:46, Haisheng Yuan a écrit : > Hi all, > > Previously, the JIRA issue e.g. [CALCITE-6789] in Calcite github PR and > commit message will be automatically linked to the JIRA site. Now there is > no link anymore. > > Does anyone know what happened? What can we do to add the link back? > > Thanks, > Haisheng Yuan >
Re: Exception parsing "SELECT language"
I understand SELECT CURRENT_TIMESTAMP is valid. I just didn't expect SELECT
x to be valid. Anyway, thanks for the clarification!
--
Michael Mior
mm...@apache.org
Le mer. 23 mars 2022 à 14:30, Julian Hyde a écrit :
> SELECT CURRENT_TIMESTAMP
>
> Is a synactically and semantically valid query (because CURRENT_TIMESTAMP
> happens to be a built-in function that doesn’t take parentheses or
> arguments). On the other hand
>
> SELECT x
>
> Is syntactically valid but semantically invalid (because there is no ‘x’
> in global scope). Lastly,
>
> SELECT language
>
> Is syntactically invalid (and gives a different error to ’SELECT x’). The
> following patch demonstrates:
>
> diff --git
> a/testkit/src/main/java/org/apache/calcite/sql/parser/SqlParserTest.java
> b/testkit/src/main/java/org/apache/calcite/sql/parser/SqlParserTest.java
> index 0dd79138ea..e45809105a 100644
> ---
> a/testkit/src/main/java/org/apache/calcite/sql/parser/SqlParserTest.java
> +++
> b/testkit/src/main/java/org/apache/calcite/sql/parser/SqlParserTest.java
> @@ -8769,6 +8769,14 @@ private static Consumer>
> checkWarnings(
> sql(sql).ok(expected);
>}
>
> + @Test void testLanguage() {
> +final String sql = "select x";
> +sql(sql).ok("SELECT `X`");
> +
> +final String sql2 = "select language";
> +sql(sql2).ok("SELECT `X`");
> + }
> +
>@Test void testMatchRecognizePatternSkip5() {
> final String sql = "select *\n"
> + " from t match_recognize\n"
>
>
>
>
> > On Mar 23, 2022, at 11:20 AM, Michael Mior wrote:
> >
> > I was hinting that there might be a syntax issue caused by a missing part
> > of the query. I would not have assumed "SELECT foobar" is a valid query
> > both in syntax and semantics.
> > --
> > Michael Mior
> > mm...@apache.org
> >
> >
> > Le mer. 23 mars 2022 à 14:12, Julian Hyde a
> écrit :
> >
> >> If there’s nowhere for LANGUAGE to come from that would make it
> >> semantically invalid but wouldn’t affect the syntactic validity. (In
> other
> >> words, you’d get an error from SqlValidator but not from SqlParser.)
> >>
> >> In this case, the problem is that LANGUAGE is a reserved keyword in both
> >> standard SQL and Calcite.
> >>
> >> Julian
> >>
> >>
> >>> On Mar 23, 2022, at 11:05 AM, Michael Mior wrote:
> >>>
> >>> What would you expect this query to return? You haven't specified a
> FROM
> >>> clause, so there's no indication where "language" should come from.
> >>>
> >>> --
> >>> Michael Mior
> >>> mm...@apache.org
> >>>
> >>>
> >>> Le mer. 23 mars 2022 à 14:02, Adolfo Ochagavía a
> >>> écrit :
> >>>
> >>>> Hi there,
> >>>>
> >>>> I am trying to find my way using Calcite to parse SQL queries, and was
> >>>> surprised to find out that parsing the query "SELECT language" fails
> >> with
> >>>> an exception.
> >>>>
> >>>> This is the code:
> >>>>> var config = SqlParser.Config.DEFAULT.withLex(Lex.MYSQL);
> >>>>> var parser = SqlParser.create("SELECT language", config);
> >>>>> var parsed = parser.parseQuery();
> >>>>
> >>>> This is the exception:
> >>>>> org.apache.calcite.sql.parser.SqlParseException: Encountered ".
> >>>> language" at line 1, column 18.
> >>>>> Was expecting one of:
> >>>>>
> >>>>> "AS" ...
> >>>>> [the rest is omitted for brevity, but about 60 more lines follow]
> >>>>
> >>>> Am I missing something or is this a bug? Note that the query is a
> >>>> simplified excerpt of an autoconfiguration query issued by MySQL's
> JDBC
> >>>> driver and seems to be handled well by MySQL servers. Below I am
> pasting
> >>>> the full query, in case someone would like to see the original:
> >>>>> /* mysql-connector-java-8.0.19 (Revision:
> >>>> a0ca826f5cdf51a98356fdfb1bf251eb042f80bf) */SELECT
> >>>> @@session.auto_increment_increment AS auto_increment_increment,
> >>>> @@character_set_client AS character_set_client,
> >>>> @@character_set_connection AS character_set_connection,
> >>>> @@character_set_results AS character_set_results,
> >>>> @@character_set_server AS character_set_server, @@collation_server AS
> >>>> collation_server, @@collation_connection AS collation_connection,
> >>>> @@init_connect AS init_connect, @@interactive_timeout AS
> >>>> interactive_timeout, @@language AS language, @@license AS license,
> >>>> @@lower_case_table_names AS lower_case_table_names,
> >>>> @@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS
> >>>> net_write_timeout, @@performance_schema AS performance_schema,
> >>>> @@query_cache_size AS query_cache_size, @@query_cache_type AS
> >>>> query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS
> >>>> system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS
> >>>> transaction_isolation, @@wait_timeout AS wait_timeout
> >>>>
> >>>> Thanks for helping out ;)
> >>>> Adolfo
> >>
> >>
>
>
Re: Exception parsing "SELECT language"
I was hinting that there might be a syntax issue caused by a missing part
of the query. I would not have assumed "SELECT foobar" is a valid query
both in syntax and semantics.
--
Michael Mior
mm...@apache.org
Le mer. 23 mars 2022 à 14:12, Julian Hyde a écrit :
> If there’s nowhere for LANGUAGE to come from that would make it
> semantically invalid but wouldn’t affect the syntactic validity. (In other
> words, you’d get an error from SqlValidator but not from SqlParser.)
>
> In this case, the problem is that LANGUAGE is a reserved keyword in both
> standard SQL and Calcite.
>
> Julian
>
>
> > On Mar 23, 2022, at 11:05 AM, Michael Mior wrote:
> >
> > What would you expect this query to return? You haven't specified a FROM
> > clause, so there's no indication where "language" should come from.
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> >
> > Le mer. 23 mars 2022 à 14:02, Adolfo Ochagavía a
> > écrit :
> >
> >> Hi there,
> >>
> >> I am trying to find my way using Calcite to parse SQL queries, and was
> >> surprised to find out that parsing the query "SELECT language" fails
> with
> >> an exception.
> >>
> >> This is the code:
> >>> var config = SqlParser.Config.DEFAULT.withLex(Lex.MYSQL);
> >>> var parser = SqlParser.create("SELECT language", config);
> >>> var parsed = parser.parseQuery();
> >>
> >> This is the exception:
> >>> org.apache.calcite.sql.parser.SqlParseException: Encountered ".
> >> language" at line 1, column 18.
> >>> Was expecting one of:
> >>>
> >>>"AS" ...
> >>>[the rest is omitted for brevity, but about 60 more lines follow]
> >>
> >> Am I missing something or is this a bug? Note that the query is a
> >> simplified excerpt of an autoconfiguration query issued by MySQL's JDBC
> >> driver and seems to be handled well by MySQL servers. Below I am pasting
> >> the full query, in case someone would like to see the original:
> >>> /* mysql-connector-java-8.0.19 (Revision:
> >> a0ca826f5cdf51a98356fdfb1bf251eb042f80bf) */SELECT
> >> @@session.auto_increment_increment AS auto_increment_increment,
> >> @@character_set_client AS character_set_client,
> >> @@character_set_connection AS character_set_connection,
> >> @@character_set_results AS character_set_results,
> >> @@character_set_server AS character_set_server, @@collation_server AS
> >> collation_server, @@collation_connection AS collation_connection,
> >> @@init_connect AS init_connect, @@interactive_timeout AS
> >> interactive_timeout, @@language AS language, @@license AS license,
> >> @@lower_case_table_names AS lower_case_table_names,
> >> @@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS
> >> net_write_timeout, @@performance_schema AS performance_schema,
> >> @@query_cache_size AS query_cache_size, @@query_cache_type AS
> >> query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS
> >> system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS
> >> transaction_isolation, @@wait_timeout AS wait_timeout
> >>
> >> Thanks for helping out ;)
> >> Adolfo
>
>
Re: Exception parsing "SELECT language"
What would you expect this query to return? You haven't specified a FROM
clause, so there's no indication where "language" should come from.
--
Michael Mior
mm...@apache.org
Le mer. 23 mars 2022 à 14:02, Adolfo Ochagavía a
écrit :
> Hi there,
>
> I am trying to find my way using Calcite to parse SQL queries, and was
> surprised to find out that parsing the query "SELECT language" fails with
> an exception.
>
> This is the code:
> > var config = SqlParser.Config.DEFAULT.withLex(Lex.MYSQL);
> > var parser = SqlParser.create("SELECT language", config);
> > var parsed = parser.parseQuery();
>
> This is the exception:
> > org.apache.calcite.sql.parser.SqlParseException: Encountered ".
> language" at line 1, column 18.
> > Was expecting one of:
> >
> > "AS" ...
> > [the rest is omitted for brevity, but about 60 more lines follow]
>
> Am I missing something or is this a bug? Note that the query is a
> simplified excerpt of an autoconfiguration query issued by MySQL's JDBC
> driver and seems to be handled well by MySQL servers. Below I am pasting
> the full query, in case someone would like to see the original:
> > /* mysql-connector-java-8.0.19 (Revision:
> a0ca826f5cdf51a98356fdfb1bf251eb042f80bf) */SELECT
> @@session.auto_increment_increment AS auto_increment_increment,
> @@character_set_client AS character_set_client,
> @@character_set_connection AS character_set_connection,
> @@character_set_results AS character_set_results,
> @@character_set_server AS character_set_server, @@collation_server AS
> collation_server, @@collation_connection AS collation_connection,
> @@init_connect AS init_connect, @@interactive_timeout AS
> interactive_timeout, @@language AS language, @@license AS license,
> @@lower_case_table_names AS lower_case_table_names,
> @@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS
> net_write_timeout, @@performance_schema AS performance_schema,
> @@query_cache_size AS query_cache_size, @@query_cache_type AS
> query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS
> system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS
> transaction_isolation, @@wait_timeout AS wait_timeout
>
> Thanks for helping out ;)
> Adolfo
Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 3)
+1 (binding) Checked hash and compiled and ran tests. Thanks for being RM! -- Michael Mior mm...@apache.org Le mar. 15 mars 2022 à 23:36, Fan Liya a écrit : > Hi all, > > I have created a build for Apache Calcite 1.30.0, release > candidate 3. > > Thanks to everyone who has contributed to this release. > > You can read the release notes here: > > https://github.com/apache/calcite/blob/calcite-1.30.0-rc3/site/_docs/history.md > > The commit to be voted upon: > > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=cd5229b9a371e20de207231d55bcbddf1ff39ec2 > > Its hash is cd5229b9a371e20de207231d55bcbddf1ff39ec2 > > Tag: > https://github.com/apache/calcite/tree/calcite-1.30.0-rc3 > > The artifacts to be voted on are located here: > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc3 > (revision 53122) > > The hashes of the artifacts are as follows: > > 6c9f65bcc75a05c226dbb0a74495e76761d738f6923086e2943fface1b8864697dff6d0de7aa0cb9b06f4f6aebe1322b5ecb829428f84d08c51dfb5773034040 > *apache-calcite-1.30.0-src.tar.gz > > A staged Maven repository is available for review at: > > https://repository.apache.org/content/repositories/orgapachecalcite-1153/org/apache/calcite/ > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/COMMITTER_ID.asc > https://www.apache.org/dist/calcite/KEYS > > To create the jars and test Apache Calcite: "gradle build" > (requires an appropriate Gradle/JDK installation) > > Please vote on releasing this package as Apache Calcite 1.30.0. > > The vote is open for the next 72 hours and passes if a majority of at > least three +1 PMC votes are cast. > > [ ] +1 Release this package as Apache Calcite 1.30.0 > [ ] 0 I don't feel strongly about it, but I'm okay with the release > [ ] -1 Do not release this package because... > > Here is my vote: > > +1 (non-binding) > > Best, > Liya Fan >
Re: ARRAY_CONCAT does not work
Exactly. I understand the problem is not CHAR(1) vs CHAR(7), but the record type. That is the point I was trying to make. -- Michael Mior mm...@apache.org Le jeu. 10 févr. 2022 à 07:47, Dmitry Sysolyatin a écrit : > Michael, the problem is not because CHAR(1) and CHAR(7). Calcite can derive > common type in this case = CHAR(7) and all will work ok. > > The problem is that one type is [ ARRAY] and another [ > ARRAY]. I see two options for resolving this problem: > > 1. Allow casting scalar type to RecordType with one field. I described it > inside https://issues.apache.org/jira/browse/CALCITE-4999 > 2. Modify `ARRAY` function in the way that it will return ARRAY of scalar > if subquery is used as argument. Like Postgres does - > > https://www.postgresql.org/docs/14/sql-expressions.html#SQL-SYNTAX-ARRAY-CONSTRUCTORS > > On Thu, Feb 10, 2022 at 2:02 PM Michael Mior wrote: > > > The two types in your example are incompatible. One is an array of > CHAR(1). > > The other is an array of records, each with a single CHAR(7) field. > > -- > > Michael Mior > > mm...@apache.org > > > > > > Le ven. 4 févr. 2022 à 11:27, Dmitry Sysolyatin > > > a > > écrit : > > > > > So, the previous case started to work. But I faced with another issue. > > This > > > query does not work: > > > > > > ``` > > > "SELECT ARRAY_CONCAT(ARRAY['1', '2'], array(select 'toast.' || x from > > > unnest(ARRAY['1','2']) x))" > > > ``` > > > > > > ``` > > > java.lang.IllegalArgumentException: Cannot infer return type for > > > ARRAY_CONCAT; operand types: [CHAR(1) ARRAY, RecordType(CHAR(7) EXPR$0) > > > ARRAY] > > > ``` > > > > > > I created an issue - > https://issues.apache.org/jira/browse/CALCITE-4999 > > > and > > > PR - https://github.com/apache/calcite/pull/2712 > > > > > > > > > On Fri, Feb 4, 2022 at 2:51 PM Dmitry Sysolyatin < > > dm.sysolya...@gmail.com> > > > wrote: > > > > > > > Actually, > > > > > > `SqlParser.Config.DEFAULT.withConformance(SqlConformanceEnum.BIG_QUERY)` > > > > helped, thanks ! > > > > I will try to understand why it doesn't work with BABEL conformance > > > > > > > > On Fri, Feb 4, 2022 at 2:35 PM Dmitry Sysolyatin < > > > dm.sysolya...@gmail.com> > > > > wrote: > > > > > > > >> How SqlParser config can help? If I understood correctly, parser > > config > > > >> affects only parser.parse stage. But code fails at the validation > > phase > > > >> > > > >> Also, I would like to use ARRAY_CONCAT function with postgres > dialect. > > > My > > > >> parser config at the moment: > > > >> > > > >> ``` > > > >> val parserConfig: SqlParser.Config = > > > >> PostgresqlSqlDialect.DEFAULT > > > >> .configureParser(SqlParser > > > >> .config() > > > >> // It is needed in order to parse PG "!~" operator > > > >> .withParserFactory(SqlBabelParserImpl.FACTORY) > > > >> ) > > > >> .withConformance(SqlConformanceEnum.BABEL) > > > >> ``` > > > >> > > > >> On Fri, Feb 4, 2022 at 2:26 PM Thomas Rebele > > > > > > > > >> wrote: > > > >> > > > >>> Hello, > > > >>> > > > >>> SqlParserTest might help you. It has some checks related to > > BIG_QUERY. > > > >>> Maybe a > > > >>> > > > >>> > > > > > > .parserConfig(SqlParser.Config.DEFAULT.withConformance(SqlConformanceEnum.BIG_QUERY)) > > > >>> could fix the problem. > > > >>> > > > >>> Cordialement / Best Regards, > > > >>> *Thomas Rebele, PhD* | R Developer | Germany | www.tibco.com > > > >>> > > > >>> > > > >>> On Fri, Feb 4, 2022 at 12:57 PM Dmitry Sysolyatin < > > > >>> dm.sysolya...@gmail.com> > > > >>> wrote: > > > >>> > > > >>> > Hi! > > > >>> > I have a problem with ARRAY_CONCAT operator. > > > >>> > > > > >>> > The following code failed with the exception ''No match found for > > > >>> function > > > >>> > signature array_concat(, )". > > > >>> > ``` > > > >>> > val config = Frameworks.newConfigBuilder() > > > >>> > .parserConfig(parserConfig) > > > >>> > .defaultSchema(CalciteSchema.createRootSchema(false, > > > >>> false).plus()) > > > >>> > .operatorTable(SqlOperatorTables.chain( > > > >>> > SqlStdOperatorTable.instance(), > > > >>> > > > > SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable( > > > >>> > SqlLibrary.BIG_QUERY > > > >>> >))) > > > >>> > .programs(Programs.standard()) > > > >>> > .build() > > > >>> > val planner = Frameworks.getPlanner(config) > > > >>> > > > > >>> > val query = "SELECT ARRAY_CONCAT(ARRAY[1, 2], ARRAY[2, 3])" > > > >>> > val parsedSqlNode = planner.parse(query) > > > >>> > planner.validate(parsedSqlNode) > > > >>> > ``` > > > >>> > > > > >>> > Am I doing something wrong or it is a bug? > > > >>> > > > > >>> > > > >> > > > > > >
Re: Contributor role request
You just ask. GitHub implemented this a while ago for first-time contributors because of the potential for abuse. It has been approved. -- Michael Mior mm...@apache.org Le jeu. 10 févr. 2022 à 07:58, Roman Puchkovskiy < roman.puchkovs...@gmail.com> a écrit : > Thanks! > I've published a PR https://github.com/apache/calcite/pull/2718 , but > it seems that someone needs to 'approve running workflows' for the PR. > How does one obtain such an approval? Sorry if I missed some obvious > mention of it. > > чт, 10 февр. 2022 г. в 16:24, Michael Mior : > > > > Welcome! I've added you as a contributor. > > -- > > Michael Mior > > mm...@apache.org > > > > Le jeu. 10 févr. 2022 à 07:15, Roman Puchkovskiy < > > roman.puchkovs...@gmail.com> a écrit : > > > > > Hello! I'd like to contribute to the Calcite project, could you please > > > grant me the Contributor role so that I can assign a JIRA issue to > > > myself? My Apache JIRA login is 'rpuch'. > > > > > > Cheers, > > > Roman Puchkovskiy > > > >
Re: Contributor role request
Welcome! I've added you as a contributor. -- Michael Mior mm...@apache.org Le jeu. 10 févr. 2022 à 07:15, Roman Puchkovskiy < roman.puchkovs...@gmail.com> a écrit : > Hello! I'd like to contribute to the Calcite project, could you please > grant me the Contributor role so that I can assign a JIRA issue to > myself? My Apache JIRA login is 'rpuch'. > > Cheers, > Roman Puchkovskiy >
Re: Visualizing Calcite data in Apache Superset
You can connect to Calcite using JDBC. I'm not familiar with Superset, but a quick glance suggests that it does connect to other JDBC sources, so this seems like a possibility. -- Michael Mior mm...@apache.org Le dim. 30 janv. 2022 à 12:42, Gunnar Morling a écrit : > Hey all, > > I'm looking into ways of visualizing data provided via Calcite using Apache > Superset. For this, I think I'd have to expose some sort of endpoint which > Superset can connect to, potentially implementing an existing wire > protocol, such as the one of Postgres. That way, Superset could connect to > my Calcite-based application just as to any other Postgres application. Has > anyone given something like this a try? > > Thanks for any hints, > > --Gunnar >
Re: ARRAY_CONCAT does not work
The two types in your example are incompatible. One is an array of CHAR(1). The other is an array of records, each with a single CHAR(7) field. -- Michael Mior mm...@apache.org Le ven. 4 févr. 2022 à 11:27, Dmitry Sysolyatin a écrit : > So, the previous case started to work. But I faced with another issue. This > query does not work: > > ``` > "SELECT ARRAY_CONCAT(ARRAY['1', '2'], array(select 'toast.' || x from > unnest(ARRAY['1','2']) x))" > ``` > > ``` > java.lang.IllegalArgumentException: Cannot infer return type for > ARRAY_CONCAT; operand types: [CHAR(1) ARRAY, RecordType(CHAR(7) EXPR$0) > ARRAY] > ``` > > I created an issue - https://issues.apache.org/jira/browse/CALCITE-4999 > and > PR - https://github.com/apache/calcite/pull/2712 > > > On Fri, Feb 4, 2022 at 2:51 PM Dmitry Sysolyatin > wrote: > > > Actually, > > `SqlParser.Config.DEFAULT.withConformance(SqlConformanceEnum.BIG_QUERY)` > > helped, thanks ! > > I will try to understand why it doesn't work with BABEL conformance > > > > On Fri, Feb 4, 2022 at 2:35 PM Dmitry Sysolyatin < > dm.sysolya...@gmail.com> > > wrote: > > > >> How SqlParser config can help? If I understood correctly, parser config > >> affects only parser.parse stage. But code fails at the validation phase > >> > >> Also, I would like to use ARRAY_CONCAT function with postgres dialect. > My > >> parser config at the moment: > >> > >> ``` > >> val parserConfig: SqlParser.Config = > >> PostgresqlSqlDialect.DEFAULT > >> .configureParser(SqlParser > >> .config() > >> // It is needed in order to parse PG "!~" operator > >> .withParserFactory(SqlBabelParserImpl.FACTORY) > >> ) > >> .withConformance(SqlConformanceEnum.BABEL) > >> ``` > >> > >> On Fri, Feb 4, 2022 at 2:26 PM Thomas Rebele > > >> wrote: > >> > >>> Hello, > >>> > >>> SqlParserTest might help you. It has some checks related to BIG_QUERY. > >>> Maybe a > >>> > >>> > .parserConfig(SqlParser.Config.DEFAULT.withConformance(SqlConformanceEnum.BIG_QUERY)) > >>> could fix the problem. > >>> > >>> Cordialement / Best Regards, > >>> *Thomas Rebele, PhD* | R Developer | Germany | www.tibco.com > >>> > >>> > >>> On Fri, Feb 4, 2022 at 12:57 PM Dmitry Sysolyatin < > >>> dm.sysolya...@gmail.com> > >>> wrote: > >>> > >>> > Hi! > >>> > I have a problem with ARRAY_CONCAT operator. > >>> > > >>> > The following code failed with the exception ''No match found for > >>> function > >>> > signature array_concat(, )". > >>> > ``` > >>> > val config = Frameworks.newConfigBuilder() > >>> > .parserConfig(parserConfig) > >>> > .defaultSchema(CalciteSchema.createRootSchema(false, > >>> false).plus()) > >>> > .operatorTable(SqlOperatorTables.chain( > >>> > SqlStdOperatorTable.instance(), > >>> > > SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable( > >>> > SqlLibrary.BIG_QUERY > >>> >))) > >>> > .programs(Programs.standard()) > >>> > .build() > >>> > val planner = Frameworks.getPlanner(config) > >>> > > >>> > val query = "SELECT ARRAY_CONCAT(ARRAY[1, 2], ARRAY[2, 3])" > >>> > val parsedSqlNode = planner.parse(query) > >>> > planner.validate(parsedSqlNode) > >>> > ``` > >>> > > >>> > Am I doing something wrong or it is a bug? > >>> > > >>> > >> >
Re: Using Calcite with Python
Flight is definitely another consideration for the future. Personally I think it would be most interesting to integrate Flight with Avatica as an alternative transport. But it would certainly also be useful to allow the Arrow adapter to connect to any Flight endpoint. -- Michael Mior mm...@apache.org Le lun. 31 janv. 2022 à 10:00, Gavin Ray a écrit : > This is really interesting stuff you've done in the example notebooks > > Nicola & Michael, I wonder if you could benefit from the recently-released > Arrow Flight SQL? > > https://www.dremio.com/subsurface/arrow-flight-and-arrow-flight-sql-accelerating-data-movement/ > > I have asked Jacques about this a bit -- it's meant to be a standardization > for communicating SQL queries and metadata with Arrow. > I'm not intimately familiar with it, but it seems like it could be a good > base to build a Calcite backend for Arrow from? > > They have a pretty thorough Java example in the repository: > > https://github.com/apache/arrow/blob/968e6ea488c939c0e1f2bfe339a5a9ed1aed603e/java/flight/flight-sql/src/test/java/org/apache/arrow/flight/sql/example/FlightSqlExample.java#L169-L180 > > On Mon, Jan 31, 2022 at 8:47 AM Michael Mior wrote: > > > You may want to keep an eye on CALCITE-2040 ( > > https://issues.apache.org/jira/browse/CALCITE-2040). I have a student > who > > is working on a Calcite adapter for Apache Arrow. We're basically hung up > > waiting on the Arrow team to release a compatible JAR. This still won't > > fully solve your problem though as the first version of the adapter is > only > > capable of reading from Arrow files. However, the goal is eventually to > > allow passing a memory reference into the adapter so that it would be > > possible to make use of Arrow data which is constructed in-memory > > elsewhere. > > -- > > Michael Mior > > mm...@apache.org > > > > > > Le dim. 30 janv. 2022 à 17:36, Nicola Vitucci > a > > écrit : > > > > > Hi all, > > > > > > What would be the best way to use Calcite with Python? I've come up > with > > > two potential solutions: > > > > > > - using the jaydebeapi package, to connect via the JDBC driver directly > > > from a JVM created via jpype; > > > - using Apache Arrow via the pyarrow package, to connect in basically > the > > > same way but creating Arrow objects with JdbcToArrowUtils (and > optionally > > > converting them to Pandas). > > > > > > Although the former is more straightforward, the latter allows to > achieve > > > better performance (see [1] for instance) since it's exactly what Arrow > > is > > > for. I've created two Jupyter notebooks [2] showing each solution. What > > > would you recommend? Is there an even better approach? > > > > > > Thanks, > > > > > > Nicola > > > > > > [1] https://uwekorn.com/2020/12/30/fast-jdbc-revisited.html > > > [2] > > https://github.com/nvitucci/calcite-sparql/tree/v0.0.2/examples/python > > > > > >
Re: Using Calcite with Python
You may want to keep an eye on CALCITE-2040 ( https://issues.apache.org/jira/browse/CALCITE-2040). I have a student who is working on a Calcite adapter for Apache Arrow. We're basically hung up waiting on the Arrow team to release a compatible JAR. This still won't fully solve your problem though as the first version of the adapter is only capable of reading from Arrow files. However, the goal is eventually to allow passing a memory reference into the adapter so that it would be possible to make use of Arrow data which is constructed in-memory elsewhere. -- Michael Mior mm...@apache.org Le dim. 30 janv. 2022 à 17:36, Nicola Vitucci a écrit : > Hi all, > > What would be the best way to use Calcite with Python? I've come up with > two potential solutions: > > - using the jaydebeapi package, to connect via the JDBC driver directly > from a JVM created via jpype; > - using Apache Arrow via the pyarrow package, to connect in basically the > same way but creating Arrow objects with JdbcToArrowUtils (and optionally > converting them to Pandas). > > Although the former is more straightforward, the latter allows to achieve > better performance (see [1] for instance) since it's exactly what Arrow is > for. I've created two Jupyter notebooks [2] showing each solution. What > would you recommend? Is there an even better approach? > > Thanks, > > Nicola > > [1] https://uwekorn.com/2020/12/30/fast-jdbc-revisited.html > [2] https://github.com/nvitucci/calcite-sparql/tree/v0.0.2/examples/python >
Re: [DISCUSS] Refactoring tests
Thanks for doing this Julian! I haven't been able to do a detailed review, but from my skim of the PR, this looks like a nice improvement. I think it's always been a bit difficult for new Calcite developers to write good tests, especially for new modules. So anything that helps that experience is very welcome. -- Michael Mior mm...@apache.org Le mer. 17 nov. 2021 à 01:15, Vladimir Sitnikov a écrit : > Jacques>This sounds like it will mean we will need to make calcite-core > test artifacts available > > Test artifacts publication is yet another anti-pattern just like "base test > class". > This change has been discussed: > https://lists.apache.org/thread/fz96p94h016p11g777otqntjxg2oxgh1 > > If you want to depend on a class from tests, consider moving it to /testkit > module: > > https://github.com/apache/calcite/tree/0899e6c157632ba1c5369a942cfe2be15fb4ed9f/testkit > > Jacques>We should think about the rules around Kotlin > > What happens in calcite-core/tests stays in calcite-core/tests :) > > It is reasonable to assume that testkit module would have dependencies, > and testkit would provide API that is usable from Java and other JVM > languages. > > In that regard, Kotlin dependency in testkit is not much different from > Quidem or commons-lang3. > Consumers might use Quidem if it fits just like they could use Kotlin if it > fits. > > Vladimir >
Re: [DISCUSS] Draft board report for Jan 2022
Thanks Haisheng! Looks good to me. -- Michael Mior mm...@apache.org Le ven. 7 janv. 2022 à 12:32, Haisheng Yuan a écrit : > Attached below is a draft of this month's board report. I plan to submit it > on Jan 11. > Please let me know if you have additions or corrections. > > ## Description: > Apache Calcite is a highly customizable framework for parsing and planning > queries on data in a wide variety of formats. It allows database-like > access, > and in particular a SQL interface and advanced query optimization, for data > not > residing in a traditional database. > > Avatica is a sub-project within Calcite and provides a framework for > building > local and remote JDBC and ODBC database drivers. Avatica has an independent > release schedule and its own repository. > > ## Issues: > There are no issues requiring board attention. > > ## Membership Data: > Apache Calcite was founded 2015-10-21 (6 years ago) > There are currently 56 committers and 23 PMC members in this project. > The Committer-to-PMC ratio is roughly 7:3. > > Community changes, past quarter: > - No new PMC members. Last addition was Ruben Q L on 2020-08-09. > - Alessandro Solimando was added as committer on 2021-12-17. > - Xiong Duan was added as committer on 2021-10-18. > > ## Project Activity: > Calcite 1.28.0 was released on 2021-10-19, with new features including > the UNIQUE sub-query predicate, the MODE aggregate function, > PERCENTILE_CONT and PERCENTILE_DISC inverse distribution functions, > an Exasol dialect for the JDBC adapter, and improvements to > materialized view recognition. > > Calcite 1.29.0 was released on 2021-12-26, which upgrades log4j2 to > 2.17.0 to fix security vulnerabilities. > > Calcite Avatica 1.19.0 was released on 2021-10-11, which adds support > for BIT and NULL data types, fixes issues with values of type ARRAY. > > Calcite Avatica 1.20.0 was released on 2021-12-13, which upgrades Log4j2 to > version 2.15.0 (to address CVE-2021-44228), and makes the SPNEGO > protocol much more efficient. > > ## Community Health: > The overall activity in the community has increased slightly in the past > few months, specifically 20% more commits, 8% more closed PRs on GitHub. > > There are some discussions about the proposal of changing workflow, e.g. > github issues vs JIRAs, merging Avatica with Calcite, people argued with > different opinions, but overall the discussion is good for community > development and health. > > The number of non-committer (contributor) commits per month: > +-+-+-+ > |year |month| contributor_commits | > +-+-+-+ > | 2021| 10 | 14 | > | 2021| 11 | 2 | > | 2021| 12 | 8 | > +-+-+-+ > > The number of active reviewers per month: > +-+-+-+ > |year |month| active_reviewers | > +-+-+-+ > | 2021| 10 | 7 | > | 2021| 11 | 2 | > | 2021| 12 | 5 | > +-+-+-+ > > Top reviewers in the last 3 months: > +---+-+ > | committer | reviews | > +---+-+ > | Julian Hyde | 7 | > | Stamatis Zampetakis | 4 | > | NobiGo | 3 | > | Jesus Camacho Rodriguez | 3 | > | rubenada | 2 | > | chunwei <37774589+chunwei...@users.noreply.github.com> | 1 > | > | Haisheng Yuan | 1 | > | chunwei.lcw | 1 | > | Wang Yanlin <1989yanlinw...@163.com> | 1 | > | Jacques Nadeau | 1 | > +---+-+ > > Thanks, > Haisheng Yuan >
Re: Release notes preparation via release-drafter
+1 for automating the process as much as possible. Also +1 for waiting until we have a discussion about it. -- Michael Mior mm...@apache.org Le mer. 22 déc. 2021 à 17:57, Julian Hyde a écrit : > > -1 > > Please don’t commit anything just yet. > > Rather than adding tools, can we discuss what’s wrong and right about the > current process. > > > On Dec 22, 2021, at 3:36 AM, Vladimir Sitnikov > > wrote: > > > > Hi, > > > > I suggest we start using Release Drafter [1] for the preparation of the > > release notes. > > I'm going to submit and merge a PR soon, however, I would appreciate it if > > somebody else > > would do that (see [2] for a minimal commit) > > > > I am sure it would significantly increase the quality of the release notes > > and it would make releases easier to prepare. > > > > Jenkins team incepted release-drafter, and they use it a lot. > > > > > > Here's a sample configuration for Jenkins projects: > > > > https://github.com/jenkinsci/.github/blob/f1a34987f2919f039282a13b8741e3c463d18bc6/.github/release-drafter.yml > > > > > > Here are the results: > > https://github.com/jenkinsci/configuration-as-code-plugin/releases > > > > [1] https://github.com/release-drafter/release-drafter > > [2] > > https://github.com/vlsi/ksar/commit/ec210ffb7d0f91631908d01976c8bb4d5f7c4e0d > > > > Vladimir >
Re: Why not make the "test" namespace public, or move to a separate Maven module that main pulls in as a "testOnly" dependency?
Thanks Julian for CALCITE-4885! I've avoided trying to do anything with the notebook repository I created a while back since test artifacts are no longer published. It would be great to get that up to date with the latest Calcite version. -- Michael Mior mm...@apache.org Le mer. 22 déc. 2021 à 18:15, Julian Hyde a écrit : > > Gavin, > > I’m working on this in https://issues.apache.org/jira/browse/CALCITE-4885 > <https://issues.apache.org/jira/browse/CALCITE-4885>. Take a look at > FixtureTest [1] to get an idea of how parser, validator, sql-to-rel or rule > tests would look in your project. > > My one concern is that once we make our test infrastructure public (as we > have already started to with the new ’testkit’ module), people will expect > that we follow semantic versioning. But we won’t. Of course semantic > versioning is very nice if you are the consumer of a project, but it has a > high cost for the project maintainers. Therefore there will be no guarantees > of compatibility of the ‘testkit’ module from one release to the next. > > Julian > > [1] > https://github.com/julianhyde/calcite/blob/4885-test-fixtures/testkit/src/test/java/org/apache/calcite/test/FixtureTest.java > > > > On Dec 22, 2021, at 6:55 AM, Gavin Ray wrote: > > > > I am still learning the fundamentals of Calcite's API, and I've found that > > there are pre-made schemas > > and other utilities meant for testing inside of the Calcite repo. > > > > Unfortunately it seems like a lot of these objects are private, because > > they aren't meant to be a part of the public API. > > > > I've taken to copy-pasting entire files to be able to get tests setup in my > > own project: > > - > > https://github.com/GavinRay97/GraphQLCalcite/blob/master/src/main/kotlin/HrClusteredSchemaKotlin.kt > > > > I think it could be valuable to others who aren't developing their code in > > the Calcite tree > > to be able to use the work that's been put into setting up premade test > > items. > > > > Or maybe there is a better way, would appreciate any advice. > > > > Thank you =) > > Gavin Ray >
Re: [QUESTION] Calcite Learning Resources
It's out of date and can't currently be easily updated since we're not publishing test artifacts, but you might also find these notebooks useful. https://github.com/michaelmior/calcite-notebooks -- Michael Mior mm...@apache.org Le jeu. 23 déc. 2021 à 21:36, Jing Zhang a écrit : > > Hi Givre, > +1 on ZheHu's response. > There are many interesting talks about Calcite at website [1]. > Among them, there is a video [2] which is especially useful for new > engineers for Calcite. > In addition, there is a blog [3] which introduces traits in Calcite. > Hope this helps. > > [1] https://calcite.apache.org/community/#talks > [2] https://youtu.be/p1O3E33FIs8 > [3] https://www.querifylabs.com/blog/custom-traits-in-apache-calcite > > Best, > Jing Zhang > > Zhe Hu 于2021年12月23日周四 23:36写道： > > > Hi, Givre. > > You can find some very useful talks at > > http://calcite.apache.org/community/#talks. > > Moreover, the website > > https://calcite.apache.org/develop/#download-source-build-and-run-tests > > is a good start for starters, which I find very instructive at least. > > > > > > Best, > > ZheHu > > > > | > > > > > > | > > Replied Message > > | From | Charles Givre | > > | Date | 12/23/2021 23:06 | > > | To | dev | > > | Subject | [QUESTION] Calcite Learning Resources | > > Hello Calcite Team, > > I hope everyone is doing well. I had a question for the team. We are > > onboarding some new engineers for our product which uses Calcite for a > > variety of purposes. As the learning curve can be a little steep, I was > > wondering if there are any additional resources other than the docs for a > > new developer to get their heads wrapped around Calcite? Tutorials, > > presentations etc.? > > > > Thanks so much and Happy Holidays to all!! > > — C > > > >
Re: [ANNOUNCE] New committer: Alessandro Solimando
Congratulations Alessandro! -- Michael Mior mm...@apache.org Le sam. 18 déc. 2021 à 09:09, Stamatis Zampetakis a écrit : > > Apache Calcite's Project Management Committee (PMC) has invited Alessandro > Solimando to become a committer, and we are pleased to announce that he has > accepted. > > Alessandro has been in the community for quite some time now with his first > contribution dating back in 2018. Since then he pushed many fixes in both > Calcite and Avatica touching parts of the core engine, > the Cassandra adapter, and materialized view matching. Apart from code > contributions, Alessandro provides valuable help to the community by doing > reviews, answering questions in the devlist, and testing releases. > > Alessandro, welcome, thank you for your contributions, and we look forward > to your further interactions with the community! If you wish, please feel > free to tell us more about yourself and what you are working on. > > Stamatis (on behalf of the Apache Calcite PMC)
Re: [VOTE] Release apache-calcite-avatica-1.20.0 (release candidate 0) (second vote)
+1 (binding) Checked release notes and hashes, compiled and ran tests. Thanks Julian! -- Michael Mior mm...@apache.org Le dim. 12 déc. 2021 à 21:45, Julian Hyde a écrit : > > Hi all, > > Yesterday I started a vote for Apache Calcite Avatica 1.20.0, release > candidate 0, and then canceled it due to CALCITE-4935. (See > https://issues.apache.org/jira/browse/CALCITE-4935.) Note that the > subject of this email ends "(release candidate 0) (second vote)". > > The artifacts are the same, but we have new information, namely the > discussion about whether CALCITE-4935 is a show-stopper. My judgment, > as release manager, that it is not a show-stopper, and therefore > release candidate 0 is good. I have restarted the vote because I would > like to give you all a chance to weigh in. > > This vote does NOT carry forward votes from the previous thread. If > you voted yesterday, please vote again. > > The remainder of this email is the same as yesterday's email, except > "The vote is open for the next 18 hours". > > > You can read the release notes here: > https://gitbox.apache.org/repos/asf/calcite-avatica-site-preview.git/avatica/docs/history.html > > The commit to be voted upon: > https://gitbox.apache.org/repos/asf?p=calcite-avatica.git;a=commit;h=8f8377551e7b07dbc6119ecf196942bc99834452 > > Its hash is 8f8377551e7b07dbc6119ecf196942bc99834452 > > Tag: > https://gitbox.apache.org/repos/asf?p=calcite-avatica.git;a=tag;h=refs/tags/avatica-1.20.0-rc0 > > The artifacts to be voted on are located here: > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-avatica-1.20.0-rc0 > (revision 51334) > > The hashes of the artifacts are as follows: > 2de57fb1823237ec2d73f2e26059b6a7f5f51ec8f6dcfdcb2eb0b562cf4ef141c4a0a8c430610815e85b0b1c1c60245062cbb075c6dc72a44a4f543a6813bff2 > *apache-calcite-avatica-1.20.0-src.tar.gz > > A staged Maven repository is available for review at: > https://repository.apache.org/content/repositories/orgapachecalcite-1122/org/apache/calcite/ > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/jhyde.asc > https://www.apache.org/dist/calcite/KEYS > > To create the jars and test Apache Calcite Avatica: "gradle build > -Prelease -PskipSign". > > If you do not have a Java/Gradle environment available, you can run the tests > using docker. To do so, install docker and docker-compose, then run > "docker-compose run test" from the root of the directory. > > Please vote on releasing this package as Apache Calcite Avatica 1.20.0. > > The vote is open for the next 18 hours and passes if a majority of at > least three +1 PMC votes are cast. (The voting period is shorter than usual.) > > [ ] +1 Release this package as Apache Calcite Avatica 1.20.0 > [ ] 0 I don't feel strongly about it, but I'm okay with the release > [ ] -1 Do not release this package because... > > > Here is my vote: > > +1 (binding)
Re: calcite-core 1.28.0 now depends on kotlin-stdlib?
It can't really be fully automatic because sometimes we may want to
legitimately add a new dependency. I would be interested in thinking
about how we could make sure unwanted dependencies don't creep in.
Part of the problem is that the release manager may not know what
dependencies should or shouldn't be added. I see a few possible ways
around this:
1. Make sure that when a new dependency is added, a commit explicitly
mentions the new dependency in some format that is grepable. Any
dependency not found in this way should be discussed.
2. Include a list of new dependencies with the RC announcement, giving
others a chance to chime in.
3. Make some step of the release process fail if new dependencies are
added since the last release unless a specific flag is given. The
purpose of this flag would be to acknowledge that the new dependencies
are in fact, intended to be added.
--
Michael Mior
mm...@apache.org
Le sam. 4 déc. 2021 à 01:27, Jacques Nadeau a écrit :
>
> Any opinions on whether we should formalize into some kind of release
> verification step (manual or automatic)?
>
> On Thu, Dec 2, 2021 at 5:22 PM Julian Hyde wrote:
>
> > It’s quite easy to script. For example,
> >
> > git checkout calcite-1.27.0
> > ./gradlew dependencies > /tmp/d27.txt
> > git checkout calcite-1.28.0
> > ./gradlew dependencies > /tmp/d28.txt
> > diff /tmp/d2{7,8}.txt
> >
> > I posted the output at
> > https://gist.github.com/julianhyde/9ca5915bfb91494b7f91405ad15d698e <
> > https://gist.github.com/julianhyde/9ca5915bfb91494b7f91405ad15d698e>.
> >
> > Julian
> >
> >
> > > On Nov 30, 2021, at 3:40 PM, Jacques Nadeau wrote:
> > >
> > > Nice.
> > >
> > > Anyone know if there is a tool to fingerprint dependencies between
> > releases
> > > so we can avoid introducing new dependencies accidentally?
> > >
> > > On Tue, Nov 30, 2021 at 1:26 PM Stamatis Zampetakis
> > > wrote:
> > >
> > >> It seems that Vladimir fixed this already in [1].
> > >>
> > >> [1]
> > >>
> > >>
> > https://github.com/apache/calcite/commit/f3e2f041567e35e65464676d3171db3b5f2ddf9c
> > >>
> > >> On Tue, Nov 30, 2021 at 7:50 PM Jacques Nadeau
> > wrote:
> > >>
> > >>> Can you file a JIRA? We should address before 1.29.
> > >>>
> > >>> On Tue, Nov 30, 2021 at 6:40 AM Zoltan Farkas
> > >> > >>>>
> > >>> wrote:
> > >>>
> > >>>> According to the published pom it is a direct dependency:
> > >>>>
> > >>>
> > >>
> > https://repo1.maven.org/maven2/org/apache/calcite/calcite-core/1.28.0/calcite-core-1.28.0.pom
> > >>>> <
> > >>>>
> > >>>
> > >>
> > https://repo1.maven.org/maven2/org/apache/calcite/calcite-core/1.28.0/calcite-core-1.28.0.pom
> > >>>>>
> > >>>>
> > >>>>
> > >>>>> On Nov 29, 2021, at 11:41 PM, Vladimir Sitnikov <
> > >>>> sitnikov.vladi...@gmail.com> wrote:
> > >>>>>
> > >>>>>> since core/src/kotlin does not exist yet.
> > >>>>>
> > >>>>> I mean core/src/main/kotlin does not exist yet
> > >>>>>
> > >>>>> Vladimir
> > >>>>
> > >>>>
> > >>>
> > >>
> >
> >
Re: Contributor role request
I have added you as a contributor. Unfortunately I won't be able to review currently. -- Michael Mior mm...@apache.org Le mer. 1 déc. 2021 à 14:11, Konstantin Orlov a écrit : > > Hello, community! > > I've created a ticket [1] and have already prepared a PR, but I need a > contributor role to be able to assign this ticket to myself. Could you > please grant me this role? My JIRA username is korlov. Also could someone > please help with review? > > [1] https://issues.apache.org/jira/browse/CALCITE-4913 > > -- > Regards, > Konstantin Orlov
Re: [DISCUSS] JIRA vs GitHub Issues
+0 to migrate to GH issues I haven't found navigating JIRA + GitHub PRs to be a significant problem, but if others think it would help, I'm for it. One question when it comes to migration: are comments and everything transferred using the ASF-linked GH account? That is, my JIRA account is mmior, but my GitHub is michaelmior. Since I have this GH account linked to my ASF account, will my GH comments end up as coming from @michaelmior? -- Michael Mior mm...@apache.org Le dim. 28 nov. 2021 à 11:43, Vladimir Sitnikov a écrit : > > Hi, > > What do you think of moving to GitHub Issues for Calcite? > > Currently, my (Calcite) development workflow focuses on pull requests. > That is all contributions I see come via pull requests. > > At the same time, issues are hosted in JIRA, which creates friction: PR > merging requires changes to both GitHub and JIRA. > I guess it would be easier if issue management was in GitHub as well. > > Then, I believe, co-locating issues and PRs would make external > contributions easier. > The links between Issues and PRs would be easier to navigate. > > There's a possibility to enable GitHub discussions as well (see > https://github.com/apache/airflow/discussions ). > Co-locating Issues, and Discussions look promising. > > I think it is not that painful, however, GitHub has a limitation of 20 > collaborators (non-committers who want to assign, edit, close issues, and > pull requests): > https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-AssigningexternalcollaboratorswiththetriageroleonGitHub > > I have no strong opinion, however, I just realized having issues in GitHub > would ease friction. > > Any thoughts? > > Just to gather opinion: > -1..+1 keep using ASF JIRA for issue management > -1..+1 migrate to GitHub Issues > > Here's my vote: > +0.5 keep using ASF JIRA for issue management. It more-or-less works, > however, merging PRs and navigating from PR to issue is far from perfect. > +1 migrate to GitHub Issues. It would simplify navigation between issues > and PRs, and I believe it would reduce friction for external contributors. > In theory, GitHub Issues can be automated with Actions (e.g. labels). I'm > not sure if we need that, however, it might be useful. > > Vladimir
Re: DISCUSS: merge calcite-avatica and calcite repositories
My mistake. I was unaware of the --allow-unrelated-histories flag :) -- Michael Mior mm...@apache.org Le mer. 10 nov. 2021 à 07:34, Dmitry Sysolyatin a écrit : > > No. hashes shouldn't change. > You can try the following steps and check: > > git pull https://github.com/apache/calcite.git > cd calcite > git remote add avatica https://github.com/apache/calcite-avatica > git fetch avatica > git merge avatica/master --allow-unrelated-histories > git add . && git commit -m "test" > > If you check the history log you can see that hash for the commit > https://github.com/apache/calcite-avatica/commit/4cf769f8ee32bb3520604e4f3284e6f233f90276 > remains the same > > On Wed, Nov 10, 2021 at 2:20 PM Michael Mior wrote: > > > > Actually, it is possible to preserve the history of commits when you > > merge > > > two repos - > > > > You can keep the history, but the hashes will necessarily change > > because they are based on a different parent. So it will be possible > > to see the changes, but the hashes would reference the old Avatica > > repository which would no longer be active. > > > > -- > > Michael Mior > > mm...@apache.org > > > > Le mer. 10 nov. 2021 à 06:47, Dmitry Sysolyatin > > a écrit : > > > > > > Actually, it is possible to preserve the history of commits when you > > merge > > > two repos - > > > > > https://stackoverflow.com/questions/17371150/moving-git-repository-content-to-another-repository-preserving-history > > > > > > On Tue, Nov 9, 2021 at 11:24 PM Julian Hyde > > wrote: > > > > > > > Merging the repositories is unnecessary, and a massive waste of time. > > > > > > > > Any supposed “time savings” due to the the merged repositories will be > > > > eaten up by dozens of small issues that will crop up long after the > > merge. > > > > > > > > I was release manager for Calcite and Avatica this time around and it > > took > > > > me about a week, because the process had changed since last time I was > > > > release manager. And now we’re going to change the process again? > > > > Absolutely fucking insane. > > > > > > > > Also, we will lose the history of the Avatica source code. All of the > > > > commit hashes for bugs fixed long ago will change. > > > > > > > > At the time that we split the repositories, I was probably against the > > > > idea, because it was work. Now that we’ve split, I am strongly against > > > > recombining the repositories. > > > > > > > > Honestly, people, we have better things to do. > > > > > > > > Julian > > > > > > > > > > > > > On Nov 9, 2021, at 1:03 PM, Stamatis Zampetakis > > > > wrote: > > > > > > > > > > The two projects have more or less the same release cadence, they're > > > > > maintained by the same community, they're written in the same > > language, > > > > and > > > > > they use the same build system. > > > > > > > > > > Although I understand the reasons the project was split in the first > > > > place, > > > > > at the moment I tend to agree that having them in the same repo > > would be > > > > > more practical. > > > > > > > > > > Truth is the merge may require quite a bit of work so we should take > > this > > > > > into consideration. > > > > > > > > > > If people prefer keeping the projects as they are I am perfectly fine > > > > with > > > > > that as well. I am leaving the decision to those who feel strongly > > about > > > > > this change. > > > > > > > > > > Best, > > > > > Stamatis > > > > > > > > > > > > > > > > > > > > On Tue, Nov 9, 2021, 7:46 PM Vladimir Ozerov > > wrote: > > > > > > > > > >> +1 for a single repo. > > > > >> > > > > >> Вт, 9 нояб. 2021 г. в 19:22, Vladimir Sitnikov < > > > > >> sitnikov.vladi...@gmail.com > > > > >>> : > > > > >> > > > > >>> Michael> is not proposing to change the > > > > >>> Michael>structure of modules within both projects, merely to have > > the > > > > >> code > >
Re: DISCUSS: merge calcite-avatica and calcite repositories
> Actually, it is possible to preserve the history of commits when you merge > two repos - You can keep the history, but the hashes will necessarily change because they are based on a different parent. So it will be possible to see the changes, but the hashes would reference the old Avatica repository which would no longer be active. -- Michael Mior mm...@apache.org Le mer. 10 nov. 2021 à 06:47, Dmitry Sysolyatin a écrit : > > Actually, it is possible to preserve the history of commits when you merge > two repos - > https://stackoverflow.com/questions/17371150/moving-git-repository-content-to-another-repository-preserving-history > > On Tue, Nov 9, 2021 at 11:24 PM Julian Hyde wrote: > > > Merging the repositories is unnecessary, and a massive waste of time. > > > > Any supposed “time savings” due to the the merged repositories will be > > eaten up by dozens of small issues that will crop up long after the merge. > > > > I was release manager for Calcite and Avatica this time around and it took > > me about a week, because the process had changed since last time I was > > release manager. And now we’re going to change the process again? > > Absolutely fucking insane. > > > > Also, we will lose the history of the Avatica source code. All of the > > commit hashes for bugs fixed long ago will change. > > > > At the time that we split the repositories, I was probably against the > > idea, because it was work. Now that we’ve split, I am strongly against > > recombining the repositories. > > > > Honestly, people, we have better things to do. > > > > Julian > > > > > > > On Nov 9, 2021, at 1:03 PM, Stamatis Zampetakis > > wrote: > > > > > > The two projects have more or less the same release cadence, they're > > > maintained by the same community, they're written in the same language, > > and > > > they use the same build system. > > > > > > Although I understand the reasons the project was split in the first > > place, > > > at the moment I tend to agree that having them in the same repo would be > > > more practical. > > > > > > Truth is the merge may require quite a bit of work so we should take this > > > into consideration. > > > > > > If people prefer keeping the projects as they are I am perfectly fine > > with > > > that as well. I am leaving the decision to those who feel strongly about > > > this change. > > > > > > Best, > > > Stamatis > > > > > > > > > > > > On Tue, Nov 9, 2021, 7:46 PM Vladimir Ozerov wrote: > > > > > >> +1 for a single repo. > > >> > > >> Вт, 9 нояб. 2021 г. в 19:22, Vladimir Sitnikov < > > >> sitnikov.vladi...@gmail.com > > >>> : > > >> > > >>> Michael> is not proposing to change the > > >>> Michael>structure of modules within both projects, merely to have the > > >> code > > >>> for > > >>> Michael>both in a single repository. > > >>> > > >>> I propose to integrate them into a single build, and keep the set of > > the > > >>> published jars. > > >>> However, the modules and dependency structure could be kept as is. > > >>> > > >>> We might want to rename folders like > > >>> calcite-core > > >>> calcite-linq4j > > >>> .. > > >>> avatica-core > > >>> avatica-server > > >>> > > >>> However, I am not sure it is that important to discuss folder names > > now. > > >>> The idea is that as you "open Calcite in IDE, you see both Avatica and > > >>> Calcite modules" > > >>> > > >>> Michael>Is there any reason we couldn't have a separate release > > schedule > > >> if > > >>> Michael>both projects are in the same repository? > > >>> > > >>> A different schedule means Calcite must support at least two different > > >>> Avatica versions. > > >>> In other words, if we allow clients to update Avatica at their will, > > then > > >>> we should allow them building Calcite with different Avatica versions, > > >>> which implies Calcite test code should succeed for multiple different > > >>> Avatica vesions. > > >>> > > >>> That makes it harder to write tests: we have to execute tests with two &
Re: DISCUSS: merge calcite-avatica and calcite repositories
> There are many reasons to divide code into modules. Unless I'm misunderstanding, Vladimir is not proposing to change the structure of modules within both projects, merely to have the code for both in a single repository. > to allow separate release schedules Is there any reason we couldn't have a separate release schedule if both projects are in the same repository? However, it does seem true that if we keep a separate release schedule, the benefits of combining the repositories are less obvious. I don't really have a strong opinion one way or the other, but I think the decision of merging repositories and restructuring code or combing release schedules are two separate decisions. -- Michael Mior mm...@apache.org Le lun. 8 nov. 2021 à 19:43, Julian Hyde a écrit : > > There are many reasons to divide code into modules. To allow separate > communities, to allow separate release schedules, to reduce coupling, to make > it easier to contribute (because contributors don’t need to understand a > large code base), to increase adoption (because people perceive that they can > use component A without using component B). > > The last of these was particularly on my mind when we split Avatica from > Calcite. I was pleased to see, for example, Apache Phoenix using Avatica > successfully (including building an ODBC driver) even though their (separate) > attempt to adopt Calcite failed. > > Splitting code into modules makes it easier to continue to splitting. If > Avatica had remained part of Calcite, both written in Java and using the same > build system and release process, it’s less likely that Avatica-go would have > happened. > > I think the split between Avatica and Calcite repositories is working great. > > Julian > > > > On Nov 8, 2021, at 3:56 PM, Josh Elser wrote: > > > > These repositories are separate because they have separate release > > schedules. Those who were working on calcite.git typically do not want to > > be bothered with changes to calcite-avatica.git, and vice versa. Further, > > there are downstream users of Avatica directly (without Calcite) who would > > be burdened by waiting for a new Calcite release as opposed to a (much more > > simple) Avatica release. > > > > > Recently there was a PR that improves error messages in Avatica: > > > > If I am interpreting your analysis correctly, you are arguing for wanting a > > shorter cycle to "implementing" all parts of a change (both in Avatica and > > Calcite). The net amount of code you have to write doesn't change with how > > it works now compared to how you are suggesting it should work. > > > > If there are breaking changes in Avatica, they would need to be accounted > > for when Calcite is bumped to the next version of Avatica. > > > > In terms of wire compatibility, there has been no breaking wire compat > > change that wasn't due to a problem with the protocol itself (where we had > > to break it). As far as API compatibility goes, I do not believe we have > > ever _had_ to break API compatibility. The untracked runtime changes (like > > the one you cite) are a bigger smell to me in Calcite but that's a > > different conversation to be had. > > > > > * Avatica has fewer commits than Calcite, so having a separate > > > calcite-avatica repository does not help for segregating PR/issue/commit > > > queue > > > > I disagree with your assessment. Having a separate repository makes it > > extremely easy for me to watch Avatica commits/pull-requests which I am > > capable of reviewing vs. Calcite pull-requests which I am not comfortable > > reviewing. > > > > > * calcite-avatica-go seems to reside in its own repository, so I do not > > > see > > > why do we split Java implementations across calcite and calcite-avatica > > > repository > > > > I have no objections to combining these two repositories together. > > > > It seems to me that combining these repositories is one possible solution > > to address the friction of making changes across these two repositories, > > rather than starting from the root problem "How can we make co-dependent > > changes easier?" > > > > I would challenge us instead of complicating releases (which are already > > complicated): what should the API/compatibility surface of Avatica be, and > > what how can be make the gaps you've experienced better? Such a fix will go > > a long way for downstream users of Avatica, too. > > > > - Josh > > > > On 11/8/21 2:39 PM, Vladimir Sitnikov wrote: > >> Hi, > >> Currently,
Re: Federated Query
To run federated queries, you simply need to define each data source in your model.json. You can see examples of model.json files for various adapters in the test code for those adapters. For example: File - https://github.com/apache/calcite/blob/master/file/src/test/resources/model.json Cassandra - https://github.com/apache/calcite/blob/master/cassandra/src/test/resources/model.json You can add as many schemas as you want from different adapters into the "schemas" array in model.json and all those schemas will be available for you to write queries including joins across schemas. -- Michael Mior mm...@apache.org Le lun. 1 nov. 2021 à 15:38, Yogendra Sharma a écrit : > > Hi Team, > > I am exploring Apache Calcite to run federated queries on three different > databases. I could not find a working example anywhere on internet. > > Can you guys confirm if it is possible and if yes, is there an example that > you can point me to? > > Thanks, > Yogendra >
Re: Job postings on the dev list ?
Those guidelines sound good to me. Somewhat off-topic, but I think your characterization of recruiting as a zero sum game misses out on the candidate. Presumably a candidate takes a new position because they benefit in some way. That said, if someone really wants to find good candidates, having an engineer reach out on a dev mailing list seems more appropriate than getting a recruiter to do it. -- Michael Mior mm...@apache.org Le ven. 29 oct. 2021 à 13:48, Julian Hyde a écrit : > > I think Apache has some kind of policy on this, but I couldn’t find a link. > > It’s a difficult question. If a company making a big investment in Calcite > (say rewriting their engine’s query optimizer) then it is good for Calcite if > someone is able to get a full-time job working on it. But we don’t want the > lists to fill up with spam job postings where SQL is one of twenty skills and > the person will probably end up writing JavaScript. > > There's a continuum between those points, I can’t find a simple way to draw a > line between “good” and “bad". Here are some guidelines that might work: > > * Jobs must be primarily working on Calcite > * We prefer posts by people who have merit in our community > * Absolutely no posts by recruiters > > In my opinion, software recruiting is often a zero-sum game (when company X > lures an employee from company Y, X wins and Y loses, and the recruiters take > a slice). I don’t want to make that merry-go-round spin any faster. A “win” > for Calcite would be when somebody moves from a job where they use Calcite > part-time to a higher paying job where they use Calcite full-time. Our > mailing list should enable those kinds of wins. > > Julian > > > > On Oct 29, 2021, at 8:28 AM, Andrei Sereda wrote: > > > > Hello, > > > > I wanted to ask what members of this list think about receiving / posting > > job opportunities (related to query optimisation / database engine) to > > calcite-dev ? > > > > Is it an appropriate usage of the dev@ list ? Should one use a different > > channel ? > > > > Thanks, > > Andrei. >
Re: [DISCUSS] Draft board report for Oct 2021
LGTM, although we have new committers to add now. -- Michael Mior mm...@apache.org Le mer. 6 oct. 2021 à 03:15, Haisheng Yuan a écrit : > > Attached below is a draft of this month's board report. I plan to submit it > on Oct 12. Please let me know if you have additions or corrections. > > ## Description: > The mission of Calcite is the creation and maintenance of software related to > Dynamic data management framework > > ## Issues: > There are no issues requiring board attention. > > ## Membership Data: > Apache Calcite was founded 2015-10-21 (6 years ago) > There are currently 53 committers and 23 PMC members in this project. > The Committer-to-PMC ratio is roughly 7:3. > > Community changes, past quarter: > - No new PMC members. Last addition was Ruben Q L on 2020-08-09. > - No new committers. Last addition was Vladimir Ozerov on 2021-06-23. > > ## Project Activity: > No new version was released. > > At ApacheCon 2021, September 22, 2021, Vladimir Ozerov gave a talk on building > modern SQL query optimizers with Apache Calcite. > > At Strange Loop 2021, September 30, 2021, Julian Hyde gave a talk on Morel, > a functional query language. > > ## Community Health: > The overall activity in the community has increased slightly in the past > few months, specifically 47% more opened PRs, 25% more closed PRs on GitHub. > But the number of active reviewers still remains small. > > Thanks, > Haisheng Yuan
Re: [ANNOUNCE] New committer: Zhaohui Xu
Welcome Zhaohui! -- Michael Mior mm...@apache.org Le mer. 6 oct. 2021 à 16:48, Stamatis Zampetakis a écrit : > > Apache Calcite's Project Management Committee (PMC) has invited Zhaohui Xu > to > become a committer, and we are pleased to announce that they have accepted. > > Numbers speak for themselves and Zhaohui has over 30 commits already in > master > and more than 20 open pull requests waiting to get in. Great record so far > including > (but not limited to) improvements and fixes in the view based > rewriting modules, > JSON serialization, metadata, and field trimming. > > Zhaohui, welcome, thank you for your contributions, and we look forward to > your > further interactions with the community! If you wish, please feel free to > tell > us more about yourself and what you are working on. > > Stamatis (on behalf of the Apache Calcite PMC)
Re: do some code contribution on calcite
I have added you as a contributor. Welcome to Calcite! -- Michael Mior mm...@apache.org Le mar. 28 sept. 2021 à 16:17, zxyoung a écrit : > > I want to fix some bug in calcite jira and assign some issue to myself. > My jira username is xuyangzhong. > Can anyone help me? > Thanks very much :)
Re: [DISCUSS] Remove contributors name from commit summary
+1 One of the reasons it was there was to make generating the changelog easier. However, this could be easily scripted to pull out the contributors name if needed. -- Michael Mior mm...@apache.org Le jeu. 23 sept. 2021 à 05:23, Stamatis Zampetakis a écrit : > > Hi all, > > Currently we are supposed to append the contributors name to the commit > summary when they are not committers of the project. The main reason for > doing this if I am not mistaken is to give some credit to those people. > > I did like this practice in the beginning but I think it adds some small > overhead to all parties involved (committers and contributors). > > The contributor quite often forgets to include the name in the end so the > burden to find and append the name goes to the committer. > > In various cases, I've seen PRs ready to merge which were actually missing > the name at the end. What usually happens afterwards is one of the > following: > * the committer merges the PR without amending the name; > * the committer rebases the PR, amends the commit, and merges it; > * the committer asks the contributor to change the commit message; > > I would prefer it if we could avoid this overhead by changing the commit > guidelines to not append the contributors name at the end. > > GitHub does a great job giving credit to contributors. Moreover in most > cases the name appears in the log under the author tag so it is very easy > to exploit if we want to extract information and statistics. > > What do you think? > > Best, > Stamatis
Re: [DISCUSS] Pronouns
+1 Thanks Julian! I'll add that in many cases, I see the possessive pronoun (e.g. his) also included but I don't know enough about the reasons for this to feel strongly either way. I've added my pronouns as well. -- Michael Mior mm...@apache.org Le mar. 21 sept. 2021 à 14:16, Julian Hyde a écrit : > > I've updated my entry in the data file of contributors [1] (from which > the table of committers and PMC members is generated on the site [2]) > adding my pronouns. I don't think that we should compel or expect > people to provide their pronouns. But I want to provide a place for > people to do so, if they wish. > > I hope we can all agree that our gender identity is at least as > important a piece of information as our employer. :) > > I plan to update the table of committers and PMC members so that > people's pronouns appear next to their name, if specified. > > Julian > > [1] > https://github.com/apache/calcite/commit/98d254d9c790669a96e7359cb449117254eaad44 > > [2] https://calcite.apache.org/community/#project-members
Re: Cassandra with Calcite integration problem
Unfortunately, I don't have a lot of advice here. The integration was designed to work, but not evaluated for performance. If you're able to do some profiling and understand where the slowdown lies, we may be able to help. -- Michael Mior mm...@apache.org Le lun. 14 juin 2021 à 11:39, santanu mohanty a écrit : > > Hello Team > We have integrated Cassandra with calcite with your given example and > getting below problem > > compare to run in spark thrift server it is petty slow > > can you suggest good git code example or tunning parameter fro driver > configuration? > we are planning to integrate in Production but not sure about performance. > > Not getting much more resources to experiment more. > Thank > santanu
Re: Minified javascript in source releases
+1 for removing these entirely. I don't believe they are currently necessary. -- Michael Mior mm...@apache.org Le mer. 2 juin 2021 à 06:13, Stamatis Zampetakis a écrit : > > Hi all, > > In the recent votes for Calcite [1] and Avatica [2] there were some -1 > votes regarding minified javascript files present in the source releases. > > The concerns are raised for the following files: > site/js/html5shiv.min.js > site/js/respond.min.js > > From discussions in other lists it seems that there are Apache source > releases including minified files so I don't think we are violating any > policy by including them but there are deferring opinions on this topic > [3,4]. > > To avoid bringing back the discussion to if minified javascript should be > present or not I would like to propose one of the following alternatives: > > 1. It seems that both of the aforementioned scripts [5,6] are used for > compatibility reasons with old browsers. I think most of the people are not > using these browsers anymore so I would suggest removing them altogether > from the code. > > 2. If for some reason we need to keep them then we could directly use the > non minified version. The difference of minified vs unminified for both > files is ~6KB so I doubt it will have any real impact with the internet > connections that we use nowadays. > > What do you think? > > Best, > Stamatis > > [1] > https://lists.apache.org/thread.html/r049cf5b953eb5824ba37df21094dd7c1e92146d659db4735ad121593%40%3Cdev.calcite.apache.org%3E > [2] > https://lists.apache.org/thread.html/r074720fc062bde09a7e69dc77f77754c7fd4c22f1357514f6b2196d0%40%3Cdev.calcite.apache.org%3E > [3] > https://lists.apache.org/thread.html/rcddc30dd1f0c7f20709e09de7202d4d6885d6235a7fce1c1ab46e4ed%40%3Clegal-discuss.apache.org%3E > [4] > http://mail-archives.apache.org/mod_mbox/incubator-general/201805.mbox/%3c89a47c0b-e74e-4571-b282-dbe505291...@classsoftware.com%3e > [5] https://github.com/scottjehl/Respond > [6] https://github.com/aFarkas/html5shiv
Fwd: [NOTICE] Git web site publishing to be done via .asf.yaml only as of July 1st
It seems like Calcite is affected by this change. I believe all that is necessary is to add .asf.yml to the calcite-site repository with the following contents: publish: whoami: master We have plenty of time to make the change, so if there are no objections or concerns, I'll commit this in the next week or two. -- Michael Mior mm...@apache.org -- Forwarded message - De : Daniel Gruno Date: lun. 31 mai 2021 à 09:41 Subject: [NOTICE] Git web site publishing to be done via .asf.yaml only as of July 1st To: Users TL;DR: if your project web site is kept in subversion, disregard this email please. If your project web site is using git, and you have not deployed it via .asf.yaml, you MUST switch before July 1st or risk your web site goes stale. Dear Apache projects, In order to simplify our web site publishing services and improve self-serve for projects and stability of deployments, we will be turning off the old 'gitwcsub' method of publishing git web sites. As of this moment, this involves 120 web sites. All web sites should switch to our self-serve method of publishing via the .asf.yaml meta-file. We aim to turn off gitwcsub around July 1st. ## How to publish via .asf.yaml: Publishing via .asf.yaml is described at: https://s.apache.org/asfyamlpublishing You can also see an example .asf.yaml with publishing and staging profiles for our own infra web site at: https://github.com/apache/infrastructure-website/blob/asf-site/.asf.yaml In short, one puts a file called .asf.yaml into the branch that needs to be published as the project's web site, with the following two-line content, in this case assuming the published branch is 'asf-site': publish: whoami: asf-site It is important to note that the .asf.yaml file MUST be present at the root of the file system in the branch you wish to publish. The 'whoami' parameter acts as a guard, ensure that only the intended branch is used for publishing. ## Is my project affected by this? The quickest way to check if you need to switch to a .asf.yaml approach is to check out site source page at https://infra-reports.apache.org/site-source/ - if your site is listed in yellow, you will need to switch. This page will also tell you which branch you are currently publishing as your web site. This is (should be) the branch that you must add a .asf.yaml meta file to. The web site source list updates every hour. If your project site appears in green, you are already using .asf.yaml for publishing and do not need to make any changes. ## What happens if we miss the deadline? If you miss the deadline, don't fret. Your site will of course still remain online as is, but new updates will not appear till you create/edit the .asf.yaml and set up publishing. ## Who do we contact if we have questions? Please contact us at us...@infra.apache.org if you have any additional questions. With regards, Daniel on behalf of ASF Infra.
Re: Question about Calcite History
I should add that while the project entered the incubator in 2014, I believe the development started a couple years earlier, but I'll leave Julian Hyde (the original author of Optiq) to answer. -- Michael Mior mm...@apache.org Le jeu. 6 mai 2021 à 09:47, Michael Mior a écrit : > > 1. The project entered the Apache incubator in 2014. > 2. Optiq was the original name of the Calcite project. The name was > changed, but the project is the same and the current code base is > derived from the code of Optiq. > 3. Calcite development is volunteer-driven so what features will be > implemented depends entirely on what volunteers choose to work on. > Some areas that have seen a lot of work over the past couple years are > JSON, geospatial, and streaming query support although there are many > other interesting things upcoming. > > This is by no means an exhaustive list, but you can see new features > which have been logged in our issue tracker below: > https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20issuetype%20%3D%20%22New%20Feature%22%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC > -- > Michael Mior > mm...@apache.org > > Le jeu. 6 mai 2021 à 00:23, Junwen Liu a écrit : > > > > Hi Mr. or Ms. : > > I'm an engineer who is using Calcite as our Optimizer in our > > project.Calcite is an amazing framework that can meet our demands. But I > > want to know mostly is the history of Calcite, we also want to create an > > open-source project. So I want to know these things about calcite: > > 1. When did you start this project? > > 2. What the difference between optiq and Calcite? > > 3. Where Calcite will go, what features Calcite will support?
Re: Question about Calcite History
1. The project entered the Apache incubator in 2014. 2. Optiq was the original name of the Calcite project. The name was changed, but the project is the same and the current code base is derived from the code of Optiq. 3. Calcite development is volunteer-driven so what features will be implemented depends entirely on what volunteers choose to work on. Some areas that have seen a lot of work over the past couple years are JSON, geospatial, and streaming query support although there are many other interesting things upcoming. This is by no means an exhaustive list, but you can see new features which have been logged in our issue tracker below: https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20issuetype%20%3D%20%22New%20Feature%22%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC -- Michael Mior mm...@apache.org Le jeu. 6 mai 2021 à 00:23, Junwen Liu a écrit : > > Hi Mr. or Ms. : > I'm an engineer who is using Calcite as our Optimizer in our > project.Calcite is an amazing framework that can meet our demands. But I > want to know mostly is the history of Calcite, we also want to create an > open-source project. So I want to know these things about calcite: > 1. When did you start this project? > 2. What the difference between optiq and Calcite? > 3. Where Calcite will go, what features Calcite will support?
Re: Support for JSON path expressions in select queries.
Just to clarify that Calcite does not explicitly attempt to provide any JSONPath support. The examples that you show just happen to work based on standard SQL syntax. -- Michael Mior mm...@apache.org Le lun. 3 mai 2021 à 14:25, Amrish Lal a écrit : > > Hello, > > Calcite is used as the parsing layer of a database I am working on. I > noticed that calcite support dot notation and array subscripts in > identifiers as in: > > SELECT json_column.person.name.email[5] FROM table > > This allows for writing rudimentary Json Path expressions. However, some > support is still missing. For example, the following queries will give > parsing errors. > > SELECT json_column.person.name.email[*] FROM table > SELECT json_column.person.name.email[:2] FROM table > SELECT json_column.person.name.email[1,3] FROM table > SELECT json_column..email FROM table > SELECT json_column.person..name FROM table > etc... (more json path examples here > <https://support.smartbear.com/alertsite/docs/monitors/api/endpoint/jsonpath.html> > ) > > From what I can see, this shouldn't be very complicated to add and will > mainly require accepting a wider range of characters in SELECT list > expression values. > > We only need basic json path expression support for now (dot operator and > array subscript operator) which calcite seems to already support, but would > like to add further json path expression support in future. I am wondering > if Calcite is open to further supporting json path expressions in SELECT > and WHERE clause expression list?
Re: Apache Arrow adapter
Yes, it was a bit of a challenge to get working in the Linux and macOS development environments we've been using. This is why I temporarily checked in the jar, but this should certainly be removed before the PR is merged. -- Michael Mior mm...@apache.org Le sam. 10 avr. 2021 à 17:05, Julian Hyde a écrit : > > I've been trying to switch over to use the official Apache Arrow > Gandiva 3.0.0 jar at Maven central. (Which means we can remove the > 3.0.0-SNAPSHOT.jar that you had checked into arrow/libs.) That jar is > built for macOS, and is a little more tricky to get running than the > previous jar, which was built for Linux. I'll post to > https://issues.apache.org/jira/browse/ARROW-11135 as I discover > things. > > (Makes me glad we don't have any C++ code in Calcite. Making artifacts > that work on multiple operating systems seems to be really > challenging.) > > Julian > > On Sat, Apr 10, 2021 at 6:31 AM Michael Mior wrote: > > > > Thanks Julian! I really appreciate the help. I think beta would be > > accurate here but it would be great to have this pushed so people can > > start trying it out. > > > > -- > > Michael Mior > > mm...@apache.org > > > > Le ven. 9 avr. 2021 à 20:37, Julian Hyde a écrit : > > > > > > Yes, thanks to Michael and Karshit for their great work. > > > > > > I am reviewing now, and doing some fix up (e.g. lint, repositories) so > > > that we could get it into master as a "beta" component. I'll add > > > updates in https://issues.apache.org/jira/browse/CALCITE-2040. > > > > > > On Wed, Apr 7, 2021 at 9:37 PM Fan Liya wrote: > > > > > > > > Hi Michael, > > > > > > > > Thanks for sharing the great work. > > > > I believe it is important work for both communities. > > > > > > > > Best, > > > > Liya Fan > > > > > > > > > > > > On Thu, Apr 8, 2021 at 3:30 AM Michael Mior wrote: > > > > > > > > > Hi all, > > > > > > > > > > I wanted to share some work one of my (now former) students, Karshit > > > > > Shah, has done with integrating Apache Arrow into Calcite. Karshit has > > > > > written an Arrow adapter that's able to perform filtering and > > > > > projections natively on Arrow data using Gandiva so these expressions > > > > > can be JITed using LLVM. The pull request[0] needs some cleanup, but > > > > > the code is in relatively good shape. > > > > > > > > > > Right now, the adapter only reads from files, but I think there are a > > > > > number of exciting extensions to this that are possible. For example, > > > > > Arrow has a client-server framework Flight which could be connected > > > > > with Calcite, perhaps via Avatica. (Andy Grove was doing some work on > > > > > this last year[1] although I'm not sure of the progress.) > > > > > > > > > > The biggest blocker on this is actually not the Calcite code, but the > > > > > availability of a suitably built Arrow dependency with Gandiva along > > > > > with the appropriate CI configuration. I opened a JIRA on the Arrow > > > > > project with some more details[2]. > > > > > > > > > > I'd love some thoughts on the approach and some help in pushing this > > > > > over the finish line. > > > > > > > > > > [0] https://github.com/apache/calcite/pull/2133 > > > > > [1] > > > > > https://mail-archives.apache.org/mod_mbox/calcite-dev/202002.mbox/%3cCAJEf=X5xvXLQpJkX_VjJk=TnNRwT52v0=p28sczmid1tyce...@mail.gmail.com%3e > > > > > [2] https://issues.apache.org/jira/browse/ARROW-11135 > > > > > -- > > > > > Michael Mior > > > > > mm...@apache.org > > > > >
Re: Apache Arrow adapter
Thanks Julian! I really appreciate the help. I think beta would be accurate here but it would be great to have this pushed so people can start trying it out. -- Michael Mior mm...@apache.org Le ven. 9 avr. 2021 à 20:37, Julian Hyde a écrit : > > Yes, thanks to Michael and Karshit for their great work. > > I am reviewing now, and doing some fix up (e.g. lint, repositories) so > that we could get it into master as a "beta" component. I'll add > updates in https://issues.apache.org/jira/browse/CALCITE-2040. > > On Wed, Apr 7, 2021 at 9:37 PM Fan Liya wrote: > > > > Hi Michael, > > > > Thanks for sharing the great work. > > I believe it is important work for both communities. > > > > Best, > > Liya Fan > > > > > > On Thu, Apr 8, 2021 at 3:30 AM Michael Mior wrote: > > > > > Hi all, > > > > > > I wanted to share some work one of my (now former) students, Karshit > > > Shah, has done with integrating Apache Arrow into Calcite. Karshit has > > > written an Arrow adapter that's able to perform filtering and > > > projections natively on Arrow data using Gandiva so these expressions > > > can be JITed using LLVM. The pull request[0] needs some cleanup, but > > > the code is in relatively good shape. > > > > > > Right now, the adapter only reads from files, but I think there are a > > > number of exciting extensions to this that are possible. For example, > > > Arrow has a client-server framework Flight which could be connected > > > with Calcite, perhaps via Avatica. (Andy Grove was doing some work on > > > this last year[1] although I'm not sure of the progress.) > > > > > > The biggest blocker on this is actually not the Calcite code, but the > > > availability of a suitably built Arrow dependency with Gandiva along > > > with the appropriate CI configuration. I opened a JIRA on the Arrow > > > project with some more details[2]. > > > > > > I'd love some thoughts on the approach and some help in pushing this > > > over the finish line. > > > > > > [0] https://github.com/apache/calcite/pull/2133 > > > [1] > > > https://mail-archives.apache.org/mod_mbox/calcite-dev/202002.mbox/%3cCAJEf=X5xvXLQpJkX_VjJk=TnNRwT52v0=p28sczmid1tyce...@mail.gmail.com%3e > > > [2] https://issues.apache.org/jira/browse/ARROW-11135 > > > -- > > > Michael Mior > > > mm...@apache.org > > >
Apache Arrow adapter
Hi all, I wanted to share some work one of my (now former) students, Karshit Shah, has done with integrating Apache Arrow into Calcite. Karshit has written an Arrow adapter that's able to perform filtering and projections natively on Arrow data using Gandiva so these expressions can be JITed using LLVM. The pull request[0] needs some cleanup, but the code is in relatively good shape. Right now, the adapter only reads from files, but I think there are a number of exciting extensions to this that are possible. For example, Arrow has a client-server framework Flight which could be connected with Calcite, perhaps via Avatica. (Andy Grove was doing some work on this last year[1] although I'm not sure of the progress.) The biggest blocker on this is actually not the Calcite code, but the availability of a suitably built Arrow dependency with Gandiva along with the appropriate CI configuration. I opened a JIRA on the Arrow project with some more details[2]. I'd love some thoughts on the approach and some help in pushing this over the finish line. [0] https://github.com/apache/calcite/pull/2133 [1] https://mail-archives.apache.org/mod_mbox/calcite-dev/202002.mbox/%3cCAJEf=X5xvXLQpJkX_VjJk=TnNRwT52v0=p28sczmid1tyce...@mail.gmail.com%3e [2] https://issues.apache.org/jira/browse/ARROW-11135 -- Michael Mior mm...@apache.org
Re: Rewrite SQL query or use an adapter?
In general I would say if you are trying to connect a new database engine to Calcite, an adapter is probably the way to go. You can use Calcite's existing JDBC driver and then just write planner rules to push down whatever is necessary. -- Michael Mior mm...@apache.org Le lun. 8 mars 2021 à 06:24, Nicola Vitucci a écrit : > > Dear all, > > I am exploring Calcite and, as a study use case, I am trying to create a JDBC > interface to a non-relational database that has its own joins, aggregates, > etc. I have been playing with a TranslatableTable and some planner rules, but > given that basically all queries including joins would be pushed down to the > underlying engine, I am wondering whether an adapter (like the core JDBC) > would still be the preferred route or a different approach (custom JDBC > driver + query rewriting engine maybe?) is advised. What would you recommend? > > Thanks, > > Nicola >
Re: [ANNOUNCE] New committer: Liya Fan
Congratulations and welcome Liya! -- Michael Mior mm...@apache.org Le mer. 10 févr. 2021 à 05:23, Francis Chuang a écrit : > > Congrats Liya and well deserved! > > Francis > > On 10/02/2021 8:53 pm, Stamatis Zampetakis wrote: > > Apache Calcite's Project Management Committee (PMC) has invited Liya Fan to > > become a committer, and we are pleased to announce that they have accepted. > > > > Liya has made a lot of contributions to the project, enhancing optimizer > > rules, > > improving the efficiency of graph traversal algorithms, as well as fixing > > various > > bugs in the project. > > > > Liya, welcome, thank you for your contributions, and we look forward to your > > further interactions with the community! If you wish, please feel free to > > tell > > us more about yourself and what you are working on. > > > > Stamatis (on behalf of the Apache Calcite PMC) > >
Re: [DISCUSS] Apache Calcite Online Meetup January 2021
With appropriate permissions, mic interruptions are not an issue as the host can prevent others from unmuting themselves. I don't think the quality of the recording is affected either except that I'm not sure about having the face superimposed on slides in regular Zoom meeting format. (This could be a positive or negative depending on preferences.) I think switching to a regular Zoom meeting for general discussion would be great, but I don't believe it's possible to do so with the same link. That is, I believe everyone would have to disconnect and rejoin a different meeting, which would work, but interrupt the flow. -- Michael Mior mm...@apache.org Le sam. 23 janv. 2021 à 18:32, Julian Hyde a écrit : > > Michael, > > From my perspective (as a presenter) the webinar worked well. The > transitions between presenters were smooth, capacity seemed to be > good, there were no interruptions from attendees' microphones, the Q > applet delivered questions efficiently, and the recordings are high > quality (good sound, and presenter's face superimposed over the the > slides). Is it possible that regular zoom would have been inferior in > one of those factors? > > I wonder, would your concerns be met if we did the presentations in a > webinar and switched to a regular zoom for the general discussion? > > Julian > > On Fri, Jan 22, 2021 at 7:48 AM Michael Mior wrote: > > > > Thanks for organizing Stamatis! I enjoyed hearing what others are > > working on. However, if we do this again I wonder if we could use a > > regular Zoom meeting instead of a webinar. I think it might be easier > > to facilitate discussion that way. > > -- > > Michael Mior > > mm...@apache.org > > > > Le ven. 22 janv. 2021 à 06:28, Stamatis Zampetakis a > > écrit : > > > > > > Hi everybody, > > > > > > Thanks again for attending the meetup and of course big thanks to the > > > speakers for the great presentations and thoughtful discussion. > > > > > > For those interested the recordings are now available on youtube: > > > https://www.youtube.com/playlist?list=PLUp_iibbIwFF_5xeLRgz4zU33mFRbS5Y6 > > > > > > Best, > > > Stamatis > > > > > > On Fri, Jan 15, 2021 at 4:07 AM Chunwei Lei > > > wrote: > > > > > > > Ignore my previous email. I found the link[1] after I read all the > > > > replies. > > > > > > > > Besides, I have sent the link to my colleagues. > > > > > > > > The online meetup is a little late in UTC+8. Would the meetup be > > > > recorded? > > > > > > > > > > > > [1] https://www.meetup.com/Apache-Calcite/events/275461117/ > > > > > > > > Best, > > > > Chunwei > > > > > > > > > > > > On Fri, Jan 15, 2021 at 10:57 AM Chunwei Lei > > > > > > > > wrote: > > > > > > > >> Hi, Stamatis. > > > >> > > > >> How can I submit my PGP key? (It's my first time to attend an online > > > >> meetup~~) > > > >> > > > >> > > > >> > > > >> Best, > > > >> Chunwei > > > >> > > > >> > > > >> -- > > > >> 发件人：Stamatis Zampetakis > > > >> 发送时间：2021年1月15日(星期五) 07:10 > > > >> 收件人：dev > > > >> 主 题：Re: [DISCUSS] Apache Calcite Online Meetup January 2021 > > > >> > > > >> I updated the agenda and added some more information regarding the key > > > >> signing party. > > > >> > > > >> For those planning to attend the party, the deadline for submitting > > > >> your > > > >> PGP key is Mon, Jan 18, 2021, 9:00 AM PST (UTC-8). > > > >> > > > >> Best, > > > >> Stamatis > > > >> > > > >> On Wed, Jan 13, 2021 at 8:28 AM Vladimir Ozerov > > > >> wrote: > > > >> > > > >> > *Talk duration*: 30 min. > > > >> > > > > >> > ср, 13 янв. 2021 г. в 10:27, Vladimir Ozerov : > > > >> > > > > >> > > Hi Stamatis, Julian, > > > >> > > > > > >> > > Thank you for releasing the slot. Just to be crystal clear - I was > > > >> very > > > >> > > late with my proposal, and have no urge to present in January. Both > > > >>
Re: [DISCUSS] Apache Calcite Online Meetup January 2021
Thanks for organizing Stamatis! I enjoyed hearing what others are working on. However, if we do this again I wonder if we could use a regular Zoom meeting instead of a webinar. I think it might be easier to facilitate discussion that way. -- Michael Mior mm...@apache.org Le ven. 22 janv. 2021 à 06:28, Stamatis Zampetakis a écrit : > > Hi everybody, > > Thanks again for attending the meetup and of course big thanks to the > speakers for the great presentations and thoughtful discussion. > > For those interested the recordings are now available on youtube: > https://www.youtube.com/playlist?list=PLUp_iibbIwFF_5xeLRgz4zU33mFRbS5Y6 > > Best, > Stamatis > > On Fri, Jan 15, 2021 at 4:07 AM Chunwei Lei wrote: > > > Ignore my previous email. I found the link[1] after I read all the replies. > > > > Besides, I have sent the link to my colleagues. > > > > The online meetup is a little late in UTC+8. Would the meetup be recorded? > > > > > > [1] https://www.meetup.com/Apache-Calcite/events/275461117/ > > > > Best, > > Chunwei > > > > > > On Fri, Jan 15, 2021 at 10:57 AM Chunwei Lei > > wrote: > > > >> Hi, Stamatis. > >> > >> How can I submit my PGP key? (It's my first time to attend an online > >> meetup~~) > >> > >> > >> > >> Best, > >> Chunwei > >> > >> > >> -- > >> 发件人：Stamatis Zampetakis > >> 发送时间：2021年1月15日(星期五) 07:10 > >> 收件人：dev > >> 主 题：Re: [DISCUSS] Apache Calcite Online Meetup January 2021 > >> > >> I updated the agenda and added some more information regarding the key > >> signing party. > >> > >> For those planning to attend the party, the deadline for submitting your > >> PGP key is Mon, Jan 18, 2021, 9:00 AM PST (UTC-8). > >> > >> Best, > >> Stamatis > >> > >> On Wed, Jan 13, 2021 at 8:28 AM Vladimir Ozerov > >> wrote: > >> > >> > *Talk duration*: 30 min. > >> > > >> > ср, 13 янв. 2021 г. в 10:27, Vladimir Ozerov : > >> > > >> > > Hi Stamatis, Julian, > >> > > > >> > > Thank you for releasing the slot. Just to be crystal clear - I was > >> very > >> > > late with my proposal, and have no urge to present in January. Both > >> > January > >> > > and April are perfectly fine with me. So please prioritize this talk > >> over > >> > > others only if you see a really good reason to do so. In any case, I > >> > > confirm that I can present in January. Please find the talk details > >> > below. > >> > > > >> > > *Speaker:* Vladimir Ozerov > >> > > *Title:* Apache Calcite integration in Hazelcast IMDG > >> > > *Abstract: * > >> > > Hazelcast IMDG is a distributed in-memory key-value store. In this > >> talk, > >> > I > >> > > will present how we used Apache Calcite to create a new distributed > >> SQL > >> > > engine that queries Hazelcast IMDG data. > >> > > We start with motivation and general design. Then we examine how > >> > Hazelcast > >> > > IMDG leverages Apache Calcite for query parsing, validation, and > >> > > optimization, and why we decided not to use Apache Calcite for JDBC > >> and > >> > > query execution. We also discuss several Apache Calcite problems that > >> > > negatively affect the integration and propose possible future > >> > improvements. > >> > > > >> > > Regards, > >> > > Vladimir. > >> > > > >> > > ср, 13 янв. 2021 г. в 02:48, Stamatis Zampetakis : > >> > > > >> > >> Yesterday, I updated our website to also include Vladimir's talk > >> about > >> > >> Hazelcast so I think it is better to keep it that way. > >> > >> Initially I had in mind something between 2 and 4 talks for this > >> meetup > >> > >> (2-3h) so I think we are good to go. > >> > >> > >> > >> Let's now freeze the agenda to avoid changing it till the last > >> minute. > >> > >> I will open up the discussion for the next meetup in another email > >> so we > >> > >> start filling the slots for April. > >> > >> > >> > >> It's defini
Re: GitHub actions disabled
I suppose that is a fair concern. The restriction may at least force folks to be aware of the potential risks. Although completely eliminated by explicitly referencing a particular commit instead of a tag. -- Michael Mior mm...@apache.org Le ven. 8 janv. 2021 à 12:30, Vladimir Sitnikov a écrit : > > Michael>Has there been a clear statement as to why the restrictions are in > Michael>place? > > They say "for security reasons". > > Michael>seems like the restriction is rather pointless. > > My feeling exactly :-( > > I guess someone submitted something like > https://julienrenaux.fr/2019/12/20/github-actions-security-risk/#the-problem > as a security issue to the ASF, and it triggered the wave :-( > > I guess they mentioned that tag-based and branch-based action > references like AdoptOpenJDK/install-jdk@v1 > could silently change (e.g. git force push), and the action would silently > capture > secrets or even push something to the ASF repository. > > Vladimir
