Re: RelFieldTrimmer misses optimization opportunity for always false condition

[chunwei]

Thanks for reporting this, Vladimir.  You can file a JIRA for it.

Best,
Chuwnei



On Wed, Aug 12, 2020 at 12:06 AM Julian Hyde  wrote:

> I’m not entirely sure why RelFieldTrimmer doesn’t touch the “*” case.
> Perhaps because its main job is to trim fields, and so if there’s no field
> trimming to be done, it tries to do no harm.
>
> The main code responsible for pruning empty relational operators is
> PruneEmptyRules. I think you should apply these.
>
> Julian
>
> > On Aug 11, 2020, at 6:13 AM, Vladimir Ozerov  wrote:
> >
> > Hi colleagues,
> >
> > Consider the following query. If the RelFieldTrimmer is applied to this,
> > then the expression is reduced to an empty LogicalValues:
> > SELECT field FROM table WHERE TRUE IS FALSE
> >
> > However, the following query will not be simplified, leaving a table scan
> > with "always false" filter:
> > SELECT * FROM table WHERE TRUE IS FALSE
> >
> > After some debugging I found that the problem is in the following piece
> of
> > code in the RelFieldTrimmer:
> >
> > // If the input is unchanged, and we need to project all columns,
> > // there's nothing we can do.
> > if (newInput == input
> >&& fieldsUsed.cardinality() == fieldCount) {
> >  return result(filter, Mappings.createIdentity(fieldCount));
> > }
> >
> > My question - is it a known issue? Looks like this early return from the
> > method misses an important optimization opportunity.  Can this check be
> > removed completely?
> >
> > Regards,
> > Vladimir
>

Re: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[Forward Xu]

Congrats, Ruben!


Best

Forward

XING JIN  于2020年8月12日周三 上午8:58写道：

> Congrats, Ruben!
>
> 953396112 <953396...@qq.com> 于2020年8月12日周三 上午7:47写道：
>
> > Congratulations,Ruben!
> >
> >
> > xzh
> > --原始邮件--
> > 发件人:
> >   "dev"
> > <
> > zabe...@gmail.com;
> > 发送时间:2020年8月12日(星期三) 凌晨5:53
> > 收件人:"dev" >
> > 主题:[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC
> >
> >
> >
> > I'm pleased to announce that Ruben has accepted an invitation to
> > join the Calcite PMC. Ruben has been a consistent and helpful
> > figure in the Calcite community for which we are very grateful. We
> > look forward to the continued contributions and support.
> >
> > Please join me in congratulating Ruben!
> >
> > - Stamatis (on behalf of the Calcite PMC)
>

Re: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[Danny Chan]

Congratulations ~

Best,
Danny Chan
在 2020年8月12日 +0800 AM9:57，chunwei ，写道：
> Congrats, Ruben!
>
> On Wed, Aug 12, 2020 at 8:58 AM XING JIN  wrote:
>
> > Congrats, Ruben!
> >
> > 953396112 <953396...@qq.com> 于2020年8月12日周三 上午7:47写道：
> >
> > > Congratulations,Ruben!
> > >
> > >
> > > xzh
> > > --原始邮件--
> > > 发件人:
> > > "dev"
> > > <
> > > zabe...@gmail.com;
> > > 发送时间:2020年8月12日(星期三) 凌晨5:53
> > > 收件人:"dev" > >
> > > 主题:[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC
> > >
> > >
> > >
> > > I'm pleased to announce that Ruben has accepted an invitation to
> > > join the Calcite PMC. Ruben has been a consistent and helpful
> > > figure in the Calcite community for which we are very grateful. We
> > > look forward to the continued contributions and support.
> > >
> > > Please join me in congratulating Ruben!
> > >
> > > - Stamatis (on behalf of the Calcite PMC)
> >

Re: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[Feng Zhu]

 Congratulations, Ruben!

chunwei  于2020年8月12日周三 上午9:57写道：

> Congrats, Ruben!
>
> On Wed, Aug 12, 2020 at 8:58 AM XING JIN  wrote:
>
> > Congrats, Ruben!
> >
> > 953396112 <953396...@qq.com> 于2020年8月12日周三 上午7:47写道：
> >
> > > Congratulations,Ruben!
> > >
> > >
> > > xzh
> > > --原始邮件--
> > > 发件人:
> > >   "dev"
> > > <
> > > zabe...@gmail.com;
> > > 发送时间:2020年8月12日(星期三) 凌晨5:53
> > > 收件人:"dev" > >
> > > 主题:[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC
> > >
> > >
> > >
> > > I'm pleased to announce that Ruben has accepted an invitation to
> > > join the Calcite PMC. Ruben has been a consistent and helpful
> > > figure in the Calcite community for which we are very grateful. We
> > > look forward to the continued contributions and support.
> > >
> > > Please join me in congratulating Ruben!
> > >
> > > - Stamatis (on behalf of the Calcite PMC)
> >
>

Re: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[chunwei]

Congrats, Ruben!

On Wed, Aug 12, 2020 at 8:58 AM XING JIN  wrote:

> Congrats, Ruben!
>
> 953396112 <953396...@qq.com> 于2020年8月12日周三 上午7:47写道：
>
> > Congratulations,Ruben!
> >
> >
> > xzh
> > --原始邮件--
> > 发件人:
> >   "dev"
> > <
> > zabe...@gmail.com;
> > 发送时间:2020年8月12日(星期三) 凌晨5:53
> > 收件人:"dev" >
> > 主题:[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC
> >
> >
> >
> > I'm pleased to announce that Ruben has accepted an invitation to
> > join the Calcite PMC. Ruben has been a consistent and helpful
> > figure in the Calcite community for which we are very grateful. We
> > look forward to the continued contributions and support.
> >
> > Please join me in congratulating Ruben!
> >
> > - Stamatis (on behalf of the Calcite PMC)
>

Re: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[XING JIN]

Congrats, Ruben!

953396112 <953396...@qq.com> 于2020年8月12日周三 上午7:47写道：

> Congratulations,Ruben!
>
>
> xzh
> --原始邮件--
> 发件人:
>   "dev"
> <
> zabe...@gmail.com;
> 发送时间:2020年8月12日(星期三) 凌晨5:53
> 收件人:"dev"
> 主题:[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC
>
>
>
> I'm pleased to announce that Ruben has accepted an invitation to
> join the Calcite PMC. Ruben has been a consistent and helpful
> figure in the Calcite community for which we are very grateful. We
> look forward to the continued contributions and support.
>
> Please join me in congratulating Ruben!
>
> - Stamatis (on behalf of the Calcite PMC)

??????[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[953396112]

Congratulations,Ruben!


xzh
----
??: 
   "dev"

Re: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[Haisheng Yuan]

Congrats, Ruben!

On 2020/08/11 21:53:47, Stamatis Zampetakis  wrote: 
> I'm pleased to announce that Ruben has accepted an invitation to
> join the Calcite PMC. Ruben has been a consistent and helpful
> figure in the Calcite community for which we are very grateful. We
> look forward to the continued contributions and support.
> 
> Please join me in congratulating Ruben!
> 
> - Stamatis (on behalf of the Calcite PMC)
> 

Re: [DISCUSS] Sarg (search argument) to generalize and replace IN in RexCall

[Haisheng Yuan]

> I am proposing to use sargs only where we would today use RexCall(IN). The 
> data structures have about the same size. Sargs are sorted. I cannot see any 
> cases that would become more expensive.

IIRC, RexCall(IN) is not supported yet.

With sargs, you have to sort it, no matter explicitly or implicitly, in case 
20k values, it will take some time. I am not saying the data structure itself 
is expensive, but how it is used, it all depends on how downstream projects 
derive stats from it. I have seen Greenplum optimizer experienced performance 
issue for large IN list when deriving all the stats info from the thousands of 
disjoint range intervals. Some downstream project may not even use histogram. 
Some may just use the number of IN constants to derive the number of distinct 
values, I would imagine people have to construct the Sarg back to constant 
list, this is an extra burden. 

Suppose we have col in (1,2,4,6,8,10, 12,13, 15,17,19, 21,22, 24, 24+2.) 
10k values, now the range sets will be {[1,2], [4,4], [10,10], [12,13], 
[15,15][21,22], [24,24]}, in the execution engine, what if I want to do 
the look up for this expression? Find out the points, extract ranges and 
disjoint them, or infer points from integer ranges?

IMHO, the interval constraint is more like a kind of logical property and can 
be strategy for simplification, people can derive the interval constraints from 
the IN list, do simplification on it (but I doubt the value in practice, which 
usually brings more trouble than benefits), but we don't have to represent it 
with sarg in RexNode world.

> Requiring sargs to support geospatial operations seems an unreasonably high 
> bar. Many techniques that we use (sorting, hashing, sketches) do not support 
> geospatial.
> Areas that have not-totally-ordered data types tend to have their own 
> operators already. For your example I’d write ST_Intersects(col, 
> ST_Collect(area1, area2, area3)), and that would be a perfectly suitable 
> representation in RexNode-land.

I am not asking sargs to support geospatial operations. And geospatial 
intersect is just an example of potential customized operation, which are not 
limited to geospatials. It can be some other binary operations. We can't 
require all the customized operation to have something like ST_Collect. 

Oracle support this:
select * from foo where a > ANY(1, 2, 3, ., 999);
select * from foo where a <= ANY(b+1, b+2, b+3, ., 999);
Although in reality that kind of query might be rare, how would Calcite 
represent in Oracle dialect? And would it be good to support operations other 
than equal, like <, >=, or even customized operation? Especially when 
downstream projects may use calcite to build their own in-house system, some 
are not sql standard. 

BTW, what is your concern about the proposal of RexListCmp [1]?

Haisheng

[1] 
https://lists.apache.org/thread.html/ra36f256e3f86f78ee126455b3ba67dbcb7bd1862214886c65ffba432%40%3Cdev.calcite.apache.org%3E

On 2020/08/11 16:44:56, Julian Hyde  wrote: 
> Let’s bring this discussion back to the original question: for these kinds of 
> predicates, is Sarg a better data structure than IN?
> 
> It absolutely is. There are algorithms in RexSimplify that, for each term in 
> an OR list, simplify using all available predicates, and when they have 
> simplified that term, add it to the list of predicates. It is therefore 
> cartesian in the size of the OR list.
> 
> The sarg data structure converts many kinds of large OR-lists and predicate 
> sets into a single term. Because that term is immutable and based on sorted 
> ranges (or points) it can be handled efficiently (e.g. two sargs can be 
> intersected using a merge, rather than nested loops). That will make our 
> simplification process more efficient.
> 
> Whether we then add new classes of optimization is a discussion for another 
> day.
> 
> Julian
> 
> 
> > On Aug 10, 2020, at 1:13 PM, Vladimir Sitnikov 
> >  wrote:
> > 
> > Julian>I cannot see any cases that would become more expensive.
> > 
> > I mean the optimization passes might be complicated, not the storage of
> > sargs themselves.
> > For instance, CALCITE-4155 converts a in (1, 2, 3, 4, 5) to a >= 1 and a <=
> > 5
> > Is it a significant improvement?
> > Is between representation always better?
> > Does the case appear very often in practice?
> > 
> > However, the optimization does take time, so it looks like extra logic with
> > doubtful gains.
> > Of course, it is unlikely sargs would be a major time consumer, however, if
> > we keep adding tiny simplifications we might
> > end up with a condition where the planning is slow and we have no way to
> > fix it.
> > 
> > I'm ok with people updating RexSimplify (and 4155 looks like an innocent
> > feature), however, I think the current design is not really scalable
> > (e.g. it might process the same expression multiple times).
> > 
> > Vladimir
> 
> 

[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[Stamatis Zampetakis]

I'm pleased to announce that Ruben has accepted an invitation to
join the Calcite PMC. Ruben has been a consistent and helpful
figure in the Calcite community for which we are very grateful. We
look forward to the continued contributions and support.

Please join me in congratulating Ruben!

- Stamatis (on behalf of the Calcite PMC)

Re: RelToSqlConverter NullPointerException

[Stamatis Zampetakis]

Thanks for reporting this. Please file a JIRA case with the necessary
information.

Also it would be helpful if you could refactor your test case as part of
RelToSqlConverterTest [1].

Best,
Stamatis

[1]
https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/rel/rel2sql/RelToSqlConverterTest.java

On Mon, Aug 10, 2020 at 11:43 AM tonytao  wrote:

> Sorry, the attached file was wrong,please ignore it.
>
> I upload a new file.
>
> On 8/10/20 4:38 PM, tonytao wrote:
> > hi team,
> >
> > I met a NullPointerException when I used calcite 1.24.0 to convert a
> > relNode to sqlNode.
> >
> > The trace:
> >
> > java.lang.NullPointerException
> > at java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > at
> >
> org.apache.calcite.rel.rel2sql.RelToSqlConverter.result(RelToSqlConverter.java:152)
> > at
> >
> org.apache.calcite.rel.rel2sql.SqlImplementor.result(SqlImplementor.java:454)
> > at
> >
> org.apache.calcite.rel.rel2sql.SqlImplementor$Builder.result(SqlImplementor.java:1822)
> > at
> >
> org.apache.calcite.rel.rel2sql.RelToSqlConverter.visit(RelToSqlConverter.java:392)
> > at server.TestRelToSqlConverter.test(TestRelToSqlConverter.java:84)
> >
> >
> > Attched file is a junit5 test case,you can repeat the issue with this
> > code.
> >
> > Maven configuration:
> >
> > 
> > org.apache.calcite
> > calcite-core
> > 1.24.0
> > 
> > 
> > org.apache.commons
> > commons-dbcp2
> > 2.7.0
> > 
> > 
> > org.postgresql
> > postgresql
> > 42.2.12
> > 
> >
> >
>
>

Re: Gather real world statistics about row count, CPU, and IO cost

[Stamatis Zampetakis]

Any help for improving the cost model is much appreciated.

I am not sure what you have in mind when you mention real-world statistics
but I think you could also start with the stats provided by synthetic
datasets
such as Foodmart, TPC-H and TPC-DS.

Foodmart, TPC-H and TPC-DS come along with a rich set of queries
(especially the latter)
that you could use to demonstrate that the changes in the cost-model are
beneficial.
The advantage is that you can also run the query and validate the
improvement.

Even if you want to emphasize on two table joins you could do that by
taking
subpatterns of the queries appearing in TPC-DS for instance.

In general, apart from the cost-model the default ruleset can be improved
and using the TPC-H/TPC-DS query plans as a baseline could help a lot.

All the datasets mentioned previously and more are already present in the
repo so you can rely on them to get started.

Best,
Stamatis


On Mon, Aug 10, 2020 at 1:48 PM Thomas Rebele 
wrote:

> Thank you for your analysis and the references. I would be more interested
> in sub-problem 2.
> A kind of tuning tool would be nice. If a project using Calcite  introduces
> custom operators, it could help to find a cost model for them.
>
> I would start with a simple case, e.g., deciding whether a particular join
> should be translated into a hash join, merge join, or correlate.
> With a different cardinality of the inputs, one algorithm or the other
> might be better. The real-world statistics might help to decide which one
> to choose.
>
> Cordialement / Best Regards,
> *Thomas Rebele* | R Developer | 18 rue du 4 septembre, 75002 Paris,
> France
> | www.tibco.com
>
>
> On Fri, Aug 7, 2020 at 11:32 PM Julian Hyde  wrote:
>
> > Also, check out the paper "Learning to Optimize Join Queries With Deep
> > Reinforcement Learning" by Krishnan, Yang, Goldberg, Hellerstein,
> > Stoica 2019 (which aims to improve the join-order benchmark and uses
> > Calcite as one of its test platforms):
> > https://arxiv.org/pdf/1808.03196.pdf
> >
> > On Fri, Aug 7, 2020 at 2:02 PM Julian Hyde  wrote:
> > >
> > > I consider there to be two fairly independent sub-problems:
> > >
> > > 1. Improve our cardinality estimates (especially for queries with
> > > complex joins and aggregation).
> > >
> > > 2. Calibrate physical operators so that, given a good estimate of the
> > > number of rows they will see, we can come up with a reasonable
> > > estimate of the physical cost (e.g. how long the query will take to
> > > execute, and how much memory).
> > >
> > > For 1, the paper you cite, and the join-order benchmark it introduces,
> > > is an excellent contribution to the field. It inspired me to do work
> > > on profiling [1]. I would encourage you to build on the work I have
> > > already done.
> > >
> > > For 2, I have not done any work personally. An approach would be to
> > > give each physical operator (e.g. EnumerableHashJoin) a cost model
> > > parameterized by certain constants, and then run experiments to
> > > determine the values of those constants empirically. Perhaps we could
> > > write a "TuningTool" that generates an "operator constants file", and
> > > thereby start to formalize the process.
> > >
> > > Julian
> > >
> > > [1]
> > https://www.slideshare.net/julianhyde/data-profiling-in-apache-calcite
> > >
> > > On Fri, Aug 7, 2020 at 6:57 AM Thomas Rebele  >
> > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I'm working on basic query optimization. I once stumbled on the case
> > that
> > > > two operators had the same row count but one had a much higher CPU
> > cost.
> > > > Unfortunately the default cost model only takes the row count into
> > account
> > > > (see [1]). Stamatis had pointed out in another mail that the row
> count
> > > > might be much more important than the other costs [2]. However, if
> > there
> > > > are two possible choices with the same row count, we should prefer
> the
> > one
> > > > with the least CPU cost. I'm wondering whether the assumption that a
> > > > smaller row count is better in most cases is actually correct. Also,
> > what
> > > > is "better" in this context? The query plan with the least execution
> > time?
> > > > Maybe there's a plan that is just <10% slower, but consumes much less
> > > > CPU/memory/etc.
> > > >
> > > > So I thought about the cost model in general, and how to improve it.
> I
> > > > assume the better the estimated cost corresponds to the real cost,
> the
> > > > better the optimized plans. So the first step would be to collect the
> > real
> > > > world statistics and the second step to adapt the cost estimation so
> > that
> > > > there's a better correspondence. For the beginning I would just
> > measure how
> > > > many rows have been in the result and how much time has passed for
> each
> > > > RelNode during query execution. Is there already a way to do this in
> > > > Calcite? Does this make sense at all?
> > > >
> > > > [1]
> > > >
> >
> 

Re: Contribute to Calcite

[Stamatis Zampetakis]

Hi Malte,

I've added you as a contributor on JIRA. Welcome to the project!

Best,
Stamatis

On Tue, Aug 11, 2020 at 7:42 PM Malte Bellmann
 wrote:

> Hello,
>
> I’d like to contribute to Calcite, could you please give me contribution
> permission on JIRA.
>
> JIRA Username: MalteBellmann
>
> Thank you,
> Malte
>
>
>

Apache Calicte learning curve

[Abhishek Dasgupta]

Hi Team,
I am working as a Software Developer in data migration sector. We use the
Apache-Calcite framework. I am trying to learn how to use it and apparently
it has a steep learning curve. I went through the documentation but as a
beginner it is difficult to grasp. Can you provide me some useful links
where to start learning Apache Calcite apart from the documentation. Can
you provide a chronological order  what to understand first i.e a step by
step list.

Thanks,
Abhishek Dasgupta

Re: [VOTE] Release apache-calcite-1.25.0 (release candidate 0)

[Andrei Sereda]

Hi All,

I plan to close the vote tomorrow (Aug 12th). If you still want to validate
RC0 please do so before Wednesday.

Regards,
Andrei.

On Tue, Aug 11, 2020 at 1:13 PM Andrei Sereda  wrote:

> Thanks, Julian, for the hint. I'll update the KEYS file.
>
> On Tue, Aug 11, 2020 at 11:59 AM Julian Hyde 
> wrote:
>
>> Andrei,
>>
>> Your key’s signature appears in KEYS because you signed Stamatis’s key.
>> But your key isn’t actually defined in the file. After I imported the file,
>> your key was still ‘unknown’ according to gpg.
>>
>> Julian
>>
>> > On Aug 11, 2020, at 8:04 AM, Andrei Sereda  wrote:
>> >
>> > 
>> >>
>> >> * Andrei, I don’t think your key is in KEYS. Be sure to add it before
>> the
>> > release announcement.
>> >
>> > I see my signing key in KEYS
>> > $ curl -s https://dist.apache.org/repos/dist/release/calcite/KEYS |
>> grep
>> > sereda
>> > sig  C41CFDDFED34C028 2019-08-19  Andrei Sereda (CODE SIGNING
>> KEY) <
>> > ser...@apache.org>
>> > sig  C41CFDDFED34C028 2019-08-19  Andrei Sereda (CODE SIGNING
>> KEY) <
>> > ser...@apache.org>
>> >
>> >> * There are several files in the source distro that are not in git: an
>> > empty ‘arrow’ directory, and a non-empty ‘licenses’ directory.
>> > Yes you're right. I presume you're still OK with RC0 (judging by your
>> +1) ?
>> >
>> >
>> >> On Tue, Aug 11, 2020 at 5:53 AM Julian Hyde 
>> wrote:
>> >>
>> >> +1 (binding)
>> >>
>> >> Checked keys, hashes, LICENSE, NOTICE, README; compiled and ran tests
>> on
>> >> Ubuntu/JDK 14; ran RAT.
>> >>
>> >> * Andrei, I don’t think your key is in KEYS. Be sure to add it before
>> the
>> >> release announcement.
>> >>
>> >> * There are several files in the source distro that are not in git: an
>> >> empty ‘arrow’ directory, and a non-empty ‘licenses’ directory.
>> >>
>> >> Julian
>> >>
>> >>
>> >>> On Aug 11, 2020, at 2:33 AM, Stamatis Zampetakis 
>> >> wrote:
>> >>>
>> >>> If I remember well the site preview is not implemented so it is normal
>> >> that
>> >>> links are broken.
>> >>> I don't know if this has changed recently but if not I think we should
>> >>> remove the site preview links from the draft email to avoid confusion.
>> >>>
>> >>> Best,
>> >>> Stamatis
>> >>>
>>  On Mon, Aug 10, 2020 at 8:08 PM Andrei Sereda 
>> wrote:
>> >>>
>>  Yes I'm aware that for some reason site artifact wasn't built /
>> >> uploaded.
>> 
>>  Meanwhile here are the release notes:
>> 
>> 
>> >>
>> https://github.com/apache/calcite/blob/calcite-1.25.0-rc0/site/_docs/history.md
>> 
>>  I'll double-check the build script / instructions, maybe I have
>> missed
>>  something.
>> 
>>  On Mon, Aug 10, 2020 at 12:59 PM Haisheng Yuan 
>> >> wrote:
>> 
>> > +1 (binding)
>> >
>> > Environment:
>> > Mac OS X 10.15.1 , JDK 1.8.0_162
>> > - Ran unit tests (./gradlew build), OK
>> > - Checked release notes, CALCITE-4156, CALCITE-4022 CALCITE-4115
>> > CALCITE-4129 CALCITE-4111 are not part of Build and test suite
>> changes,
>> > those can be updated after release.
>> >
>> > - Haisheng
>> >
>> > On 2020/08/09 03:22:28, Andrei Sereda  wrote:
>> >> Hi all,
>> >>
>> >> I have created a build for Apache Calcite 1.25.0, release
>> >> candidate 0.
>> >>
>> >> Thanks to everyone who has contributed to this release.
>> >>
>> >> You can read the release notes here:
>> >> https://apache.github.io/calcite-site-preview/docs/history.html
>> >>
>> >> The commit to be voted upon:
>> >>
>> >
>> 
>> >>
>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=68b02dfd4af15bc94a91a0cd2a30655d04439555
>> >>
>> >> Its hash is 68b02dfd4af15bc94a91a0cd2a30655d04439555
>> >>
>> >> Tag:
>> >> https://github.com/apache/calcite/tree/calcite-1.25.0-rc0
>> >>
>> >> The artifacts to be voted on are located here:
>> >>
>> 
>> >>
>> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.25.0-rc0
>> >> (revision 40922)
>> >>
>> >> RAT report:
>> >> https://apache.github.io/calcite-site-preview/rat/rat-report.txt
>> >>
>> >> Site preview is here:
>> >> https://apache.github.io/calcite-site-preview/
>> >>
>> >> JavaDoc API preview is here:
>> >> https://apache.github.io/calcite-site-preview/api
>> >>
>> >> The hashes of the artifacts are as follows:
>> >>
>> >
>> 
>> >>
>> a5e61bd93657a274ee8a1d1ecbde68e3e471fd27b85bea179991b372f094ae3cdf692672245506a08b996534f9136be26569de81af2bb7d8f026799313957e87
>> >> *apache-calcite-1.25.0-src.tar.gz
>> >>
>> >> A staged Maven repository is available for review at:
>> >>
>> >
>> 
>> >>
>> https://repository.apache.org/content/repositories/orgapachecalcite-1097/org/apache/calcite/
>> >>
>> >> Release artifacts are signed with the following key:
>> >> https://people.apache.org/keys/committer/sereda.asc
>> >> 

Re: [VOTE] Release apache-calcite-1.25.0 (release candidate 0)

[Andrei Sereda]

Thanks, Julian, for the hint. I'll update the KEYS file.

On Tue, Aug 11, 2020 at 11:59 AM Julian Hyde  wrote:

> Andrei,
>
> Your key’s signature appears in KEYS because you signed Stamatis’s key.
> But your key isn’t actually defined in the file. After I imported the file,
> your key was still ‘unknown’ according to gpg.
>
> Julian
>
> > On Aug 11, 2020, at 8:04 AM, Andrei Sereda  wrote:
> >
> > 
> >>
> >> * Andrei, I don’t think your key is in KEYS. Be sure to add it before
> the
> > release announcement.
> >
> > I see my signing key in KEYS
> > $ curl -s https://dist.apache.org/repos/dist/release/calcite/KEYS | grep
> > sereda
> > sig  C41CFDDFED34C028 2019-08-19  Andrei Sereda (CODE SIGNING
> KEY) <
> > ser...@apache.org>
> > sig  C41CFDDFED34C028 2019-08-19  Andrei Sereda (CODE SIGNING
> KEY) <
> > ser...@apache.org>
> >
> >> * There are several files in the source distro that are not in git: an
> > empty ‘arrow’ directory, and a non-empty ‘licenses’ directory.
> > Yes you're right. I presume you're still OK with RC0 (judging by your
> +1) ?
> >
> >
> >> On Tue, Aug 11, 2020 at 5:53 AM Julian Hyde 
> wrote:
> >>
> >> +1 (binding)
> >>
> >> Checked keys, hashes, LICENSE, NOTICE, README; compiled and ran tests on
> >> Ubuntu/JDK 14; ran RAT.
> >>
> >> * Andrei, I don’t think your key is in KEYS. Be sure to add it before
> the
> >> release announcement.
> >>
> >> * There are several files in the source distro that are not in git: an
> >> empty ‘arrow’ directory, and a non-empty ‘licenses’ directory.
> >>
> >> Julian
> >>
> >>
> >>> On Aug 11, 2020, at 2:33 AM, Stamatis Zampetakis 
> >> wrote:
> >>>
> >>> If I remember well the site preview is not implemented so it is normal
> >> that
> >>> links are broken.
> >>> I don't know if this has changed recently but if not I think we should
> >>> remove the site preview links from the draft email to avoid confusion.
> >>>
> >>> Best,
> >>> Stamatis
> >>>
>  On Mon, Aug 10, 2020 at 8:08 PM Andrei Sereda 
> wrote:
> >>>
>  Yes I'm aware that for some reason site artifact wasn't built /
> >> uploaded.
> 
>  Meanwhile here are the release notes:
> 
> 
> >>
> https://github.com/apache/calcite/blob/calcite-1.25.0-rc0/site/_docs/history.md
> 
>  I'll double-check the build script / instructions, maybe I have missed
>  something.
> 
>  On Mon, Aug 10, 2020 at 12:59 PM Haisheng Yuan 
> >> wrote:
> 
> > +1 (binding)
> >
> > Environment:
> > Mac OS X 10.15.1 , JDK 1.8.0_162
> > - Ran unit tests (./gradlew build), OK
> > - Checked release notes, CALCITE-4156, CALCITE-4022 CALCITE-4115
> > CALCITE-4129 CALCITE-4111 are not part of Build and test suite
> changes,
> > those can be updated after release.
> >
> > - Haisheng
> >
> > On 2020/08/09 03:22:28, Andrei Sereda  wrote:
> >> Hi all,
> >>
> >> I have created a build for Apache Calcite 1.25.0, release
> >> candidate 0.
> >>
> >> Thanks to everyone who has contributed to this release.
> >>
> >> You can read the release notes here:
> >> https://apache.github.io/calcite-site-preview/docs/history.html
> >>
> >> The commit to be voted upon:
> >>
> >
> 
> >>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=68b02dfd4af15bc94a91a0cd2a30655d04439555
> >>
> >> Its hash is 68b02dfd4af15bc94a91a0cd2a30655d04439555
> >>
> >> Tag:
> >> https://github.com/apache/calcite/tree/calcite-1.25.0-rc0
> >>
> >> The artifacts to be voted on are located here:
> >>
> 
> >>
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.25.0-rc0
> >> (revision 40922)
> >>
> >> RAT report:
> >> https://apache.github.io/calcite-site-preview/rat/rat-report.txt
> >>
> >> Site preview is here:
> >> https://apache.github.io/calcite-site-preview/
> >>
> >> JavaDoc API preview is here:
> >> https://apache.github.io/calcite-site-preview/api
> >>
> >> The hashes of the artifacts are as follows:
> >>
> >
> 
> >>
> a5e61bd93657a274ee8a1d1ecbde68e3e471fd27b85bea179991b372f094ae3cdf692672245506a08b996534f9136be26569de81af2bb7d8f026799313957e87
> >> *apache-calcite-1.25.0-src.tar.gz
> >>
> >> A staged Maven repository is available for review at:
> >>
> >
> 
> >>
> https://repository.apache.org/content/repositories/orgapachecalcite-1097/org/apache/calcite/
> >>
> >> Release artifacts are signed with the following key:
> >> https://people.apache.org/keys/committer/sereda.asc
> >> https://www.apache.org/dist/calcite/KEYS
> >>
> >> N.B.
> >> To create the jars and test Apache Calcite: "./gradlew build".
> >>
> >> If you do not have a Java environment available, you can run the
> tests
> >> using docker. To do so, install docker and docker-compose, then run
> >> "docker-compose run test" from the root of the directory.
> 

Re: [DISCUSS] Sarg (search argument) to generalize and replace IN in RexCall

[Julian Hyde]

Let’s bring this discussion back to the original question: for these kinds of 
predicates, is Sarg a better data structure than IN?

It absolutely is. There are algorithms in RexSimplify that, for each term in an 
OR list, simplify using all available predicates, and when they have simplified 
that term, add it to the list of predicates. It is therefore cartesian in the 
size of the OR list.

The sarg data structure converts many kinds of large OR-lists and predicate 
sets into a single term. Because that term is immutable and based on sorted 
ranges (or points) it can be handled efficiently (e.g. two sargs can be 
intersected using a merge, rather than nested loops). That will make our 
simplification process more efficient.

Whether we then add new classes of optimization is a discussion for another day.

Julian


> On Aug 10, 2020, at 1:13 PM, Vladimir Sitnikov  
> wrote:
> 
> Julian>I cannot see any cases that would become more expensive.
> 
> I mean the optimization passes might be complicated, not the storage of
> sargs themselves.
> For instance, CALCITE-4155 converts a in (1, 2, 3, 4, 5) to a >= 1 and a <=
> 5
> Is it a significant improvement?
> Is between representation always better?
> Does the case appear very often in practice?
> 
> However, the optimization does take time, so it looks like extra logic with
> doubtful gains.
> Of course, it is unlikely sargs would be a major time consumer, however, if
> we keep adding tiny simplifications we might
> end up with a condition where the planning is slow and we have no way to
> fix it.
> 
> I'm ok with people updating RexSimplify (and 4155 looks like an innocent
> feature), however, I think the current design is not really scalable
> (e.g. it might process the same expression multiple times).
> 
> Vladimir

Contribute to Calcite

[Malte Bellmann]

Hello, 

I’d like to contribute to Calcite, could you please give me contribution 
permission on JIRA.

JIRA Username: MalteBellmann

Thank you,
Malte

Re: RelFieldTrimmer misses optimization opportunity for always false condition

[Julian Hyde]

I’m not entirely sure why RelFieldTrimmer doesn’t touch the “*” case. Perhaps 
because its main job is to trim fields, and so if there’s no field trimming to 
be done, it tries to do no harm. 

The main code responsible for pruning empty relational operators is 
PruneEmptyRules. I think you should apply these.

Julian

> On Aug 11, 2020, at 6:13 AM, Vladimir Ozerov  wrote:
> 
> Hi colleagues,
> 
> Consider the following query. If the RelFieldTrimmer is applied to this,
> then the expression is reduced to an empty LogicalValues:
> SELECT field FROM table WHERE TRUE IS FALSE
> 
> However, the following query will not be simplified, leaving a table scan
> with "always false" filter:
> SELECT * FROM table WHERE TRUE IS FALSE
> 
> After some debugging I found that the problem is in the following piece of
> code in the RelFieldTrimmer:
> 
> // If the input is unchanged, and we need to project all columns,
> // there's nothing we can do.
> if (newInput == input
>&& fieldsUsed.cardinality() == fieldCount) {
>  return result(filter, Mappings.createIdentity(fieldCount));
> }
> 
> My question - is it a known issue? Looks like this early return from the
> method misses an important optimization opportunity.  Can this check be
> removed completely?
> 
> Regards,
> Vladimir

Re: [VOTE] Release apache-calcite-1.25.0 (release candidate 0)

[Julian Hyde]

Andrei,

Your key’s signature appears in KEYS because you signed Stamatis’s key. But 
your key isn’t actually defined in the file. After I imported the file, your 
key was still ‘unknown’ according to gpg. 

Julian

> On Aug 11, 2020, at 8:04 AM, Andrei Sereda  wrote:
> 
> 
>> 
>> * Andrei, I don’t think your key is in KEYS. Be sure to add it before the
> release announcement.
> 
> I see my signing key in KEYS
> $ curl -s https://dist.apache.org/repos/dist/release/calcite/KEYS | grep
> sereda
> sig  C41CFDDFED34C028 2019-08-19  Andrei Sereda (CODE SIGNING KEY) <
> ser...@apache.org>
> sig  C41CFDDFED34C028 2019-08-19  Andrei Sereda (CODE SIGNING KEY) <
> ser...@apache.org>
> 
>> * There are several files in the source distro that are not in git: an
> empty ‘arrow’ directory, and a non-empty ‘licenses’ directory.
> Yes you're right. I presume you're still OK with RC0 (judging by your +1) ?
> 
> 
>> On Tue, Aug 11, 2020 at 5:53 AM Julian Hyde  wrote:
>> 
>> +1 (binding)
>> 
>> Checked keys, hashes, LICENSE, NOTICE, README; compiled and ran tests on
>> Ubuntu/JDK 14; ran RAT.
>> 
>> * Andrei, I don’t think your key is in KEYS. Be sure to add it before the
>> release announcement.
>> 
>> * There are several files in the source distro that are not in git: an
>> empty ‘arrow’ directory, and a non-empty ‘licenses’ directory.
>> 
>> Julian
>> 
>> 
>>> On Aug 11, 2020, at 2:33 AM, Stamatis Zampetakis 
>> wrote:
>>> 
>>> If I remember well the site preview is not implemented so it is normal
>> that
>>> links are broken.
>>> I don't know if this has changed recently but if not I think we should
>>> remove the site preview links from the draft email to avoid confusion.
>>> 
>>> Best,
>>> Stamatis
>>> 
 On Mon, Aug 10, 2020 at 8:08 PM Andrei Sereda  wrote:
>>> 
 Yes I'm aware that for some reason site artifact wasn't built /
>> uploaded.
 
 Meanwhile here are the release notes:
 
 
>> https://github.com/apache/calcite/blob/calcite-1.25.0-rc0/site/_docs/history.md
 
 I'll double-check the build script / instructions, maybe I have missed
 something.
 
 On Mon, Aug 10, 2020 at 12:59 PM Haisheng Yuan 
>> wrote:
 
> +1 (binding)
> 
> Environment:
> Mac OS X 10.15.1 , JDK 1.8.0_162
> - Ran unit tests (./gradlew build), OK
> - Checked release notes, CALCITE-4156, CALCITE-4022 CALCITE-4115
> CALCITE-4129 CALCITE-4111 are not part of Build and test suite changes,
> those can be updated after release.
> 
> - Haisheng
> 
> On 2020/08/09 03:22:28, Andrei Sereda  wrote:
>> Hi all,
>> 
>> I have created a build for Apache Calcite 1.25.0, release
>> candidate 0.
>> 
>> Thanks to everyone who has contributed to this release.
>> 
>> You can read the release notes here:
>> https://apache.github.io/calcite-site-preview/docs/history.html
>> 
>> The commit to be voted upon:
>> 
> 
 
>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=68b02dfd4af15bc94a91a0cd2a30655d04439555
>> 
>> Its hash is 68b02dfd4af15bc94a91a0cd2a30655d04439555
>> 
>> Tag:
>> https://github.com/apache/calcite/tree/calcite-1.25.0-rc0
>> 
>> The artifacts to be voted on are located here:
>> 
 
>> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.25.0-rc0
>> (revision 40922)
>> 
>> RAT report:
>> https://apache.github.io/calcite-site-preview/rat/rat-report.txt
>> 
>> Site preview is here:
>> https://apache.github.io/calcite-site-preview/
>> 
>> JavaDoc API preview is here:
>> https://apache.github.io/calcite-site-preview/api
>> 
>> The hashes of the artifacts are as follows:
>> 
> 
 
>> a5e61bd93657a274ee8a1d1ecbde68e3e471fd27b85bea179991b372f094ae3cdf692672245506a08b996534f9136be26569de81af2bb7d8f026799313957e87
>> *apache-calcite-1.25.0-src.tar.gz
>> 
>> A staged Maven repository is available for review at:
>> 
> 
 
>> https://repository.apache.org/content/repositories/orgapachecalcite-1097/org/apache/calcite/
>> 
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/sereda.asc
>> https://www.apache.org/dist/calcite/KEYS
>> 
>> N.B.
>> To create the jars and test Apache Calcite: "./gradlew build".
>> 
>> If you do not have a Java environment available, you can run the tests
>> using docker. To do so, install docker and docker-compose, then run
>> "docker-compose run test" from the root of the directory.
>> 
>> Please vote on releasing this package as Apache Calcite 1.25.0.
>> 
>> The vote is open for the next 72 hours and passes if a majority of at
>> least three +1 PMC votes are cast.
>> 
>> [ ] +1 Release this package as Apache Calcite 1.25.0
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> [ ] -1 Do not 

Re: [VOTE] Release apache-calcite-1.25.0 (release candidate 0)

[Andrei Sereda]

> * Andrei, I don’t think your key is in KEYS. Be sure to add it before the
release announcement.

I see my signing key in KEYS
$ curl -s https://dist.apache.org/repos/dist/release/calcite/KEYS | grep
sereda
sig  C41CFDDFED34C028 2019-08-19  Andrei Sereda (CODE SIGNING KEY) <
ser...@apache.org>
sig  C41CFDDFED34C028 2019-08-19  Andrei Sereda (CODE SIGNING KEY) <
ser...@apache.org>

> * There are several files in the source distro that are not in git: an
empty ‘arrow’ directory, and a non-empty ‘licenses’ directory.
Yes you're right. I presume you're still OK with RC0 (judging by your +1) ?


On Tue, Aug 11, 2020 at 5:53 AM Julian Hyde  wrote:

> +1 (binding)
>
> Checked keys, hashes, LICENSE, NOTICE, README; compiled and ran tests on
> Ubuntu/JDK 14; ran RAT.
>
> * Andrei, I don’t think your key is in KEYS. Be sure to add it before the
> release announcement.
>
> * There are several files in the source distro that are not in git: an
> empty ‘arrow’ directory, and a non-empty ‘licenses’ directory.
>
> Julian
>
>
> > On Aug 11, 2020, at 2:33 AM, Stamatis Zampetakis 
> wrote:
> >
> > If I remember well the site preview is not implemented so it is normal
> that
> > links are broken.
> > I don't know if this has changed recently but if not I think we should
> > remove the site preview links from the draft email to avoid confusion.
> >
> > Best,
> > Stamatis
> >
> > On Mon, Aug 10, 2020 at 8:08 PM Andrei Sereda  wrote:
> >
> >> Yes I'm aware that for some reason site artifact wasn't built /
> uploaded.
> >>
> >> Meanwhile here are the release notes:
> >>
> >>
> https://github.com/apache/calcite/blob/calcite-1.25.0-rc0/site/_docs/history.md
> >>
> >> I'll double-check the build script / instructions, maybe I have missed
> >> something.
> >>
> >> On Mon, Aug 10, 2020 at 12:59 PM Haisheng Yuan 
> wrote:
> >>
> >>> +1 (binding)
> >>>
> >>> Environment:
> >>> Mac OS X 10.15.1 , JDK 1.8.0_162
> >>> - Ran unit tests (./gradlew build), OK
> >>> - Checked release notes, CALCITE-4156, CALCITE-4022 CALCITE-4115
> >>> CALCITE-4129 CALCITE-4111 are not part of Build and test suite changes,
> >>> those can be updated after release.
> >>>
> >>> - Haisheng
> >>>
> >>> On 2020/08/09 03:22:28, Andrei Sereda  wrote:
>  Hi all,
> 
>  I have created a build for Apache Calcite 1.25.0, release
>  candidate 0.
> 
>  Thanks to everyone who has contributed to this release.
> 
>  You can read the release notes here:
>  https://apache.github.io/calcite-site-preview/docs/history.html
> 
>  The commit to be voted upon:
> 
> >>>
> >>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=68b02dfd4af15bc94a91a0cd2a30655d04439555
> 
>  Its hash is 68b02dfd4af15bc94a91a0cd2a30655d04439555
> 
>  Tag:
>  https://github.com/apache/calcite/tree/calcite-1.25.0-rc0
> 
>  The artifacts to be voted on are located here:
> 
> >>
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.25.0-rc0
>  (revision 40922)
> 
>  RAT report:
>  https://apache.github.io/calcite-site-preview/rat/rat-report.txt
> 
>  Site preview is here:
>  https://apache.github.io/calcite-site-preview/
> 
>  JavaDoc API preview is here:
>  https://apache.github.io/calcite-site-preview/api
> 
>  The hashes of the artifacts are as follows:
> 
> >>>
> >>
> a5e61bd93657a274ee8a1d1ecbde68e3e471fd27b85bea179991b372f094ae3cdf692672245506a08b996534f9136be26569de81af2bb7d8f026799313957e87
>  *apache-calcite-1.25.0-src.tar.gz
> 
>  A staged Maven repository is available for review at:
> 
> >>>
> >>
> https://repository.apache.org/content/repositories/orgapachecalcite-1097/org/apache/calcite/
> 
>  Release artifacts are signed with the following key:
>  https://people.apache.org/keys/committer/sereda.asc
>  https://www.apache.org/dist/calcite/KEYS
> 
>  N.B.
>  To create the jars and test Apache Calcite: "./gradlew build".
> 
>  If you do not have a Java environment available, you can run the tests
>  using docker. To do so, install docker and docker-compose, then run
>  "docker-compose run test" from the root of the directory.
> 
>  Please vote on releasing this package as Apache Calcite 1.25.0.
> 
>  The vote is open for the next 72 hours and passes if a majority of at
>  least three +1 PMC votes are cast.
> 
>  [ ] +1 Release this package as Apache Calcite 1.25.0
>  [ ]  0 I don't feel strongly about it, but I'm okay with the release
>  [ ] -1 Do not release this package because...
> 
> 
>  Here is my vote:
> 
>  +1 (non binding)
> 
> >>>
> >>
>
>

RelFieldTrimmer misses optimization opportunity for always false condition

[Vladimir Ozerov]

Hi colleagues,

Consider the following query. If the RelFieldTrimmer is applied to this,
then the expression is reduced to an empty LogicalValues:
SELECT field FROM table WHERE TRUE IS FALSE

However, the following query will not be simplified, leaving a table scan
with "always false" filter:
SELECT * FROM table WHERE TRUE IS FALSE

After some debugging I found that the problem is in the following piece of
code in the RelFieldTrimmer:

// If the input is unchanged, and we need to project all columns,
// there's nothing we can do.
if (newInput == input
&& fieldsUsed.cardinality() == fieldCount) {
  return result(filter, Mappings.createIdentity(fieldCount));
}

My question - is it a known issue? Looks like this early return from the
method misses an important optimization opportunity.  Can this check be
removed completely?

Regards,
Vladimir

Re: [DISCUSS] Publish test JARS

[Michael Mior]

It will have to wait until the release is complete, but yes, I’d like to
get this in. Thanks for the patch Vladimir!

On Tue, Aug 11, 2020 at 04:38 Danny Chan  wrote:

> Michael, do you plan to push this function/patch ? We actually need this
> too ~
>
> Best,
> Danny Chan
> 在 2020年8月11日 +0800 AM3:06，Michael Mior ，写道：
> > Thanks for pointing back to the previous discussion. I'm fine with
> > publishing separate modules. No real preference on my end since I
> > haven't found consuming test artifacts to cause any problems. Although
> > I'm not sure what the changes to the Gradle config would look like to
> > support this.
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> > Le lun. 10 août 2020 à 14:49, Vladimir Sitnikov
> >  a écrit :
> > >
> > > Hi Michael,
> > >
> > > I suggest we go with adding explicitly published modules rather than
> > > publishing tests.
> > > Test artifacts do not have their own pom.xml, so they are not really
> > > convenient for consumers.
> > >
> > > Here's the relevant thread:
> > >
> https://lists.apache.org/thread.html/2ab9652ea855dce0b5b75cc221a5f74ebec536818847ae5ce6d284f5%40%3Cdev.calcite.apache.org%3E
> > >
> > > I guess the naming could be calcite-testkit or calcite-test-framework.
> > >
> > > Vladimir
>
-- 
--
Michael Mior
michael.m...@gmail.com

Re: master locked during 1.25.0 release

[Julian Hyde]

Danny,

Master branch is still locked.

I have backed out your commit 
https://github.com/apache/calcite/commit/aadb605decd6bb6a853e23fac4b0f479b2397e06
 
.

Julian


> On Aug 7, 2020, at 7:03 PM, Andrei Sereda  wrote:
> 
> Hello,
> 
> Please avoid committing anything to master while 1.25.0 release is in
> progress.
> 
> It will be unlocked after the voting / successful release.
> 
> Many thanks,
> Andrei.

Re: [VOTE] Release apache-calcite-1.25.0 (release candidate 0)

[Julian Hyde]

+1 (binding)

Checked keys, hashes, LICENSE, NOTICE, README; compiled and ran tests on 
Ubuntu/JDK 14; ran RAT.

* Andrei, I don’t think your key is in KEYS. Be sure to add it before the 
release announcement.

* There are several files in the source distro that are not in git: an empty 
‘arrow’ directory, and a non-empty ‘licenses’ directory.

Julian


> On Aug 11, 2020, at 2:33 AM, Stamatis Zampetakis  wrote:
> 
> If I remember well the site preview is not implemented so it is normal that
> links are broken.
> I don't know if this has changed recently but if not I think we should
> remove the site preview links from the draft email to avoid confusion.
> 
> Best,
> Stamatis
> 
> On Mon, Aug 10, 2020 at 8:08 PM Andrei Sereda  wrote:
> 
>> Yes I'm aware that for some reason site artifact wasn't built / uploaded.
>> 
>> Meanwhile here are the release notes:
>> 
>> https://github.com/apache/calcite/blob/calcite-1.25.0-rc0/site/_docs/history.md
>> 
>> I'll double-check the build script / instructions, maybe I have missed
>> something.
>> 
>> On Mon, Aug 10, 2020 at 12:59 PM Haisheng Yuan  wrote:
>> 
>>> +1 (binding)
>>> 
>>> Environment:
>>> Mac OS X 10.15.1 , JDK 1.8.0_162
>>> - Ran unit tests (./gradlew build), OK
>>> - Checked release notes, CALCITE-4156, CALCITE-4022 CALCITE-4115
>>> CALCITE-4129 CALCITE-4111 are not part of Build and test suite changes,
>>> those can be updated after release.
>>> 
>>> - Haisheng
>>> 
>>> On 2020/08/09 03:22:28, Andrei Sereda  wrote:
 Hi all,
 
 I have created a build for Apache Calcite 1.25.0, release
 candidate 0.
 
 Thanks to everyone who has contributed to this release.
 
 You can read the release notes here:
 https://apache.github.io/calcite-site-preview/docs/history.html
 
 The commit to be voted upon:
 
>>> 
>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=68b02dfd4af15bc94a91a0cd2a30655d04439555
 
 Its hash is 68b02dfd4af15bc94a91a0cd2a30655d04439555
 
 Tag:
 https://github.com/apache/calcite/tree/calcite-1.25.0-rc0
 
 The artifacts to be voted on are located here:
 
>> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.25.0-rc0
 (revision 40922)
 
 RAT report:
 https://apache.github.io/calcite-site-preview/rat/rat-report.txt
 
 Site preview is here:
 https://apache.github.io/calcite-site-preview/
 
 JavaDoc API preview is here:
 https://apache.github.io/calcite-site-preview/api
 
 The hashes of the artifacts are as follows:
 
>>> 
>> a5e61bd93657a274ee8a1d1ecbde68e3e471fd27b85bea179991b372f094ae3cdf692672245506a08b996534f9136be26569de81af2bb7d8f026799313957e87
 *apache-calcite-1.25.0-src.tar.gz
 
 A staged Maven repository is available for review at:
 
>>> 
>> https://repository.apache.org/content/repositories/orgapachecalcite-1097/org/apache/calcite/
 
 Release artifacts are signed with the following key:
 https://people.apache.org/keys/committer/sereda.asc
 https://www.apache.org/dist/calcite/KEYS
 
 N.B.
 To create the jars and test Apache Calcite: "./gradlew build".
 
 If you do not have a Java environment available, you can run the tests
 using docker. To do so, install docker and docker-compose, then run
 "docker-compose run test" from the root of the directory.
 
 Please vote on releasing this package as Apache Calcite 1.25.0.
 
 The vote is open for the next 72 hours and passes if a majority of at
 least three +1 PMC votes are cast.
 
 [ ] +1 Release this package as Apache Calcite 1.25.0
 [ ]  0 I don't feel strongly about it, but I'm okay with the release
 [ ] -1 Do not release this package because...
 
 
 Here is my vote:
 
 +1 (non binding)
 
>>> 
>> 

Re: [VOTE] Release apache-calcite-1.25.0 (release candidate 0)

[Stamatis Zampetakis]

If I remember well the site preview is not implemented so it is normal that
links are broken.
I don't know if this has changed recently but if not I think we should
remove the site preview links from the draft email to avoid confusion.

Best,
Stamatis

On Mon, Aug 10, 2020 at 8:08 PM Andrei Sereda  wrote:

> Yes I'm aware that for some reason site artifact wasn't built / uploaded.
>
> Meanwhile here are the release notes:
>
> https://github.com/apache/calcite/blob/calcite-1.25.0-rc0/site/_docs/history.md
>
> I'll double-check the build script / instructions, maybe I have missed
> something.
>
> On Mon, Aug 10, 2020 at 12:59 PM Haisheng Yuan  wrote:
>
> > +1 (binding)
> >
> > Environment:
> > Mac OS X 10.15.1 , JDK 1.8.0_162
> > - Ran unit tests (./gradlew build), OK
> > - Checked release notes, CALCITE-4156, CALCITE-4022 CALCITE-4115
> > CALCITE-4129 CALCITE-4111 are not part of Build and test suite changes,
> > those can be updated after release.
> >
> > - Haisheng
> >
> > On 2020/08/09 03:22:28, Andrei Sereda  wrote:
> > > Hi all,
> > >
> > > I have created a build for Apache Calcite 1.25.0, release
> > > candidate 0.
> > >
> > > Thanks to everyone who has contributed to this release.
> > >
> > > You can read the release notes here:
> > > https://apache.github.io/calcite-site-preview/docs/history.html
> > >
> > > The commit to be voted upon:
> > >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=68b02dfd4af15bc94a91a0cd2a30655d04439555
> > >
> > > Its hash is 68b02dfd4af15bc94a91a0cd2a30655d04439555
> > >
> > > Tag:
> > > https://github.com/apache/calcite/tree/calcite-1.25.0-rc0
> > >
> > > The artifacts to be voted on are located here:
> > >
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.25.0-rc0
> > > (revision 40922)
> > >
> > > RAT report:
> > > https://apache.github.io/calcite-site-preview/rat/rat-report.txt
> > >
> > > Site preview is here:
> > > https://apache.github.io/calcite-site-preview/
> > >
> > > JavaDoc API preview is here:
> > > https://apache.github.io/calcite-site-preview/api
> > >
> > > The hashes of the artifacts are as follows:
> > >
> >
> a5e61bd93657a274ee8a1d1ecbde68e3e471fd27b85bea179991b372f094ae3cdf692672245506a08b996534f9136be26569de81af2bb7d8f026799313957e87
> > > *apache-calcite-1.25.0-src.tar.gz
> > >
> > > A staged Maven repository is available for review at:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1097/org/apache/calcite/
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/sereda.asc
> > > https://www.apache.org/dist/calcite/KEYS
> > >
> > > N.B.
> > > To create the jars and test Apache Calcite: "./gradlew build".
> > >
> > > If you do not have a Java environment available, you can run the tests
> > > using docker. To do so, install docker and docker-compose, then run
> > > "docker-compose run test" from the root of the directory.
> > >
> > > Please vote on releasing this package as Apache Calcite 1.25.0.
> > >
> > > The vote is open for the next 72 hours and passes if a majority of at
> > > least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Calcite 1.25.0
> > > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > >
> > > Here is my vote:
> > >
> > > +1 (non binding)
> > >
> >
>

Re: [DISCUSS] Publish test JARS

[Danny Chan]

Michael, do you plan to push this function/patch ? We actually need this too ~

Best,
Danny Chan
在 2020年8月11日 +0800 AM3:06，Michael Mior ，写道：
> Thanks for pointing back to the previous discussion. I'm fine with
> publishing separate modules. No real preference on my end since I
> haven't found consuming test artifacts to cause any problems. Although
> I'm not sure what the changes to the Gradle config would look like to
> support this.
>
> --
> Michael Mior
> mm...@apache.org
>
> Le lun. 10 août 2020 à 14:49, Vladimir Sitnikov
>  a écrit :
> >
> > Hi Michael,
> >
> > I suggest we go with adding explicitly published modules rather than
> > publishing tests.
> > Test artifacts do not have their own pom.xml, so they are not really
> > convenient for consumers.
> >
> > Here's the relevant thread:
> > https://lists.apache.org/thread.html/2ab9652ea855dce0b5b75cc221a5f74ebec536818847ae5ce6d284f5%40%3Cdev.calcite.apache.org%3E
> >
> > I guess the naming could be calcite-testkit or calcite-test-framework.
> >
> > Vladimir