date:20180214

Re: Jekyll container configured for Github site generation

2018-02-14 Thread Enrico Olivelli

Il gio 15 feb 2018, 04:34 Edmon Begoli  ha scritto:

> Does anyone have a docker container with
>
> Jekyll configured for Github site generation like described in site/
> readme.md and published?
>
>
> If not, would it make sense to build few containers and publish them
> official on Dockerhub for various development and testing tasks?
>

In Apache BookKeeper community we are using Jekyll, and we have a bunch of
scripts to run it inside containers, no need to push to docker hub.
See
https://github.com/apache/bookkeeper/tree/master/site/docker

Hope that helps
Enrico

-- 


-- Enrico Olivelli

Jekyll container configured for Github site generation

2018-02-14 Thread Edmon Begoli

Does anyone have a docker container with

Jekyll configured for Github site generation like described in site/
readme.md and published?


If not, would it make sense to build few containers and publish them
official on Dockerhub for various development and testing tasks?

Re: copy method in RelSubset class

2018-02-14 Thread Alessandro Solimando

Hi,
while preparing some additional unit tests for the spark adapter (
https://github.com/asolimando/calcite/tree/SPARK-TESTS) I have stumbled
upon the issue several times.

This is the list of the tests that in my opinion should be succeeding but
are failing because of an invocation of *copy* method for *RelSubset* class:
- testFilterBetween
- testFilterIsIn
- testFilterTrue
- testFilterFalse
- testFilterOr
- testFilterIsNotNull
- testFilterIsNull

As you can infer from the name, the common trait among them is the presence
of a "complex" filtering condition in the where clause.

Just as a side note (not implying this is a solution), by replacing the
exception throwing with "return this;" inside *RelSubset.copy, *the
aforementioned tests pass.

Can you please acknowledge the issue (if any) so I can open a ticket, and
reference it in the "@Ignore" of those tests, so I can advance with the PR?

Best regards,
Alessandro

On 12 February 2018 at 09:56, Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

> Hello Julian,
> If I got it right, trimming the query plan for unused fields is a top-down
> procedure removing any reference to unused fields in the subplan rooted at
> the considered tree node.
>
> This, in principle, can affect also those coming from elements of
> *RelSubset*, independently from the fact that they are in an equivalence
> class, and that their result is "immutable". The only source of problem I
> see, is that the very concept of *RelSubset* suggests a "global scope",
> and updating it according to the contextual information of a specific
> subplan could break its correctness (the relational expressions composing
> *RelSubset* would be fitting only some of original contexts in which they
> were originally equivalent).
>
> However, *trimUnusedFields*, in my example, tries to update the traits of
> RelSubset's elements.
>
> So, if *RelSubset* should be immutable (traits included), then the
> *trimUnusedFields* method should never call *copy* on it, but it does,
> and the exception is thrown.
>
> The fact that implementing copy for *RelSubset* as the identity (that is,
> simply returning "this", ignoring any modification to the traits) did not
> introduce any problem reinforces the immutability hypothesis.
>
> Is my understanding correct?
> Given that the query looks legal, the problem looks "real".
> If this is confirmed, how do you suggest to address it?
>
> On 12 February 2018 at 00:04, Julian Hyde  wrote:
>
>> Can you tell me why you want to copy a RelSubset?
>>
>> A RelSubset is an equivalence class - a set of relational expressions
>> that always return the same results. So if you made a copy you’d be
>> creating another equivalent relational expression - that by definition
>> should be in the original RelSubset.
>>
>> > On Feb 11, 2018, at 1:18 PM, Alessandro Solimando <
>> alessandro.solima...@gmail.com> wrote:
>> >
>> > Hello community,
>> >
>> > I am adding a SparkAdapter test with the following query:
>> >
>> >> select *
>> >> from *(values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as
>> t(x, y)*
>> >> where x between 3 and 4
>> >>
>> >> When executed, an exception is thrown (the full stack trace is at the
>> end
>> > of the email) in *copy* method in *RelSubset* class, while Calcite is
>> > trying to get rid of unused terms (specifically, *trimUnusedFields*
>> method
>> > from *SqlToRelConverted* class).
>> >
>> > The signature of copy is as follows: public RelNode copy(RelTraitSet
>> > traitSet, List inputs)
>> >
>> > First of all, I don't understand the reason for the
>> > *UnsupportedOperationException* in the first place. Why a RelSubset
>> > shouldn't be copied?
>> >
>> > Assuming that the functionality is simply missing, I have considered two
>> > alternatives for implementing it:
>> > 1) copy as the identity function -> all Calcite tests pass, but I am
>> > ignoring the *traitSet* parameter in this way, looks odd
>> > 2) I have tried to build a new *RelSubset* by reusing the cluster and
>> set
>> > information from the object, and the trait argument of copy -> assert
>> > traits.allSimple();
>> > fails in the constructor
>> >
>> > In my example, the trait "[1]" (ordering detected at tuple level on the
>> 2nd
>> > component) is transformed into a composite trait "[[1]]", this makes the
>> > assertion fail.
>> > While I know what a trait is, I don't understand what a composite one
>> is.
>> > Do you have a concrete example?
>> >
>> > So the problem here is the introduction of the composite trait, which is
>> > caused by the *replace* method in *trimUnusedFields*:
>> >
>> > if (isTrimUnusedFields()) {
>> >>>  final RelFieldTrimmer trimmer = newFieldTrimmer();
>> >>>  final List collations =
>> >>>rootRel.getTraitSet().getTraits(RelCollationTraitDef.INSTANCE);
>> >>>  rootRel = trimmer.trim(rootRel);
>> >>>  if (!ordered
>> >>>  && collations != null
>> >>>  && !collations.isEmpty()
>> >>>  &&

subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"

2018-02-14 Thread Alessandro Solimando

Hello,
while executing this query Calcite tries to cast the subquery (RexSubQuery)
to a local reference (RexLocalRef), resulting in a ClassCastException.

Here is the query:

>
> *select **
> *from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)*
> *where exists (*
> *  select **
> *  from (values (1, 'a'), (2, 'b')) as v(w, z)*
> *  where w < x**)*


But the same happens with other (similar) queries:

> *select x*

*from (values (1, 'a'), (2, 'b')) as t(x, y)*

*where x <= all (*

*  select x*

*  from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x,
> y)*

*)*


Can you confirm the issue?
If so I will open a ticket, so I can mark as ignored the related tests with
a reference to such ticket (until the problem gets fixed).

If needed, they can inspected here:
https://github.com/asolimando/calcite/tree/SPARK-TESTS

The tests reproducing the issue are:
-) testFilterExists
-) testFilterNotExists
-) testSubqueryAny
-) testSubqueryAll

Below the full stack trace (for the first query mentioned above):

java.lang.RuntimeException: With materializationsEnabled=false, limit=0
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
> at
> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
> at
> org.apache.calcite.test.SparkAdapterTest.testFilterExists(SparkAdapterTest.java:720)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.sql.SQLException: Error while executing SQL "select *
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
> where exists (
>   select *
>   from (values (1, 'a'), (2, 'b')) as v(w, z)
>   where w < x
> )": org.apache.calcite.rex.RexSubQuery cannot be cast to
> org.apache.calcite.rex.RexLocalRef
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
> ... 27 more
> Caused by: java.lang.ClassCastException:
> org.apache.calcite.rex.RexSubQuery cannot be cast to
> org.apache.calcite.rex.RexLocalRef
> at
> org.apache.calcite.rex.RexProgramBuilder.registerInput(RexProgramBuilder.java:298)
> at
> org.apache.calcite.rex.RexProgramBuilder.addCondition(RexProgramBuilder.java:272)
> at
> org.apache.calcite.adapter.enumerable.EnumerableFilterToCalcRule.onMatch(EnumerableFilterToCalcRule.java:50)
> at
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:317)
> at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:555)
> at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:415)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:252)
> at
>

Re: "org.codehaus.commons.compiler.CompileException"

2018-02-14 Thread Alessandro Solimando

Hi again,
I have re-checked more carefully, the issue has indeed been fixed, sorry
about that.

On 14 February 2018 at 23:14, Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

> Hi Julian,
> I exclude that because in that branch I have got the two mentioned in the
> JIRA ticket (+ ~28 new tests) passing, and this one failing.
>
> Also the exception is different, in the ticket it was an assertion on the
> plan/resultset failing, while in this new case it is the code compilation
> module.
>
> I think I should open file a JIRA ticket if you agree.
>
> On 14 February 2018 at 20:27, Julian Hyde  wrote:
>
>>  https://issues.apache.org/jira/browse/CALCITE-2075 <
>> https://issues.apache.org/jira/browse/CALCITE-2075> was recently fixed.
>> Is this a duplicate?
>>
>>
>> > On Feb 13, 2018, at 1:29 PM, Alessandro Solimando <
>> alessandro.solima...@gmail.com> wrote:
>> >
>> > Hello,
>> > the exception in the object is raised when the following query is
>> issued to
>> > Calcite:
>> >
>> > *select x*
>> >
>> >
>> >>
>> >>
>> >>
>> >>
>> >> *from (values (1, 'a'), (2, 'b')) as t(x, y)where x > 1 unionselect
>> xfrom
>> >> (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x,
>> y)where x
>> >>> 1*
>> >
>> >
>> > I found several issues involving code compilation but none of them
>> seemed
>> > relevant for this case.
>> >
>> > The query looks fine to me, can you please check whether this it is an
>> > issue?
>> >
>> > You can check it here by launching *testUnionWithFiltersProject()
>> *method in
>> > *SparkAdapterTest.java*:
>> > https://github.com/asolimando/calcite/tree/SPARK-TESTS
>> >
>> > Full stack trace below:
>> >
>> > java.lang.RuntimeException: With materializationsEnabled=false, limit=0
>> > at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAss
>> ert.java:600)
>> > at
>> > org.apache.calcite.test.CalciteAssert$AssertQuery.returns(
>> CalciteAssert.java:1346)
>> > at
>> > org.apache.calcite.test.CalciteAssert$AssertQuery.returns(
>> CalciteAssert.java:1329)
>> > at
>> > org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUno
>> rdered(CalciteAssert.java:1357)
>> > at
>> > org.apache.calcite.test.SparkAdapterTest.commonTester(SparkA
>> dapterTest.java:93)
>> > at
>> > org.apache.calcite.test.SparkAdapterTest.testUnionWithFilter
>> sProject(SparkAdapterTest.java:651)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>> > at
>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> > at java.lang.reflect.Method.invoke(Method.java:498)
>> > at
>> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
>> FrameworkMethod.java:50)
>> > at
>> > org.junit.internal.runners.model.ReflectiveCallable.run(Refl
>> ectiveCallable.java:12)
>> > at
>> > org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr
>> ameworkMethod.java:47)
>> > at
>> > org.junit.internal.runners.statements.InvokeMethod.evaluate(
>> InvokeMethod.java:17)
>> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>> > at
>> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
>> 4ClassRunner.java:78)
>> > at
>> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
>> 4ClassRunner.java:57)
>> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>> > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>> > at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>> > at
>> > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs
>> (JUnit4IdeaTestRunner.java:68)
>> > at
>> > com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.star
>> tRunnerWithArgs(IdeaTestRunner.java:47)
>> > at
>> > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsA
>> ndStart(JUnitStarter.java:242)
>> > at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStart
>> er.java:70)
>> > Caused by: java.sql.SQLException: Error while executing SQL "select x
>> > from (values (1, 'a'), (2, 'b')) as t(x, y)
>> > where x > 1
>> > union
>> > select x
>> > from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x,
>> y)
>> > where x > 1
>> > ": Error while compiling generated Java code:
>> > public static class Record2_0 implements java.io.Serializable {
>> >  public int X;
>> >  public String Y;
>> >  public Record2_0() {}
>> >  public boolean equals(Object o) {
>> >if (this == o) {
>> >  return true;
>> >}
>> >if (!(o instanceof Record2_0)) {
>> >  return false;
>> >}
>> >return this.X == ((Record2_0) o).X && java.util.Objects.equals(this.
>> Y,
>> > ((Record2_0) o).Y);
>> >

[jira] [Created] (CALCITE-2179) General improvements for materialized view rewriting rule

2018-02-14 Thread Jesus Camacho Rodriguez (JIRA)

Jesus Camacho Rodriguez created CALCITE-2179:


 Summary: General improvements for materialized view rewriting rule
 Key: CALCITE-2179
 URL: https://issues.apache.org/jira/browse/CALCITE-2179
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
 Fix For: 1.16.0


This issue is for extending {{AbstractMaterializedViewRule}} rule:
- Support for rolling up date nodes. For instance, rewrite in the following 
case:
{code}
Materialization:
select "empid", floor(cast('1997-01-20' as timestamp) to month), count(*) + 1 
as c, sum("empid") as s
from "emps" group by "empid", floor(cast('1997-01-20' as timestamp) to month);
Query:
select floor(cast('1997-01-20' as timestamp) to year), sum("empid") as s
from "emps" group by floor(cast('1997-01-20' as timestamp) to year);
{code}
- Add flag to enable/disable fast bail out for joins. By default it is true, 
and thus, we were only creating the rewriting in the minimal subtree of plan 
operators. For instance:
{code}
View: (A JOIN B) JOIN C
Query: (((A JOIN B) JOIN D) JOIN C) JOIN E
{code}
We produce it at:
{code}
((A JOIN B) JOIN D) JOIN C
{code}
But not at:
{code}
(((A JOIN B) JOIN D) JOIN C) JOIN E
{code}
This is important when the rule is used with the Volcano planner together with 
other rules, e.g. join reordering, as it prevents that the search space grows 
unnecessarily. However, if we use the rewriting rule in isolation, fast bail 
out can lead to missing rewriting opportunities (e.g. for bushy join trees).
- Possibility to provide a HepProgram to optimize query branch in union 
rewritings. Note that when we produce a partial rewriting with a Union, the 
branch that will execute the (partial) query can be fully rewritten so we can 
add the compensation predicate. (We cannot do the same for views because the 
expression might not be computable if the needed subexpressions are not 
available in the view output). If we use Volcano with a determined set of 
rules, this might not be needed, hence providing this program is optional.
- Multiple small fixes discovered while testing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-2178) Extend expression simplifier to work on datetime FLOOR functions

2018-02-14 Thread Jesus Camacho Rodriguez (JIRA)

Jesus Camacho Rodriguez created CALCITE-2178:


 Summary: Extend expression simplifier to work on datetime FLOOR 
functions
 Key: CALCITE-2178
 URL: https://issues.apache.org/jira/browse/CALCITE-2178
 Project: Calcite
  Issue Type: Improvement
  Components: core
 Environment: Extend expression simplifier to support:
{code}
FLOOR(FLOOR(CAST('2010-01-10 00:00:00' AS TIMESTAMP) TO HOUR) TO DAY) => 
FLOOR(CAST('2010-01-10 00:00:00' AS TIMESTAMP) TO DAY)
{code}
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
 Fix For: 1.16.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: "org.codehaus.commons.compiler.CompileException"

2018-02-14 Thread Alessandro Solimando

Hi Julian,
I exclude that because in that branch I have got the two mentioned in the
JIRA ticket (+ ~28 new tests) passing, and this one failing.

Also the exception is different, in the ticket it was an assertion on the
plan/resultset failing, while in this new case it is the code compilation
module.

I think I should open file a JIRA ticket if you agree.

On 14 February 2018 at 20:27, Julian Hyde  wrote:

>  https://issues.apache.org/jira/browse/CALCITE-2075 <
> https://issues.apache.org/jira/browse/CALCITE-2075> was recently fixed.
> Is this a duplicate?
>
>
> > On Feb 13, 2018, at 1:29 PM, Alessandro Solimando <
> alessandro.solima...@gmail.com> wrote:
> >
> > Hello,
> > the exception in the object is raised when the following query is issued
> to
> > Calcite:
> >
> > *select x*
> >
> >
> >>
> >>
> >>
> >>
> >> *from (values (1, 'a'), (2, 'b')) as t(x, y)where x > 1 unionselect
> xfrom
> >> (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x,
> y)where x
> >>> 1*
> >
> >
> > I found several issues involving code compilation but none of them seemed
> > relevant for this case.
> >
> > The query looks fine to me, can you please check whether this it is an
> > issue?
> >
> > You can check it here by launching *testUnionWithFiltersProject()
> *method in
> > *SparkAdapterTest.java*:
> > https://github.com/asolimando/calcite/tree/SPARK-TESTS
> >
> > Full stack trace below:
> >
> > java.lang.RuntimeException: With materializationsEnabled=false, limit=0
> > at org.apache.calcite.test.CalciteAssert.assertQuery(
> CalciteAssert.java:600)
> > at
> > org.apache.calcite.test.CalciteAssert$AssertQuery.
> returns(CalciteAssert.java:1346)
> > at
> > org.apache.calcite.test.CalciteAssert$AssertQuery.
> returns(CalciteAssert.java:1329)
> > at
> > org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(
> CalciteAssert.java:1357)
> > at
> > org.apache.calcite.test.SparkAdapterTest.commonTester(
> SparkAdapterTest.java:93)
> > at
> > org.apache.calcite.test.SparkAdapterTest.testUnionWithFiltersProject(
> SparkAdapterTest.java:651)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> FrameworkMethod.java:50)
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(
> ReflectiveCallable.java:12)
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(
> FrameworkMethod.java:47)
> > at
> > org.junit.internal.runners.statements.InvokeMethod.
> evaluate(InvokeMethod.java:17)
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:78)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:57)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> > at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> > at
> > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(
> JUnit4IdeaTestRunner.java:68)
> > at
> > com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.
> startRunnerWithArgs(IdeaTestRunner.java:47)
> > at
> > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(
> JUnitStarter.java:242)
> > at com.intellij.rt.execution.junit.JUnitStarter.main(
> JUnitStarter.java:70)
> > Caused by: java.sql.SQLException: Error while executing SQL "select x
> > from (values (1, 'a'), (2, 'b')) as t(x, y)
> > where x > 1
> > union
> > select x
> > from (values (1, 'a'), (2, 'b'), (1, 'b'), (2, 'c'), (2, 'c')) as t(x, y)
> > where x > 1
> > ": Error while compiling generated Java code:
> > public static class Record2_0 implements java.io.Serializable {
> >  public int X;
> >  public String Y;
> >  public Record2_0() {}
> >  public boolean equals(Object o) {
> >if (this == o) {
> >  return true;
> >}
> >if (!(o instanceof Record2_0)) {
> >  return false;
> >}
> >return this.X == ((Record2_0) o).X && java.util.Objects.equals(this.
> Y,
> > ((Record2_0) o).Y);
> >  }
> >  public int hashCode() {
> >int h = 0;
> >h = org.apache.calcite.runtime.Utilities.hash(h, this.X);
> >h = org.apache.calcite.runtime.Utilities.hash(h, this.Y);
> >return h;
> >  }
> >  public int compareTo(Record2_0 that) {
> >int c;
> >c = org.apache.calcite.runtime.Utilities.compare(this.X, that.X);
> >

Use Calcite adapter with Elastic search and Java

2018-02-14 Thread Saurabh Pathak

Hello,

I want to use calcite adapter with elastic search and java. But i haven't idea 
how to use calcite elastic search adapter could you please let us know how to 
use calcite adapter with elastic search and java.

Thanks in advance for your co operation

Thanks & Regards
Saurabh Pathak

[jira] [Created] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match

2018-02-14 Thread Andrew Pilloud (JIRA)

Andrew Pilloud created CALCITE-2177:
---

 Summary: TUMBLE_START does not respect AS when SELECT and GROUP BY 
match
 Key: CALCITE-2177
 URL: https://issues.apache.org/jira/browse/CALCITE-2177
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.13.0
Reporter: Andrew Pilloud
Assignee: Julian Hyde


When the order of SELECT and GROUP BY arguments match, the output of 
TUMBLE_START and HOP_START can not be named with AS. Found while trying to 
implement Nexmark benchmark queries in Apache Beam.

This query fails with "An exception occured while executing the Java class. 
null: InvocationTargetException: Type 'RecordType(TIMESTAMP(0) $f0)' has no 
field 'starttime'"
{code:sql}
SELECT B.starttime FROM
   (SELECT
 TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
  FROM Bid B1
  GROUP BY
 TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
{code}
This more useful query also fails with "An exception occured while executing 
the Java class. null: InvocationTargetException: Type 'RecordType(BIGINT 
auction, BIGINT price, TIMESTAMP(0) $f2)' has no field 'starttime'"
{code:sql}
SELECT B.starttime FROM
   (SELECT B1.auction, B1.price,
 TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
  FROM Bid B1
  GROUP BY B1.auction, B1.price,
 TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
{code}
 However swap the order of the select arguments and it works as expected:
{code:sql}
SELECT B.starttime FROM
   (SELECT B1.price, B1.auction,
 TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
  FROM Bid B1
  GROUP BY B1.auction, B1.price,
  TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
{code}
 

More detailed stack from second query:
{code:java}
Caused by: java.lang.AssertionError: Type 'RecordType(TIMESTAMP(0) $f0, BIGINT 
auction, BIGINT price)' has no field 'starttime'
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:175)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3533)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:210)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4545)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3904)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4438)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3744)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:619)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3054)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:555)
at 
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:232)
at 
org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.convertToRelNode(BeamQueryPlanner.java:164)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Using Calcite for query planning

2018-02-14 Thread Julian Hyde

It sounds like a good fit. Parse the SQL, translate to relational algebra, 
apply some query transformation rules on the algebra.

If you have a few simple transformations in mind, you may be able to achieve it 
without a cost model. Or, as you propose, a simple model based on cardinality.

To convert a union of conjunctive queries to a conjunction of possibly-union 
queries, you would probably need a rule called UnionJoinTransposeRule that 
converts

  (Union (Join X Y) (Join X Y))

into

  (Join X (Union Y Z))

or something like that, and combines it with some existing rules to push 
unions. There is currently no such rule but it would not be hard to write.

Julian


> On Feb 14, 2018, at 5:24 AM, Guohui Xiao  wrote:
> 
> Hi,
> 
> We are considering using Calcite to perform cost-based query optimization
> in our project.
> 
> Specifically, we can already generate some SQL queries expressed in some
> relational algebra expression through our API, and we want to optimize the
> generated expressions using Calcite.
> 
> We have a cost model based on cardinality estimation. We want to use it to
> convert e.g., a union of conjunctive queries (UCQ) into a a join of UCQs.
> 
> We would like to understand how much efforts are needed to realize our idea
> using Calcite.
> 
> Do you have suggestions about this?
> 
> Thanks in advance.
> 
> Best regards,
> 
> Guohui & Davide
> 
> 
> 
> -- 
> Guohui Xiao, PhD
> Assistant Professor with a fixed-term contract
> KRDB - Faculty of Computer Science
> Free University of Bozen-Bolzano
> Piazza Domenicani, 3
> I-39100 Bolzano, Italy
> 
> http://www.ghxiao.org

Using Calcite for query planning

2018-02-14 Thread Guohui Xiao

Hi,

We are considering using Calcite to perform cost-based query optimization
in our project.

Specifically, we can already generate some SQL queries expressed in some
relational algebra expression through our API, and we want to optimize the
generated expressions using Calcite.

We have a cost model based on cardinality estimation. We want to use it to
convert e.g., a union of conjunctive queries (UCQ) into a a join of UCQs.

We would like to understand how much efforts are needed to realize our idea
using Calcite.

Do you have suggestions about this?

Thanks in advance.

Best regards,

Guohui & Davide



-- 
Guohui Xiao, PhD
Assistant Professor with a fixed-term contract
KRDB - Faculty of Computer Science
Free University of Bozen-Bolzano
Piazza Domenicani, 3
I-39100 Bolzano, Italy

http://www.ghxiao.org

Apache EU Roadshow CFP Closing Soon (23 February)

2018-02-14 Thread Sharan F


Hello Everyone

This is an initial reminder to let you all know that we are holding an 
Apache EU Roadshow co-located with FOSS Backstage in Berlin on 13^th and 
14^th June 2018. https://s.apache.org/tCHx


The Call for Proposals (CFP) for the Apache EU Roadshow is currently 
open and will close at the end of next week, so if you have been 
delaying making a submission because the closing date seemed a long way 
off, then it's time to start getting your proposals submitted.


So what are we looking for?
We will have 2 Apache Devrooms available during the 2 day Roadshow so 
are looking for projects including incubating ones, to submit 
presentations, panel discussions, BoFs, or workshop proposals. The main 
focus of the Roadshow will be IoT, Cloud, Httpd and Tomcat so if your 
project is involved in or around any of these technologies at Apache 
then we are very interested in hearing from you.


Community and collaboration is important at Apache so if your project is 
interested in organising a project sprint, meetup or hackathon during 
the Roadshow, then please submit it inthe CFP as we do have some space 
available to allocate for these.


If you are wanting to submit a talk on open source community related 
topics such as the Apache Way, governance or legal aspects then please 
submit these to the CFP for FOSS Backstage.


Tickets for the Apache EU Roadshow are included as part of the 
registration for FOSS Backstage, so to attend the Roadshow you will need 
to register for FOSS Backstage. Early Bird tickets are still available 
until the 21^st February 2018.


Please see below for important URLs to remember:

-  To submit a CFP for the Apache EU Roadshow 
:http://apachecon.com/euroadshow18/ 


-  To submit a CFP for FOSS Backstage : 
https://foss-backstage.de/call-papers


-  To register to attend the Apache EU Roadshow and/or FOSS Backstage : 
https://foss-backstage.de/tickets


For further updates and information about the Apache EU Roadshowplease 
check http://apachecon.com/euroadshow18/


Thanks
Sharan Foga, VP Apache Community Development

Fwd: Travel Assistance applications open. Please inform your communities

2018-02-14 Thread Michael Mior

See information below on travel assistance for ApacheCon 2018. It would be
great to see some representation for Calcite. I'm hoping to attend myself,
although not yet confirmed.

--
Michael Mior
mm...@apache.org

-- Forwarded message --
From: Gavin McDonald 
Date: 2018-02-14 4:34 GMT-05:00
Subject: Travel Assistance applications open. Please inform your communities
To: travel-assista...@apache.org


Hello PMCs.

Please could you forward on the below email to your dev and user lists.

Thanks

Gav…

—
The Travel Assistance Committee (TAC) are pleased to announce that travel
assistance applications for ApacheCon NA 2018 are now open!

We will be supporting ApacheCon NA Montreal, Canada on 24th - 29th
September 2018

 TAC exists to help those that would like to attend ApacheCon events, but
are unable to do so for financial reasons.
For more info on this years applications and qualifying criteria, please
visit the TAC website at < http://www.apache.org/travel/ >. Applications
are now open and will close 1st May.

*Important*: Applications close on May 1st, 2018. Applicants have until the
closing date above to submit their applications (which should contain as
much supporting material as required to efficiently and accurately process
their request), this will enable TAC to announce successful awards shortly
afterwards.

As usual, TAC expects to deal with a range of applications from a diverse
range of backgrounds. We therefore encourage (as always) anyone thinking
about sending in an application to do so ASAP.
We look forward to greeting many of you in Montreal

Kind Regards,
Gavin - (On behalf of the Travel Assistance Committee)
—

Multi Product support in a single RelNode tree

2018-02-14 Thread Abbas Gadhia

Hi,
I want to build a RelNode tree with different conventions on different RelNodes 
(for example: in the following select query "select * from t1,t2", t1 is a 
table from Oracle and t2 is a table from SqlServer). 

I'm confused whether i should be using a single SchemaPlus to hold table 
references from both Oracle and SqlServer or I should be creating a different 
SchemaPlus for each product. Different SchemaPlus would force me to use a 
different RelBuilder, so my guess is that a single SchemaPlus with the 
following hierarchy may suffice ("oracle" -> "database1" -> "schema1" -> "t1"). 
However, I suspect this single hierarchy (with the product name inside) may not 
play well with other parts of Calcite.
Any thoughts, however small would be appreciated.
Thanks
Abbas

Re: Jekyll container configured for Github site generation

Jekyll container configured for Github site generation

Re: copy method in RelSubset class

subquery leading to a "java.lang.ClassCastException: RexSubQuery cannot be cast to RexLocalRef"

Re: "org.codehaus.commons.compiler.CompileException"

[jira] [Created] (CALCITE-2179) General improvements for materialized view rewriting rule

[jira] [Created] (CALCITE-2178) Extend expression simplifier to work on datetime FLOOR functions

Re: "org.codehaus.commons.compiler.CompileException"

Use Calcite adapter with Elastic search and Java

[jira] [Created] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match

Re: Using Calcite for query planning

Using Calcite for query planning

Apache EU Roadshow CFP Closing Soon (23 February)

Fwd: Travel Assistance applications open. Please inform your communities

Multi Product support in a single RelNode tree

15 matches

Site Navigation

Mail list logo

Footer information