date:20190830

[jira] [Comment Edited] (CALCITE-3315) Multiple failures in Druid IT tests due to implicit casts

2019-08-30 Thread Danny Chan (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920015#comment-16920015
 ] 

Danny Chan edited comment on CALCITE-3315 at 8/31/19 4:25 AM:
--

[~zabetak] I'm so sorry to be a blocker.

I have tried many times to install the  calcite-test-dataset but failed, many 
times because of the files downloading. If i do mvn install skipping tests, it 
succeed, but i still can not connect to the Druid.

I tried to fire the PR first to fix the plan change(expected). Can you please 
help to make sure the tests result are still correct ?


was (Author: danny0405):
[~zabetak] I'm so sorry to be a blocker, but i have tried many times to install 
the 

calcite-test-dataset but failed, many times because of the files downloading. 
If i install skipping tests, if succeed, but i still can not connect to the 
Druid.

I tried to fire the PR first to fix the plan change(expected). Can you please 
help to make sure the tests result are still correct ?

> Multiple failures in Druid IT tests due to implicit casts 
> --
>
> Key: CALCITE-3315
> URL: https://issues.apache.org/jira/browse/CALCITE-3315
> Project: Calcite
>  Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> After merging CALCITE-2302 there are 12 test failures in DruidAdapterIT and 
> DruidAdapter2IT.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3315) Multiple failures in Druid IT tests due to implicit casts

2019-08-30 Thread Danny Chan (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920015#comment-16920015
 ] 

Danny Chan commented on CALCITE-3315:
-

[~zabetak] I'm so sorry to be a blocker, but i have tried many times to install 
the 

calcite-test-dataset but failed, many times because of the files downloading. 
If i install skipping tests, if succeed, but i still can not connect to the 
Druid.

I tried to fire the PR first to fix the plan change(expected). Can you please 
help to make sure the tests result are still correct ?

> Multiple failures in Druid IT tests due to implicit casts 
> --
>
> Key: CALCITE-3315
> URL: https://issues.apache.org/jira/browse/CALCITE-3315
> Project: Calcite
>  Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> After merging CALCITE-2302 there are 12 test failures in DruidAdapterIT and 
> DruidAdapter2IT.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3310) Approximate and exact aggregate calls are recognized as the same during sql-to-rel conversion

2019-08-30 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3310:

Labels: pull-request-available  (was: )

> Approximate and exact aggregate calls are recognized as the same during 
> sql-to-rel conversion
> -
>
> Key: CALCITE-3310
> URL: https://issues.apache.org/jira/browse/CALCITE-3310
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>
> For sql:
> {code:sql}
> SELECT empno, count(distinct ename)
> approx_count_distinct(ename)
> FROM emp
> GROUP BY empno
> {code}
> After sql-to-rel conversion, the plan is:
> {code:sql}
> LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$1])
>   LogicalAggregate(group=[{0}], EXPR$1=[COUNT(DISTINCT $1)])
> LogicalProject(EMPNO=[$0], ENAME=[$1])
>   LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (CALCITE-3292) SqlToRelConverter#substituteSubQuery fails with NullPointerException when converting SqlUpdate.

2019-08-30 Thread Haisheng Yuan (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haisheng Yuan resolved CALCITE-3292.

Fix Version/s: 1.21.0
   Resolution: Fixed

Fixed in 
https://github.com/apache/calcite/commit/b8d24a6f7f2b4c7cb8093eea158f1fa00dbc6ffc,
 thanks for the PR, [~jinxing6...@126.com]!

> SqlToRelConverter#substituteSubQuery fails with NullPointerException when 
> converting SqlUpdate.
> ---
>
> Key: CALCITE-3292
> URL: https://issues.apache.org/jira/browse/CALCITE-3292
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Current code fails below test
> {code:java}
> // code placeholder
> @Test public void testUpdateSubQueryWithIn1() {
>   final String sql = "update emp\n"
>   + "set empno = 1 where emp.empno in (\n"
>   + "  select emp.empno from emp where emp.empno=2)";
>   sql(sql).ok();
> }
> java.lang.NullPointerExceptionjava.lang.NullPointerException at 
> org.apache.calcite.rel.logical.LogicalJoin.create(LogicalJoin.java:146) at 
> org.apache.calcite.rel.logical.LogicalJoin.create(LogicalJoin.java:163) at 
> org.apache.calcite.sql2rel.SqlToRelConverter.substituteSubQuery(SqlToRelConverter.java:1130)
>  at 
> org.apache.calcite.sql2rel.SqlToRelConverter.replaceSubQueries(SqlToRelConverter.java:1014)
>  at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertUpdate(SqlToRelConverter.java:3574)
>  at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3176)
>  at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:563)
>  at 
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:616)
>  at 
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:731)
>  at 
> org.apache.calcite.test.SqlToRelConverterTest$Sql.convertsTo(SqlToRelConverterTest.java:3601)
>  at 
> org.apache.calcite.test.SqlToRelConverterTest$Sql.ok(SqlToRelConverterTest.java:3593)
> {code}
>  In above case, *Subquery* is used as *{{SqlUpdate#condition}}*, when 
> converting and trying to replace the subquery in *{{SqlUpdate#condition}}*, 
> *{{BalckBoard#root}}* is *{{null}}* and it makes no sense to do the subquery 
> substitution.
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3315) Multiple failures in Druid IT tests due to implicit casts

2019-08-30 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3315:

Labels: pull-request-available  (was: )

> Multiple failures in Druid IT tests due to implicit casts 
> --
>
> Key: CALCITE-3315
> URL: https://issues.apache.org/jira/browse/CALCITE-3315
> Project: Calcite
>  Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>
> After merging CALCITE-2302 there are 12 test failures in DruidAdapterIT and 
> DruidAdapter2IT.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (CALCITE-3311) Add doc to site for implicit type coercion

2019-08-30 Thread Danny Chan (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Danny Chan resolved CALCITE-3311.
-
Resolution: Fixed

Fixed in 
[72680df|https://github.com/apache/calcite/commit/72680df93ef089abd0a7c0ac8cdc6253619c2ebe]
 !

> Add doc to site for implicit type coercion
> --
>
> Key: CALCITE-3311
> URL: https://issues.apache.org/jira/browse/CALCITE-3311
> Project: Calcite
>  Issue Type: Sub-task
>  Components: site
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3315) Multiple failures in Druid IT tests due to implicit casts

2019-08-30 Thread Danny Chan (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919626#comment-16919626
 ] 

Danny Chan commented on CALCITE-3315:
-

Thanks [~zabetak] for reporting, fixing ~

> Multiple failures in Druid IT tests due to implicit casts 
> --
>
> Key: CALCITE-3315
> URL: https://issues.apache.org/jira/browse/CALCITE-3315
> Project: Calcite
>  Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Priority: Blocker
> Fix For: 1.21.0
>
>
> After merging CALCITE-2302 there are 12 test failures in DruidAdapterIT and 
> DruidAdapter2IT.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3315) Multiple failures in Druid IT tests due to implicit casts

2019-08-30 Thread Stamatis Zampetakis (Jira)

Stamatis Zampetakis created CALCITE-3315:


 Summary: Multiple failures in Druid IT tests due to implicit casts 
 Key: CALCITE-3315
 URL: https://issues.apache.org/jira/browse/CALCITE-3315
 Project: Calcite
  Issue Type: Bug
Reporter: Stamatis Zampetakis
 Fix For: 1.21.0


After merging CALCITE-2302 there are 12 test failures in DruidAdapterIT and 
DruidAdapter2IT.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919575#comment-16919575
 ] 

Ruben Quesada Lopez commented on CALCITE-3307:
--

BTW, I just noticed that there is another "skip for Windows" in 
PigRelBuilderStyleTest:
{code}
public class PigRelBuilderStyleTest extends AbstractPigTest {
  public PigRelBuilderStyleTest() {
Assume.assumeThat("Pigs don't like Windows", File.separatorChar, is('/'));
  }
  ...
{code}
If I remove that line, I get the same exception {{java.io.IOException: Could 
not locate executable null\bin\winutils.exe in the Hadoop binaries.}} in those 
tests.



> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Ruben Quesada Lopez
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919575#comment-16919575
 ] 

Ruben Quesada Lopez edited comment on CALCITE-3307 at 8/30/19 2:05 PM:
---

BTW, I just noticed that there was already another "skip for Windows" in 
PigRelBuilderStyleTest:
{code}
public class PigRelBuilderStyleTest extends AbstractPigTest {
  public PigRelBuilderStyleTest() {
Assume.assumeThat("Pigs don't like Windows", File.separatorChar, is('/'));
  }
  ...
{code}
If I remove that line, I get the same exception {{java.io.IOException: Could 
not locate executable null\bin\winutils.exe in the Hadoop binaries.}} in those 
tests.




was (Author: rubenql):
BTW, I just noticed that there is another "skip for Windows" in 
PigRelBuilderStyleTest:
{code}
public class PigRelBuilderStyleTest extends AbstractPigTest {
  public PigRelBuilderStyleTest() {
Assume.assumeThat("Pigs don't like Windows", File.separatorChar, is('/'));
  }
  ...
{code}
If I remove that line, I get the same exception {{java.io.IOException: Could 
not locate executable null\bin\winutils.exe in the Hadoop binaries.}} in those 
tests.



> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Ruben Quesada Lopez
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at

[jira] [Commented] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919520#comment-16919520
 ] 

Ruben Quesada Lopez commented on CALCITE-3307:
--

You're welcome! :)

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Ruben Quesada Lopez
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Stamatis Zampetakis (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stamatis Zampetakis resolved CALCITE-3307.
--
Resolution: Fixed

Fixed in 
[e2300d297d291514906ca002d045664732ec928c|https://github.com/apache/calcite/commit/e2300d297d291514906ca002d045664732ec928c]!
 Thanks [~rubenql] ;)

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Ruben Quesada Lopez
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3312) TIMESTAMPDIFF cannot be converted to SQL

2019-08-30 Thread Danny Chan (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919517#comment-16919517
 ] 

Danny Chan commented on CALCITE-3312:
-

Thanks [~Pavel Gubin] for reporting this, maybe we can overwrite the 
unparseCall logic to handle this special function.

> TIMESTAMPDIFF cannot be converted to SQL
> 
>
> Key: CALCITE-3312
> URL: https://issues.apache.org/jira/browse/CALCITE-3312
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Pavel Gubin
>Priority: Major
>
> The following test in {{RelToSqlConverterTest}}:
> {code}
>   @Test public void testTimestampDiff() {
> String query = "SELECT {fn TIMESTAMPDIFF(SQL_TSI_YEAR,TIMESTAMP 
> '2016-01-01 00:00:00', TIMESTAMP '2017-01-01 00:00:00')}";
> String expected = "";
> sql(query).ok(expected);
>   }
> {code}
> fails with:
> {noformat}
> java.lang.UnsupportedOperationException: class 
> org.apache.calcite.sql.SqlSyntax$6: SPECIAL
>   at org.apache.calcite.util.Util.needToImplement(Util.java:967)
>   at org.apache.calcite.sql.SqlSyntax$6.unparse(SqlSyntax.java:116)
>   at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:105)
>   at org.apache.calcite.sql.SqlUtil.unparseBinarySyntax(SqlUtil.java:382)
>   at org.apache.calcite.sql.SqlSyntax$3.unparse(SqlSyntax.java:65)
>   at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
>   at 
> org.apache.calcite.sql.fun.SqlCastFunction.unparse(SqlCastFunction.java:174)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
>   at org.apache.calcite.sql.SqlNodeList.commaList(SqlNodeList.java:121)
>   at 
> org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:365)
>   at 
> org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:354)
>   at 
> org.apache.calcite.sql.SqlSelectOperator.unparse(SqlSelectOperator.java:152)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlSelect.unparse(SqlSelect.java:230)
>   at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:152)
>   at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:157)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.toSql(RelToSqlConverterTest.java:145)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.access$100(RelToSqlConverterTest.java:77)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.exec(RelToSqlConverterTest.java:3634)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.ok(RelToSqlConverterTest.java:3609)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testTimestampDiff(RelToSqlConverterTest.java:1942)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>   at 
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
>   at 
>

[jira] [Commented] (CALCITE-3310) Approximate and exact aggregate calls are recognized as the same during sql-to-rel conversion

2019-08-30 Thread Danny Chan (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919513#comment-16919513
 ] 

Danny Chan commented on CALCITE-3310:
-

The plan is wrong and the agg output should not be reused.

> Approximate and exact aggregate calls are recognized as the same during 
> sql-to-rel conversion
> -
>
> Key: CALCITE-3310
> URL: https://issues.apache.org/jira/browse/CALCITE-3310
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> For sql:
> {code:sql}
> SELECT empno, count(distinct ename)
> approx_count_distinct(ename)
> FROM emp
> GROUP BY empno
> {code}
> After sql-to-rel conversion, the plan is:
> {code:sql}
> LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$1])
>   LogicalAggregate(group=[{0}], EXPR$1=[COUNT(DISTINCT $1)])
> LogicalProject(EMPNO=[$0], ENAME=[$1])
>   LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3282) Make every SqlDialect unparse their own data type

2019-08-30 Thread Danny Chan (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919451#comment-16919451
 ] 

Danny Chan commented on CALCITE-3282:
-

Let's just ignore unparsing the DDL first, Calcite's DDL has much difference 
for different sql engines, especially Hive, i think it needs much more effort 
if we want to unparse the DDL based on sql dialects.

> Make every SqlDialect unparse their own data type
> -
>
> Key: CALCITE-3282
> URL: https://issues.apache.org/jira/browse/CALCITE-3282
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: feng huang
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Every database might have different type or same type but different type 
> name, therefore making every SqlDialect unparse their own data type is a 
> suitable way.
> For example, there is a sql “select cast(col as int) from table” change to 
> hive sql "select cast(col as integer) from table", but "integer" is not 
> allowed in hive.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3311) Add doc to site for implicit type coercion

2019-08-30 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3311:

Labels: pull-request-available  (was: )

> Add doc to site for implicit type coercion
> --
>
> Key: CALCITE-3311
> URL: https://issues.apache.org/jira/browse/CALCITE-3311
> Project: Calcite
>  Issue Type: Sub-task
>  Components: site
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3314) CVSS dependency-check-maven fails for calcite-pig, calcite-piglet, calcite-spark

2019-08-30 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3314:

Labels: pull-request-available  (was: )

> CVSS dependency-check-maven fails for calcite-pig, calcite-piglet, 
> calcite-spark
> 
>
> Key: CALCITE-3314
> URL: https://issues.apache.org/jira/browse/CALCITE-3314
> Project: Calcite
>  Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>
> Calcite build fails if the CVSS dependency check is active since there are 
> serious vulnerabilties in calcite-pig, calcite-piglet, calcite-spark. 
> Running mvn install -Ppedantic -fn gives the following errors:
> {noformat}
> ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
> (default) on project calcite-pig: 
> [ERROR] 
> [ERROR] One or more dependencies were identified with vulnerabilities that 
> have a CVSS score greater than or equal to '8.0': 
> [ERROR] 
> [ERROR] jetty-6.1.26.jar: CVE-2017-7658, CVE-2017-7657
> [ERROR] groovy-all-1.8.6.jar: CVE-2015-3253, CVE-2016-6814
> [ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
> (default) on project calcite-piglet: 
> [ERROR] 
> [ERROR] One or more dependencies were identified with vulnerabilities that 
> have a CVSS score greater than or equal to '8.0': 
> [ERROR] 
> [ERROR] jetty-6.1.26.jar: CVE-2017-7658, CVE-2017-7657
> [ERROR] jackson-core-asl-1.8.8.jar: CVE-2017-17485, CVE-2017-7525, 
> CVE-2017-15095
> [ERROR] groovy-all-1.8.6.jar: CVE-2015-3253, CVE-2016-6814
> [ERROR] jackson-xc-1.8.3.jar: CVE-2017-17485, CVE-2017-7525, CVE-2017-15095
> [ERROR] hadoop-auth-2.7.5.jar: CVE-2018-8029, CVE-2018-11766, CVE-2018-8009
> [ERROR] api-util-1.0.0-M20.jar: CVE-2018-1337
> [ERROR] zookeeper-3.4.6.jar: CVE-2016-5017
> [ERROR] 
> htrace-core-3.1.0-incubating.jar/META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml:
>  CVE-2017-17485, CVE-2018-5968, CVE-2017-15095, CVE-2019-14379, 
> CVE-2018-19362, CVE-2018-19361, CVE-2018-19360, CVE-2017-7525, 
> CVE-2018-11307, CVE-2018-14718, CVE-2018-7489, CVE-2018-14719, 
> CVE-2018-14721, CVE-2018-14720
> [ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
> (default) on project calcite-spark: 
> [ERROR] 
> [ERROR] One or more dependencies were identified with vulnerabilities that 
> have a CVSS score greater than or equal to '8.0': 
> [ERROR] 
> [ERROR] spark-core_2.10-2.2.0.jar: CVE-2018-17190
> [ERROR] api-util-1.0.0-M20.jar: CVE-2018-1337
> [ERROR] hadoop-mapreduce-client-core-2.7.5.jar: CVE-2018-8029, 
> CVE-2018-11766, CVE-2018-8009
> [ERROR] bcprov-jdk15on-1.51.jar: CVE-2018-1000613
> [ERROR] zookeeper-3.4.6.jar: CVE-2016-5017
> [ERROR] unused-1.0.0.jar: CVE-2018-17190
> [ERROR] 
> htrace-core-3.1.0-incubating.jar/META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml:
>  CVE-2017-17485, CVE-2018-5968, CVE-2017-15095, CVE-2019-14379, 
> CVE-2018-19362, CVE-2018-19361, CVE-2018-19360, CVE-2017-7525, 
> CVE-2018-11307, CVE-2018-14718, CVE-2018-7489, CVE-2018-14719, 
> CVE-2018-14721, CVE-2018-14720
> [ERROR] 
> spark-core_2.10-2.2.0.jar/META-INF/maven/org.eclipse.jetty/jetty-plus/pom.xml:
>  CVE-2017-7658, CVE-2017-7657
> {noformat}
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3313) AssertionError for using an invalid type parameter in REGEXP_REPLACE

2019-08-30 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3313:

Labels: pull-request-available  (was: )

> AssertionError for using an invalid type parameter in REGEXP_REPLACE
> 
>
> Key: CALCITE-3313
> URL: https://issues.apache.org/jira/browse/CALCITE-3313
> Project: Calcite
>  Issue Type: Bug
>Reporter: Wang Yanlin
>Priority: Minor
>  Labels: pull-request-available
>
> When using REGEXP_REPLACE function with an invalid type parameter, like this
> {code:sql}
> select regexp_replace(12, 'b', 'X', 1, 3, 'i')
> {code}
>  we got
> {code:java}
> java.lang.AssertionError: If you see this, assign operandTypeChecker a value 
> or override this function
>   at 
> org.apache.calcite.sql.SqlOperator.getAllowedSignatures(SqlOperator.java:730)
>   at 
> org.apache.calcite.sql.SqlOperator.getAllowedSignatures(SqlOperator.java:721)
>   at 
> org.apache.calcite.sql.SqlCallBinding.newValidationSignatureError(SqlCallBinding.java:283)
>   at 
> org.apache.calcite.sql.type.FamilyOperandTypeChecker.checkSingleOperandType(FamilyOperandTypeChecker.java:96)
>   at 
> org.apache.calcite.sql.fun.SqlRegexpReplaceFunction.checkOperandTypes(SqlRegexpReplaceFunction.java:56)
>   at 
> org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
>   at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:298)
>   at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
>   at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:476)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4104)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3392)
>   at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
>   at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1005)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:965)
>   at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
> {code}
> Better to give a more detailed message of the allowed signatures.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3314) CVSS dependency-check-maven fails for calcite-pig, calcite-piglet, calcite-spark

2019-08-30 Thread Stamatis Zampetakis (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919391#comment-16919391
 ] 

Stamatis Zampetakis commented on CALCITE-3314:
--

Regarding the nature of vulnerabilities they originate from the following 
direct dependencies: 
 * org.apache.hadoop:hadoop-client:jar:2.7.5:test
 * org.apache.hadoop:hadoop-common:jar:2.7.5:test
 * org.apache.pig:pig:jar:h2:0.16.0:compile
 * org.apache.spark:spark-core_2.10:jar:2.2.0:compile

Simply updating those to the latest version does not solve the problem.  Due to 
this we decided to disable the OWASP check for pig, piglet, and spark modules 
delegating the responsibility of choosing the appropriate versions to the 
clients using these modules.

> CVSS dependency-check-maven fails for calcite-pig, calcite-piglet, 
> calcite-spark
> 
>
> Key: CALCITE-3314
> URL: https://issues.apache.org/jira/browse/CALCITE-3314
> Project: Calcite
>  Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Blocker
> Fix For: 1.21.0
>
>
> Calcite build fails if the CVSS dependency check is active since there are 
> serious vulnerabilties in calcite-pig, calcite-piglet, calcite-spark. 
> Running mvn install -Ppedantic -fn gives the following errors:
> {noformat}
> ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
> (default) on project calcite-pig: 
> [ERROR] 
> [ERROR] One or more dependencies were identified with vulnerabilities that 
> have a CVSS score greater than or equal to '8.0': 
> [ERROR] 
> [ERROR] jetty-6.1.26.jar: CVE-2017-7658, CVE-2017-7657
> [ERROR] groovy-all-1.8.6.jar: CVE-2015-3253, CVE-2016-6814
> [ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
> (default) on project calcite-piglet: 
> [ERROR] 
> [ERROR] One or more dependencies were identified with vulnerabilities that 
> have a CVSS score greater than or equal to '8.0': 
> [ERROR] 
> [ERROR] jetty-6.1.26.jar: CVE-2017-7658, CVE-2017-7657
> [ERROR] jackson-core-asl-1.8.8.jar: CVE-2017-17485, CVE-2017-7525, 
> CVE-2017-15095
> [ERROR] groovy-all-1.8.6.jar: CVE-2015-3253, CVE-2016-6814
> [ERROR] jackson-xc-1.8.3.jar: CVE-2017-17485, CVE-2017-7525, CVE-2017-15095
> [ERROR] hadoop-auth-2.7.5.jar: CVE-2018-8029, CVE-2018-11766, CVE-2018-8009
> [ERROR] api-util-1.0.0-M20.jar: CVE-2018-1337
> [ERROR] zookeeper-3.4.6.jar: CVE-2016-5017
> [ERROR] 
> htrace-core-3.1.0-incubating.jar/META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml:
>  CVE-2017-17485, CVE-2018-5968, CVE-2017-15095, CVE-2019-14379, 
> CVE-2018-19362, CVE-2018-19361, CVE-2018-19360, CVE-2017-7525, 
> CVE-2018-11307, CVE-2018-14718, CVE-2018-7489, CVE-2018-14719, 
> CVE-2018-14721, CVE-2018-14720
> [ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
> (default) on project calcite-spark: 
> [ERROR] 
> [ERROR] One or more dependencies were identified with vulnerabilities that 
> have a CVSS score greater than or equal to '8.0': 
> [ERROR] 
> [ERROR] spark-core_2.10-2.2.0.jar: CVE-2018-17190
> [ERROR] api-util-1.0.0-M20.jar: CVE-2018-1337
> [ERROR] hadoop-mapreduce-client-core-2.7.5.jar: CVE-2018-8029, 
> CVE-2018-11766, CVE-2018-8009
> [ERROR] bcprov-jdk15on-1.51.jar: CVE-2018-1000613
> [ERROR] zookeeper-3.4.6.jar: CVE-2016-5017
> [ERROR] unused-1.0.0.jar: CVE-2018-17190
> [ERROR] 
> htrace-core-3.1.0-incubating.jar/META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml:
>  CVE-2017-17485, CVE-2018-5968, CVE-2017-15095, CVE-2019-14379, 
> CVE-2018-19362, CVE-2018-19361, CVE-2018-19360, CVE-2017-7525, 
> CVE-2018-11307, CVE-2018-14718, CVE-2018-7489, CVE-2018-14719, 
> CVE-2018-14721, CVE-2018-14720
> [ERROR] 
> spark-core_2.10-2.2.0.jar/META-INF/maven/org.eclipse.jetty/jetty-plus/pom.xml:
>  CVE-2017-7658, CVE-2017-7657
> {noformat}
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3314) CVSS dependency-check-maven fails for calcite-pig, calcite-piglet, calcite-spark

2019-08-30 Thread Stamatis Zampetakis (Jira)

Stamatis Zampetakis created CALCITE-3314:


 Summary: CVSS dependency-check-maven fails for calcite-pig, 
calcite-piglet, calcite-spark
 Key: CALCITE-3314
 URL: https://issues.apache.org/jira/browse/CALCITE-3314
 Project: Calcite
  Issue Type: Bug
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
 Fix For: 1.21.0


Calcite build fails if the CVSS dependency check is active since there are 
serious vulnerabilties in calcite-pig, calcite-piglet, calcite-spark. 

Running mvn install -Ppedantic -fn gives the following errors:

{noformat}
ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
(default) on project calcite-pig: 
[ERROR] 
[ERROR] One or more dependencies were identified with vulnerabilities that have 
a CVSS score greater than or equal to '8.0': 
[ERROR] 
[ERROR] jetty-6.1.26.jar: CVE-2017-7658, CVE-2017-7657
[ERROR] groovy-all-1.8.6.jar: CVE-2015-3253, CVE-2016-6814

[ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
(default) on project calcite-piglet: 
[ERROR] 
[ERROR] One or more dependencies were identified with vulnerabilities that have 
a CVSS score greater than or equal to '8.0': 
[ERROR] 
[ERROR] jetty-6.1.26.jar: CVE-2017-7658, CVE-2017-7657
[ERROR] jackson-core-asl-1.8.8.jar: CVE-2017-17485, CVE-2017-7525, 
CVE-2017-15095
[ERROR] groovy-all-1.8.6.jar: CVE-2015-3253, CVE-2016-6814
[ERROR] jackson-xc-1.8.3.jar: CVE-2017-17485, CVE-2017-7525, CVE-2017-15095
[ERROR] hadoop-auth-2.7.5.jar: CVE-2018-8029, CVE-2018-11766, CVE-2018-8009
[ERROR] api-util-1.0.0-M20.jar: CVE-2018-1337
[ERROR] zookeeper-3.4.6.jar: CVE-2016-5017
[ERROR] 
htrace-core-3.1.0-incubating.jar/META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml:
 CVE-2017-17485, CVE-2018-5968, CVE-2017-15095, CVE-2019-14379, CVE-2018-19362, 
CVE-2018-19361, CVE-2018-19360, CVE-2017-7525, CVE-2018-11307, CVE-2018-14718, 
CVE-2018-7489, CVE-2018-14719, CVE-2018-14721, CVE-2018-14720

[ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.2.1:check 
(default) on project calcite-spark: 
[ERROR] 
[ERROR] One or more dependencies were identified with vulnerabilities that have 
a CVSS score greater than or equal to '8.0': 
[ERROR] 
[ERROR] spark-core_2.10-2.2.0.jar: CVE-2018-17190
[ERROR] api-util-1.0.0-M20.jar: CVE-2018-1337
[ERROR] hadoop-mapreduce-client-core-2.7.5.jar: CVE-2018-8029, CVE-2018-11766, 
CVE-2018-8009
[ERROR] bcprov-jdk15on-1.51.jar: CVE-2018-1000613
[ERROR] zookeeper-3.4.6.jar: CVE-2016-5017
[ERROR] unused-1.0.0.jar: CVE-2018-17190
[ERROR] 
htrace-core-3.1.0-incubating.jar/META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml:
 CVE-2017-17485, CVE-2018-5968, CVE-2017-15095, CVE-2019-14379, CVE-2018-19362, 
CVE-2018-19361, CVE-2018-19360, CVE-2017-7525, CVE-2018-11307, CVE-2018-14718, 
CVE-2018-7489, CVE-2018-14719, CVE-2018-14721, CVE-2018-14720
[ERROR] 
spark-core_2.10-2.2.0.jar/META-INF/maven/org.eclipse.jetty/jetty-plus/pom.xml: 
CVE-2017-7658, CVE-2017-7657

{noformat}
 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3313) AssertionError for using an invalid type parameter in REGEXP_REPLACE

2019-08-30 Thread Wang Yanlin (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wang Yanlin updated CALCITE-3313:
-
Summary: AssertionError for using an invalid type parameter in 
REGEXP_REPLACE  (was: AssertionError for an invalid type when using 
REGEXP_REPLACE)

> AssertionError for using an invalid type parameter in REGEXP_REPLACE
> 
>
> Key: CALCITE-3313
> URL: https://issues.apache.org/jira/browse/CALCITE-3313
> Project: Calcite
>  Issue Type: Bug
>Reporter: Wang Yanlin
>Priority: Minor
>
> When using REGEXP_REPLACE function with an invalid type parameter, like this
> {code:sql}
> select regexp_replace(12, 'b', 'X', 1, 3, 'i')
> {code}
>  we got
> {code:java}
> java.lang.AssertionError: If you see this, assign operandTypeChecker a value 
> or override this function
>   at 
> org.apache.calcite.sql.SqlOperator.getAllowedSignatures(SqlOperator.java:730)
>   at 
> org.apache.calcite.sql.SqlOperator.getAllowedSignatures(SqlOperator.java:721)
>   at 
> org.apache.calcite.sql.SqlCallBinding.newValidationSignatureError(SqlCallBinding.java:283)
>   at 
> org.apache.calcite.sql.type.FamilyOperandTypeChecker.checkSingleOperandType(FamilyOperandTypeChecker.java:96)
>   at 
> org.apache.calcite.sql.fun.SqlRegexpReplaceFunction.checkOperandTypes(SqlRegexpReplaceFunction.java:56)
>   at 
> org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
>   at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:298)
>   at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
>   at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:476)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4104)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3392)
>   at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
>   at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1005)
>   at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:965)
>   at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
> {code}
> Better to give a more detailed message of the allowed signatures.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3313) AssertionError for an invalid type when using REGEXP_REPLACE

2019-08-30 Thread Wang Yanlin (Jira)

Wang Yanlin created CALCITE-3313:


 Summary: AssertionError for an invalid type when using 
REGEXP_REPLACE
 Key: CALCITE-3313
 URL: https://issues.apache.org/jira/browse/CALCITE-3313
 Project: Calcite
  Issue Type: Bug
Reporter: Wang Yanlin


When using REGEXP_REPLACE function with an invalid type parameter, like this

{code:sql}
select regexp_replace(12, 'b', 'X', 1, 3, 'i')
{code}

 we got
{code:java}
java.lang.AssertionError: If you see this, assign operandTypeChecker a value or 
override this function
at 
org.apache.calcite.sql.SqlOperator.getAllowedSignatures(SqlOperator.java:730)
at 
org.apache.calcite.sql.SqlOperator.getAllowedSignatures(SqlOperator.java:721)
at 
org.apache.calcite.sql.SqlCallBinding.newValidationSignatureError(SqlCallBinding.java:283)
at 
org.apache.calcite.sql.type.FamilyOperandTypeChecker.checkSingleOperandType(FamilyOperandTypeChecker.java:96)
at 
org.apache.calcite.sql.fun.SqlRegexpReplaceFunction.checkOperandTypes(SqlRegexpReplaceFunction.java:56)
at 
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:298)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:476)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4104)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3392)
at 
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at 
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1005)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:965)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
{code}

Better to give a more detailed message of the allowed signatures.




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CALCITE-3312) TIMESTAMPDIFF cannot be converted to SQL

2019-08-30 Thread Pavel Gubin (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919373#comment-16919373
 ] 

Pavel Gubin edited comment on CALCITE-3312 at 8/30/19 9:45 AM:
---

Function call above is converted to 
{noformat}
CAST(/INT(Reinterpret(-(2017-01-01 00:00:00, 2016-01-01 00:00:00)), 
12)):INTEGER NOT NULL
{noformat}
 in {{SqlToRelConverter}} which is then passed all the way through to 
{{RelToSqlConverter}} and to actual SQL serialization.


was (Author: pavel gubin):
Function call above is converted to {{CAST(/INT(Reinterpret(-(2017-01-01 
00:00:00, 2016-01-01 00:00:00)), 12)):INTEGER NOT NULL}} in 
{{SqlToRelConverter}} which is then passed all the way through to 
{{RelToSqlConverter}} and to actual SQL serialization.

> TIMESTAMPDIFF cannot be converted to SQL
> 
>
> Key: CALCITE-3312
> URL: https://issues.apache.org/jira/browse/CALCITE-3312
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Pavel Gubin
>Priority: Major
>
> The following test in {{RelToSqlConverterTest}}:
> {code}
>   @Test public void testTimestampDiff() {
> String query = "SELECT {fn TIMESTAMPDIFF(SQL_TSI_YEAR,TIMESTAMP 
> '2016-01-01 00:00:00', TIMESTAMP '2017-01-01 00:00:00')}";
> String expected = "";
> sql(query).ok(expected);
>   }
> {code}
> fails with:
> {noformat}
> java.lang.UnsupportedOperationException: class 
> org.apache.calcite.sql.SqlSyntax$6: SPECIAL
>   at org.apache.calcite.util.Util.needToImplement(Util.java:967)
>   at org.apache.calcite.sql.SqlSyntax$6.unparse(SqlSyntax.java:116)
>   at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:105)
>   at org.apache.calcite.sql.SqlUtil.unparseBinarySyntax(SqlUtil.java:382)
>   at org.apache.calcite.sql.SqlSyntax$3.unparse(SqlSyntax.java:65)
>   at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
>   at 
> org.apache.calcite.sql.fun.SqlCastFunction.unparse(SqlCastFunction.java:174)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
>   at org.apache.calcite.sql.SqlNodeList.commaList(SqlNodeList.java:121)
>   at 
> org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:365)
>   at 
> org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:354)
>   at 
> org.apache.calcite.sql.SqlSelectOperator.unparse(SqlSelectOperator.java:152)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlSelect.unparse(SqlSelect.java:230)
>   at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:152)
>   at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:157)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.toSql(RelToSqlConverterTest.java:145)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.access$100(RelToSqlConverterTest.java:77)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.exec(RelToSqlConverterTest.java:3634)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.ok(RelToSqlConverterTest.java:3609)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testTimestampDiff(RelToSqlConverterTest.java:1942)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at

[jira] [Commented] (CALCITE-3312) TIMESTAMPDIFF cannot be converted to SQL

2019-08-30 Thread Pavel Gubin (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919373#comment-16919373
 ] 

Pavel Gubin commented on CALCITE-3312:
--

Function call above is converted to {{CAST(/INT(Reinterpret(-(2017-01-01 
00:00:00, 2016-01-01 00:00:00)), 12)):INTEGER NOT NULL}} in 
{{SqlToRelConverter}} which is then passed all the way through to 
{{RelToSqlConverter}} and to actual SQL serialization.

> TIMESTAMPDIFF cannot be converted to SQL
> 
>
> Key: CALCITE-3312
> URL: https://issues.apache.org/jira/browse/CALCITE-3312
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Pavel Gubin
>Priority: Major
>
> The following test in {{RelToSqlConverterTest}}:
> {code}
>   @Test public void testTimestampDiff() {
> String query = "SELECT {fn TIMESTAMPDIFF(SQL_TSI_YEAR,TIMESTAMP 
> '2016-01-01 00:00:00', TIMESTAMP '2017-01-01 00:00:00')}";
> String expected = "";
> sql(query).ok(expected);
>   }
> {code}
> fails with:
> {noformat}
> java.lang.UnsupportedOperationException: class 
> org.apache.calcite.sql.SqlSyntax$6: SPECIAL
>   at org.apache.calcite.util.Util.needToImplement(Util.java:967)
>   at org.apache.calcite.sql.SqlSyntax$6.unparse(SqlSyntax.java:116)
>   at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:105)
>   at org.apache.calcite.sql.SqlUtil.unparseBinarySyntax(SqlUtil.java:382)
>   at org.apache.calcite.sql.SqlSyntax$3.unparse(SqlSyntax.java:65)
>   at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
>   at 
> org.apache.calcite.sql.fun.SqlCastFunction.unparse(SqlCastFunction.java:174)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
>   at org.apache.calcite.sql.SqlNodeList.commaList(SqlNodeList.java:121)
>   at 
> org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:365)
>   at 
> org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:354)
>   at 
> org.apache.calcite.sql.SqlSelectOperator.unparse(SqlSelectOperator.java:152)
>   at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
>   at org.apache.calcite.sql.SqlSelect.unparse(SqlSelect.java:230)
>   at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:152)
>   at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:157)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.toSql(RelToSqlConverterTest.java:145)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.access$100(RelToSqlConverterTest.java:77)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.exec(RelToSqlConverterTest.java:3634)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.ok(RelToSqlConverterTest.java:3609)
>   at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testTimestampDiff(RelToSqlConverterTest.java:1942)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>   at 
>

[jira] [Created] (CALCITE-3312) TIMESTAMPDIFF cannot be converted to SQL

2019-08-30 Thread Pavel Gubin (Jira)

Pavel Gubin created CALCITE-3312:


 Summary: TIMESTAMPDIFF cannot be converted to SQL
 Key: CALCITE-3312
 URL: https://issues.apache.org/jira/browse/CALCITE-3312
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Pavel Gubin


The following test in {{RelToSqlConverterTest}}:
{code}
  @Test public void testTimestampDiff() {
String query = "SELECT {fn TIMESTAMPDIFF(SQL_TSI_YEAR,TIMESTAMP '2016-01-01 
00:00:00', TIMESTAMP '2017-01-01 00:00:00')}";
String expected = "";
sql(query).ok(expected);
  }
{code}

fails with:
{noformat}
java.lang.UnsupportedOperationException: class 
org.apache.calcite.sql.SqlSyntax$6: SPECIAL

at org.apache.calcite.util.Util.needToImplement(Util.java:967)
at org.apache.calcite.sql.SqlSyntax$6.unparse(SqlSyntax.java:116)
at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:105)
at org.apache.calcite.sql.SqlUtil.unparseBinarySyntax(SqlUtil.java:382)
at org.apache.calcite.sql.SqlSyntax$3.unparse(SqlSyntax.java:65)
at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:348)
at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
at 
org.apache.calcite.sql.fun.SqlCastFunction.unparse(SqlCastFunction.java:174)
at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:108)
at org.apache.calcite.sql.SqlNodeList.commaList(SqlNodeList.java:121)
at 
org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:365)
at 
org.apache.calcite.sql.SqlOperator.unparseListClause(SqlOperator.java:354)
at 
org.apache.calcite.sql.SqlSelectOperator.unparse(SqlSelectOperator.java:152)
at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:402)
at org.apache.calcite.sql.SqlSelect.unparse(SqlSelect.java:230)
at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:152)
at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:157)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.toSql(RelToSqlConverterTest.java:145)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.access$100(RelToSqlConverterTest.java:77)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.exec(RelToSqlConverterTest.java:3634)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.ok(RelToSqlConverterTest.java:3609)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testTimestampDiff(RelToSqlConverterTest.java:1942)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at 
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at 
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at 
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3310) Approximate and exact aggregate calls are recognized as the same during sql-to-rel conversion

2019-08-30 Thread Chris Baynes (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919340#comment-16919340
 ] 

Chris Baynes commented on CALCITE-3310:
---

I may be wrong here but I seem to remember that the approximate flag is kept on 
the RelNode, it's just not printed in the plan.

> Approximate and exact aggregate calls are recognized as the same during 
> sql-to-rel conversion
> -
>
> Key: CALCITE-3310
> URL: https://issues.apache.org/jira/browse/CALCITE-3310
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> For sql:
> {code:sql}
> SELECT empno, count(distinct ename)
> approx_count_distinct(ename)
> FROM emp
> GROUP BY empno
> {code}
> After sql-to-rel conversion, the plan is:
> {code:sql}
> LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$1])
>   LogicalAggregate(group=[{0}], EXPR$1=[COUNT(DISTINCT $1)])
> LogicalProject(EMPNO=[$0], ENAME=[$1])
>   LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3307:

Labels: pull-request-available  (was: )

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Ruben Quesada Lopez
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruben Quesada Lopez updated CALCITE-3307:
-
Fix Version/s: 1.21.0

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Ruben Quesada Lopez
>Priority: Major
> Fix For: 1.21.0
>
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919308#comment-16919308
 ] 

Ruben Quesada Lopez edited comment on CALCITE-3307 at 8/30/19 8:11 AM:
---

Following the suggestion by [~julianhyde] in CALCITE-3122, I can take care of 
this by just ignoring the:
- 29 tests in PigRelExTest
- 32 tests in PigRelOpTest
- 1 test in PigScriptTest

By adding:
{code}
Assume.assumeFalse("Skip: Pig/Hadoop tests do not work on Windows", 
isWindows());
{code}

Plus the corresponding {{isWindows}} method in a common location (probably 
{{PigRelTestBase}}):
{code}
static boolean isWindows() {
return System.getProperty("os.name").startsWith("Windows");
}
{code}


was (Author: rubenql):
Following the suggestion by [~julianhyde] in CALCITE-3122, I can take care of 
this by just ignoring the:
- 32 tests in PigRelOpTest
- 29 tests in PigRelExTest
- 1 test in PigScriptTest

By adding:
{code}
Assume.assumeFalse("Skip: Pig/Hadoop tests do not work on Windows", 
isWindows());
{code}

Plus the corresponding {{isWindows}} method in a common location (probably 
{{PigRelTestBase}}):
{code}
static boolean isWindows() {
return System.getProperty("os.name").startsWith("Windows");
}
{code}

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Ruben Quesada Lopez
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at

[jira] [Assigned] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruben Quesada Lopez reassigned CALCITE-3307:


Assignee: Ruben Quesada Lopez

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Ruben Quesada Lopez
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919308#comment-16919308
 ] 

Ruben Quesada Lopez edited comment on CALCITE-3307 at 8/30/19 8:09 AM:
---

Following the suggestion by [~julianhyde] in CALCITE-3122, I can take care of 
this by just ignoring the:
- 32 tests in PigRelOpTest
- 29 tests in PigRelExTest
- 1 test in PigScriptTest

By adding:
{code}
Assume.assumeFalse("Skip: Pig/Hadoop tests do not work on Windows", 
isWindows());
{code}

Plus the corresponding {{isWindows}} method in a common location (probably 
{{PigRelTestBase}}):
{code}
static boolean isWindows() {
return System.getProperty("os.name").startsWith("Windows");
  }
{code}


was (Author: rubenql):
Following the suggestion by [~julianhyde] in CALCITE-3122, I can take care of 
this by just ignoring the:
- 32 tests in PigRelOpTest
- 29 tests in PigRelExTest
- 1 test in PigScriptTest

By adding:
{code}
Assume.assumeFalse("Skip: Pig/Hadoop tests do not work on Windows", 
isWindows());
{code}

Plus the corresponding {{isWindows}} method in a common location (probably 
PigRelTestBas):
{code}
static boolean isWindows() {
return System.getProperty("os.name").startsWith("Windows");
  }
{code}

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at

[jira] [Comment Edited] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919308#comment-16919308
 ] 

Ruben Quesada Lopez edited comment on CALCITE-3307 at 8/30/19 8:09 AM:
---

Following the suggestion by [~julianhyde] in CALCITE-3122, I can take care of 
this by just ignoring the:
- 32 tests in PigRelOpTest
- 29 tests in PigRelExTest
- 1 test in PigScriptTest

By adding:
{code}
Assume.assumeFalse("Skip: Pig/Hadoop tests do not work on Windows", 
isWindows());
{code}

Plus the corresponding {{isWindows}} method in a common location (probably 
{{PigRelTestBase}}):
{code}
static boolean isWindows() {
return System.getProperty("os.name").startsWith("Windows");
}
{code}


was (Author: rubenql):
Following the suggestion by [~julianhyde] in CALCITE-3122, I can take care of 
this by just ignoring the:
- 32 tests in PigRelOpTest
- 29 tests in PigRelExTest
- 1 test in PigScriptTest

By adding:
{code}
Assume.assumeFalse("Skip: Pig/Hadoop tests do not work on Windows", 
isWindows());
{code}

Plus the corresponding {{isWindows}} method in a common location (probably 
{{PigRelTestBase}}):
{code}
static boolean isWindows() {
return System.getProperty("os.name").startsWith("Windows");
  }
{code}

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at

[jira] [Comment Edited] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919308#comment-16919308
 ] 

Ruben Quesada Lopez edited comment on CALCITE-3307 at 8/30/19 8:09 AM:
---

Following the suggestion by [~julianhyde] in CALCITE-3122, I can take care of 
this by just ignoring the:
- 32 tests in PigRelOpTest
- 29 tests in PigRelExTest
- 1 test in PigScriptTest

By adding:
{code}
Assume.assumeFalse("Skip: Pig/Hadoop tests do not work on Windows", 
isWindows());
{code}

Plus the corresponding {{isWindows}} method in a common location (probably 
PigRelTestBas):
{code}
static boolean isWindows() {
return System.getProperty("os.name").startsWith("Windows");
  }
{code}


was (Author: rubenql):
Following the suggestion by [~julianhyde] in CALCITE-3122, I can take care of 
this by just ignoring the:
- 32 tests in PigRelOpTest
- 29 tests in PigRelExTest
- 1 test in PigScriptTest
By adding:
{code}
Assume.assumeFalse("Skip: Pig/Hadoop tests do not work on Windows", 
isWindows());
{code}

Plus the corresponding {{isWindows}} method in a common location (probably 
PigRelTestBas):
{code}
static boolean isWindows() {
return System.getProperty("os.name").startsWith("Windows");
  }
{code}

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at

[jira] [Updated] (CALCITE-3307) PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruben Quesada Lopez updated CALCITE-3307:
-
Summary: PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows  
(was: PigRelOpTest, PigRelExTest fails on Windows)

> PigRelExTest, PigRelOpTest and PigScriptTest fail on Windows
> 
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3311) Add doc to site for implicit type coercion

2019-08-30 Thread Danny Chan (Jira)

Danny Chan created CALCITE-3311:
---

 Summary: Add doc to site for implicit type coercion
 Key: CALCITE-3311
 URL: https://issues.apache.org/jira/browse/CALCITE-3311
 Project: Calcite
  Issue Type: Sub-task
  Components: site
Affects Versions: 1.20.0
Reporter: Danny Chan
Assignee: Danny Chan
 Fix For: 1.21.0






--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3307) PigRelOpTest, PigRelExTest fails on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruben Quesada Lopez updated CALCITE-3307:
-
Summary: PigRelOpTest, PigRelExTest fails on Windows  (was: PigRelExTest 
fails on Windows)

> PigRelOpTest, PigRelExTest fails on Windows
> ---
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3307) PigRelExTest fails on Windows

2019-08-30 Thread Danny Chan (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919296#comment-16919296
 ] 

Danny Chan commented on CALCITE-3307:
-

Thanks for the confirm, Ruben !

> PigRelExTest fails on Windows
> -
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3309) Refactor generatePredicate method from EnumerableNestedLoopJoin/EnumerableHashJoin/EnumerableBatchNestedLoopJoin into a single location

2019-08-30 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3309:

Labels: pull-request-available  (was: )

> Refactor generatePredicate method from 
> EnumerableNestedLoopJoin/EnumerableHashJoin/EnumerableBatchNestedLoopJoin 
> into a single location
> ---
>
> Key: CALCITE-3309
> URL: https://issues.apache.org/jira/browse/CALCITE-3309
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Ruben Quesada Lopez
>Assignee: Ruben Quesada Lopez
>Priority: Trivial
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>
> The method {{EnumerableNestedLoopJoin#predicate}} (that generates a Predicate 
> Expression based on a RexNode condition) has been copied pasted as 
> {{EnumerableBatchNestedLoopJoin#generatePredicate}} due to CALCITE-2979, and 
> {{EnumerableHashJoin#generatePredicate}} due to CALCITE-2973.
> The goal of this ticket is to refactor that method into a single location 
> (e.g. EnumUtils) where it can be accessible by all 3 Enumerable Joins.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3310) Approximate and exact aggregate calls are recognized as the same during sql-to-rel conversion

2019-08-30 Thread Danny Chan (Jira)

Danny Chan created CALCITE-3310:
---

 Summary: Approximate and exact aggregate calls are recognized as 
the same during sql-to-rel conversion
 Key: CALCITE-3310
 URL: https://issues.apache.org/jira/browse/CALCITE-3310
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.20.0
Reporter: Danny Chan
Assignee: Danny Chan
 Fix For: 1.21.0


For sql:
{code:sql}
SELECT empno, count(distinct ename)
approx_count_distinct(ename)
FROM emp
GROUP BY empno
{code}
After sql-to-rel conversion, the plan is:
{code:sql}
LogicalProject(EMPNO=[$0], EXPR$1=[$1], EXPR$2=[$1])
  LogicalAggregate(group=[{0}], EXPR$1=[COUNT(DISTINCT $1)])
LogicalProject(EMPNO=[$0], ENAME=[$1])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3306) Add REGEXP_SPLIT_TO_ARRAY function

2019-08-30 Thread Wang Yanlin (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919285#comment-16919285
 ] 

Wang Yanlin commented on CALCITE-3306:
--

OK, I'll update the PR.

> Add REGEXP_SPLIT_TO_ARRAY function
> --
>
> Key: CALCITE-3306
> URL: https://issues.apache.org/jira/browse/CALCITE-3306
> Project: Calcite
>  Issue Type: New Feature
>Reporter: Wang Yanlin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In work, I need to split a string into array. But I found there is no 
> coresponding *split* function in calcite like the *regexp_split_to_array* in 
> [PostgreSQL|https://www.postgresql.org/docs/9.1/functions-matching.html].
> Although we can solve this by creating a udf based on calcite, but a built in 
> function might be better, in case some others may have the same need.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3309) Refactor generatePredicate method from EnumerableNestedLoopJoin/EnumerableHashJoin/EnumerableBatchNestedLoopJoin into a single location

2019-08-30 Thread Ruben Quesada Lopez (Jira)

Ruben Quesada Lopez created CALCITE-3309:


 Summary: Refactor generatePredicate method from 
EnumerableNestedLoopJoin/EnumerableHashJoin/EnumerableBatchNestedLoopJoin into 
a single location
 Key: CALCITE-3309
 URL: https://issues.apache.org/jira/browse/CALCITE-3309
 Project: Calcite
  Issue Type: Improvement
Reporter: Ruben Quesada Lopez
Assignee: Ruben Quesada Lopez
 Fix For: 1.21.0


The method {{EnumerableNestedLoopJoin#predicate}} (that generates a Predicate 
Expression based on a RexNode condition) has been copied pasted as 
{{EnumerableBatchNestedLoopJoin#generatePredicate}} due to CALCITE-2979, and 
{{EnumerableHashJoin#generatePredicate}} due to CALCITE-2973.
The goal of this ticket is to refactor that method into a single location (e.g. 
EnumUtils) where it can be accessible by all 3 Enumerable Joins.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3307) PigRelExTest fails on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919278#comment-16919278
 ] 

Ruben Quesada Lopez commented on CALCITE-3307:
--

I confirm that (on my local Windows environment) by ignoring those 3 tests 
mentioned above we get a successful build.

> PigRelExTest fails on Windows
> -
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CALCITE-3307) PigRelExTest fails on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919278#comment-16919278
 ] 

Ruben Quesada Lopez edited comment on CALCITE-3307 at 8/30/19 7:01 AM:
---

I confirm that (on my local Windows environment) by ignoring those 3 tests 
mentioned above Iget a successful build.


was (Author: rubenql):
I confirm that (on my local Windows environment) by ignoring those 3 tests 
mentioned above we get a successful build.

> PigRelExTest fails on Windows
> -
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CALCITE-3307) PigRelExTest fails on Windows

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919278#comment-16919278
 ] 

Ruben Quesada Lopez edited comment on CALCITE-3307 at 8/30/19 7:01 AM:
---

I confirm that (on my local Windows environment) by ignoring those 3 tests 
mentioned above I get a successful build.


was (Author: rubenql):
I confirm that (on my local Windows environment) by ignoring those 3 tests 
mentioned above Iget a successful build.

> PigRelExTest fails on Windows
> -
>
> Key: CALCITE-3307
> URL: https://issues.apache.org/jira/browse/CALCITE-3307
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Running {{PigRelExTest}} on Windows, after CALCITE-3122 was merged, gives the 
> following error:
> {noformat}
> 2019-08-29 15:33:23,229 [main] ERROR - Failed to locate the winutils binary 
> in the hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
> Hadoop binaries.
>   at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:382)
>   at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:397)
>   at org.apache.hadoop.util.Shell.(Shell.java:390)
>   at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
>   at 
> org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
>   at 
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
>   at 
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:338)
>   at 
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:332)
>   at 
> org.apache.pig.backend.hadoop.HKerberos.tryKerberosKeytabLogin(HKerberos.java:70)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:220)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:112)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:305)
>   at org.apache.pig.PigServer.(PigServer.java:231)
>   at org.apache.pig.PigServer.(PigServer.java:219)
>   at org.apache.pig.PigServer.(PigServer.java:211)
>   at org.apache.pig.PigServer.(PigServer.java:207)
>   at org.apache.calcite.piglet.PigConverter.(PigConverter.java:107)
>   at org.apache.calcite.piglet.PigConverter.create(PigConverter.java:112)
>   at 
> org.apache.calcite.test.PigRelTestBase.testSetup(PigRelTestBase.java:34)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> {noformat}
> This issue has arisen many times; see HADOOP-10775 and SPARK-2356.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CALCITE-3284) Enumerable hash semijoin / antijoin support non-equi join conditions

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919274#comment-16919274
 ] 

Ruben Quesada Lopez edited comment on CALCITE-3284 at 8/30/19 6:56 AM:
---

Fixed (together with CALCITE-2973) via 
https://github.com/apache/calcite/commit/6cec81178e6739d9b1f3474dc5dbd7488dda2464
Thanks for the PR [~hhlai1990]!


was (Author: rubenql):
Fixed (together with CALCITE-2973) via 
https://github.com/apache/calcite/commit/6cec81178e6739d9b1f3474dc5dbd7488dda2464
Thanks for the Pr Lai Zhou!

> Enumerable hash semijoin / antijoin support non-equi join conditions
> 
>
> Key: CALCITE-3284
> URL: https://issues.apache.org/jira/browse/CALCITE-3284
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Haisheng Yuan
>Priority: Major
> Fix For: 1.21.0
>
>
> Calcite should be able to generate enumerable hash semijoin / antijoin with 
> non-equi join conditions, as long as there are equi-join condtions, so that 
> we can do hash look up.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CALCITE-2973) Allow theta joins that have equi conditions to be executed using a hash join algorithm

2019-08-30 Thread Ruben Quesada Lopez (Jira)



[ 
https://issues.apache.org/jira/browse/CALCITE-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919273#comment-16919273
 ] 

Ruben Quesada Lopez edited comment on CALCITE-2973 at 8/30/19 6:55 AM:
---

Fixed via 
https://github.com/apache/calcite/commit/6cec81178e6739d9b1f3474dc5dbd7488dda2464
Thanks for the PR [~hhlai1990]!


was (Author: rubenql):
Fixed via 
https://github.com/apache/calcite/commit/6cec81178e6739d9b1f3474dc5dbd7488dda2464
Thanks for the Pr [~hhlai1990]!

> Allow theta joins that have equi conditions to be executed using a hash join 
> algorithm
> --
>
> Key: CALCITE-2973
> URL: https://issues.apache.org/jira/browse/CALCITE-2973
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Affects Versions: 1.19.0
>Reporter: Lai Zhou
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 9.5h
>  Remaining Estimate: 0h
>
> Now the EnumerableMergeJoinRule only supports an inner and equi join.
> If users make a theta-join query  for a large dataset (such as 1*1), 
> the nested-loop join process will take dozens of time than the sort-merge 
> join process .
> So if we can apply merge-join or hash-join rule for a theta join, it will 
> improve the performance greatly.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (CALCITE-3284) Enumerable hash semijoin / antijoin support non-equi join conditions

2019-08-30 Thread Ruben Quesada Lopez (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruben Quesada Lopez resolved CALCITE-3284.
--
Resolution: Fixed

Fixed (together with CALCITE-2973) via 
https://github.com/apache/calcite/commit/6cec81178e6739d9b1f3474dc5dbd7488dda2464
Thanks for the Pr Lai Zhou!

> Enumerable hash semijoin / antijoin support non-equi join conditions
> 
>
> Key: CALCITE-3284
> URL: https://issues.apache.org/jira/browse/CALCITE-3284
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Haisheng Yuan
>Priority: Major
> Fix For: 1.21.0
>
>
> Calcite should be able to generate enumerable hash semijoin / antijoin with 
> non-equi join conditions, as long as there are equi-join condtions, so that 
> we can do hash look up.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (CALCITE-2973) Allow theta joins that have equi conditions to be executed using a hash join algorithm

2019-08-30 Thread Ruben Quesada Lopez (Jira)



 [ 
https://issues.apache.org/jira/browse/CALCITE-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruben Quesada Lopez resolved CALCITE-2973.
--
Resolution: Fixed

Fixed via 
https://github.com/apache/calcite/commit/6cec81178e6739d9b1f3474dc5dbd7488dda2464
Thanks for the Pr [~hhlai1990]!

> Allow theta joins that have equi conditions to be executed using a hash join 
> algorithm
> --
>
> Key: CALCITE-2973
> URL: https://issues.apache.org/jira/browse/CALCITE-2973
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Affects Versions: 1.19.0
>Reporter: Lai Zhou
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.21.0
>
>  Time Spent: 9.5h
>  Remaining Estimate: 0h
>
> Now the EnumerableMergeJoinRule only supports an inner and equi join.
> If users make a theta-join query  for a large dataset (such as 1*1), 
> the nested-loop join process will take dozens of time than the sort-merge 
> join process .
> So if we can apply merge-join or hash-join rule for a theta join, it will 
> improve the performance greatly.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

48 matches

Mail list logo