[jira] [Commented] (HIVE-14908) Upgrade ANTLR to 3.5.2

2016-11-07 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15645671#comment-15645671
 ] 

Hive QA commented on HIVE-14908:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12837820/HIVE-14908.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10628 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
 (batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
 (batchId=145)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5] 
(batchId=90)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[columnstats_partlvl_multiple_part_clause]
 (batchId=83)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[missing_overwrite]
 (batchId=223)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2009/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2009/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2009/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12837820 - PreCommit-HIVE-Build

> Upgrade ANTLR to 3.5.2
> --
>
> Key: HIVE-14908
> URL: https://issues.apache.org/jira/browse/HIVE-14908
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14908.01.patch, HIVE-14908.02.patch
>
>
> Antlr v4 is also available but it does not support "->" which is widely used 
> in our grammar. Antlr 3.5.2 is the latest v3 version. It will reduce the code 
> size:
> {code}
> Here is summary of current parser code size
> 422345  HiveLexer.java
> 2436601  HiveParser.java
> 814184  HiveParser_FromClauseParser.java
> 2705920  HiveParser_IdentifiersParser.java
> 777665 HiveParser_SelectClauseParser.java
>After change, it will become
> 319589 HiveLexer.java
> 1853104 HiveParser.java
> 574156 HiveParser_FromClauseParser.java
> 1799195 HiveParser_IdentifiersParser.java
> 587305 HiveParser_SelectClauseParser.java
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14908) Upgrade ANTLR to 3.5.2

2016-10-07 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1699#comment-1699
 ] 

Pengcheng Xiong commented on HIVE-14908:


Hi [~kgyrtkirk], if you have time, you can open a jira and try to upgrade it to 
v4 and then we can see if it benefits. I can expect that the major work is to 
rewrite "->" in v4. :)

> Upgrade ANTLR to 3.5.2
> --
>
> Key: HIVE-14908
> URL: https://issues.apache.org/jira/browse/HIVE-14908
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14908.01.patch
>
>
> Antlr v4 is also available but it does not support "->" which is widely used 
> in our grammar. Antlr 3.5.2 is the latest v3 version. It will reduce the code 
> size:
> {code}
> Here is summary of current parser code size
> 422345  HiveLexer.java
> 2436601  HiveParser.java
> 814184  HiveParser_FromClauseParser.java
> 2705920  HiveParser_IdentifiersParser.java
> 777665 HiveParser_SelectClauseParser.java
>After change, it will become
> 319589 HiveLexer.java
> 1853104 HiveParser.java
> 574156 HiveParser_FromClauseParser.java
> 1799195 HiveParser_IdentifiersParser.java
> 587305 HiveParser_SelectClauseParser.java
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14908) Upgrade ANTLR to 3.5.2

2016-10-07 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1535#comment-1535
 ] 

Ashutosh Chauhan commented on HIVE-14908:
-

yeah.. benefits are worth the upgrade. 
Patch LGTM +1

> Upgrade ANTLR to 3.5.2
> --
>
> Key: HIVE-14908
> URL: https://issues.apache.org/jira/browse/HIVE-14908
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14908.01.patch
>
>
> Antlr v4 is also available but it does not support "->" which is widely used 
> in our grammar. Antlr 3.5.2 is the latest v3 version. It will reduce the code 
> size:
> {code}
> Here is summary of current parser code size
> 422345  HiveLexer.java
> 2436601  HiveParser.java
> 814184  HiveParser_FromClauseParser.java
> 2705920  HiveParser_IdentifiersParser.java
> 777665 HiveParser_SelectClauseParser.java
>After change, it will become
> 319589 HiveLexer.java
> 1853104 HiveParser.java
> 574156 HiveParser_FromClauseParser.java
> 1799195 HiveParser_IdentifiersParser.java
> 587305 HiveParser_SelectClauseParser.java
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14908) Upgrade ANTLR to 3.5.2

2016-10-07 Thread Eugene Koifman (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1365#comment-1365
 ] 

Eugene Koifman commented on HIVE-14908:
---

Given how much this upgrade reduced the generated file sizes it seems worth 
doing regardless of what we decide with respect to reserved words

> Upgrade ANTLR to 3.5.2
> --
>
> Key: HIVE-14908
> URL: https://issues.apache.org/jira/browse/HIVE-14908
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14908.01.patch
>
>
> Antlr v4 is also available but it does not support "->" which is widely used 
> in our grammar. Antlr 3.5.2 is the latest v3 version. It will reduce the code 
> size:
> {code}
> Here is summary of current parser code size
> 422345  HiveLexer.java
> 2436601  HiveParser.java
> 814184  HiveParser_FromClauseParser.java
> 2705920  HiveParser_IdentifiersParser.java
> 777665 HiveParser_SelectClauseParser.java
>After change, it will become
> 319589 HiveLexer.java
> 1853104 HiveParser.java
> 574156 HiveParser_FromClauseParser.java
> 1799195 HiveParser_IdentifiersParser.java
> 587305 HiveParser_SelectClauseParser.java
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14908) Upgrade ANTLR to 3.5.2

2016-10-07 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554804#comment-15554804
 ] 

Hive QA commented on HIVE-14908:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12832091/HIVE-14908.01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10656 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[columnstats_partlvl_multiple_part_clause]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1425/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1425/console
Test logs: 
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1425/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12832091 - PreCommit-HIVE-Build

> Upgrade ANTLR to 3.5.2
> --
>
> Key: HIVE-14908
> URL: https://issues.apache.org/jira/browse/HIVE-14908
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14908.01.patch
>
>
> Antlr v4 is also available but it does not support "->" which is widely used 
> in our grammar. Antlr 3.5.2 is the latest v3 version. It will reduce the code 
> size:
> {code}
> Here is summary of current parser code size
> 422345  HiveLexer.java
> 2436601  HiveParser.java
> 814184  HiveParser_FromClauseParser.java
> 2705920  HiveParser_IdentifiersParser.java
> 777665 HiveParser_SelectClauseParser.java
>After change, it will become
> 319589 HiveLexer.java
> 1853104 HiveParser.java
> 574156 HiveParser_FromClauseParser.java
> 1799195 HiveParser_IdentifiersParser.java
> 587305 HiveParser_SelectClauseParser.java
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14908) Upgrade ANTLR to 3.5.2

2016-10-07 Thread Zoltan Haindrich (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554436#comment-15554436
 ] 

Zoltan Haindrich commented on HIVE-14908:
-

I've also tried 3.5.2  while I experimented with interval fixes..it didnt 
helped in that neither ;)

But I was not sure about antlr upgrade preferences ... afaik antlr4 does syn 
rules automatically..
So i think it might worth a try - I think i will try it.
Antlr4 has much better dev tooling support which would be useful during 
development

althrough the removal of the optional reservation of these keywords will help - 
but I think identifierparser should be split up
 [~pxiong] 

> Upgrade ANTLR to 3.5.2
> --
>
> Key: HIVE-14908
> URL: https://issues.apache.org/jira/browse/HIVE-14908
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14908.01.patch
>
>
> Antlr v4 is also available but it does not support "->" which is widely used 
> in our grammar. Antlr 3.5.2 is the latest v3 version. It will reduce the code 
> size:
> {code}
> Here is summary of current parser code size
> 422345  HiveLexer.java
> 2436601  HiveParser.java
> 814184  HiveParser_FromClauseParser.java
> 2705920  HiveParser_IdentifiersParser.java
> 777665 HiveParser_SelectClauseParser.java
>After change, it will become
> 319589 HiveLexer.java
> 1853104 HiveParser.java
> 574156 HiveParser_FromClauseParser.java
> 1799195 HiveParser_IdentifiersParser.java
> 587305 HiveParser_SelectClauseParser.java
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14908) Upgrade ANTLR to 3.5.2

2016-10-07 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554267#comment-15554267
 ] 

Pengcheng Xiong commented on HIVE-14908:


Thanks for [~ekoifman]'s suggestion. However, it is still not enough if we 
would like to support set operator without "distinct" keywords. We still need 
to drop the configuration for SQL2011 keywords used as identifier. ccing 
[~ashutoshc] and [~alangates]

> Upgrade ANTLR to 3.5.2
> --
>
> Key: HIVE-14908
> URL: https://issues.apache.org/jira/browse/HIVE-14908
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>
> Antlr v4 is also available but it does not support "->" which is widely used 
> in our grammar. Antlr 3.5.2 is the latest v3 version. It will reduce the code 
> size:
> {code}
> Here is summary of current parser code size
> 422345  HiveLexer.java
> 2436601  HiveParser.java
> 814184  HiveParser_FromClauseParser.java
> 2705920  HiveParser_IdentifiersParser.java
> 777665 HiveParser_SelectClauseParser.java
>After change, it will become
> 319589 HiveLexer.java
> 1853104 HiveParser.java
> 574156 HiveParser_FromClauseParser.java
> 1799195 HiveParser_IdentifiersParser.java
> 587305 HiveParser_SelectClauseParser.java
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)