subject:"\[jira\] \[Commented\] \(HIVE\-20284\) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error \"No partition predicate for Alias\""

[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error "No partition predicate for Alias"

2018-08-01 Thread Hive QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565096#comment-16565096
 ] 

Hive QA commented on HIVE-20284:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12933882/HIVE-20284.1.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 14839 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_string_bigint]
 (batchId=97)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_string_bigint_2]
 (batchId=97)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input4] 
(batchId=97)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input4_2] 
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[strict_join] 
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[strict_join_2] 
(batchId=97)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[strict_orderby] 
(batchId=97)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[strict_orderby_2]
 (batchId=97)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12981/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12981/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12981/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12933882 - PreCommit-HIVE-Build

> In strict mode, if constant propagation is enable, the partition filter is 
> folded before partition pruner lead to error "No partition predicate for 
> Alias"  
> 
>
> Key: HIVE-20284
> URL: https://issues.apache.org/jira/browse/HIVE-20284
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Affects Versions: 1.2.1, 2.3.3
>Reporter: Hui Huang
>Assignee: Hui Huang
>Priority: Trivial
> Fix For: 2.3.3, 4.0.0
>
> Attachments: HIVE-20284.1.patch, HIVE-20284.patch
>
>
> In strict mode and the hive.optimize.constant.propagation is set to true, the 
> following sql will failed:
> {code:java}
> hive> desc employee_part;
> OK
> col_name  data_type   comment
> eid   int
> name  string
> dept  string
> year  string
> month string
> # Partition Information
> # col_namedata_type   comment
> year  string
> month string
> Time taken: 0.564 seconds, Fetched: 11 row(s)
> hive> set hive.mapred.mode=strict;
> hive> select * from employee_part where false and concat(year,month)='201807';
> FAILED: SemanticException Queries against partitioned tables without a 
> partition filter are disabled for safety reasons. If you know what you are 
> doing, please sethive.strict.checks.large.query to false and that 
> hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get 
> errors or incorrect results if you make a mistake while using some of the 
> unsafe features. No partition predicate for Alias "employee_part" Table 
> "employee_part"
> {code}
> The above error msg is confusing，  concat(year,month)='201807' is the 
> partition filter。
> The reason is during logic optimization, the ConstantPropagate optimizer is 
> running before partitionPruner optimizer, when found a express like 'false 
> and concat(year,month)=', the express is replaced with 'fasle' , and the 
> partition filter is dropped. So the PartitionPruner can not get the partition 
> filter.
> Users can remove the constant express that always has true/false values to 
> work around.
> When views used, if some columns are constant values, users  will be  
> confusing.
> So we should add some more message in the error msg returned.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error "No partition predicate for Alias"

2018-08-01 Thread Hive QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565064#comment-16565064
 ] 

Hive QA commented on HIVE-20284:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
34s{color} | {color:blue} common in master has 64 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
8s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  5m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 31s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12981/dev-support/hive-personality.sh
 |
| git revision | master / 4d43695 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12981/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> In strict mode, if constant propagation is enable, the partition filter is 
> folded before partition pruner lead to error "No partition predicate for 
> Alias"  
> 
>
> Key: HIVE-20284
> URL: https://issues.apache.org/jira/browse/HIVE-20284
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Affects Versions: 1.2.1, 2.3.3
>Reporter: Hui Huang
>Assignee: Hui Huang
>Priority: Trivial
> Fix For: 2.3.3, 4.0.0
>
> Attachments: HIVE-20284.1.patch, HIVE-20284.patch
>
>
> In strict mode and the hive.optimize.constant.propagation is set to true, the 
> following sql will failed:
> {code:java}
> hive> desc employee_part;
> OK
> col_name  data_type   comment
> eid   int
> name  string
> dept  string
> year  string
> month string
> # Partition Information
> # col_namedata_type   comment
> year  string
> month string
> Time

[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error "No partition predicate for Alias"

2018-08-01 Thread Hive QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564782#comment-16564782
 ] 

Hive QA commented on HIVE-20284:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12933861/HIVE-20284.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12980/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12980/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12980/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-08-01 06:01:16.260
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-12980/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-08-01 06:01:16.263
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 4d43695 HIVE-18201 : Disable XPROD_EDGE for sq_count_check()  
created for scalar subqueries (Ashutosh Chauhan via Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 4d43695 HIVE-18201 : Disable XPROD_EDGE for sq_count_check()  
created for scalar subqueries (Ashutosh Chauhan via Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-08-01 06:01:17.302
+ rm -rf ../yetus_PreCommit-HIVE-Build-12980
+ mkdir ../yetus_PreCommit-HIVE-Build-12980
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-12980
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12980/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: patch failed: 
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:4535
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java' 
with conflicts.
error: patch failed: 
ql/src/test/results/clientnegative/alter_view_failure6.q.out:19
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientnegative/alter_view_failure6.q.out' 
with conflicts.
error: patch failed: ql/src/test/results/clientnegative/input_part0_neg.q.out:1
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientnegative/input_part0_neg.q.out' 
with conflicts.
error: patch failed: ql/src/test/results/clientnegative/strict_pruning.q.out:1
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientnegative/strict_pruning.q.out' with 
conflicts.
Going to apply patch with: git apply -p0
error: patch failed: 
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:4535
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java' 
with conflicts.
error: patch failed: 
ql/src/test/results/clientnegative/alter_view_failure6.q.out:19
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientnegative/alter_view_failure6.q.out' 
with conflicts.
error: patch failed: ql/src/test/results/clientnegative/input_part0_neg.q.out:1
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientnegative/input_part0_neg.q.out' 
with conflicts.
error: patch failed: ql/src/test/results/clientnegative/strict_pruning.q.out:1
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientnegative/strict_pruning.q.out' with 
conflicts.
U common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
U ql/src/test/results/clientnegative/alter_view_failure6.q.out
U

[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error "No partition predicate for Alias"

2018-07-31 Thread Hui Huang (JIRA)



[ 
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564771#comment-16564771
 ] 

Hui Huang commented on HIVE-20284:
--

The patch add some message into the returned error string.

> In strict mode, if constant propagation is enable, the partition filter is 
> folded before partition pruner lead to error "No partition predicate for 
> Alias"  
> 
>
> Key: HIVE-20284
> URL: https://issues.apache.org/jira/browse/HIVE-20284
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Affects Versions: 1.2.1, 2.3.3
>Reporter: Hui Huang
>Assignee: Hui Huang
>Priority: Trivial
> Fix For: 2.3.3
>
> Attachments: HIVE-20284.patch
>
>
> In strict mode and the hive.optimize.constant.propagation is set to true, the 
> following sql will failed:
> {code:java}
> hive> desc employee_part;
> OK
> col_name  data_type   comment
> eid   int
> name  string
> dept  string
> year  string
> month string
> # Partition Information
> # col_namedata_type   comment
> year  string
> month string
> Time taken: 0.564 seconds, Fetched: 11 row(s)
> hive> set hive.mapred.mode=strict;
> hive> select * from employee_part where false and concat(year,month)='201807';
> FAILED: SemanticException Queries against partitioned tables without a 
> partition filter are disabled for safety reasons. If you know what you are 
> doing, please sethive.strict.checks.large.query to false and that 
> hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get 
> errors or incorrect results if you make a mistake while using some of the 
> unsafe features. No partition predicate for Alias "employee_part" Table 
> "employee_part"
> {code}
> The above error msg is confusing，  concat(year,month)='201807' is the 
> partition filter。
> The reason is during logic optimization, the ConstantPropagate optimizer is 
> running before partitionPruner optimizer, when found a express like 'false 
> and concat(year,month)=', the express is replaced with 'fasle' , and the 
> partition filter is dropped. So the PartitionPruner can not get the partition 
> filter.
> Users can remove the constant express that always has true/false values to 
> work around.
> When views used, if some columns are constant values, users  will be  
> confusing.
> So we should add some more message in the error msg returned.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error "No partition predicate for Alias"

[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error "No partition predicate for Alias"

[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error "No partition predicate for Alias"

[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter is folded before partition pruner lead to error "No partition predicate for Alias"

4 matches

Site Navigation

Mail list logo

Footer information