[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398015#comment-15398015 ] Matt McCline commented on HIVE-14214: - Also committed to branch-2.1 > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 2.2.0 > > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.06.patch, HIVE-14214.07.patch, HIVE-14214.08.patch, > HIVE-14214.09.patch, HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397969#comment-15397969 ] Prasanth Jayachandran commented on HIVE-14214: -- [~mmccline] Can you plz backport this to branch-2.1? This is required for HIVE-14333 backport. > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 2.2.0 > > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.06.patch, HIVE-14214.07.patch, HIVE-14214.08.patch, > HIVE-14214.09.patch, HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390992#comment-15390992 ] Matt McCline commented on HIVE-14214: - An internal ptest pass succeeded with no related failures (#153). Committed to master. > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.06.patch, HIVE-14214.07.patch, HIVE-14214.08.patch, > HIVE-14214.09.patch, HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390232#comment-15390232 ] Owen O'Malley commented on HIVE-14214: -- I should point out that an array is a better type than the Map. :) > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.06.patch, HIVE-14214.07.patch, HIVE-14214.08.patch, > HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390227#comment-15390227 ] Owen O'Malley commented on HIVE-14214: -- This patch still introduces the redundant SchemaEvolution.readerFileTypes, which exactly duplicates the values in SchemaEvolution.readerFileTypes. It also has spurious trailing spaces at the end of some of the qfile tests. > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.06.patch, HIVE-14214.07.patch, HIVE-14214.08.patch, > HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[
https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389156#comment-15389156
]
Hive QA commented on HIVE-14214:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12819507/HIVE-14214.08.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 10349 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testCheckPermissions
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testGetToken
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testConnections
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testVectorizationWithAcid
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderIncompleteDelta
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderNewBaseAndDelta
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/596/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/596/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-596/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12819507 - PreCommit-HIVE-MASTER-Build
> ORC Schema Evolution and Predicate Push Down do not work together (no rows
> returned)
>
>
> Key: HIVE-14214
> URL: https://issues.apache.org/jira/browse/HIVE-14214
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch,
> HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch,
> HIVE-14214.06.patch, HIVE-14214.07.patch, HIVE-14214.08.patch,
> HIVE-14214.WIP.patch
>
>
> In Schema Evolution, the reader schema is different than the file schema
> which is used to evaluate predicate push down.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388575#comment-15388575 ] Gunther Hagleitner commented on HIVE-14214: --- I've spoken to [~prasanth_j] offline. He's willing to take up the refactoring and enablement of safe PPD in HIVE-14310, based on this patch. With this in mind, I'm +1 on this patch. > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.06.patch, HIVE-14214.07.patch, HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387086#comment-15387086 ] Matt McCline commented on HIVE-14214: - Agreed except I don't have time to rework it right now. Currently the RecordReader has the SchemaEvolution not the Reader. I'll create another JIRA for reworking this later. > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.06.patch, HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386665#comment-15386665 ] Prasanth Jayachandran commented on HIVE-14214: -- Left some comments in RB. I think many of the complications in the patch can be avoided if we just provide a Reader api that returns true if there is conversion. Reader.hasConversion(). Behind the scenes we should do all the magic of determining if the conversion is required based on reader schema, file schema and included. In OrcInputFormat the only place we need to disable PPD is in ETL strategy, which creates ORC reader. If this reader returns hasConversion() then we should disable PPD. Similarly for task side. > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385169#comment-15385169 ] Matt McCline commented on HIVE-14214: - Ok, review board created. New patch submitted for Hive QA. > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.03.patch, HIVE-14214.04.patch, HIVE-14214.05.patch, > HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[
https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381481#comment-15381481
]
Hive QA commented on HIVE-14214:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12818466/HIVE-14214.03.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10333 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_join
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_distinct_gby
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testCheckPermissions
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testGetToken
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testConnections
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion2
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion3
org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.testExternalFooterCache
org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.testExternalFooterCachePpd
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/556/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/556/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-556/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12818466 - PreCommit-HIVE-MASTER-Build
> ORC Schema Evolution and Predicate Push Down do not work together (no rows
> returned)
>
>
> Key: HIVE-14214
> URL: https://issues.apache.org/jira/browse/HIVE-14214
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch,
> HIVE-14214.03.patch, HIVE-14214.WIP.patch
>
>
> In Schema Evolution, the reader schema is different than the file schema
> which is used to evaluate predicate push down.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[
https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381358#comment-15381358
]
Hive QA commented on HIVE-14214:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12818441/HIVE-14214.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/554/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/554/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-554/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.8.0_25 ]]
+ export JAVA_HOME=/usr/java/jdk1.8.0_25
+ JAVA_HOME=/usr/java/jdk1.8.0_25
+ export
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-MASTER-Build-554/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 9764966 HIVE-14226 : Invalid check on an ASTNode#toStringTree in
CalcitePlanner (Barna Zsombor Klara via Jesus Camacho Rodriguez)
+ git clean -f -d
Removing common/src/test/org/apache/hadoop/hive/common/TestFileUtils.java
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 9764966 HIVE-14226 : Invalid check on an ASTNode#toStringTree in
CalcitePlanner (Barna Zsombor Klara via Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12818441 - PreCommit-HIVE-MASTER-Build
> ORC Schema Evolution and Predicate Push Down do not work together (no rows
> returned)
>
>
> Key: HIVE-14214
> URL: https://issues.apache.org/jira/browse/HIVE-14214
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch,
> HIVE-14214.WIP.patch
>
>
> In Schema Evolution, the reader schema is different than the file schema
> which is used to evaluate predicate push down.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381166#comment-15381166 ] Matt McCline commented on HIVE-14214: - (Tolerate no MapWork) > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.02.patch, > HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[
https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380681#comment-15380681
]
Hive QA commented on HIVE-14214:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12818138/HIVE-14214.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 101 failed/errored test(s), 10327 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_join
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mergejoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge_incompat2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_ppd_basic
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_vectorization_ppd
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_pushdown
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_distinct_gby
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_dynamic_partition_pruning
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testCheckPermissions
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testGetToken
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testConnections
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion2
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion3
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderFooterSerialize
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderFooterSerializeWithDeltas
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderNoFooterSerialize
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderNoFooterSerializeWithDeltas
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testDefaultTypes
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testDoAs
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testEmptyFile
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testInOutFormat
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testMROutput
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testNonVectorReaderFooterSerialize
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testNonVectorReaderNoFooterSerialize
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testSplitElimination
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testSplitEliminationNullStats
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testSplitGenReadOps
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testSplitGenReadOpsLocalCache
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testSplitGenReadOpsLocalCacheChangeFileLen
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testSplitGenReadOpsLocalCacheChangeModificationTime
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testVectorReaderFooterSerialize
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testVectorReaderNoFooterSerialize
org.apache.hadoop.hive.ql.io.orc.TestNewInputOutputFormat.testNewInputFormat
org.apache.hadoop.hive.ql.io.orc.TestNewInputOutputFormat.testNewInputFormatPruning
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderDelta
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderIncompleteDelta
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderNewBaseAndDelta
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderOldBaseAndDelta
org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.testExternalFooterCache
org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.testExternalFooterCachePpd
org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.testSplitEliminationComplexExpr
org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.testSplitEliminationLargeMaxSplit
org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.testSplitEliminationSmallMaxSplit
org.apache.hive.hcatalog.mapreduce.TestHCatDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask[3]
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15379200#comment-15379200 ] Matt McCline commented on HIVE-14214: - Thanks Sergey for the advice. Patch #1 still needs a test for split elimination (i.e. test new code in OrcInputFormat.SplitGenerator.callInternal) > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.01.patch, HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15375539#comment-15375539 ] Sergey Shelukhin commented on HIVE-14214: - HBase PPD feature is not complete right now. Unless I am missing something it should be ok to break it as long as it breaks in a reasonable manner (e.g. throws suggesting the user not enable it, it's off by default), and there's a followup JIRA > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
[ https://issues.apache.org/jira/browse/HIVE-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374775#comment-15374775 ] Matt McCline commented on HIVE-14214: - Work-In-Progress patch. Areas that need work: 1) Using readerTypes to check and not do PPD in LocalCache. 2) OrcFileFormatProxy.applySargToMetadata -- need to pass down or generate included boolean array. 3) HiveMetaStore.get_file_metadata_by_expr -- need to deserialize the readerTypes. And, modify the Thrift source to add the readerTypes. And add unit tests and/or Q file tests. > ORC Schema Evolution and Predicate Push Down do not work together (no rows > returned) > > > Key: HIVE-14214 > URL: https://issues.apache.org/jira/browse/HIVE-14214 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-14214.WIP.patch > > > In Schema Evolution, the reader schema is different than the file schema > which is used to evaluate predicate push down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
