[jira] [Commented] (HIVE-15192) Use Calcite to de-correlate and plan subqueries
[
https://issues.apache.org/jira/browse/HIVE-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15719403#comment-15719403
]
Hive QA commented on HIVE-15192:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12841661/HIVE-15192.3.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 41 failed/errored test(s), 10762 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_subq_exists]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog_partitioner]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_3] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_4] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin4] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin5] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subq_where_serialization]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_exists]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_exists_having]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_in_having]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_unqualcolumnrefs]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_mapjoin_reduce]
(batchId=71)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_subq_in]
(batchId=138)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_subq_not_in]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_partition_pruning]
(batchId=142)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage3]
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_exists]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in]
(batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_nested_subquery]
(batchId=142)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_views]
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_mapjoin_reduce]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_partition_pruning]
(batchId=142)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[constprog_partitioner]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_restrictions]
(batchId=84)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query70]
(batchId=220)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_exists]
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in]
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_mapjoin_reduce]
(batchId=127)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema[2]
(batchId=171)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2404/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2404/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2404/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 41 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12841661 -
[jira] [Commented] (HIVE-15323) allow the user to turn off reduce-side SMB join
[ https://issues.apache.org/jira/browse/HIVE-15323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15719388#comment-15719388 ] Lefty Leverenz commented on HIVE-15323: --- Doc note: *hive.auto.convert.sortmerge.join.reduce.side* needs to be documented in the wiki for release 2.2.0. It can go after *hive.auto.convert.sortmerge.join* & *hive.auto.convert.sortmerge.join.bigtable.selection.policy* from HIVE-3403 and *hive.auto.convert.sortmerge.join.to.mapjoin* from HIVE-3891, none of which are documented yet. * [Configuration Properties – Query and DDL Execution | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution] Added a TODOC2.2 label. > allow the user to turn off reduce-side SMB join > > > Key: HIVE-15323 > URL: https://issues.apache.org/jira/browse/HIVE-15323 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-15323.01.patch, HIVE-15323.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-3403) user should not specify mapjoin to perform sort-merge bucketed join
[ https://issues.apache.org/jira/browse/HIVE-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15719384#comment-15719384 ] Lefty Leverenz commented on HIVE-3403: -- Doc note: This adds *hive.auto.convert.sortmerge.join* and *hive.auto.convert.sortmerge.join.bigtable.selection.policy* to HiveConf.java in release 0.11.0, so they need to be documented in the wiki. * [Configuration Properties -- Query and DDL Execution | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution] Added a TODOC11 label. > user should not specify mapjoin to perform sort-merge bucketed join > --- > > Key: HIVE-3403 > URL: https://issues.apache.org/jira/browse/HIVE-3403 > Project: Hive > Issue Type: Bug >Reporter: Namit Jain >Assignee: Namit Jain > Labels: TODOC11 > Fix For: 0.11.0 > > Attachments: auto_sortmerge_join_1_modified.q, hive.3403.1.patch, > hive.3403.10.patch, hive.3403.11.patch, hive.3403.12.patch, > hive.3403.13.patch, hive.3403.14.patch, hive.3403.15.patch, > hive.3403.16.patch, hive.3403.17.patch, hive.3403.18.patch, > hive.3403.19.patch, hive.3403.2.patch, hive.3403.21.patch, > hive.3403.22.patch, hive.3403.23.patch, hive.3403.24.patch, > hive.3403.25.patch, hive.3403.26.patch, hive.3403.27.patch, > hive.3403.28.patch, hive.3403.29.patch, hive.3403.3.patch, > hive.3403.30.patch, hive.3403.4.patch, hive.3403.5.patch, hive.3403.6.patch, > hive.3403.7.patch, hive.3403.8.patch, hive.3403.9.patch > > > Currently, in order to perform a sort merge bucketed join, the user needs > to set hive.optimize.bucketmapjoin.sortedmerge to true, and also specify the > mapjoin hint. > The user should not specify any hints. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join
[ https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15719381#comment-15719381 ] Lefty Leverenz commented on HIVE-3891: -- Doc note: This adds *hive.auto.convert.sortmerge.join.to.mapjoin* to HiveConf.java in release 0.12.0, so it needs to be documented in the wiki. * [Configuration Properties -- Query and DDL Execution | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution] Added a TODOC12 label. > physical optimizer changes for auto sort-merge join > --- > > Key: HIVE-3891 > URL: https://issues.apache.org/jira/browse/HIVE-3891 > Project: Hive > Issue Type: Bug >Reporter: Namit Jain >Assignee: Namit Jain > Labels: TODOC12 > Fix For: 0.12.0 > > Attachments: HIVE-3891_8.patch, auto_sortmerge_join_1.q, > auto_sortmerge_join_1.q.out, hive.3891.1.patch, hive.3891.10.patch, > hive.3891.11.patch, hive.3891.12.patch, hive.3891.13.patch, > hive.3891.14.patch, hive.3891.15.patch, hive.3891.16.patch, > hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, hive.3891.5.patch, > hive.3891.6.patch, hive.3891.7.patch, hive.3891.9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-3891) physical optimizer changes for auto sort-merge join
[ https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-3891: - Labels: TODOC12 (was: ) > physical optimizer changes for auto sort-merge join > --- > > Key: HIVE-3891 > URL: https://issues.apache.org/jira/browse/HIVE-3891 > Project: Hive > Issue Type: Bug >Reporter: Namit Jain >Assignee: Namit Jain > Labels: TODOC12 > Fix For: 0.12.0 > > Attachments: HIVE-3891_8.patch, auto_sortmerge_join_1.q, > auto_sortmerge_join_1.q.out, hive.3891.1.patch, hive.3891.10.patch, > hive.3891.11.patch, hive.3891.12.patch, hive.3891.13.patch, > hive.3891.14.patch, hive.3891.15.patch, hive.3891.16.patch, > hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, hive.3891.5.patch, > hive.3891.6.patch, hive.3891.7.patch, hive.3891.9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-3403) user should not specify mapjoin to perform sort-merge bucketed join
[ https://issues.apache.org/jira/browse/HIVE-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-3403: - Labels: TODOC11 (was: ) > user should not specify mapjoin to perform sort-merge bucketed join > --- > > Key: HIVE-3403 > URL: https://issues.apache.org/jira/browse/HIVE-3403 > Project: Hive > Issue Type: Bug >Reporter: Namit Jain >Assignee: Namit Jain > Labels: TODOC11 > Fix For: 0.11.0 > > Attachments: auto_sortmerge_join_1_modified.q, hive.3403.1.patch, > hive.3403.10.patch, hive.3403.11.patch, hive.3403.12.patch, > hive.3403.13.patch, hive.3403.14.patch, hive.3403.15.patch, > hive.3403.16.patch, hive.3403.17.patch, hive.3403.18.patch, > hive.3403.19.patch, hive.3403.2.patch, hive.3403.21.patch, > hive.3403.22.patch, hive.3403.23.patch, hive.3403.24.patch, > hive.3403.25.patch, hive.3403.26.patch, hive.3403.27.patch, > hive.3403.28.patch, hive.3403.29.patch, hive.3403.3.patch, > hive.3403.30.patch, hive.3403.4.patch, hive.3403.5.patch, hive.3403.6.patch, > hive.3403.7.patch, hive.3403.8.patch, hive.3403.9.patch > > > Currently, in order to perform a sort merge bucketed join, the user needs > to set hive.optimize.bucketmapjoin.sortedmerge to true, and also specify the > mapjoin hint. > The user should not specify any hints. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15323) allow the user to turn off reduce-side SMB join
[ https://issues.apache.org/jira/browse/HIVE-15323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-15323: -- Labels: TODOC2.2 (was: ) > allow the user to turn off reduce-side SMB join > > > Key: HIVE-15323 > URL: https://issues.apache.org/jira/browse/HIVE-15323 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-15323.01.patch, HIVE-15323.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15192) Use Calcite to de-correlate and plan subqueries
[ https://issues.apache.org/jira/browse/HIVE-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15719342#comment-15719342 ] Vineet Garg commented on HIVE-15192: Work in progress @ https://github.com/vineetgarg02/hive/tree/HIVE-15192 > Use Calcite to de-correlate and plan subqueries > --- > > Key: HIVE-15192 > URL: https://issues.apache.org/jira/browse/HIVE-15192 > Project: Hive > Issue Type: Task > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg > Labels: sub-query > Attachments: HIVE-15192.2.patch, HIVE-15192.3.patch, HIVE-15192.patch > > > Currently support of subqueries is limited [Link to original spec | > https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf]. > Using Calcite to plan and de-correlate subqueries will help Hive get rid of > these limitations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15192) Use Calcite to de-correlate and plan subqueries
[ https://issues.apache.org/jira/browse/HIVE-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-15192: --- Status: Patch Available (was: Open) Know issue with this patch (HIVE-15192.3.patch) * Following tests fails since NOT IN plan is wrong ** subquery_notin ** subquery_notin_having ** vector_groupby_mapjoin ** cbo_subq_not_in * Following return path test fails ** cbo_rp_subq_in ** cbo_rp_subq_not_in ** cbo_rp_subq_exists > Use Calcite to de-correlate and plan subqueries > --- > > Key: HIVE-15192 > URL: https://issues.apache.org/jira/browse/HIVE-15192 > Project: Hive > Issue Type: Task > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg > Labels: sub-query > Attachments: HIVE-15192.2.patch, HIVE-15192.3.patch, HIVE-15192.patch > > > Currently support of subqueries is limited [Link to original spec | > https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf]. > Using Calcite to plan and de-correlate subqueries will help Hive get rid of > these limitations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15192) Use Calcite to de-correlate and plan subqueries
[ https://issues.apache.org/jira/browse/HIVE-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-15192: --- Attachment: HIVE-15192.3.patch > Use Calcite to de-correlate and plan subqueries > --- > > Key: HIVE-15192 > URL: https://issues.apache.org/jira/browse/HIVE-15192 > Project: Hive > Issue Type: Task > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg > Labels: sub-query > Attachments: HIVE-15192.2.patch, HIVE-15192.3.patch, HIVE-15192.patch > > > Currently support of subqueries is limited [Link to original spec | > https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf]. > Using Calcite to plan and de-correlate subqueries will help Hive get rid of > these limitations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15192) Use Calcite to de-correlate and plan subqueries
[ https://issues.apache.org/jira/browse/HIVE-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-15192: --- Status: Open (was: Patch Available) > Use Calcite to de-correlate and plan subqueries > --- > > Key: HIVE-15192 > URL: https://issues.apache.org/jira/browse/HIVE-15192 > Project: Hive > Issue Type: Task > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg > Labels: sub-query > Attachments: HIVE-15192.2.patch, HIVE-15192.3.patch, HIVE-15192.patch > > > Currently support of subqueries is limited [Link to original spec | > https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf]. > Using Calcite to plan and de-correlate subqueries will help Hive get rid of > these limitations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15334) HIVE-13945 changed scale rules for division
[ https://issues.apache.org/jira/browse/HIVE-15334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-15334: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to master > HIVE-13945 changed scale rules for division > --- > > Key: HIVE-15334 > URL: https://issues.apache.org/jira/browse/HIVE-15334 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Jason Dere >Assignee: Jason Dere > Fix For: 2.2.0 > > Attachments: HIVE-15334.1.patch, HIVE-15334.2.patch, > HIVE-15334.3.patch > > > Looks like HIVE-13945 change the decimal division precision/scale rules - the > explanation being "Changed the default decimal precision in division, not > sure why it was so low by default." > (https://issues.apache.org/jira/browse/HIVE-13945?focusedCommentId=15354403=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15354403) > As far as I can tell this causes decimal division to have a minimum scale of > 18. > cc [~sershe] - the rules that were in place were based on the SQL Server > precision/scale rules in > https://msdn.microsoft.com/en-us/library/ms190476.aspx > I'd like to revert this change to precision/scale rules -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15057) Nested column pruning: support all operators
[ https://issues.apache.org/jira/browse/HIVE-15057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718829#comment-15718829 ] ASF GitHub Bot commented on HIVE-15057: --- Github user sunchao closed the pull request at: https://github.com/apache/hive/pull/115 > Nested column pruning: support all operators > > > Key: HIVE-15057 > URL: https://issues.apache.org/jira/browse/HIVE-15057 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Physical Optimizer >Reporter: Chao Sun >Assignee: Chao Sun > Fix For: 2.2.0 > > Attachments: HIVE-15057.1.patch, HIVE-15057.2.patch, > HIVE-15057.3.patch > > > Currently only SELECT operators are supported for nested column pruning. We > should add support for other types of operators so the optimization can work > for complex queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15130) Fixing shifting index issue related to object inspector caching
[
https://issues.apache.org/jira/browse/HIVE-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718830#comment-15718830
]
ASF GitHub Bot commented on HIVE-15130:
---
Github user sunchao closed the pull request at:
https://github.com/apache/hive/pull/112
> Fixing shifting index issue related to object inspector caching
> ---
>
> Key: HIVE-15130
> URL: https://issues.apache.org/jira/browse/HIVE-15130
> Project: Hive
> Issue Type: Sub-task
> Components: Reader
>Reporter: Chao Sun
>Assignee: Chao Sun
> Fix For: 2.2.0
>
> Attachments: HIVE-15130.1.patch, HIVE-15130.2.patch
>
>
> Hive caches object inspector in memory to avoid recreating the same object.
> However, in {{ArrayWritableObjectInspector}}, the {{equals}} and {{hashCode}}
> doesn't take account on all attributes of the class, therefore, even though
> the internal state (e.g., {{adjustedIndex}}) is different, it could still be
> mapped to the same result. This could cause wrong field index being used when
> reading Parquet data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15353) Metastore throws NPE if StorageDescriptor.cols is null
[
https://issues.apache.org/jira/browse/HIVE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718691#comment-15718691
]
Hive QA commented on HIVE-15353:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12841638/HIVE-15353.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10713 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=143)
[vectorized_rcfile_columnar.q,vector_elt.q,explainuser_1.q,multi_insert.q,tez_dml.q,vector_bround.q,schema_evol_orc_acid_table.q,vector_when_case_null.q,orc_ppd_schema_evol_1b.q,vector_join30.q,vectorization_11.q,cte_3.q,update_tmp_table.q,vector_interval_mapjoin.q,vector_decimal_cast.q,groupby_grouping_id2.q,vector_decimal_round.q,tez_smb_empty.q,orc_merge6.q,vector_decimal_trailing.q,cte_5.q,tez_union.q,cbo_rp_subq_not_in.q,columnStatsUpdateForStatsOptimizer_1.q,vector_outer_join3.q,schema_evol_text_vec_part_all_complex.q,tez_dynpart_hashjoin_2.q,auto_sortmerge_join_12.q,offset_limit.q,tez_union_multiinsert.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=123)
[table_access_keys_stats.q,bucketmapjoin11.q,auto_join4.q,join34.q,nullgroup.q,mergejoins_mixed.q,sort.q,join_nullsafe.q,stats8.q,auto_join28.q,join17.q,union17.q,skewjoinopt11.q,groupby1_map.q,load_dyn_part11.q]
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=92)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testPartition
(batchId=191)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testPartition
(batchId=194)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testPartition
(batchId=190)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testPartition
(batchId=188)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartition
(batchId=199)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2403/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2403/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2403/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12841638 - PreCommit-HIVE-Build
> Metastore throws NPE if StorageDescriptor.cols is null
> --
>
> Key: HIVE-15353
> URL: https://issues.apache.org/jira/browse/HIVE-15353
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Anthony Hsu
>Assignee: Anthony Hsu
> Attachments: HIVE-15353.1.patch
>
>
> When using the HiveMetaStoreClient API directly to talk to the metastore, you
> get NullPointerExceptions when StorageDescriptor.cols is null in the
> Table/Partition object in the following calls:
> * create_table
> * alter_table
> * alter_partition
> Calling add_partition with StorageDescriptor.cols set to null causes null to
> be stored in the metastore database and subsequent calls to alter_partition
> for that partition to fail with an NPE.
> The simplest way to fix these NPEs seems to be to update the
> StorageDescriptor.cols Thrift definition and set a default value of empty
> list. Some null checks will also have to be added to handle existing nulls in
> the metastore database.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15057) Nested column pruning: support all operators
[ https://issues.apache.org/jira/browse/HIVE-15057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-15057: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to master. Thanks [~Ferd] for the review! > Nested column pruning: support all operators > > > Key: HIVE-15057 > URL: https://issues.apache.org/jira/browse/HIVE-15057 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Physical Optimizer >Reporter: Chao Sun >Assignee: Chao Sun > Fix For: 2.2.0 > > Attachments: HIVE-15057.1.patch, HIVE-15057.2.patch, > HIVE-15057.3.patch > > > Currently only SELECT operators are supported for nested column pruning. We > should add support for other types of operators so the optimization can work > for complex queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15057) Nested column pruning: support all operators
[ https://issues.apache.org/jira/browse/HIVE-15057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-15057: Summary: Nested column pruning: support all operators (was: Support other types of operators (other than SELECT)) > Nested column pruning: support all operators > > > Key: HIVE-15057 > URL: https://issues.apache.org/jira/browse/HIVE-15057 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Physical Optimizer >Reporter: Chao Sun >Assignee: Chao Sun > Attachments: HIVE-15057.1.patch, HIVE-15057.2.patch, > HIVE-15057.3.patch > > > Currently only SELECT operators are supported for nested column pruning. We > should add support for other types of operators so the optimization can work > for complex queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15353) Metastore throws NPE if StorageDescriptor.cols is null
[ https://issues.apache.org/jira/browse/HIVE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Hsu updated HIVE-15353: --- Status: Patch Available (was: Open) > Metastore throws NPE if StorageDescriptor.cols is null > -- > > Key: HIVE-15353 > URL: https://issues.apache.org/jira/browse/HIVE-15353 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.2.0 >Reporter: Anthony Hsu >Assignee: Anthony Hsu > Attachments: HIVE-15353.1.patch > > > When using the HiveMetaStoreClient API directly to talk to the metastore, you > get NullPointerExceptions when StorageDescriptor.cols is null in the > Table/Partition object in the following calls: > * create_table > * alter_table > * alter_partition > Calling add_partition with StorageDescriptor.cols set to null causes null to > be stored in the metastore database and subsequent calls to alter_partition > for that partition to fail with an NPE. > The simplest way to fix these NPEs seems to be to update the > StorageDescriptor.cols Thrift definition and set a default value of empty > list. Some null checks will also have to be added to handle existing nulls in > the metastore database. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15353) Metastore throws NPE if StorageDescriptor.cols is null
[ https://issues.apache.org/jira/browse/HIVE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Hsu updated HIVE-15353: --- Attachment: HIVE-15353.1.patch Uploaded patch. Also posted to RB: https://reviews.apache.org/r/54341/ > Metastore throws NPE if StorageDescriptor.cols is null > -- > > Key: HIVE-15353 > URL: https://issues.apache.org/jira/browse/HIVE-15353 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.2.0 >Reporter: Anthony Hsu >Assignee: Anthony Hsu > Attachments: HIVE-15353.1.patch > > > When using the HiveMetaStoreClient API directly to talk to the metastore, you > get NullPointerExceptions when StorageDescriptor.cols is null in the > Table/Partition object in the following calls: > * create_table > * alter_table > * alter_partition > Calling add_partition with StorageDescriptor.cols set to null causes null to > be stored in the metastore database and subsequent calls to alter_partition > for that partition to fail with an NPE. > The simplest way to fix these NPEs seems to be to update the > StorageDescriptor.cols Thrift definition and set a default value of empty > list. Some null checks will also have to be added to handle existing nulls in > the metastore database. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15353) Metastore throws NPE if StorageDescriptor.cols is null
[
https://issues.apache.org/jira/browse/HIVE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718515#comment-15718515
]
Anthony Hsu commented on HIVE-15353:
Example stack trace:
{noformat}
2016-12-02T14:25:36,994 ERROR [pool-6-thread-6] metastore.RetryingHMSHandler:
MetaException(message:java.lang.NullPointerException)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:6152)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.rename_partition(HiveMetaStore.java:3828)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.rename_partition(HiveMetaStore.java:3765)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partition(HiveMetaStore.java:3748)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
at com.sun.proxy.$Proxy21.alter_partition(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partition.getResult(ThriftHiveMetastore.java:12394)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_partition.getResult(ThriftHiveMetastore.java:12378)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:103)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.areSameColumns(MetaStoreUtils.java:629)
at
org.apache.hadoop.hive.metastore.HiveAlterHandler.updatePartColumnStats(HiveAlterHandler.java:770)
at
org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartition(HiveAlterHandler.java:405)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.rename_partition(HiveMetaStore.java:3799)
... 17 more
{noformat}
> Metastore throws NPE if StorageDescriptor.cols is null
> --
>
> Key: HIVE-15353
> URL: https://issues.apache.org/jira/browse/HIVE-15353
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Anthony Hsu
>Assignee: Anthony Hsu
>
> When using the HiveMetaStoreClient API directly to talk to the metastore, you
> get NullPointerExceptions when StorageDescriptor.cols is null in the
> Table/Partition object in the following calls:
> * create_table
> * alter_table
> * alter_partition
> Calling add_partition with StorageDescriptor.cols set to null causes null to
> be stored in the metastore database and subsequent calls to alter_partition
> for that partition to fail with an NPE.
> The simplest way to fix these NPEs seems to be to update the
> StorageDescriptor.cols Thrift definition and set a default value of empty
> list. Some null checks will also have to be added to handle existing nulls in
> the metastore database.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12504) TxnHandler.abortTxn() should check if already aborted to improve message
[ https://issues.apache.org/jira/browse/HIVE-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718425#comment-15718425 ] Eugene Koifman commented on HIVE-12504: --- failures have age > 1 except org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a] which is a known flaky test [~wzheng] could you review please > TxnHandler.abortTxn() should check if already aborted to improve message > > > Key: HIVE-12504 > URL: https://issues.apache.org/jira/browse/HIVE-12504 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-12504.01.patch, HIVE-12504.patch > > > If the update statement misses, it would be useful for abortTxn() to > run another query to see if the requested txnid is in Aborted state to > improve error msg. > Note: get a separate read_committed connection for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15337) Need to specify starttime when putting Compaction job into "attempted" state
[ https://issues.apache.org/jira/browse/HIVE-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718421#comment-15718421 ] Eugene Koifman commented on HIVE-15337: --- failures have age > 1 [~wzheng] could you review please > Need to specify starttime when putting Compaction job into "attempted" state > > > Key: HIVE-15337 > URL: https://issues.apache.org/jira/browse/HIVE-15337 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15337.01.patch > > > W/o this SHOW COMPACTIONS output is not as useful > Also, add Hadoop Job ID to SHOW COMPACTIONS output -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15276) CLIs spell "substitution" as "subsitution" and "auxiliary" as "auxillary"
[ https://issues.apache.org/jira/browse/HIVE-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718310#comment-15718310 ] Grant Sohn commented on HIVE-15276: --- Thanks, fixed the wiki! > CLIs spell "substitution" as "subsitution" and "auxiliary" as "auxillary" > - > > Key: HIVE-15276 > URL: https://issues.apache.org/jira/browse/HIVE-15276 > Project: Hive > Issue Type: Bug > Components: CLI >Affects Versions: 1.1.0 >Reporter: Grant Sohn >Assignee: Grant Sohn >Priority: Trivial > Fix For: 2.2.0 > > Attachments: HIVE-15276.1.patch, HIVE-15276.2.patch, > HIVE-15276.3.patch, HIVE-15276.4.patch, HIVE-15276.5.patch > > > Found some obvious spelling typos in the CLI help. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15352) MVCC (Multi Versioned Concurrency Control) in Hive
[ https://issues.apache.org/jira/browse/HIVE-15352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garima Dosi updated HIVE-15352: --- Attachment: Hive MVCC - Requirement & Design.pdf > MVCC (Multi Versioned Concurrency Control) in Hive > -- > > Key: HIVE-15352 > URL: https://issues.apache.org/jira/browse/HIVE-15352 > Project: Hive > Issue Type: New Feature >Reporter: Garima Dosi > Attachments: Hive MVCC - Requirement & Design.pdf > > > Use Case > While working with providing solutions for various applications, we see that > there is at times, a need to provide multi version concurrency support for > certain datasets. The requirement of multi versioned concurrency is mainly > due to two reasons – > • Simultaneous querying and loading from tables or datasets, which requires > maintaining versions for reading and writing (Locking is not the right option > here) > • Maintaining historical load of tables/datasets upto some extent > Both of these requirements are seen in data management systems (warehouses > etc). > What happens without MVCC in Hive? > In cases, where MVCC had to be done, design similar to this - > https://dzone.com/articles/zookeeper-a-real-world-example-of-how-to-use-it > was followed to make it work. Zookeeper was used to maintain versions and > provide MVCC support. However, this design poses a limitation if a normal > user would like to query a hive table because he will not be aware of the > current version to be queried. The additional layer to match versions in > zookeeper with the dataset to be queried introduces a bit of an overhead for > normal users and hence, the request to make this feature available in Hive. > Hive Design for Support of MVCC > The hive design for MVCC support can be as described below (It would somewhat > follow the article mentioned in the previous section) – > 1. The first thing should be the ability for the user to specify that this is > a MVCC table. So, a DDL something like this – > create table ( ) MULTI_VERSIONED ON [sequence, > time] > Internally this DDL can be translated to a partitioned table either on a > sequence number (auto-generated by Hive) or a timestamp. The metastore would > keep this information. > 2. DMLs related to inserting or loading data to the table would remain the > same for an end user. However, internally Hive would automatically detect > that a table is a multi-versioned table and write the new data to a new > partition with a new version of the dataset. The Hive Metastore would also be > updated with the current version. > 3. DMLs related to querying data from the table would remain the same for a > user. However, internally Hive would use the latest version for queries. > Latest version is always stored in the metastore. > Management of obsolete versions > The obsolete versions can be deleted based on the following – > 1.Either a setting which simply says delete the version which is older than a > threshold and is not active, OR > 2.By tracking the count of queries running on older versions and deleting the > ones which are not the latest and are not being used by any query. This would > require some sort of a background thread monitoring the table for obsolete > versions. As shown in the article mentioned above, this would also require > incrementing version count whenever a version is queried and decrement it > once the query is done. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15337) Need to specify starttime when putting Compaction job into "attempted" state
[
https://issues.apache.org/jira/browse/HIVE-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15717903#comment-15717903
]
Hive QA commented on HIVE-15337:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12841595/HIVE-15337.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10758 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=134)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2401/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2401/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2401/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12841595 - PreCommit-HIVE-Build
> Need to specify starttime when putting Compaction job into "attempted" state
>
>
> Key: HIVE-15337
> URL: https://issues.apache.org/jira/browse/HIVE-15337
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-15337.01.patch
>
>
> W/o this SHOW COMPACTIONS output is not as useful
> Also, add Hadoop Job ID to SHOW COMPACTIONS output
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15337) Need to specify starttime when putting Compaction job into "attempted" state
[
https://issues.apache.org/jira/browse/HIVE-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15717833#comment-15717833
]
Hive QA commented on HIVE-15337:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12841595/HIVE-15337.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10758 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=134)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2400/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2400/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2400/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12841595 - PreCommit-HIVE-Build
> Need to specify starttime when putting Compaction job into "attempted" state
>
>
> Key: HIVE-15337
> URL: https://issues.apache.org/jira/browse/HIVE-15337
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-15337.01.patch
>
>
> W/o this SHOW COMPACTIONS output is not as useful
> Also, add Hadoop Job ID to SHOW COMPACTIONS output
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15351) Disable vectorized VectorUDFAdaptor usage with non-column or constant parameters
[
https://issues.apache.org/jira/browse/HIVE-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15717765#comment-15717765
]
Hive QA commented on HIVE-15351:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12841591/HIVE-15351.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 10746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog_when_case]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[foldts] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_between_columns]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_2]
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_3]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_math_funcs]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_when_case_null]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_case]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_casts]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_math_funcs]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_timestamp_ints_casts]
(batchId=44)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_adaptor_usage_mode]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_between_columns]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_coalesce_2]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_when_case_null]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_case]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_casts]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=91)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_case]
(batchId=117)
org.apache.hive.jdbc.TestJdbcWithLocalClusterSpark.testSparkQuery (batchId=214)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2399/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2399/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2399/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12841591 - PreCommit-HIVE-Build
> Disable vectorized VectorUDFAdaptor usage with non-column or constant
> parameters
>
>
> Key: HIVE-15351
> URL: https://issues.apache.org/jira/browse/HIVE-15351
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Blocker
> Attachments: HIVE-15351.01.patch
>
>
> Vectorization using VectorUDFAdaptor is broken and produces wrong results
> when the parameter(s) have vectorized expressions that allocate scratch
> columns. So, for now, we restrict VectorUDFAdaptor usage to columns or
> constant expressions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15348) Add more specific error codes to ReplicationSemanticAnalyzer's SemanticExceptions
[
https://issues.apache.org/jira/browse/HIVE-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15717701#comment-15717701
]
Hive QA commented on HIVE-15348:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12841588/HIVE-15348.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10740 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1]
(batchId=91)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=92)
org.apache.hive.hcatalog.api.TestHCatClient.org.apache.hive.hcatalog.api.TestHCatClient
(batchId=167)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2398/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2398/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2398/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12841588 - PreCommit-HIVE-Build
> Add more specific error codes to ReplicationSemanticAnalyzer's
> SemanticExceptions
> -
>
> Key: HIVE-15348
> URL: https://issues.apache.org/jira/browse/HIVE-15348
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-15348.1.patch
>
>
> Also remove TODOS/FIXME in ReplicationSemanticAnalyzer and create
> corresponding jiras or fix minor isues
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14582) Add trunc(numeric) udf
[ https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15717669#comment-15717669 ] Lefty Leverenz commented on HIVE-14582: --- Doc note: The trunc(numeric) UDF needs to be documented in the wiki for release 2.2.0. * [Hive Operators and UDFs -- Mathematical Functions | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-MathematicalFunctions] Added a TODOC2.2 label. > Add trunc(numeric) udf > -- > > Key: HIVE-14582 > URL: https://issues.apache.org/jira/browse/HIVE-14582 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Ashutosh Chauhan >Assignee: Chinna Rao Lalam > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-14582.1.patch, HIVE-14582.2.patch, > HIVE-14582.3.patch, HIVE-14582.4.patch, HIVE-14582.patch > > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
