[jira] [Commented] (HIVE-19374) Parse and process ALTER TABLE SET OWNER command syntax
[
https://issues.apache.org/jira/browse/HIVE-19374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470016#comment-16470016
]
Hive QA commented on HIVE-19374:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922555/HIVE-19374.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 45 failed/errored test(s), 13545 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.q,l
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470051#comment-16470051
]
Hive QA commented on HIVE-19306:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} serde in master has 190 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 588 new + 0 unchanged - 0
fixed = 588 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
55s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10797/dev-support/hive-personality.sh
|
| git revision | master / 1cd5274 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10797/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10797/yetus/diff-javadoc-javadoc-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10797/yetus/patch-asflicense-problems.txt
|
| modules | C: common serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10797/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Arrow batch serializer
> --
>
> Key: HIVE-19306
> URL: https://issues.apache.org/jira/browse/HIVE-19306
> Project: Hive
> Issue Type: Task
> Components: Serializers/Deserializers
>Reporter: Eric Wohlstadter
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-19306.2.patch, HIVE-19306.3.patch,
> HIVE-19306.4.patch
>
>
> Leverage the ThriftJDBCBinarySerDe code path that already exists in
> S
[jira] [Commented] (HIVE-19435) Incremental replication cause data loss if a table is dropped followed by create and insert-into with different partition type.
[ https://issues.apache.org/jira/browse/HIVE-19435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470077#comment-16470077 ] mahesh kumar behera commented on HIVE-19435: code changes looks fine to me > Incremental replication cause data loss if a table is dropped followed by > create and insert-into with different partition type. > --- > > Key: HIVE-19435 > URL: https://issues.apache.org/jira/browse/HIVE-19435 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 3.1.0 > > Attachments: HIVE-19435.01.patch, HIVE-19435.02.patch, > HIVE-19435.03.patch > > > If the incremental dump have drop of partitioned table followed by > create/insert on non-partitioned table with same name, doesn't replicate the > data. Explained below. > Let's say we have a partitioned table T1 which was already replicated to > target. > DROP_TABLE(T1)->CREATE_TABLE(T1) (Non-partitioned) -> INSERT(T1)(10) > After REPL LOAD, T1 doesn't have any data. > Same is valid for non-partitioned to partitioned and partition spec mismatch > case as well. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470122#comment-16470122
]
Hive QA commented on HIVE-19306:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922558/HIVE-19306.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 43 failed/errored test(s), 13574 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.q,l
[jira] [Commented] (HIVE-19472) HiveStreamingConnection swallows exception on partition creation
[
https://issues.apache.org/jira/browse/HIVE-19472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470139#comment-16470139
]
Hive QA commented on HIVE-19472:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10800/dev-support/hive-personality.sh
|
| git revision | master / 1cd5274 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10800/yetus/patch-asflicense-problems.txt
|
| modules | C: streaming U: streaming |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10800/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveStreamingConnection swallows exception on partition creation
>
>
> Key: HIVE-19472
> URL: https://issues.apache.org/jira/browse/HIVE-19472
> Project: Hive
> Issue Type: Sub-task
> Components: Streaming
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19472.1.patch
>
>
> HiveStreamingConnection swallows exception on partition creation
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19485) dump directory for non native tables should not be created
[ https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-19485: -- > dump directory for non native tables should not be created > -- > > Key: HIVE-19485 > URL: https://issues.apache.org/jira/browse/HIVE-19485 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Fix For: 3.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError
[
https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470204#comment-16470204
]
Alisha Prabhu commented on HIVE-19454:
--
At Line no. 499 and 532 of TestTxnCommands2.java the below assertion tries to
compare the bucket file name at the 0th index to "bucket_1".
{code:java}
FileStatus[] buckets = fs.listStatus(status[i].getPath(),
FileUtils.HIDDEN_FILES_PATH_FILTER);
Assert.assertEquals(2, buckets.length);
Assert.assertTrue(buckets[0].getPath().getName().matches("bucket_1"));
{code}
However, on our local platform we get the bucket file name at the 0th index as
"bucket_0" and hence the test case fails.
Also would like to bring to your notice that, the listStatus function of the
FileSystem does not guarantee to return the List of files/directories status in
a sorted order.
Please update me with any possible solution on the above issue.
> Test failure :
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails
> with java.lang.AssertionError
> --
>
> Key: HIVE-19454
> URL: https://issues.apache.org/jira/browse/HIVE-19454
> Project: Hive
> Issue Type: Bug
> Environment: uname -a
> Linux pts00433-vm40 3.10.0-862.el7.ppc64le #1 SMP Wed Mar 21 22:20:19 UTC
> 2018 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: HIVE-19454.1.patch
>
>
> org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
> refers to the same test case.
> Maven command used is : mvn -Dtest=TestTxnCommands2 test
> Error:
> {code:java}
> [INFO] Running org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3, Time elapsed:
> 618.215 s <<< FAILURE! - in org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR]
> testNonAcidToAcidConversion1(org.apache.hadoop.hive.ql.TestTxnCommands2)
> Time elapsed: 17.557 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1(TestTxnCommands2.java:499)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.b
[jira] [Comment Edited] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError
[
https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470204#comment-16470204
]
Alisha Prabhu edited comment on HIVE-19454 at 5/10/18 10:42 AM:
Hi [~djaiswal],
at Line no. 499 and 532 of TestTxnCommands2.java the below assertion tries to
compare the bucket file name at the 0th index to "bucket_1".
{code:java}
FileStatus[] buckets = fs.listStatus(status[i].getPath(),
FileUtils.HIDDEN_FILES_PATH_FILTER);
Assert.assertEquals(2, buckets.length);
Assert.assertTrue(buckets[0].getPath().getName().matches("bucket_1"));
{code}
However, on our local platform we get the bucket file name at the 0th index as
"bucket_0" and hence the test case fails.
Also would like to bring to your notice that, the listStatus function of the
FileSystem does not guarantee to return the List of files/directories status in
a sorted order.
Please update me with any possible solution on the above issue.
was (Author: alishap):
At Line no. 499 and 532 of TestTxnCommands2.java the below assertion tries to
compare the bucket file name at the 0th index to "bucket_1".
{code:java}
FileStatus[] buckets = fs.listStatus(status[i].getPath(),
FileUtils.HIDDEN_FILES_PATH_FILTER);
Assert.assertEquals(2, buckets.length);
Assert.assertTrue(buckets[0].getPath().getName().matches("bucket_1"));
{code}
However, on our local platform we get the bucket file name at the 0th index as
"bucket_0" and hence the test case fails.
Also would like to bring to your notice that, the listStatus function of the
FileSystem does not guarantee to return the List of files/directories status in
a sorted order.
Please update me with any possible solution on the above issue.
> Test failure :
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails
> with java.lang.AssertionError
> --
>
> Key: HIVE-19454
> URL: https://issues.apache.org/jira/browse/HIVE-19454
> Project: Hive
> Issue Type: Bug
> Environment: uname -a
> Linux pts00433-vm40 3.10.0-862.el7.ppc64le #1 SMP Wed Mar 21 22:20:19 UTC
> 2018 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: HIVE-19454.1.patch
>
>
> org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
> refers to the same test case.
> Maven command used is : mvn -Dtest=TestTxnCommands2 test
> Error:
> {code:java}
> [INFO] Running org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3, Time elapsed:
> 618.215 s <<< FAILURE! - in org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR]
> testNonAcidToAcidConversion1(org.apache.hadoop.hive.ql.TestTxnCommands2)
> Time elapsed: 17.557 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1(TestTxnCommands2.java:499)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.
[jira] [Commented] (HIVE-19472) HiveStreamingConnection swallows exception on partition creation
[
https://issues.apache.org/jira/browse/HIVE-19472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470208#comment-16470208
]
Hive QA commented on HIVE-19472:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922567/HIVE-19472.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 47 failed/errored test(s), 13420 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cb
[jira] [Commented] (HIVE-19471) bucket_map_join_tez1 and bucket_map_join_tez2 are failing
[
https://issues.apache.org/jira/browse/HIVE-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470212#comment-16470212
]
Hive QA commented on HIVE-19471:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-10801/patches/PreCommit-HIVE-Build-10801.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10801/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> bucket_map_join_tez1 and bucket_map_join_tez2 are failing
> --
>
> Key: HIVE-19471
> URL: https://issues.apache.org/jira/browse/HIVE-19471
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Deepak Jaiswal
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HIVE-19471.01-branch-3.patch, HIVE-19471.1.patch
>
>
> https://builds.apache.org/job/PreCommit-HIVE-Build/10766/testReport/
> TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
> TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez2]
> Both are failing. Probably need golden file update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created
[ https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-19485: --- Attachment: HIVE-19485.0.patch > dump directory for non native tables should not be created > -- > > Key: HIVE-19485 > URL: https://issues.apache.org/jira/browse/HIVE-19485 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19485.0.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created
[ https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-19485: --- Status: Patch Available (was: Open) > dump directory for non native tables should not be created > -- > > Key: HIVE-19485 > URL: https://issues.apache.org/jira/browse/HIVE-19485 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19485.0.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19097) related equals and in operators may cause inaccurate stats estimations
[
https://issues.apache.org/jira/browse/HIVE-19097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470235#comment-16470235
]
Zoltan Haindrich commented on HIVE-19097:
-
I've re-checked the current state of this fix:
* for branch-2
** I would recommend to stay with the "localReferencedColumns" approach what
I've started in patch 01 (without that small refactor)
* for branch-3/master
** CALCITE-2247 could enable the simplification of (a=1 7& (a=2 || a=1)) to
(a=1)
** but since Calcite is *always* breaking up "IN" clauses during sql parsing;
Hive would need a facility to do that; I would like to suggest to add an IN
opener to {{HivePointLookupOptimizerRule}}; since that rule already has an IN
closer which closes large ORs.
[link to my current wip
tree|https://github.com/apache/hive/compare/master...kgyrtkirk:HIVE-19097-related-equals#diff-85205e52a019624957e9ce940ff7cbeeR145]
to add the in opener to hive
[~jcamachorodriguez], [~ashutoshc]: what do you think about the above?
> related equals and in operators may cause inaccurate stats estimations
> --
>
> Key: HIVE-19097
> URL: https://issues.apache.org/jira/browse/HIVE-19097
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19097.01.patch, HIVE-19097.partial.patch
>
>
> tpcds#74 is optimized in a way that for date_dim the condition contains IN
> and = for the same column
> {code:java}
> | Map Operator Tree: |
> | TableScan |
> | alias: date_dim |
> | filterExpr: (((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) or ((d_year) IN (2001, 2002) and (d_year =
> 2001) and d_date_sk is not null)) (type: boolean) |
> | Statistics: Num rows: 73049 Data size: 876588 Basic
> stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator |
> | predicate: ((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) (type: boolean) |
> | Statistics: Num rows: 4 Data size: 48 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
> the "real" row count will be 365
> for separate {{IN}} and {{=}} the estimation is very good; but if both are
> present it becomes (very) underestimated.
> {code:java}
> set hive.query.results.cache.enabled=false;
> drop table if exists t1;
> drop table if exists t8;
> create table t1 (a integer,b integer);
> create table t8 like t1;
> insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5);
> insert into t8
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1 union all
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1
> ;
> analyze table t1 compute statistics for columns;
> analyze table t8 compute statistics for columns;
> explain analyze select sum(a) from t8 where b in (2,3) group by b;
> explain analyze select sum(a) from t8 where b=2 group by b;
> explain analyze select sum(a) from t1 where b in (2,3) and b=2 group by b;
> explain analyze select sum(a) from t8 where b in (2,3) and b=2 group by b;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19471) bucket_map_join_tez1 and bucket_map_join_tez2 are failing
[
https://issues.apache.org/jira/browse/HIVE-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470313#comment-16470313
]
Hive QA commented on HIVE-19471:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922691/HIVE-19471.01-branch-3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 13373 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authori
[jira] [Commented] (HIVE-18652) Print Spark metrics on console
[
https://issues.apache.org/jira/browse/HIVE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470342#comment-16470342
]
Sahil Takiar commented on HIVE-18652:
-
[~vihangk1] could you take a look?
> Print Spark metrics on console
> --
>
> Key: HIVE-18652
> URL: https://issues.apache.org/jira/browse/HIVE-18652
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18652.1.patch, HIVE-18652.2.patch
>
>
> For Hive-on-MR, each MR job launched prints out some stats about the job:
> {code}
> INFO : 2018-02-07 17:51:11,218 Stage-1 map = 0%, reduce = 0%
> INFO : 2018-02-07 17:51:18,396 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.87 sec
> INFO : 2018-02-07 17:51:25,742 Stage-1 map = 100%, reduce = 100%,
> Cumulative CPU 4.34 sec
> INFO : MapReduce Total cumulative CPU time: 4 seconds 340 msec
> INFO : Ended Job = job_1517865654989_0004
> INFO : MapReduce Jobs Launched:
> INFO : Stage-Stage-1: Map: 1 Reduce: 1 Cumulative CPU: 4.34 sec HDFS
> Read: 7353 HDFS Write: 151 SUCCESS
> INFO : Total MapReduce CPU Time Spent: 4 seconds 340 msec
> {code}
> We should do the same for Hive-on-Spark.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19258) add originals support to MM tables (and make the conversion a metadata only operation)
[
https://issues.apache.org/jira/browse/HIVE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470350#comment-16470350
]
Hive QA commented on HIVE-19258:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} itests/hive-unit: The patch generated 5 new + 164
unchanged - 0 fixed = 169 total (was 164) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 13 new + 889 unchanged - 7
fixed = 902 total (was 896) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
6s{color} | {color:red} ql generated 1 new + 2321 unchanged - 0 fixed = 2322
total (was 2321) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | null could be null and is guaranteed to be dereferenced in
org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat.getSplits(JobConf, int)
Dereferenced at BucketizedHiveInputFormat.java:is guaranteed to be dereferenced
in org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat.getSplits(JobConf,
int) Dereferenced at BucketizedHiveInputFormat.java:[line 146] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10802/dev-support/hive-personality.sh
|
| git revision | master / 1cd5274 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10802/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10802/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10802/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10802/yetus/new-findbugs-ql.html
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10802/yetus/patch-asf
[jira] [Assigned] (HIVE-19486) Discrepancy between the config and the code in Hikari connectionPoolingType
[ https://issues.apache.org/jira/browse/HIVE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Sinkovits reassigned HIVE-19486: -- > Discrepancy between the config and the code in Hikari connectionPoolingType > --- > > Key: HIVE-19486 > URL: https://issues.apache.org/jira/browse/HIVE-19486 > Project: Hive > Issue Type: Bug >Reporter: Antal Sinkovits >Assignee: Antal Sinkovits >Priority: Major > > MetaStoreConf contains datanucleus.connectionPoolingType "HikariCP" not > "Hikari". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17318) Make Hikari CP configurable using hive properties in hive-site.xml
[ https://issues.apache.org/jira/browse/HIVE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470368#comment-16470368 ] Antal Sinkovits commented on HIVE-17318: Yes you are right [~pvary] Apart from the MetaStoreConf change, the [https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L4169] needs to be updated as well, to use the constant defined in HikariCPDataSourceProvider, instead of the hardcoded value. I've created a jira for it: https://issues.apache.org/jira/browse/HIVE-19486 > Make Hikari CP configurable using hive properties in hive-site.xml > -- > > Key: HIVE-17318 > URL: https://issues.apache.org/jira/browse/HIVE-17318 > Project: Hive > Issue Type: Sub-task >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-17318.01.patch, HIVE-17318.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19460) Improve stats estimations for NOT IN operator
[ https://issues.apache.org/jira/browse/HIVE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-19460: Attachment: HIVE-19460.01wip01.patch > Improve stats estimations for NOT IN operator > - > > Key: HIVE-19460 > URL: https://issues.apache.org/jira/browse/HIVE-19460 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-19460.01wip01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19460) Improve stats estimations for NOT IN operator
[ https://issues.apache.org/jira/browse/HIVE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-19460: Status: Patch Available (was: Open) > Improve stats estimations for NOT IN operator > - > > Key: HIVE-19460 > URL: https://issues.apache.org/jira/browse/HIVE-19460 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-19460.01wip01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19371) Add table ownerType to HMS thrift API
[
https://issues.apache.org/jira/browse/HIVE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470404#comment-16470404
]
Peter Vary commented on HIVE-19371:
---
Hi [~spena],
I know I am a little late, but I think it would be useful to modify the HMS API
tests to the check the default value, and the modification of the {{OwnerType}}
field as well?
{{TestTablesCreateDropAlterTruncate.testCreateTableDefaultValues}}
[https://github.com/apache/hive/blob/master/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestTablesCreateDropAlterTruncate.java#L244]
and
{{TestTablesCreateDropAlterTruncate.testAlterTable}}
[https://github.com/apache/hive/blob/master/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestTablesCreateDropAlterTruncate.java#L662]
What do you think?
Thanks,
Peter
> Add table ownerType to HMS thrift API
> -
>
> Key: HIVE-19371
> URL: https://issues.apache.org/jira/browse/HIVE-19371
> Project: Hive
> Issue Type: Sub-task
> Components: Hive, Metastore
>Affects Versions: 3.0.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19371.1.patch, HIVE-19371.2.patch
>
>
> Subtasks that adds the ownerType field to the Table object of the HMS Thrift
> API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19258) add originals support to MM tables (and make the conversion a metadata only operation)
[
https://issues.apache.org/jira/browse/HIVE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470406#comment-16470406
]
Hive QA commented on HIVE-19258:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922577/HIVE-19258.04.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 224 failed/errored test(s), 13547 tests
executed
*Failed tests:*
{noformat}
TestCustomAuthentication - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete
[jira] [Commented] (HIVE-19474) Decimal type should be casted as part of the CTAS or INSERT Clause.
[
https://issues.apache.org/jira/browse/HIVE-19474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470419#comment-16470419
]
Hive QA commented on HIVE-19474:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} druid-handler in master has 12 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} common: The patch generated 0 new + 427 unchanged -
1 fixed = 427 total (was 428) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} druid-handler: The patch generated 1 new + 180
unchanged - 8 fixed = 181 total (was 188) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10803/dev-support/hive-personality.sh
|
| git revision | master / 1cd5274 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10803/yetus/diff-checkstyle-druid-handler.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10803/yetus/patch-asflicense-problems.txt
|
| modules | C: common druid-handler U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10803/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Decimal type should be casted as part of the CTAS or INSERT Clause.
> ---
>
> Key: HIVE-19474
> URL: https://issues.apache.org/jira/browse/HIVE-19474
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Labels: druid
> Fix For: 3.0.0
>
> Attachments: HIVE-19474.patch
>
>
> HIVE-18569 introduced a runtime config variable to allow the indexing of
> Decimal as Double, this leads to kind of messy state, Hive metadata think the
> column is still decimal while it is stored
[jira] [Commented] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470481#comment-16470481
]
Matt McCline commented on HIVE-19347:
-
#10819
> TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Blocker
> Attachments: HIVE-19347.01.patch, HIVE-19347.02.patch
>
>
> Caused by the patch which turned on vectorization. Following tests are
> failing due to the patch:
> * org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData
> *
> org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
> *
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19384) Vectorization: IfExprTimestamp* do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470483#comment-16470483
]
Matt McCline commented on HIVE-19384:
-
#10824
> Vectorization: IfExprTimestamp* do not handle NULLs correctly
> -
>
> Key: HIVE-19384
> URL: https://issues.apache.org/jira/browse/HIVE-19384
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Blocker
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19384.01.patch, HIVE-19384.02.patch,
> HIVE-19384.04.patch
>
>
> HIVE-18622: "Vectorization: IF Statements, Comparisons, and more do not
> handle NULLs correctly" didn't quite fix the IfExprTimestamp* classes
> right
> {noformat}
> // Carefully handle NULLs...
> outputColVector.noNulls = false;{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19374) Parse and process ALTER TABLE SET OWNER command syntax
[ https://issues.apache.org/jira/browse/HIVE-19374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-19374: --- Attachment: HIVE-19374.3.patch > Parse and process ALTER TABLE SET OWNER command syntax > -- > > Key: HIVE-19374 > URL: https://issues.apache.org/jira/browse/HIVE-19374 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Major > Attachments: HIVE-19374.1.patch, HIVE-19374.2.patch, > HIVE-19374.3.patch > > > Subtask that parses the new alter table set owner syntax and implements code > to call HMS to change the owner of a table to a user or a role. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[ https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470533#comment-16470533 ] Vihang Karajgaonkar commented on HIVE-19108: +1 Thanks [~jerrychenhf] for fixing this. [~vgarg] [~mmccline] Can we merge this into branch-3 as well? > Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q > causes Wrong Query Results > --- > > Key: HIVE-19108 > URL: https://issues.apache.org/jira/browse/HIVE-19108 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Haifeng Chen >Priority: Critical > Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch, > HIVE-19108.03.patch, HIVE-19108.04.patch > > > Found in vectorization enable by default experiment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[ https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470571#comment-16470571 ] Vihang Karajgaonkar commented on HIVE-19108: patch merged in master branch. Waiting on [~vgarg] to get back before merging this in branch-3. This is related to result correctness and since we turned on vectorization on by default in Hive 3.0 we should merge this fix in branch-3 as well. > Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q > causes Wrong Query Results > --- > > Key: HIVE-19108 > URL: https://issues.apache.org/jira/browse/HIVE-19108 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Haifeng Chen >Priority: Critical > Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch, > HIVE-19108.03.patch, HIVE-19108.04.patch > > > Found in vectorization enable by default experiment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[
https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470575#comment-16470575
]
Vihang Karajgaonkar commented on HIVE-19108:
[~jerrychenhf] I found that there was a minor compilation issue with the latest
patch. The newly added test {{testCastDecimalToFloat}} should throw
{{HiveException}} so that it compiles. I made the change and it worked fine
after that.
> Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q
> causes Wrong Query Results
> ---
>
> Key: HIVE-19108
> URL: https://issues.apache.org/jira/browse/HIVE-19108
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Haifeng Chen
>Priority: Critical
> Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch,
> HIVE-19108.03.patch, HIVE-19108.04.patch
>
>
> Found in vectorization enable by default experiment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16042) special characters in the comment of sql file cause ParseException
[ https://issues.apache.org/jira/browse/HIVE-16042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470580#comment-16470580 ] Yi Zhou commented on HIVE-16042: Hi [~pxiong] Thank you for your response. I am confused why it have the diff in Hive 1.x vs Hive 2.x. I meant that if the same Hive SQL can successfully run on Hive 1.x , it should run on Hive 2.0 instead of reporting error to user... It seem to be a Hive issue. > special characters in the comment of sql file cause ParseException > -- > > Key: HIVE-16042 > URL: https://issues.apache.org/jira/browse/HIVE-16042 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.2.0 > Environment: Hive2.2 (commit: 2768361) > TPCx-BB v1.2 >Reporter: KaiXu >Priority: Major > Attachments: q04.sql, q17.sql, q18.sql, q23.sql > > > current Hive upstream(commit: 2768361) failed to parse some > queries(q04,q17,q18,q23) in TPCx-BB v1.2, while it's ok with Hive(commit: > ac68aed). > Q04: FAILED: ParseException line 24:0 missing EOF at ';' near > 'abandonedShoppingCartsPageCountsPerSession' > Q17: > NoViableAltException(350@[]) > at > org.apache.hadoop.hive.ql.parse.HiveParser.limitClause(HiveParser.java:38898) > at > org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:37002) > at > org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:36404) > at > org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:35722) > at > org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:35610) > at > org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2279) > at > org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1328) > at > org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204) > at > org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:75) > at > org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:68) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) > at > org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:474) > at > org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:490) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > FAILED: ParseException line 39:0 cannot recognize input near 'LIMIT' '100' > ';' in limit clause > Q18: > NoViableAltException(350@[()* loopback of 424:20: ( ( LSQUARE ^ expression > RSQUARE !) | ( DOT ^ identifier ) )*]) > at org.antlr.runtime.DFA.noViableAlt(DFA.java:158) > at org.antlr.runtime.DFA.predict(DFA.java:116) > at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceFieldExpression(HiveParser_IdentifiersParser.java:6665) > at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceUnaryPrefixExpression(HiveParser_IdentifiersParser.java:6992) > at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceUnarySuffixExpression(HiveParser_IdentifiersParser.java:7048) > at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceBitwiseXorExpression(HiveParser_IdentifiersParser.java:7210) > at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceStarExpression(HiveParser_IdentifiersParser.java:7353) > at > org.apache.hadoop.hive.ql.parse.Hiv
[jira] [Commented] (HIVE-19474) Decimal type should be casted as part of the CTAS or INSERT Clause.
[
https://issues.apache.org/jira/browse/HIVE-19474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470623#comment-16470623
]
Hive QA commented on HIVE-19474:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922588/HIVE-19474.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 43 failed/errored test(s), 13544 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.q,local_
[jira] [Updated] (HIVE-19460) Improve stats estimations for NOT IN operator
[ https://issues.apache.org/jira/browse/HIVE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-19460: Attachment: HIVE-19460.01wip02.patch > Improve stats estimations for NOT IN operator > - > > Key: HIVE-19460 > URL: https://issues.apache.org/jira/browse/HIVE-19460 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-19460.01wip01.patch, HIVE-19460.01wip02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19371) Add table ownerType to HMS thrift API
[ https://issues.apache.org/jira/browse/HIVE-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470646#comment-16470646 ] Sergio Peña commented on HIVE-19371: There is a test that I added in HIVE-19374 named TestAlterTableMetadata.java that tests this. This Jira was only to expose the API and make the review easier. > Add table ownerType to HMS thrift API > - > > Key: HIVE-19371 > URL: https://issues.apache.org/jira/browse/HIVE-19371 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore >Affects Versions: 3.0.0 >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19371.1.patch, HIVE-19371.2.patch > > > Subtasks that adds the ownerType field to the Table object of the HMS Thrift > API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19097) related equals and in operators may cause inaccurate stats estimations
[
https://issues.apache.org/jira/browse/HIVE-19097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470661#comment-16470661
]
Jesus Camacho Rodriguez commented on HIVE-19097:
[~kgyrtkirk], could you create a different issue for branch-3 linked to this
one then? And an RB with the solution that you are proposing to push to
branch-2 so we can review it and push it?
Concerning your solution for branch-3, I like the fact that it is integrated
within existing rule, but I think it may not be the most efficient solution.
For instance, if you integrate the opener within the
HivePointLookupOptimizerRule, this decomposition/composition will be done
multiple times during that stage in the planning phase, which will incur some
overhead. In addition, we already have in that stage (reduce expressions) rules
that can handle the OR simplification. Thus, I was thinking whether we should
take a similar approach as Calcite, and break the IN clause when we are
generating the Calcite tree? This is a one time process, and we already have
code that does something very similar but for IN clause with a single element:
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java#L342
Creating an OR clause over there should make the trick.
What do you think?
> related equals and in operators may cause inaccurate stats estimations
> --
>
> Key: HIVE-19097
> URL: https://issues.apache.org/jira/browse/HIVE-19097
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19097.01.patch, HIVE-19097.partial.patch
>
>
> tpcds#74 is optimized in a way that for date_dim the condition contains IN
> and = for the same column
> {code:java}
> | Map Operator Tree: |
> | TableScan |
> | alias: date_dim |
> | filterExpr: (((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) or ((d_year) IN (2001, 2002) and (d_year =
> 2001) and d_date_sk is not null)) (type: boolean) |
> | Statistics: Num rows: 73049 Data size: 876588 Basic
> stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator |
> | predicate: ((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) (type: boolean) |
> | Statistics: Num rows: 4 Data size: 48 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
> the "real" row count will be 365
> for separate {{IN}} and {{=}} the estimation is very good; but if both are
> present it becomes (very) underestimated.
> {code:java}
> set hive.query.results.cache.enabled=false;
> drop table if exists t1;
> drop table if exists t8;
> create table t1 (a integer,b integer);
> create table t8 like t1;
> insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5);
> insert into t8
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1 union all
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1
> ;
> analyze table t1 compute statistics for columns;
> analyze table t8 compute statistics for columns;
> explain analyze select sum(a) from t8 where b in (2,3) group by b;
> explain analyze select sum(a) from t8 where b=2 group by b;
> explain analyze select sum(a) from t1 where b in (2,3) and b=2 group by b;
> explain analyze select sum(a) from t8 where b in (2,3) and b=2 group by b;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19041) Thrift deserialization of Partition objects should intern fields
[
https://issues.apache.org/jira/browse/HIVE-19041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470664#comment-16470664
]
Vihang Karajgaonkar commented on HIVE-19041:
Hi [~mi...@cloudera.com] Thanks for the review. The M* classes are used to map
the database results into java objects. These are relatively short lived. I am
not sure how much of an advantage would there be to intern these fields since
they generally go out of scope very soon. Here is one example. You can see
these objects get created and destroyed with the function of one method.
{noformat}
List toPersist = new ArrayList<>();
for (Partition part : parts) {
if (!part.getTableName().equals(tblName) ||
!part.getDbName().equals(dbName)) {
throw new MetaException("Partition does not belong to target table "
+ dbName + "." + tblName + ": " + part);
}
MPartition mpart = convertToMPart(part, true);
toPersist.add(mpart);
int now = (int)(System.currentTimeMillis()/1000);
if (tabGrants != null) {
for (MTablePrivilege tab: tabGrants) {
toPersist.add(new MPartitionPrivilege(tab.getPrincipalName(),
tab.getPrincipalType(), mpart, tab.getPrivilege(), now,
tab.getGrantor(), tab.getGrantorType(), tab.getGrantOption()));
}
}
if (tabColumnGrants != null) {
for (MTableColumnPrivilege col : tabColumnGrants) {
toPersist.add(new MPartitionColumnPrivilege(col.getPrincipalName(),
col.getPrincipalType(), mpart, col.getColumnName(),
col.getPrivilege(),
now, col.getGrantor(), col.getGrantorType(),
col.getGrantOption()));
}
}
}
if (CollectionUtils.isNotEmpty(toPersist)) {
pm.makePersistentAll(toPersist);
pm.flush();
}
success = commitTransaction();
{noformat}
> Thrift deserialization of Partition objects should intern fields
>
>
> Key: HIVE-19041
> URL: https://issues.apache.org/jira/browse/HIVE-19041
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-19041.01.patch, HIVE-19041.02.patch,
> HIVE-19041.03.patch, HIVE-19041.04.patch
>
>
> When a client is creating large number of partitions, the thrift objects are
> deserialized into Partition objects. The read method of these objects does
> not intern the inputformat, location, outputformat which cause large number
> of duplicate Strings in the HMS memory. We should intern these objects while
> deserialization to reduce memory pressure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[ https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470675#comment-16470675 ] Vineet Garg commented on HIVE-19108: [~vihangk1] Sure go ahead. I suggest uploading a patch for branch-3 and getting a test run before you push this in. > Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q > causes Wrong Query Results > --- > > Key: HIVE-19108 > URL: https://issues.apache.org/jira/browse/HIVE-19108 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Haifeng Chen >Priority: Critical > Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch, > HIVE-19108.03.patch, HIVE-19108.04.patch > > > Found in vectorization enable by default experiment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19041) Thrift deserialization of Partition objects should intern fields
[
https://issues.apache.org/jira/browse/HIVE-19041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470664#comment-16470664
]
Vihang Karajgaonkar edited comment on HIVE-19041 at 5/10/18 4:31 PM:
-
Hi [~mi...@cloudera.com] Thanks for the review. The M* classes are used to map
the database results into java objects. These are relatively short lived. I am
not sure how much of an advantage would there be to intern these fields since
they generally go out of scope very soon. Here is one example. You can see
these objects get created and destroyed within the scope of one method.
{noformat}
List toPersist = new ArrayList<>();
for (Partition part : parts) {
if (!part.getTableName().equals(tblName) ||
!part.getDbName().equals(dbName)) {
throw new MetaException("Partition does not belong to target table "
+ dbName + "." + tblName + ": " + part);
}
MPartition mpart = convertToMPart(part, true);
toPersist.add(mpart);
int now = (int)(System.currentTimeMillis()/1000);
if (tabGrants != null) {
for (MTablePrivilege tab: tabGrants) {
toPersist.add(new MPartitionPrivilege(tab.getPrincipalName(),
tab.getPrincipalType(), mpart, tab.getPrivilege(), now,
tab.getGrantor(), tab.getGrantorType(), tab.getGrantOption()));
}
}
if (tabColumnGrants != null) {
for (MTableColumnPrivilege col : tabColumnGrants) {
toPersist.add(new MPartitionColumnPrivilege(col.getPrincipalName(),
col.getPrincipalType(), mpart, col.getColumnName(),
col.getPrivilege(),
now, col.getGrantor(), col.getGrantorType(),
col.getGrantOption()));
}
}
}
if (CollectionUtils.isNotEmpty(toPersist)) {
pm.makePersistentAll(toPersist);
pm.flush();
}
success = commitTransaction();
{noformat}
was (Author: vihangk1):
Hi [~mi...@cloudera.com] Thanks for the review. The M* classes are used to map
the database results into java objects. These are relatively short lived. I am
not sure how much of an advantage would there be to intern these fields since
they generally go out of scope very soon. Here is one example. You can see
these objects get created and destroyed with the function of one method.
{noformat}
List toPersist = new ArrayList<>();
for (Partition part : parts) {
if (!part.getTableName().equals(tblName) ||
!part.getDbName().equals(dbName)) {
throw new MetaException("Partition does not belong to target table "
+ dbName + "." + tblName + ": " + part);
}
MPartition mpart = convertToMPart(part, true);
toPersist.add(mpart);
int now = (int)(System.currentTimeMillis()/1000);
if (tabGrants != null) {
for (MTablePrivilege tab: tabGrants) {
toPersist.add(new MPartitionPrivilege(tab.getPrincipalName(),
tab.getPrincipalType(), mpart, tab.getPrivilege(), now,
tab.getGrantor(), tab.getGrantorType(), tab.getGrantOption()));
}
}
if (tabColumnGrants != null) {
for (MTableColumnPrivilege col : tabColumnGrants) {
toPersist.add(new MPartitionColumnPrivilege(col.getPrincipalName(),
col.getPrincipalType(), mpart, col.getColumnName(),
col.getPrivilege(),
now, col.getGrantor(), col.getGrantorType(),
col.getGrantOption()));
}
}
}
if (CollectionUtils.isNotEmpty(toPersist)) {
pm.makePersistentAll(toPersist);
pm.flush();
}
success = commitTransaction();
{noformat}
> Thrift deserialization of Partition objects should intern fields
>
>
> Key: HIVE-19041
> URL: https://issues.apache.org/jira/browse/HIVE-19041
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-19041.01.patch, HIVE-19041.02.patch,
> HIVE-19041.03.patch, HIVE-19041.04.patch
>
>
> When a client is creating large number of partitions, the thrift objects are
> deserialized into Partition objects. The read method of these objects does
> not intern the inputformat, location, outputformat which cause large number
> of duplicate Strings in the HMS memory. We should intern these objects while
> deserialization to reduce memory pressure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19438) Test failure: org.apache.hadoop.hive.llap.daemon.services.impl.TestLlapWebServices.testContextRootUrlRewrite
[
https://issues.apache.org/jira/browse/HIVE-19438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470679#comment-16470679
]
Hive QA commented on HIVE-19438:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10804/dev-support/hive-personality.sh
|
| git revision | master / 9124ee1 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10804/yetus/patch-asflicense-problems.txt
|
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10804/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Test failure:
> org.apache.hadoop.hive.llap.daemon.services.impl.TestLlapWebServices.testContextRootUrlRewrite
>
>
> Key: HIVE-19438
> URL: https://issues.apache.org/jira/browse/HIVE-19438
> Project: Hive
> Issue Type: Bug
> Components: llap, Test, Tests
>Affects Versions: 3.1.0
>Reporter: Pravin Dsilva
>Priority: Major
> Attachments: HIVE-19438.1.patch
>
>
> *Error Message*
> {code:java}
> expected:<200> but was:<500>
> {code}
> *Stacktrace*
> {code:java}
> java.lang.AssertionError: expected:<200> but was:<500> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:743) at
> org.junit.Assert.assertEquals(Assert.java:118) at
> org.junit.Assert.assertEquals(Assert.java:555) at
> org.junit.Assert.assertEquals(Assert.java:542) at
> org.apache.hadoop.hive.llap.daemon.services.impl.TestLlapWebServices.getURLResponseAsString(TestLlapWebServices.java:59)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.
[ https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera reassigned HIVE-19488: -- > enable CM root based on db parameter, identifying a db as source of > replication. > > > Key: HIVE-19488 > URL: https://issues.apache.org/jira/browse/HIVE-19488 > Project: Hive > Issue Type: Task > Components: Hive, HiveServer2, repl >Affects Versions: 3.1.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.1.0 > > > * add a parameter at db level to identify if its a source of replication. > beacon will set this. > * Enable CM root only for databases that are a source of a replication > policy, for other db's skip the CM root functionality. > * prevent database drop if the parameter indicating its source of a > replication, is set. > * as an upgrade to this version, beacon should set the property on all > existing database policies, in affect. > * the parameter should be of the form . – repl.source.for : List < policy > ids > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19489) Disable stats autogather for external tables
[ https://issues.apache.org/jira/browse/HIVE-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-19489: - > Disable stats autogather for external tables > > > Key: HIVE-19489 > URL: https://issues.apache.org/jira/browse/HIVE-19489 > Project: Hive > Issue Type: Sub-task > Components: Statistics >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > > Hive auto-gather of table statistics can result in incorrect generation of > stats (and the stats being marked as accurate) in the case of external tables > where the data is being written by external apps. > To avoid this issue, stats autogather will be disabled on external tables > when loading/inserting into a table with existing data, if > HIVE_DISABLE_UNSAFE_EXTERNALTABLE_OPERATIONS is enabled. In this situation, > users should rely on explicitly calling ANALYZE TABLE on their external > tables to make sure the stats are kept up-to-date. > Autogather of stats will still be allowed to occur on external tables in the > case of INSERT OVERWRITE or LOAD DATA OVERWRITE, since the existing data is > being removed and so the stats calculated on the inserted/loaded data should > be accurate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19178) TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
[
https://issues.apache.org/jira/browse/HIVE-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470736#comment-16470736
]
Vineet Garg commented on HIVE-19178:
+1
> TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
>
>
> Key: HIVE-19178
> URL: https://issues.apache.org/jira/browse/HIVE-19178
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HIVE-19178.patch
>
>
> I have verified that this failure is due to HIVE-18825.
> Error stack:
> {code}
> java.lang.IllegalStateException: calling recordValidTxn() more than once in
> the same txnid:5
> at org.apache.hadoop.hive.ql.Driver.acquireLocks(Driver.java:1439)
> at org.apache.hadoop.hive.ql.Driver.lockAndRespond(Driver.java:1624)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1794)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1538)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1527)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:137)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:287)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:635)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1655)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1602)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1597)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:200)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1455)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1429)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:177)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> at
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver(TestMiniTezCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:73)
> at org.juni
[jira] [Commented] (HIVE-19270) TestAcidOnTez tests are failing
[
https://issues.apache.org/jira/browse/HIVE-19270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470743#comment-16470743
]
Vineet Garg commented on HIVE-19270:
[~ekoifman] Can you confirm if Is this test only issue? I am trying to figure
out if this is blocker for 3.0.
> TestAcidOnTez tests are failing
> ---
>
> Key: HIVE-19270
> URL: https://issues.apache.org/jira/browse/HIVE-19270
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Vineet Garg
>Assignee: Eugene Koifman
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HIVE-19270.01.patch
>
>
> Following tests are failing:
> * testCtasTezUnion
> * testNonStandardConversion01
> * testAcidInsertWithRemoveUnion
> All of them have the similar failure:
> {noformat}
> Actual line 0 ac: {"writeid":1,"bucketid":536870913,"rowid":1} 1 2
> file:/home/hiveptest/35.193.47.6-hiveptest-1/apache-github-source-source/itests/hive-unit/target/tmp/org.apache.hadoop.hive.ql.TestAcidOnTez-1524409020904/warehouse/t/delta_001_001_0001/bucket_0
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.
[ https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-19488: --- Attachment: HIVE-19488.01.patch > enable CM root based on db parameter, identifying a db as source of > replication. > > > Key: HIVE-19488 > URL: https://issues.apache.org/jira/browse/HIVE-19488 > Project: Hive > Issue Type: Task > Components: Hive, HiveServer2, repl >Affects Versions: 3.1.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19488.01.patch > > > * add a parameter at db level to identify if its a source of replication. > beacon will set this. > * Enable CM root only for databases that are a source of a replication > policy, for other db's skip the CM root functionality. > * prevent database drop if the parameter indicating its source of a > replication, is set. > * as an upgrade to this version, beacon should set the property on all > existing database policies, in affect. > * the parameter should be of the form . – repl.source.for : List < policy > ids > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.
[ https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-19488: --- Status: Patch Available (was: Open) > enable CM root based on db parameter, identifying a db as source of > replication. > > > Key: HIVE-19488 > URL: https://issues.apache.org/jira/browse/HIVE-19488 > Project: Hive > Issue Type: Task > Components: Hive, HiveServer2, repl >Affects Versions: 3.1.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19488.01.patch > > > * add a parameter at db level to identify if its a source of replication. > beacon will set this. > * Enable CM root only for databases that are a source of a replication > policy, for other db's skip the CM root functionality. > * prevent database drop if the parameter indicating its source of a > replication, is set. > * as an upgrade to this version, beacon should set the property on all > existing database policies, in affect. > * the parameter should be of the form . – repl.source.for : List < policy > ids > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.
[ https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470758#comment-16470758 ] ASF GitHub Bot commented on HIVE-19488: --- GitHub user maheshk114 opened a pull request: https://github.com/apache/hive/pull/345 HIVE-19488 : enable CM root based on db parameter, identifying a db as source of replication … You can merge this pull request into a Git repository by running: $ git pull https://github.com/maheshk114/hive BUG-102289 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/345.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #345 commit 2e86b6cd8a894355771029966ba6f59ee5f447ec Author: Mahesh Kumar Behera Date: 2018-05-10T17:03:58Z HIVE-19488 : enable CM root based on db parameter, identifying a db as source of replication > enable CM root based on db parameter, identifying a db as source of > replication. > > > Key: HIVE-19488 > URL: https://issues.apache.org/jira/browse/HIVE-19488 > Project: Hive > Issue Type: Task > Components: Hive, HiveServer2, repl >Affects Versions: 3.1.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0 > > Attachments: HIVE-19488.01.patch > > > * add a parameter at db level to identify if its a source of replication. > beacon will set this. > * Enable CM root only for databases that are a source of a replication > policy, for other db's skip the CM root functionality. > * prevent database drop if the parameter indicating its source of a > replication, is set. > * as an upgrade to this version, beacon should set the property on all > existing database policies, in affect. > * the parameter should be of the form . – repl.source.for : List < policy > ids > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.
[ https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-19488: -- Labels: pull-request-available (was: ) > enable CM root based on db parameter, identifying a db as source of > replication. > > > Key: HIVE-19488 > URL: https://issues.apache.org/jira/browse/HIVE-19488 > Project: Hive > Issue Type: Task > Components: Hive, HiveServer2, repl >Affects Versions: 3.1.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0 > > Attachments: HIVE-19488.01.patch > > > * add a parameter at db level to identify if its a source of replication. > beacon will set this. > * Enable CM root only for databases that are a source of a replication > policy, for other db's skip the CM root functionality. > * prevent database drop if the parameter indicating its source of a > replication, is set. > * as an upgrade to this version, beacon should set the property on all > existing database policies, in affect. > * the parameter should be of the form . – repl.source.for : List < policy > ids > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19490) Locking on Insert into for non native and managed tables.
[ https://issues.apache.org/jira/browse/HIVE-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-19490: - Assignee: slim bouguerra > Locking on Insert into for non native and managed tables. > - > > Key: HIVE-19490 > URL: https://issues.apache.org/jira/browse/HIVE-19490 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > > Current state of the art: > Managed non native table like Druid Tables, will need to get a Lock on Insert > into or insert Over write. The nature of this lock is set to Exclusive by > default for any non native table. > This implies that Inserts into Druid table will Lock any read query as well > during the execution of the insert into. IMO this lock (on insert into) is > not needed since the insert statement is appending data and the state of > loading it is managed partially by Hive Storage handler hook and part of it > by Druid. > What i am proposing is to relax the lock level to shared for all non native > tables on insert into operations and keep it as Exclusive Write for insert > Overwrite for now. > > Any feedback is welcome. > cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn] > Also am not sure what is the best way to unit test this currently am using > debugger to check of locks are what i except, please let me know if there is > a better way to do this. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19490) Locking on Insert into for non native and managed tables.
[ https://issues.apache.org/jira/browse/HIVE-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-19490: -- Status: Patch Available (was: Open) > Locking on Insert into for non native and managed tables. > - > > Key: HIVE-19490 > URL: https://issues.apache.org/jira/browse/HIVE-19490 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > > Current state of the art: > Managed non native table like Druid Tables, will need to get a Lock on Insert > into or insert Over write. The nature of this lock is set to Exclusive by > default for any non native table. > This implies that Inserts into Druid table will Lock any read query as well > during the execution of the insert into. IMO this lock (on insert into) is > not needed since the insert statement is appending data and the state of > loading it is managed partially by Hive Storage handler hook and part of it > by Druid. > What i am proposing is to relax the lock level to shared for all non native > tables on insert into operations and keep it as Exclusive Write for insert > Overwrite for now. > > Any feedback is welcome. > cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn] > Also am not sure what is the best way to unit test this currently am using > debugger to check of locks are what i except, please let me know if there is > a better way to do this. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19483) Metastore cleaner tasks that run periodically are created more than once
[ https://issues.apache.org/jira/browse/HIVE-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470772#comment-16470772 ] Alan Gates commented on HIVE-19483: --- +1 > Metastore cleaner tasks that run periodically are created more than once > > > Key: HIVE-19483 > URL: https://issues.apache.org/jira/browse/HIVE-19483 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19483.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19490) Locking on Insert into for non native and managed tables.
[ https://issues.apache.org/jira/browse/HIVE-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-19490: -- Labels: druid locking (was: ) > Locking on Insert into for non native and managed tables. > - > > Key: HIVE-19490 > URL: https://issues.apache.org/jira/browse/HIVE-19490 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: druid, locking > Attachments: HIVE-19490.patch > > > Current state of the art: > Managed non native table like Druid Tables, will need to get a Lock on Insert > into or insert Over write. The nature of this lock is set to Exclusive by > default for any non native table. > This implies that Inserts into Druid table will Lock any read query as well > during the execution of the insert into. IMO this lock (on insert into) is > not needed since the insert statement is appending data and the state of > loading it is managed partially by Hive Storage handler hook and part of it > by Druid. > What i am proposing is to relax the lock level to shared for all non native > tables on insert into operations and keep it as Exclusive Write for insert > Overwrite for now. > > Any feedback is welcome. > cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn] > Also am not sure what is the best way to unit test this currently am using > debugger to check of locks are what i except, please let me know if there is > a better way to do this. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19490) Locking on Insert into for non native and managed tables.
[ https://issues.apache.org/jira/browse/HIVE-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-19490: -- Attachment: HIVE-19490.patch > Locking on Insert into for non native and managed tables. > - > > Key: HIVE-19490 > URL: https://issues.apache.org/jira/browse/HIVE-19490 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: druid, locking > Attachments: HIVE-19490.patch > > > Current state of the art: > Managed non native table like Druid Tables, will need to get a Lock on Insert > into or insert Over write. The nature of this lock is set to Exclusive by > default for any non native table. > This implies that Inserts into Druid table will Lock any read query as well > during the execution of the insert into. IMO this lock (on insert into) is > not needed since the insert statement is appending data and the state of > loading it is managed partially by Hive Storage handler hook and part of it > by Druid. > What i am proposing is to relax the lock level to shared for all non native > tables on insert into operations and keep it as Exclusive Write for insert > Overwrite for now. > > Any feedback is welcome. > cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn] > Also am not sure what is the best way to unit test this currently am using > debugger to check of locks are what i except, please let me know if there is > a better way to do this. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19475) Issue when streaming data to Azure Data Lake Store
[
https://issues.apache.org/jira/browse/HIVE-19475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470804#comment-16470804
]
Thomas Nys commented on HIVE-19475:
---
[~prasanth_j] can you confirm these are the correct ones?
> Issue when streaming data to Azure Data Lake Store
> --
>
> Key: HIVE-19475
> URL: https://issues.apache.org/jira/browse/HIVE-19475
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 2.2.0
> Environment: HDInsight 3.6 on Ubuntu 16.04.4 LTS (GNU/Linux
> 4.13.0-1012-azure x86_64)
> Used java libraries:
> {code:java}
> libraryDependencies += "org.apache.hive.hcatalog" % "hive-hcatalog-streaming"
> % "2.2.0"
> libraryDependencies += "org.apache.hive.hcatalog" % "hive-hcatalog-core" %
> "2.2.0"
> libraryDependencies += "org.apache.hadoop" % "hadoop-client" % "2.8.0"
> {code}
> Please let me know if more details are needed.
>Reporter: Thomas Nys
>Priority: Major
>
> I am trying to stream data from a Java (Play2 api) to HDInsight Hive
> interactive query with Azure Data Lake Store as storage back-end. The
> following code is ran on one of the head nodes of the cluster.
> When fetching a transaction-batch:
> {code:java}
> TransactionBatch txnBatch = this.connection.fetchTransactionBatch(10,
> (RecordWriter)writer);
> {code}
> I receive the following error:
> {code:java}
> play.api.UnexpectedException: Unexpected exception[StreamingIOFailure: Failed
> creating RecordUpdaterS for
> adl://home/hive/warehouse/raw_telemetry_data/ingest_date=2018-05-07
> txnIds[506,515]]
> at
> play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:251)
> at
> play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:182)
> at
> play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:343)
> at
> play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:341)
> at scala.concurrent.Future.$anonfun$recoverWith$1(Future.scala:414)
> at
> scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
> at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
> at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> at
> akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
> Caused by: org.apache.hive.hcatalog.streaming.StreamingIOFailure: Failed
> creating RecordUpdaterS for
> adl://home/hive/warehouse/raw_telemetry_data/ingest_date=2018-05-07
> txnIds[506,515]
> at
> org.apache.hive.hcatalog.streaming.AbstractRecordWriter.newBatch(AbstractRecordWriter.java:208)
> at
> org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.(HiveEndPoint.java:608)
> at
> org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.(HiveEndPoint.java:556)
> at
> org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.fetchTransactionBatchImpl(HiveEndPoint.java:442)
> at
> org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.fetchTransactionBatch(HiveEndPoint.java:422)
> at hive.HiveRepository.createMany(HiveRepository.java:76)
> at controllers.HiveController.create(HiveController.java:40)
> at router.Routes$$anonfun$routes$1.$anonfun$applyOrElse$2(Routes.scala:70)
> at
> play.core.routing.HandlerInvokerFactory$$anon$4.resultCall(HandlerInvoker.scala:137)
> at
> play.core.routing.HandlerInvokerFactory$JavaActionInvokerFactory$$anon$8$$anon$2$$anon$1.invocation(HandlerInvoker.scala:108)
> Caused by: java.io.IOException: No FileSystem for scheme: adl
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2798)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2809)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2848)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2830)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.(OrcRecordUpdater.java:187)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat.getRecordUpdater(OrcOutputFormat.java:278)
> at
> org.apache.hive.hcatalog.streaming.AbstractRecordWriter.createRecordUpdater(AbstractRecordWriter.java:268){code}
>
> Any help would be greatly appreciated.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19424) NPE In MetaDataFormatters
[
https://issues.apache.org/jira/browse/HIVE-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alice Fan reassigned HIVE-19424:
Assignee: Alice Fan
> NPE In MetaDataFormatters
> -
>
> Key: HIVE-19424
> URL: https://issues.apache.org/jira/browse/HIVE-19424
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Metastore, Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Alice Fan
>Priority: Minor
>
> h2. Overview
> According to the Hive Schema definition, a table's {{INPUT_FORMAT}} class can
> be set to NULL. However, there are places in the code where we do not
> account for this NULL value, in particular the {{MetaDataFormatters}} classes
> {{TextMetaDataFormatter}} and {{JsonMetaDataFormatter}}. In addition, there
> is no debug level logging in the {{MetaDataFormatters}} classes to tell me
> which table in particular is causing the problem.
> {code:sql|title=hive-schema-2.2.0.mysql.sql}
> CREATE TABLE IF NOT EXISTS `SDS` (
> `SD_ID` bigint(20) NOT NULL,
> `CD_ID` bigint(20) DEFAULT NULL,
> `INPUT_FORMAT` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin
> DEFAULT NULL,
> `IS_COMPRESSED` bit(1) NOT NULL,
> ...
> {code}
> {code:java|title=TextMetaDataFormatter.java}
> // Not checking for a null return from getInputFormatClass
> inputFormattCls = par.getInputFormatClass().getName();
> outputFormattCls = par.getOutputFormatClass().getName();
> {code}
> h2. Reproduction
> {code:sql}
> -- MySQL Backend
> update SDS SET INPUT_FORMAT=NULL WHERE SD_ID=XXX;
> {code}
> {code}
> // Hive
> SHOW TABLE EXTENDED FROM default LIKE '*';
> // HS2 Logs
> [HiveServer2-Background-Pool: Thread-464]: Error running hive query:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing show table
> status
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:400)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:238)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89)
> at
> org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at
> org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Exception while
> processing show table status
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.showTableStatus(DDLTask.java:3025)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:405)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2052)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1748)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1501)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1285)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1280)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236)
> ... 11 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.metadata.formatting.TextMetaDataFormatter.showTableStatus(TextMetaDataFormatter.java:202)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.showTableStatus(DDLTask.java:3020)
> ... 20 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19489) Disable stats autogather for external tables
[ https://issues.apache.org/jira/browse/HIVE-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470827#comment-16470827 ] Jason Dere commented on HIVE-19489: --- It looks like ALTER TABLE (which occurs after every load/insert) is responsible for computing numFiles/totalSize, and does so by checking the underlying files on the FS. These stats look like they should be accurate, so it looks like we can keep this portion of the stats. So autogather of row/column stats (generated by the autogather stats tasks) will be disabled for external tables. > Disable stats autogather for external tables > > > Key: HIVE-19489 > URL: https://issues.apache.org/jira/browse/HIVE-19489 > Project: Hive > Issue Type: Sub-task > Components: Statistics >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > > Hive auto-gather of table statistics can result in incorrect generation of > stats (and the stats being marked as accurate) in the case of external tables > where the data is being written by external apps. > To avoid this issue, stats autogather will be disabled on external tables > when loading/inserting into a table with existing data, if > HIVE_DISABLE_UNSAFE_EXTERNALTABLE_OPERATIONS is enabled. In this situation, > users should rely on explicitly calling ANALYZE TABLE on their external > tables to make sure the stats are kept up-to-date. > Autogather of stats will still be allowed to occur on external tables in the > case of INSERT OVERWRITE or LOAD DATA OVERWRITE, since the existing data is > being removed and so the stats calculated on the inserted/loaded data should > be accurate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19438) Test failure: org.apache.hadoop.hive.llap.daemon.services.impl.TestLlapWebServices.testContextRootUrlRewrite
[
https://issues.apache.org/jira/browse/HIVE-19438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470837#comment-16470837
]
Hive QA commented on HIVE-19438:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922606/HIVE-19438.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 43 failed/errored test(s), 13546 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.q,loca
[jira] [Commented] (HIVE-19468) Add Apache license to TestTxnConcatenate
[ https://issues.apache.org/jira/browse/HIVE-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470836#comment-16470836 ] Igor Kryvenko commented on HIVE-19468: -- [~kgyrtkirk] [~vgarg] Hi. Could you review, please? > Add Apache license to TestTxnConcatenate > > > Key: HIVE-19468 > URL: https://issues.apache.org/jira/browse/HIVE-19468 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Igor Kryvenko >Assignee: Igor Kryvenko >Priority: Major > Attachments: HIVE-19468.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19287) parse error with semicolon in comment which starts with whitespace in file
[
https://issues.apache.org/jira/browse/HIVE-19287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470844#comment-16470844
]
Hive QA commented on HIVE-19287:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922614/HIVE-19287.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10805/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10805/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10805/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-05-10 17:47:43.128
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10805/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-05-10 17:47:43.132
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 9124ee1 HIVE-19108 : Vectorization and Parquet: Turning on
vectorization in parquet_ppd_decimal.q causes Wrong Query Results (Haifeng Chen
reviewed by Matt McCline and Vihang Karajgaonkar)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 9124ee1 HIVE-19108 : Vectorization and Parquet: Turning on
vectorization in parquet_ppd_decimal.q causes Wrong Query Results (Haifeng Chen
reviewed by Matt McCline and Vihang Karajgaonkar)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-05-10 17:47:44.445
+ rm -rf ../yetus_PreCommit-HIVE-Build-10805
+ mkdir ../yetus_PreCommit-HIVE-Build-10805
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10805
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10805/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java: does not exist
in index
error:
a/common/src/java/org/apache/hadoop/hive/common/cli/HiveFileProcessor.java:
does not exist in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc462490391072919430.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc462490391072919430.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(process-resource-bundles) on project hive-shims: Execution
process-resource-bundles of goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process failed.
ConcurrentModificationException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enabl
[jira] [Commented] (HIVE-18862) qfiles: prepare .q files for using datasets
[
https://issues.apache.org/jira/browse/HIVE-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470853#comment-16470853
]
Hive QA commented on HIVE-18862:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-10806/patches/PreCommit-HIVE-Build-10806.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10806/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> qfiles: prepare .q files for using datasets
> ---
>
> Key: HIVE-18862
> URL: https://issues.apache.org/jira/browse/HIVE-18862
> Project: Hive
> Issue Type: Sub-task
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-18862.01.patch, HIVE-18862.02.patch,
> HIVE-18862.03.patch, HIVE-18862.04.patch, HIVE-18862.05.patch,
> HIVE-18862.06.patch, HIVE-18862.07.patch, HIVE-18862.08.patch,
> HIVE-18862.09-branch-3.patch, HIVE-18862.09.patch,
> HIVE-18862.10-branch-3.patch
>
>
> # Parse .q files for source table usage
> # Add needed dataset annotations
> # Remove create table statements from "q_test_init.sql" like files
> # Handle oncoming issues related to dataset introduction
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[ https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-19108: -- Assignee: Haifeng Chen (was: Vihang Karajgaonkar) > Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q > causes Wrong Query Results > --- > > Key: HIVE-19108 > URL: https://issues.apache.org/jira/browse/HIVE-19108 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Haifeng Chen >Priority: Critical > Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch, > HIVE-19108.03.patch, HIVE-19108.04.patch, HIVE-19108.05-branch-3.patch > > > Found in vectorization enable by default experiment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[ https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-19108: -- Assignee: Vihang Karajgaonkar (was: Haifeng Chen) > Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q > causes Wrong Query Results > --- > > Key: HIVE-19108 > URL: https://issues.apache.org/jira/browse/HIVE-19108 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch, > HIVE-19108.03.patch, HIVE-19108.04.patch, HIVE-19108.05-branch-3.patch > > > Found in vectorization enable by default experiment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[ https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-19108: --- Attachment: HIVE-19108.05-branch-3.patch > Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q > causes Wrong Query Results > --- > > Key: HIVE-19108 > URL: https://issues.apache.org/jira/browse/HIVE-19108 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch, > HIVE-19108.03.patch, HIVE-19108.04.patch, HIVE-19108.05-branch-3.patch > > > Found in vectorization enable by default experiment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[ https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470898#comment-16470898 ] Vihang Karajgaonkar commented on HIVE-19108: attached branch-3 patch. Had to reassign the Jira to myself inorder to attach the file. Reassigned it back to [~jerrychenhf] > Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q > causes Wrong Query Results > --- > > Key: HIVE-19108 > URL: https://issues.apache.org/jira/browse/HIVE-19108 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Haifeng Chen >Priority: Critical > Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch, > HIVE-19108.03.patch, HIVE-19108.04.patch, HIVE-19108.05-branch-3.patch > > > Found in vectorization enable by default experiment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError
[
https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470908#comment-16470908
]
Deepak Jaiswal commented on HIVE-19454:
---
If an order is not guaranteed then maybe we should always sort them before
asserting?
> Test failure :
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails
> with java.lang.AssertionError
> --
>
> Key: HIVE-19454
> URL: https://issues.apache.org/jira/browse/HIVE-19454
> Project: Hive
> Issue Type: Bug
> Environment: uname -a
> Linux pts00433-vm40 3.10.0-862.el7.ppc64le #1 SMP Wed Mar 21 22:20:19 UTC
> 2018 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: HIVE-19454.1.patch
>
>
> org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
> refers to the same test case.
> Maven command used is : mvn -Dtest=TestTxnCommands2 test
> Error:
> {code:java}
> [INFO] Running org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3, Time elapsed:
> 618.215 s <<< FAILURE! - in org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR]
> testNonAcidToAcidConversion1(org.apache.hadoop.hive.ql.TestTxnCommands2)
> Time elapsed: 17.557 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1(TestTxnCommands2.java:499)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestTxnCommands2.testNonAcidToAcidConversion1:499
> [INFO]
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19471) bucket_map_join_tez1 and bucket_map_join_tez2 are failing
[ https://issues.apache.org/jira/browse/HIVE-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-19471: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to master and branch-3 thanks for the reviews. > bucket_map_join_tez1 and bucket_map_join_tez2 are failing > -- > > Key: HIVE-19471 > URL: https://issues.apache.org/jira/browse/HIVE-19471 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Deepak Jaiswal >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HIVE-19471.01-branch-3.patch, HIVE-19471.1.patch > > > https://builds.apache.org/job/PreCommit-HIVE-Build/10766/testReport/ > TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] > TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez2] > Both are failing. Probably need golden file update. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19491) Branch-3 Start using storage-api 2.6.1 once available.
[ https://issues.apache.org/jira/browse/HIVE-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal reassigned HIVE-19491: - > Branch-3 Start using storage-api 2.6.1 once available. > -- > > Key: HIVE-19491 > URL: https://issues.apache.org/jira/browse/HIVE-19491 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Blocker > > branch-3 needs storage-api 2.6.1 which is in the process of being released. > > cc. [~vgarg] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19492) Update Wiki with load data extension syntax
[ https://issues.apache.org/jira/browse/HIVE-19492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal reassigned HIVE-19492: - > Update Wiki with load data extension syntax > --- > > Key: HIVE-19492 > URL: https://issues.apache.org/jira/browse/HIVE-19492 > Project: Hive > Issue Type: Sub-task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19474) Decimal type should be casted as part of the CTAS or INSERT Clause.
[
https://issues.apache.org/jira/browse/HIVE-19474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470940#comment-16470940
]
Jesus Camacho Rodriguez commented on HIVE-19474:
[~bslim], could we change the message in {{DruidStorageHandlerUtils}} to "Cast
to any numeric type supported by Druid: x, y, z, t"? Thanks
> Decimal type should be casted as part of the CTAS or INSERT Clause.
> ---
>
> Key: HIVE-19474
> URL: https://issues.apache.org/jira/browse/HIVE-19474
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Labels: druid
> Fix For: 3.0.0
>
> Attachments: HIVE-19474.patch
>
>
> HIVE-18569 introduced a runtime config variable to allow the indexing of
> Decimal as Double, this leads to kind of messy state, Hive metadata think the
> column is still decimal while it is stored as double. Since the Hive metadata
> of the column is Decimal the logical optimizer will not push down aggregates.
> i tried to fix this by adding some logic to the application but it makes the
> code very clumsy with lot of branches. Instead i propose to revert
> HIVE-18569 and let the user introduce an explicit cast this will be better
> since the metada reflects actual storage type and push down aggregates will
> kick in and there is no config needed without adding any code or bug.
> cc [~ashutoshc] and [~nishantbangarwa]
> You can see the difference with the following DDL
> {code:java}
> create table test_base_table(`timecolumn` timestamp, `interval_marker`
> string, `num_l` DECIMAL(10,2));
> insert into test_base_table values ('2015-03-08 00:00:00', 'i1-start', 4.5);
> set hive.druid.approx.result=true;
> CREATE TABLE druid_test_table
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.segment.granularity" = "DAY")
> AS
> select cast(`timecolumn` as timestamp with local time zone) as `__time`,
> `interval_marker`, cast(`num_l` as double)
> FROM test_base_table;
> describe druid_test_table;
> explain select sum(num_l), min(num_l) FROM druid_test_table;
> CREATE TABLE druid_test_table_2
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.segment.granularity" = "DAY")
> AS
> select cast(`timecolumn` as timestamp with local time zone) as `__time`,
> `interval_marker`, `num_l`
> FROM test_base_table;
> describe druid_test_table_2;
> explain select sum(num_l), min(num_l) FROM druid_test_table_2;
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18652) Print Spark metrics on console
[
https://issues.apache.org/jira/browse/HIVE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470943#comment-16470943
]
Vihang Karajgaonkar commented on HIVE-18652:
[~stakiar] thanks for the patch. Overall LGTM. I left some minor suggestions.
Rest looks good. +1
> Print Spark metrics on console
> --
>
> Key: HIVE-18652
> URL: https://issues.apache.org/jira/browse/HIVE-18652
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18652.1.patch, HIVE-18652.2.patch
>
>
> For Hive-on-MR, each MR job launched prints out some stats about the job:
> {code}
> INFO : 2018-02-07 17:51:11,218 Stage-1 map = 0%, reduce = 0%
> INFO : 2018-02-07 17:51:18,396 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.87 sec
> INFO : 2018-02-07 17:51:25,742 Stage-1 map = 100%, reduce = 100%,
> Cumulative CPU 4.34 sec
> INFO : MapReduce Total cumulative CPU time: 4 seconds 340 msec
> INFO : Ended Job = job_1517865654989_0004
> INFO : MapReduce Jobs Launched:
> INFO : Stage-Stage-1: Map: 1 Reduce: 1 Cumulative CPU: 4.34 sec HDFS
> Read: 7353 HDFS Write: 151 SUCCESS
> INFO : Total MapReduce CPU Time Spent: 4 seconds 340 msec
> {code}
> We should do the same for Hive-on-Spark.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19395) OutStream.write hangs with misconfigured bufferSize
[
https://issues.apache.org/jira/browse/HIVE-19395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Doe updated HIVE-19395:
Description:
When the bufferSize is configured to be 0 in the class initialization, the
while loop in OutStream.write function hangs endlessly.
This is because when the bufferSize is 0, current.remaining is 0, length will
always > 0.
Here is the code snippet.
{code:java}
OutStream(String name, int bufferSize, CompressionCodec codec,
OutputReceiver receiver) throws IOException {
...
this.bufferSize = bufferSize; //bufferSize can be configured with 0
...
}
private void getNewInputBuffer() throws IOException {
...
current = ByteBuffer.allocate(bufferSize);
...
}
public void write(byte[] bytes, int offset, int length) throws IOException {
if (current == null) {
getNewInputBuffer();
}
int remaining = Math.min(current.remaining(), length);
current.put(bytes, offset, remaining);
uncompressedBytes += remaining;
length -= remaining;
while (length != 0) {//length > 0
spill();
offset += remaining;
remaining = Math.min(current.remaining(), length);//current.remaining()
== 0
current.put(bytes, offset, remaining);
uncompressedBytes += remaining;
length -= remaining;
}
}
{code}
The similar case is HDFS-13513, HDFS-13514
was:
When the bufferSize is configured to be 0 in the class initialization, the
while loop in OutStream.write function hangs endlessly.
This is because when the bufferSize is 0, current.remaining is 0, length will
always > 0.
Here is the code snippet.
{code:java}
OutStream(String name, int bufferSize, CompressionCodec codec,
OutputReceiver receiver) throws IOException {
...
this.bufferSize = bufferSize; //bufferSize can be configured with 0
...
}
private void getNewInputBuffer() throws IOException {
...
current = ByteBuffer.allocate(bufferSize);
...
}
public void write(byte[] bytes, int offset, int length) throws IOException {
if (current == null) {
getNewInputBuffer();
}
int remaining = Math.min(current.remaining(), length);
current.put(bytes, offset, remaining);
uncompressedBytes += remaining;
length -= remaining;
while (length != 0) {//length > 0
spill();
offset += remaining;
remaining = Math.min(current.remaining(), length);//current.remaining()
== 0
current.put(bytes, offset, remaining);
uncompressedBytes += remaining;
length -= remaining;
}
}
{code}
The similar case is
[HDFS-13513|https://issues.apache.org/jira/browse/HDFS-13513],
[HDFS-13514|https://issues.apache.org/jira/browse/HDFS-13514]
> OutStream.write hangs with misconfigured bufferSize
> ---
>
> Key: HIVE-19395
> URL: https://issues.apache.org/jira/browse/HIVE-19395
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.0.0
>Reporter: John Doe
>Priority: Minor
>
> When the bufferSize is configured to be 0 in the class initialization, the
> while loop in OutStream.write function hangs endlessly.
> This is because when the bufferSize is 0, current.remaining is 0, length will
> always > 0.
> Here is the code snippet.
> {code:java}
>OutStream(String name, int bufferSize, CompressionCodec codec,
> OutputReceiver receiver) throws IOException {
> ...
> this.bufferSize = bufferSize; //bufferSize can be configured with 0
> ...
> }
> private void getNewInputBuffer() throws IOException {
> ...
> current = ByteBuffer.allocate(bufferSize);
> ...
> }
> public void write(byte[] bytes, int offset, int length) throws IOException {
> if (current == null) {
> getNewInputBuffer();
> }
> int remaining = Math.min(current.remaining(), length);
> current.put(bytes, offset, remaining);
> uncompressedBytes += remaining;
> length -= remaining;
> while (length != 0) {//length > 0
> spill();
> offset += remaining;
> remaining = Math.min(current.remaining(), length);//current.remaining()
> == 0
> current.put(bytes, offset, remaining);
> uncompressedBytes += remaining;
> length -= remaining;
> }
> }
> {code}
> The similar case is HDFS-13513, HDFS-13514
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19178) TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
[
https://issues.apache.org/jira/browse/HIVE-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-19178:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master, branch-3. Thanks [~vgarg]
> TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
>
>
> Key: HIVE-19178
> URL: https://issues.apache.org/jira/browse/HIVE-19178
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HIVE-19178.patch
>
>
> I have verified that this failure is due to HIVE-18825.
> Error stack:
> {code}
> java.lang.IllegalStateException: calling recordValidTxn() more than once in
> the same txnid:5
> at org.apache.hadoop.hive.ql.Driver.acquireLocks(Driver.java:1439)
> at org.apache.hadoop.hive.ql.Driver.lockAndRespond(Driver.java:1624)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1794)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1538)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1527)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:137)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:287)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:635)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1655)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1602)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1597)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:200)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1455)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1429)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:177)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> at
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver(TestMiniTezCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.apache.hadoop.h
[jira] [Commented] (HIVE-19041) Thrift deserialization of Partition objects should intern fields
[
https://issues.apache.org/jira/browse/HIVE-19041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470954#comment-16470954
]
Misha Dmitriev commented on HIVE-19041:
---
Thank you for looking into the details, [~vihangk1] I've checked the code and I
agree with you - these strings are really short-lived. Furthemore, the worst
offenders,
org.apache.hadoop.hive.metastore.model.MStorageDescriptor.inputFormat,outputFormat,
are actually copies of the corresponding fields of {{StorageDescriptor}}. That
is, they reference the same string instances. So as soon as you intern the
strings in {{StorageDescriptor}}, {{MStorageDescriptor}} will stop referencing
duplicate strings as well.
Thus, this patch is good to go from my prospective.
> Thrift deserialization of Partition objects should intern fields
>
>
> Key: HIVE-19041
> URL: https://issues.apache.org/jira/browse/HIVE-19041
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-19041.01.patch, HIVE-19041.02.patch,
> HIVE-19041.03.patch, HIVE-19041.04.patch
>
>
> When a client is creating large number of partitions, the thrift objects are
> deserialized into Partition objects. The read method of these objects does
> not intern the inputformat, location, outputformat which cause large number
> of duplicate Strings in the HMS memory. We should intern these objects while
> deserialization to reduce memory pressure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19472) HiveStreamingConnection swallows exception on partition creation
[ https://issues.apache.org/jira/browse/HIVE-19472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19472: - Resolution: Fixed Fix Version/s: 3.1.0 3.0.0 Status: Resolved (was: Patch Available) Test failures are unrelated. Committed to branch-3 and master. > HiveStreamingConnection swallows exception on partition creation > > > Key: HIVE-19472 > URL: https://issues.apache.org/jira/browse/HIVE-19472 > Project: Hive > Issue Type: Sub-task > Components: Streaming >Affects Versions: 3.0.0, 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19472.1.patch > > > HiveStreamingConnection swallows exception on partition creation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19493) VectorUDFDateDiffColCol copySelected does not handle nulls correctly
[
https://issues.apache.org/jira/browse/HIVE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar reassigned HIVE-19493:
--
> VectorUDFDateDiffColCol copySelected does not handle nulls correctly
>
>
> Key: HIVE-19493
> URL: https://issues.apache.org/jira/browse/HIVE-19493
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> The {{copySelected}} method in {{VectorUDFDateDiffColCol}} class was missed
> during HIVE-18622
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19493) VectorUDFDateDiffColCol copySelected does not handle nulls correctly
[
https://issues.apache.org/jira/browse/HIVE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470964#comment-16470964
]
Vihang Karajgaonkar commented on HIVE-19493:
cc: [~mmccline]
> VectorUDFDateDiffColCol copySelected does not handle nulls correctly
>
>
> Key: HIVE-19493
> URL: https://issues.apache.org/jira/browse/HIVE-19493
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> The {{copySelected}} method in {{VectorUDFDateDiffColCol}} class was missed
> during HIVE-18622
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19258) add originals support to MM tables (and make the conversion a metadata only operation)
[ https://issues.apache.org/jira/browse/HIVE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19258: Attachment: HIVE-19258.05.patch > add originals support to MM tables (and make the conversion a metadata only > operation) > -- > > Key: HIVE-19258 > URL: https://issues.apache.org/jira/browse/HIVE-19258 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19258.01.patch, HIVE-19258.02.patch, > HIVE-19258.03.patch, HIVE-19258.04.patch, HIVE-19258.05.patch, > HIVE-19258.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19258) add originals support to MM tables (and make the conversion a metadata only operation)
[ https://issues.apache.org/jira/browse/HIVE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470970#comment-16470970 ] Sergey Shelukhin commented on HIVE-19258: - Fixed two more issues in BHIF, NPE and the same final path being added twice for non MM case > add originals support to MM tables (and make the conversion a metadata only > operation) > -- > > Key: HIVE-19258 > URL: https://issues.apache.org/jira/browse/HIVE-19258 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19258.01.patch, HIVE-19258.02.patch, > HIVE-19258.03.patch, HIVE-19258.04.patch, HIVE-19258.05.patch, > HIVE-19258.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18862) qfiles: prepare .q files for using datasets
[
https://issues.apache.org/jira/browse/HIVE-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470981#comment-16470981
]
Hive QA commented on HIVE-18862:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922622/HIVE-18862.10-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 2062 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 47 failed/errored test(s), 13373 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,a
[jira] [Commented] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils
[
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471021#comment-16471021
]
Hive QA commented on HIVE-19476:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10807/dev-support/hive-personality.sh
|
| git revision | master / 2d8e48c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10807/yetus/patch-asflicense-problems.txt
|
| modules | C: itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10807/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix failures in TestReplicationScenariosAcidTables,
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils
> -
>
> Key: HIVE-19476
> URL: https://issues.apache.org/jira/browse/HIVE-19476
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: Tests, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19476.01.patch
>
>
> TestReplicationScenariosAcidTables broken due to HIVE-17657.
> cc [~sershe]
> TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to
> auto-stats gathered which thrown exception irrelevant to theses tests.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19494) Accept shade prefix during reflective instantiation of output format
[ https://issues.apache.org/jira/browse/HIVE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-19494: > Accept shade prefix during reflective instantiation of output format > > > Key: HIVE-19494 > URL: https://issues.apache.org/jira/browse/HIVE-19494 > Project: Hive > Issue Type: Sub-task > Components: Streaming >Affects Versions: 3.0.0, 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > Hive Streaming API jars are sometimes shaded with a different prefix when > used in environments where another version of hive already exists (spark for > example). In most cases, shading is done with rename of classes with some > prefix. If an uber/assembly jar is generated with renamed prefix, Hive > Streaming API will not work as Hive Streaming API will reflectively > instantiate outputformat class using FQCN string provided by metastore table > storage descriptor object. > For example: > RecordWriter will create instance of OutputFormat using string > "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat". When a shaded jar with > renames class references are used, this class will not be found by the > classloader. > We can optionally accept a shade prefix from user which will be tried (as > fallback) when ClassNotFoundException is thrown. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19490) Locking on Insert into for non native and managed tables.
[ https://issues.apache.org/jira/browse/HIVE-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471031#comment-16471031 ] Eugene Koifman commented on HIVE-19490: --- TestDbTxnManager2 has many tests to verify locks I think using shared lock for Druid insert makes sense but I'm not sure if that is appropriate for all non-native tables. > Locking on Insert into for non native and managed tables. > - > > Key: HIVE-19490 > URL: https://issues.apache.org/jira/browse/HIVE-19490 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: druid, locking > Attachments: HIVE-19490.patch > > > Current state of the art: > Managed non native table like Druid Tables, will need to get a Lock on Insert > into or insert Over write. The nature of this lock is set to Exclusive by > default for any non native table. > This implies that Inserts into Druid table will Lock any read query as well > during the execution of the insert into. IMO this lock (on insert into) is > not needed since the insert statement is appending data and the state of > loading it is managed partially by Hive Storage handler hook and part of it > by Druid. > What i am proposing is to relax the lock level to shared for all non native > tables on insert into operations and keep it as Exclusive Write for insert > Overwrite for now. > > Any feedback is welcome. > cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn] > Also am not sure what is the best way to unit test this currently am using > debugger to check of locks are what i except, please let me know if there is > a better way to do this. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17463) ORC: include orc-shims in hive-exec.jar
[ https://issues.apache.org/jira/browse/HIVE-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471033#comment-16471033 ] Jesus Camacho Rodriguez commented on HIVE-17463: [~gopalv], [~owen.omalley], I will incorporate this into HIVE-19465? > ORC: include orc-shims in hive-exec.jar > --- > > Key: HIVE-17463 > URL: https://issues.apache.org/jira/browse/HIVE-17463 > Project: Hive > Issue Type: Bug > Components: ORC >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Attachments: HIVE-17463.1.patch > > > ORC-234 added a new shims module - this needs to be part of hive-exec shading > to use ORC-1.5.x branch in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17463) ORC: include orc-shims in hive-exec.jar
[ https://issues.apache.org/jira/browse/HIVE-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471036#comment-16471036 ] Gopal V commented on HIVE-17463: Yes, +1 > ORC: include orc-shims in hive-exec.jar > --- > > Key: HIVE-17463 > URL: https://issues.apache.org/jira/browse/HIVE-17463 > Project: Hive > Issue Type: Bug > Components: ORC >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Attachments: HIVE-17463.1.patch > > > ORC-234 added a new shims module - this needs to be part of hive-exec shading > to use ORC-1.5.x branch in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19465) Upgrade ORC to 1.5.0
[ https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471038#comment-16471038 ] Gopal V commented on HIVE-19465: My nightly runs on LLAP have finished without any issue after rebuilding against orc-1.5 branch. LGTM - +1 tests pending > Upgrade ORC to 1.5.0 > > > Key: HIVE-19465 > URL: https://issues.apache.org/jira/browse/HIVE-19465 > Project: Hive > Issue Type: Improvement > Components: File Formats >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19465.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-17463) ORC: include orc-shims in hive-exec.jar
[ https://issues.apache.org/jira/browse/HIVE-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved HIVE-17463. Resolution: Fixed Fix Version/s: 3.0.0 Thanks [~gopalv]! > ORC: include orc-shims in hive-exec.jar > --- > > Key: HIVE-17463 > URL: https://issues.apache.org/jira/browse/HIVE-17463 > Project: Hive > Issue Type: Bug > Components: ORC >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Fix For: 3.0.0 > > Attachments: HIVE-17463.1.patch > > > ORC-234 added a new shims module - this needs to be part of hive-exec shading > to use ORC-1.5.x branch in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19465) Upgrade ORC to 1.5.0
[ https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19465: --- Attachment: HIVE-19465.01.patch > Upgrade ORC to 1.5.0 > > > Key: HIVE-19465 > URL: https://issues.apache.org/jira/browse/HIVE-19465 > Project: Hive > Issue Type: Improvement > Components: File Formats >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19465.01.patch, HIVE-19465.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19465) Upgrade ORC to 1.5.0
[ https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19465: --- Attachment: HIVE-19465.01.patch > Upgrade ORC to 1.5.0 > > > Key: HIVE-19465 > URL: https://issues.apache.org/jira/browse/HIVE-19465 > Project: Hive > Issue Type: Improvement > Components: File Formats >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19465.01.patch, HIVE-19465.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19465) Upgrade ORC to 1.5.0
[ https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19465: --- Attachment: (was: HIVE-19465.01.patch) > Upgrade ORC to 1.5.0 > > > Key: HIVE-19465 > URL: https://issues.apache.org/jira/browse/HIVE-19465 > Project: Hive > Issue Type: Improvement > Components: File Formats >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19465.01.patch, HIVE-19465.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19464) Upgrade Parquet to 1.10.0
[ https://issues.apache.org/jira/browse/HIVE-19464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19464: --- Attachment: (was: HIVE-19464.patch) > Upgrade Parquet to 1.10.0 > - > > Key: HIVE-19464 > URL: https://issues.apache.org/jira/browse/HIVE-19464 > Project: Hive > Issue Type: Improvement > Components: File Formats >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19464.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19464) Upgrade Parquet to 1.10.0
[ https://issues.apache.org/jira/browse/HIVE-19464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19464: --- Attachment: HIVE-19464.01.patch > Upgrade Parquet to 1.10.0 > - > > Key: HIVE-19464 > URL: https://issues.apache.org/jira/browse/HIVE-19464 > Project: Hive > Issue Type: Improvement > Components: File Formats >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19464.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19494) Accept shade prefix during reflective instantiation of output format
[ https://issues.apache.org/jira/browse/HIVE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19494: - Description: Hive Streaming API jars are sometimes shaded with a different prefix when used in environments where another version of hive already exists (spark for example). In most cases, shading is done with rename of classes with some prefix. If an uber/assembly jar is generated with renamed prefix, Hive Streaming API will not work as Hive Streaming API will reflectively instantiate outputformat class using FQCN string provided by metastore table storage descriptor object. For example: RecordWriter will create instance of OutputFormat using string "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat". When a shaded jar with renamed class references are used, this class will not be found by the classloader. We can optionally accept a shade prefix from user which will be tried (as fallback) when ClassNotFoundException is thrown. was: Hive Streaming API jars are sometimes shaded with a different prefix when used in environments where another version of hive already exists (spark for example). In most cases, shading is done with rename of classes with some prefix. If an uber/assembly jar is generated with renamed prefix, Hive Streaming API will not work as Hive Streaming API will reflectively instantiate outputformat class using FQCN string provided by metastore table storage descriptor object. For example: RecordWriter will create instance of OutputFormat using string "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat". When a shaded jar with renames class references are used, this class will not be found by the classloader. We can optionally accept a shade prefix from user which will be tried (as fallback) when ClassNotFoundException is thrown. > Accept shade prefix during reflective instantiation of output format > > > Key: HIVE-19494 > URL: https://issues.apache.org/jira/browse/HIVE-19494 > Project: Hive > Issue Type: Sub-task > Components: Streaming >Affects Versions: 3.0.0, 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > Hive Streaming API jars are sometimes shaded with a different prefix when > used in environments where another version of hive already exists (spark for > example). In most cases, shading is done with rename of classes with some > prefix. If an uber/assembly jar is generated with renamed prefix, Hive > Streaming API will not work as Hive Streaming API will reflectively > instantiate outputformat class using FQCN string provided by metastore table > storage descriptor object. > For example: > RecordWriter will create instance of OutputFormat using string > "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat". When a shaded jar with > renamed class references are used, this class will not be found by the > classloader. > We can optionally accept a shade prefix from user which will be tried (as > fallback) when ClassNotFoundException is thrown. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19250) Schema column definitions inconsistencies in MySQL
[ https://issues.apache.org/jira/browse/HIVE-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-19250: - Status: Patch Available (was: Open) > Schema column definitions inconsistencies in MySQL > -- > > Key: HIVE-19250 > URL: https://issues.apache.org/jira/browse/HIVE-19250 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-19250.patch, HIVE-19250.patch > > > There are some inconsistencies in column definitions in MySQL between a > schema that was upgraded to 2.1 (from an older release) vs installing the > 2.1.0 schema directly. > > `CQ_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 117d117 > < `CQ_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 135a136 > > `CC_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 143d143 > < `CC_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 156c156 > < `CTC_TXNID` bigint(20) DEFAULT NULL, > --- > > `CTC_TXNID` bigint(20) NOT NULL, > 158c158 > < `CTC_TABLE` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT > NULL, > --- > > `CTC_TABLE` varchar(256) DEFAULT NULL, > 476c476 > < `TBL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT > NULL, > --- > > `TBL_NAME` varchar(256) DEFAULT NULL, > 664c664 > < KEY `PCS_STATS_IDX` > (`DB_NAME`,`TABLE_NAME`,`COLUMN_NAME`,`PARTITION_NAME`), > --- > > KEY `PCS_STATS_IDX` > > (`DB_NAME`,`TABLE_NAME`,`COLUMN_NAME`,`PARTITION_NAME`) USING BTREE, > 768c768 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 814c814 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 934c934 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 1066d1065 > < `TXN_HEARTBEAT_COUNT` int(11) DEFAULT NULL, > 1067a1067 > > `TXN_HEARTBEAT_COUNT` int(11) DEFAULT NULL, > 1080c1080 > < `TC_TXNID` bigint(20) DEFAULT NULL, > --- > > `TC_TXNID` bigint(20) NOT NULL, > 1082c1082 > < `TC_TABLE` varchar(128) DEFAULT NULL, > --- > > `TC_TABLE` varchar(128) NOT NULL, > 1084c1084 > < `TC_OPERATION_TYPE` char(1) DEFAULT NULL, > --- > > `TC_OPERATION_TYPE` char(1) NOT NULL, -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19250) Schema column definitions inconsistencies in MySQL
[ https://issues.apache.org/jira/browse/HIVE-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-19250: - Attachment: HIVE-19250.patch > Schema column definitions inconsistencies in MySQL > -- > > Key: HIVE-19250 > URL: https://issues.apache.org/jira/browse/HIVE-19250 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-19250.patch, HIVE-19250.patch > > > There are some inconsistencies in column definitions in MySQL between a > schema that was upgraded to 2.1 (from an older release) vs installing the > 2.1.0 schema directly. > > `CQ_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 117d117 > < `CQ_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 135a136 > > `CC_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 143d143 > < `CC_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 156c156 > < `CTC_TXNID` bigint(20) DEFAULT NULL, > --- > > `CTC_TXNID` bigint(20) NOT NULL, > 158c158 > < `CTC_TABLE` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT > NULL, > --- > > `CTC_TABLE` varchar(256) DEFAULT NULL, > 476c476 > < `TBL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT > NULL, > --- > > `TBL_NAME` varchar(256) DEFAULT NULL, > 664c664 > < KEY `PCS_STATS_IDX` > (`DB_NAME`,`TABLE_NAME`,`COLUMN_NAME`,`PARTITION_NAME`), > --- > > KEY `PCS_STATS_IDX` > > (`DB_NAME`,`TABLE_NAME`,`COLUMN_NAME`,`PARTITION_NAME`) USING BTREE, > 768c768 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 814c814 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 934c934 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 1066d1065 > < `TXN_HEARTBEAT_COUNT` int(11) DEFAULT NULL, > 1067a1067 > > `TXN_HEARTBEAT_COUNT` int(11) DEFAULT NULL, > 1080c1080 > < `TC_TXNID` bigint(20) DEFAULT NULL, > --- > > `TC_TXNID` bigint(20) NOT NULL, > 1082c1082 > < `TC_TABLE` varchar(128) DEFAULT NULL, > --- > > `TC_TABLE` varchar(128) NOT NULL, > 1084c1084 > < `TC_OPERATION_TYPE` char(1) DEFAULT NULL, > --- > > `TC_OPERATION_TYPE` char(1) NOT NULL, -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19250) Schema column definitions inconsistencies in MySQL
[ https://issues.apache.org/jira/browse/HIVE-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-19250: - Status: Open (was: Patch Available) pre-commits did not run. Retrying > Schema column definitions inconsistencies in MySQL > -- > > Key: HIVE-19250 > URL: https://issues.apache.org/jira/browse/HIVE-19250 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-19250.patch, HIVE-19250.patch > > > There are some inconsistencies in column definitions in MySQL between a > schema that was upgraded to 2.1 (from an older release) vs installing the > 2.1.0 schema directly. > > `CQ_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 117d117 > < `CQ_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 135a136 > > `CC_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 143d143 > < `CC_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 156c156 > < `CTC_TXNID` bigint(20) DEFAULT NULL, > --- > > `CTC_TXNID` bigint(20) NOT NULL, > 158c158 > < `CTC_TABLE` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT > NULL, > --- > > `CTC_TABLE` varchar(256) DEFAULT NULL, > 476c476 > < `TBL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT > NULL, > --- > > `TBL_NAME` varchar(256) DEFAULT NULL, > 664c664 > < KEY `PCS_STATS_IDX` > (`DB_NAME`,`TABLE_NAME`,`COLUMN_NAME`,`PARTITION_NAME`), > --- > > KEY `PCS_STATS_IDX` > > (`DB_NAME`,`TABLE_NAME`,`COLUMN_NAME`,`PARTITION_NAME`) USING BTREE, > 768c768 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 814c814 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 934c934 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 1066d1065 > < `TXN_HEARTBEAT_COUNT` int(11) DEFAULT NULL, > 1067a1067 > > `TXN_HEARTBEAT_COUNT` int(11) DEFAULT NULL, > 1080c1080 > < `TC_TXNID` bigint(20) DEFAULT NULL, > --- > > `TC_TXNID` bigint(20) NOT NULL, > 1082c1082 > < `TC_TABLE` varchar(128) DEFAULT NULL, > --- > > `TC_TABLE` varchar(128) NOT NULL, > 1084c1084 > < `TC_OPERATION_TYPE` char(1) DEFAULT NULL, > --- > > `TC_OPERATION_TYPE` char(1) NOT NULL, -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19494) Accept shade prefix during reflective instantiation of output format
[ https://issues.apache.org/jira/browse/HIVE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19494: - Description: Hive Streaming API jars are sometimes shaded with a different prefix when used in environments where another version of hive already exists (spark for example). In most cases, shading is done with rename of classes with some prefix. If an uber/assembly jar is generated with renamed prefix, Hive Streaming API will not work as Hive Streaming API will reflectively instantiate outputformat class using FQCN string provided by metastore table storage descriptor object. For example: RecordWriter will create instance of OutputFormat using string "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat". When a shaded jar with renamed class references are used, this class will not be found by the classloader. We can optionally accept a shade prefix from user via config which will be tried (as fallback) when ClassNotFoundException is thrown. was: Hive Streaming API jars are sometimes shaded with a different prefix when used in environments where another version of hive already exists (spark for example). In most cases, shading is done with rename of classes with some prefix. If an uber/assembly jar is generated with renamed prefix, Hive Streaming API will not work as Hive Streaming API will reflectively instantiate outputformat class using FQCN string provided by metastore table storage descriptor object. For example: RecordWriter will create instance of OutputFormat using string "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat". When a shaded jar with renamed class references are used, this class will not be found by the classloader. We can optionally accept a shade prefix from user which will be tried (as fallback) when ClassNotFoundException is thrown. > Accept shade prefix during reflective instantiation of output format > > > Key: HIVE-19494 > URL: https://issues.apache.org/jira/browse/HIVE-19494 > Project: Hive > Issue Type: Sub-task > Components: Streaming >Affects Versions: 3.0.0, 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > Hive Streaming API jars are sometimes shaded with a different prefix when > used in environments where another version of hive already exists (spark for > example). In most cases, shading is done with rename of classes with some > prefix. If an uber/assembly jar is generated with renamed prefix, Hive > Streaming API will not work as Hive Streaming API will reflectively > instantiate outputformat class using FQCN string provided by metastore table > storage descriptor object. > For example: > RecordWriter will create instance of OutputFormat using string > "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat". When a shaded jar with > renamed class references are used, this class will not be found by the > classloader. > We can optionally accept a shade prefix from user via config which will be > tried (as fallback) when ClassNotFoundException is thrown. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19250) Schema column definitions inconsistencies in MySQL
[ https://issues.apache.org/jira/browse/HIVE-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471057#comment-16471057 ] Naveen Gangam commented on HIVE-19250: -- Actually they ran .. there has been only 1 version of the patch. There were some failures but I cannot analyze them as the build is too old. So a re-run would help. > Schema column definitions inconsistencies in MySQL > -- > > Key: HIVE-19250 > URL: https://issues.apache.org/jira/browse/HIVE-19250 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-19250.patch, HIVE-19250.patch > > > There are some inconsistencies in column definitions in MySQL between a > schema that was upgraded to 2.1 (from an older release) vs installing the > 2.1.0 schema directly. > > `CQ_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 117d117 > < `CQ_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 135a136 > > `CC_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 143d143 > < `CC_TBLPROPERTIES` varchar(2048) DEFAULT NULL, > 156c156 > < `CTC_TXNID` bigint(20) DEFAULT NULL, > --- > > `CTC_TXNID` bigint(20) NOT NULL, > 158c158 > < `CTC_TABLE` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT > NULL, > --- > > `CTC_TABLE` varchar(256) DEFAULT NULL, > 476c476 > < `TBL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT > NULL, > --- > > `TBL_NAME` varchar(256) DEFAULT NULL, > 664c664 > < KEY `PCS_STATS_IDX` > (`DB_NAME`,`TABLE_NAME`,`COLUMN_NAME`,`PARTITION_NAME`), > --- > > KEY `PCS_STATS_IDX` > > (`DB_NAME`,`TABLE_NAME`,`COLUMN_NAME`,`PARTITION_NAME`) USING BTREE, > 768c768 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 814c814 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 934c934 > < `PARAM_VALUE` mediumtext, > --- > > `PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin, > 1066d1065 > < `TXN_HEARTBEAT_COUNT` int(11) DEFAULT NULL, > 1067a1067 > > `TXN_HEARTBEAT_COUNT` int(11) DEFAULT NULL, > 1080c1080 > < `TC_TXNID` bigint(20) DEFAULT NULL, > --- > > `TC_TXNID` bigint(20) NOT NULL, > 1082c1082 > < `TC_TABLE` varchar(128) DEFAULT NULL, > --- > > `TC_TABLE` varchar(128) NOT NULL, > 1084c1084 > < `TC_OPERATION_TYPE` char(1) DEFAULT NULL, > --- > > `TC_OPERATION_TYPE` char(1) NOT NULL, -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19490) Locking on Insert into for non native and managed tables.
[ https://issues.apache.org/jira/browse/HIVE-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-19490: -- Description: Current state of the art: Managed non native table like Druid Tables, will need to get a Lock on Insert into or insert Over write. The nature of this lock is set to Exclusive by default for any non native table. This implies that Inserts into Druid table will Lock any read query as well during the execution of the insert into. IMO this lock (on insert into) is not needed since the insert statement is appending data and the state of loading it is managed partially by Hive Storage handler hook and part of it by Druid. What i am proposing is to relax the lock level to shared for all non native tables on insert into operations and keep it as Exclusive Write for insert Overwrite for now. Any feedback is welcome. cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn] Also am not sure what is the best way to unit test this currently am using debugger to check if locks are what i except, please let me know if there is a better way to do this. was: Current state of the art: Managed non native table like Druid Tables, will need to get a Lock on Insert into or insert Over write. The nature of this lock is set to Exclusive by default for any non native table. This implies that Inserts into Druid table will Lock any read query as well during the execution of the insert into. IMO this lock (on insert into) is not needed since the insert statement is appending data and the state of loading it is managed partially by Hive Storage handler hook and part of it by Druid. What i am proposing is to relax the lock level to shared for all non native tables on insert into operations and keep it as Exclusive Write for insert Overwrite for now. Any feedback is welcome. cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn] Also am not sure what is the best way to unit test this currently am using debugger to check of locks are what i except, please let me know if there is a better way to do this. > Locking on Insert into for non native and managed tables. > - > > Key: HIVE-19490 > URL: https://issues.apache.org/jira/browse/HIVE-19490 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: druid, locking > Attachments: HIVE-19490.patch > > > Current state of the art: > Managed non native table like Druid Tables, will need to get a Lock on Insert > into or insert Over write. The nature of this lock is set to Exclusive by > default for any non native table. > This implies that Inserts into Druid table will Lock any read query as well > during the execution of the insert into. IMO this lock (on insert into) is > not needed since the insert statement is appending data and the state of > loading it is managed partially by Hive Storage handler hook and part of it > by Druid. > What i am proposing is to relax the lock level to shared for all non native > tables on insert into operations and keep it as Exclusive Write for insert > Overwrite for now. > > Any feedback is welcome. > cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn] > Also am not sure what is the best way to unit test this currently am using > debugger to check if locks are what i except, please let me know if there is > a better way to do this. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils
[
https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471092#comment-16471092
]
Hive QA commented on HIVE-19476:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922637/HIVE-19476.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 39 failed/errored test(s), 13546 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.q,
[jira] [Updated] (HIVE-19466) Update constraint violation error message
[
https://issues.apache.org/jira/browse/HIVE-19466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19466:
---
Attachment: HIVE-19466.01-branch-3.patch
> Update constraint violation error message
> -
>
> Key: HIVE-19466
> URL: https://issues.apache.org/jira/browse/HIVE-19466
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19466.01-branch-3.patch, HIVE-19466.1.patch
>
>
> Currently for both CHECK and NOT NULL constraint violation hive throws {{NOT
> NULL Constraint violated}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19421) Upgrade version of Jetty to 9.3.20.v20170531
[ https://issues.apache.org/jira/browse/HIVE-19421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janaki Lahorani updated HIVE-19421: --- Attachment: HIVE-19421.3.patch > Upgrade version of Jetty to 9.3.20.v20170531 > > > Key: HIVE-19421 > URL: https://issues.apache.org/jira/browse/HIVE-19421 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Attachments: HIVE-19421.1.patch, HIVE-19421.2.patch, > HIVE-19421.3.patch, HIVE-19421.3.patch > > > Move Jetty up to 9.3.20.v20170531 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19466) Update constraint violation error message
[
https://issues.apache.org/jira/browse/HIVE-19466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19466:
---
Status: Open (was: Patch Available)
> Update constraint violation error message
> -
>
> Key: HIVE-19466
> URL: https://issues.apache.org/jira/browse/HIVE-19466
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19466.01-branch-3.patch, HIVE-19466.1.patch
>
>
> Currently for both CHECK and NOT NULL constraint violation hive throws {{NOT
> NULL Constraint violated}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
