[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-20 Thread Prasanth Jayachandran (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260206#comment-16260206
 ] 

Prasanth Jayachandran commented on HIVE-18072:
--

+1

> WM - fix various bugs based on cluster testing - part 2
> ---
>
> Key: HIVE-18072
> URL: https://issues.apache.org/jira/browse/HIVE-18072
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18072.01.patch, HIVE-18072.02.patch, 
> HIVE-18072.03.patch, HIVE-18072.03.patch, HIVE-18072.04.patch, 
> HIVE-18072.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-20 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260084#comment-16260084
 ] 

Hive QA commented on HIVE-18072:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898535/HIVE-18072.04.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 11387 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks] 
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] 
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
 (batchId=162)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints 
(batchId=223)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7927/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7927/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7927/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12898535 - PreCommit-HIVE-Build

> WM - fix various bugs based on cluster testing - part 2
> ---
>
> Key: HIVE-18072
> URL: https://issues.apache.org/jira/browse/HIVE-18072
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18072.01.patch, HIVE-18072.02.patch, 
> HIVE-18072.03.patch, HIVE-18072.03.patch, HIVE-18072.04.patch, 
> HIVE-18072.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-17 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257460#comment-16257460
 ] 

Sergey Shelukhin commented on HIVE-18072:
-

Nm, could be because new taskInfo map is not cleaned up...

> WM - fix various bugs based on cluster testing - part 2
> ---
>
> Key: HIVE-18072
> URL: https://issues.apache.org/jira/browse/HIVE-18072
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18072.01.patch, HIVE-18072.02.patch, 
> HIVE-18072.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-17 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257456#comment-16257456
 ] 

Sergey Shelukhin commented on HIVE-18072:
-

Hmm, something about this patch does cause tests to oom. Need to take a look 
locally with the same args.

> WM - fix various bugs based on cluster testing - part 2
> ---
>
> Key: HIVE-18072
> URL: https://issues.apache.org/jira/browse/HIVE-18072
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18072.01.patch, HIVE-18072.02.patch, 
> HIVE-18072.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-17 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256712#comment-16256712
 ] 

Hive QA commented on HIVE-18072:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898107/HIVE-18072.02.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 79 failed/errored test(s), 11189 tests 
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=150)

[skewjoinopt15.q,vector_coalesce.q,dp_counter_mm.q,orc_ppd_decimal.q,cbo_rp_lineage2.q,insert_into_with_schema.q,join_emit_interval.q,join1.q,join32_lessSize.q,vector_null_projection.q,vector_groupby_grouping_sets_grouping.q,vector_cast_constant.q,mapjoin2.q,orc_predicate_pushdown.q,cte_mat_4.q,correlationoptimizer4.q,vector_windowing_navfn.q,vectorization_12.q,vector_number_compare_projection.q,orc_merge_incompat3.q,dynamic_semijoin_reduction_sw.q,vector_leftsemi_mapjoin.q,update_all_non_partitioned.q,tez_insert_overwrite_local_directory_1.q,column_table_stats_orc.q,schema_evol_text_vecrow_table.q,vector_if_expr.q,delete_whole_partition.q,bucketsortoptimize_insert_6.q,ctas.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=155)

[mapjoin_emit_interval.q,vector_auto_smb_mapjoin_14.q,deleteAnalyze.q,tez_dml.q,database.q,vector_bround.q,vector_join30.q,smb_mapjoin_6.q,vector_reduce_groupby_decimal.q,vectorized_dynamic_partition_pruning.q,cbo_views.q,vectorization_part.q,dynamic_partition_pruning.q,cte_mat_1.q,cluster.q,vector_char_mapjoin1.q,cte_5.q,subquery_shared_alias.q,vector_decimal_2.q,bucketmapjoin7.q,vector_outer_join3.q,vector_string_concat.q,subquery_nested_subquery.q,limit_pushdown3.q,vector_outer_join2.q,vectorized_context.q,metadata_only_queries.q,auto_sortmerge_join_12.q,union6.q,vectorized_timestamp_funcs.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=161)

[load_dyn_part2.q,parquet_types.q,vectorization_5.q,mapjoin46.q,jdbc_handler.q,column_access_stats.q,vector_between_in.q,subquery_in_having.q,vectorized_string_funcs.q,vectorization_1.q,auto_join_filters.q,join0.q,vector_coalesce_3.q,mergejoin.q,stats_noscan_1.q,orc_split_elimination.q,vector_decimal_10_0.q,vector_outer_join0.q,vector_udf1.q,vector_groupby_reduce.q,union8.q,vector_complex_all.q,insert_values_orig_table.q,correlationoptimizer1.q,windowing_windowspec2.q,vector_decimal_mapjoin.q,auto_smb_mapjoin_14.q,vectorized_case.q,vector_join_filters.q,vector_windowing_rank.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=162)

[auto_sortmerge_join_13.q,union_top_level.q,schema_evol_orc_acid_table_update.q,vector_left_outer_join2.q,schema_evol_text_vecrow_part_all_primitive.q,constprog_semijoin.q,update_where_partitioned.q,vectorized_multi_output_select.q,drop_partition_with_stats.q,smb_mapjoin_14.q,skiphf_aggr.q,vectorized_ptf.q,alter_merge_orc.q,vector_char_cast.q,join_reordering_no_stats.q,insert_orig_table.q,join_filters.q,subquery_in.q,schema_evol_text_vec_part_all_primitive.q,auto_sortmerge_join_4.q,bucket_many.q,vectorization_15.q,union3.q,union7.q,vectorization_nested_udf.q,vector_groupby_3.q,insert_values_orig_table_use_metadata.q,vector_mr_diff_schema_alias.q,cbo_rp_gby.q,reduce_deduplicate_extended.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=163)

[vector_interval_2.q,schema_evol_orc_acid_part_update.q,orc_ppd_varchar.q,metadataonly1.q,count_dist_rewrite.q,limit_pushdown.q,auto_join_nulls.q,metadata_only_queries_with_filters.q,vector_inner_join.q,subquery_notin.q,schema_evol_text_nonvec_part_all_complex.q,vector_coalesce_2.q,vector_between_columns.q,vector_groupby_grouping_sets6.q,table_access_keys_stats.q,udaf_collect_set_2.q,subquery_null_agg.q,update_after_multiple_inserts.q,offset_limit_ppd_optimizer.q,mapjoin_decimal.q,orc_merge_incompat1.q,columnstats_part_coltype.q,vectorized_parquet_types.q,vector_windowing_gby2.q,vector_char_2.q,schema_evol_orc_acidvec_part.q,acid_no_buckets.q,extrapolate_part_stats_partial_ndv.q,auto_sortmerge_join_9.q,vector_groupby_grouping_id2.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=166)

[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-16 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256189#comment-16256189
 ] 

Hive QA commented on HIVE-18072:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898057/HIVE-18072.01.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7867/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7867/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7867/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:10:46.425
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7867/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:10:46.428
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
   254e216..477e9c5  master -> origin/master
+ git reset --hard HEAD
HEAD is now at 254e216 HIVE-15018: ALTER rewriting flag in materialized view 
(Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java.orig
Removing ql/src/test/queries/clientpositive/alter_view_as_select_view.q
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 477e9c5 HIVE-18071 : add HS2 jmx information about pools and 
current resource plan (Sergey Shelukhin, reviewed by Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:10:51.584
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: patch failed: 
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/GuaranteedTasksAllocator.java:167
error: 
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/GuaranteedTasksAllocator.java: 
patch does not apply
error: patch failed: 
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java:18
error: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java: 
patch does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12898057 - PreCommit-HIVE-Build

> WM - fix various bugs based on cluster testing - part 2
> ---
>
> Key: HIVE-18072
> URL: https://issues.apache.org/jira/browse/HIVE-18072
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18072.01.patch, HIVE-18072.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-16 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255892#comment-16255892
 ] 

Sergey Shelukhin commented on HIVE-18072:
-

Hmm, I cannot repro most of these.. looks like all cluster tests OOMed in 
various ways and lots of minor tests timed out. I'll see what I can repro and 
fix that, then do another run.

> WM - fix various bugs based on cluster testing - part 2
> ---
>
> Key: HIVE-18072
> URL: https://issues.apache.org/jira/browse/HIVE-18072
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18072.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-16 Thread Prasanth Jayachandran (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255860#comment-16255860
 ] 

Prasanth Jayachandran commented on HIVE-18072:
--

Left minor comments in RB

> WM - fix various bugs based on cluster testing - part 2
> ---
>
> Key: HIVE-18072
> URL: https://issues.apache.org/jira/browse/HIVE-18072
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18072.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2

2017-11-15 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254790#comment-16254790
 ] 

Hive QA commented on HIVE-18072:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897912/HIVE-18072.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 73 failed/errored test(s), 11138 tests 
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=150)

[skewjoinopt15.q,vector_coalesce.q,dp_counter_mm.q,orc_ppd_decimal.q,cbo_rp_lineage2.q,insert_into_with_schema.q,join_emit_interval.q,join1.q,join32_lessSize.q,vector_null_projection.q,vector_groupby_grouping_sets_grouping.q,vector_cast_constant.q,mapjoin2.q,orc_predicate_pushdown.q,cte_mat_4.q,correlationoptimizer4.q,vector_windowing_navfn.q,vectorization_12.q,vector_number_compare_projection.q,orc_merge_incompat3.q,dynamic_semijoin_reduction_sw.q,vector_leftsemi_mapjoin.q,update_all_non_partitioned.q,tez_insert_overwrite_local_directory_1.q,column_table_stats_orc.q,schema_evol_text_vecrow_table.q,vector_if_expr.q,delete_whole_partition.q,bucketsortoptimize_insert_6.q,ctas.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=155)

[mapjoin_emit_interval.q,vector_auto_smb_mapjoin_14.q,deleteAnalyze.q,tez_dml.q,database.q,vector_bround.q,vector_join30.q,smb_mapjoin_6.q,vector_reduce_groupby_decimal.q,vectorized_dynamic_partition_pruning.q,cbo_views.q,vectorization_part.q,dynamic_partition_pruning.q,cte_mat_1.q,cluster.q,vector_char_mapjoin1.q,cte_5.q,subquery_shared_alias.q,vector_decimal_2.q,bucketmapjoin7.q,vector_outer_join3.q,vector_string_concat.q,subquery_nested_subquery.q,limit_pushdown3.q,vector_outer_join2.q,vectorized_context.q,metadata_only_queries.q,auto_sortmerge_join_12.q,union6.q,vectorized_timestamp_funcs.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=159)

[dynamic_semijoin_reduction.q,vectorization_pushdown.q,correlationoptimizer2.q,cbo_subq_not_in.q,cbo_gby_empty.q,resourceplan.q,vectorization_short_regress.q,identity_project_remove_skip.q,mapjoin3.q,cross_product_check_1.q,bucket2.q,cbo_join.q,schema_evol_text_nonvec_part_all_primitive.q,correlationoptimizer6.q,union_remove_26.q,cbo_rp_limit.q,vector_groupby_cube1.q,vectorization_13.q,mapjoin_mapjoin.q,union2.q,groupby2.q,vector_decimal_precision.q,dynpart_sort_opt_vectorization.q,exchgpartition2lel.q,sample10.q,vectorized_timestamp_ints_casts.q,auto_sortmerge_join_2.q,bucketizedhiveinputformat.q,cte_mat_2.q,vectorization_8.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=161)

[load_dyn_part2.q,parquet_types.q,vectorization_5.q,mapjoin46.q,jdbc_handler.q,column_access_stats.q,vector_between_in.q,subquery_in_having.q,vectorized_string_funcs.q,vectorization_1.q,auto_join_filters.q,join0.q,vector_coalesce_3.q,mergejoin.q,stats_noscan_1.q,orc_split_elimination.q,vector_decimal_10_0.q,vector_outer_join0.q,vector_udf1.q,vector_groupby_reduce.q,union8.q,vector_complex_all.q,insert_values_orig_table.q,correlationoptimizer1.q,windowing_windowspec2.q,vector_decimal_mapjoin.q,auto_smb_mapjoin_14.q,vectorized_case.q,vector_join_filters.q,vector_windowing_rank.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=162)

[auto_sortmerge_join_13.q,union_top_level.q,schema_evol_orc_acid_table_update.q,vector_left_outer_join2.q,schema_evol_text_vecrow_part_all_primitive.q,constprog_semijoin.q,update_where_partitioned.q,vectorized_multi_output_select.q,drop_partition_with_stats.q,smb_mapjoin_14.q,skiphf_aggr.q,vectorized_ptf.q,alter_merge_orc.q,vector_char_cast.q,join_reordering_no_stats.q,insert_orig_table.q,join_filters.q,subquery_in.q,schema_evol_text_vec_part_all_primitive.q,auto_sortmerge_join_4.q,bucket_many.q,vectorization_15.q,union3.q,union7.q,vectorization_nested_udf.q,vector_groupby_3.q,insert_values_orig_table_use_metadata.q,vector_mr_diff_schema_alias.q,cbo_rp_gby.q,reduce_deduplicate_extended.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=163)