[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883956#comment-15883956
]
Sergey Shelukhin edited comment on HIVE-14990 at 2/28/17 1:59 AM:
--
Many of the above tests no longer fail or turned out to be expected given the
hacky nature of this testing.
Remaining tests to look at, not covered by other subtasks of the parent:
{panel}
Minimr:
parallel_orderby
CliDriver:
-avro_partitioned-
-extrapolate_part_stats_full,extrapolate_part_stats_partial-
-skewjoin- (filed a jira)
MiniLlapLocal:
-dynpart_sort_opt_vectorization,dynpart_sort_optimization,dynpart_sort_optimization2-
-exchgpartition2lel- (filed a jira, may become fixed automatically when we
integrate with ACID)
-extrapolate_part_stats_partial_ndv-
{panel}
was (Author: sershe):
Many of the above tests no longer fail or turned out to be expected given the
hacky nature of this testing.
Remaining tests to look at, not covered by other subtasks of the parent:
{panel}
Minimr:
parallel_orderby
CliDriver:
-avro_partitioned-
-extrapolate_part_stats_full,extrapolate_part_stats_partial-
-skewjoin- (filed a jira)
MiniLlapLocal:
-dynpart_sort_opt_vectorization,dynpart_sort_optimization,dynpart_sort_optimization2-
exchgpartition2lel
extrapolate_part_stats_partial_ndv
{panel}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.12.patch, HIVE-14990.13.patch,
> HIVE-14990.14.patch, HIVE-14990.15.patch, HIVE-14990.16.patch,
> HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff in partition "base file name" (path
> changes).
> 7) TestTxnCommands - all the conversion tests, as they check for bucket count
> using file lists (path changes).
> 8) HBase metastore tests cause methods are not implemented.
> 9) Some load and ExIm tests that export a table and then rely on specific
> path for load (path changes).
> 10) Bucket map join/etc. - diffs; disabled the optimization for MM tables due
> to how it accounts for buckets
> 11) rand - different results due to different sequence of processing.
> 12) many (not all i.e. not the ones with just one insert) tests that have
> stats output, such as file count, for obvious reasons
> 13) materialized views, not handled by design - the test check erroneously
> makes them "mm", no easy way to tell them apart, I don't want to plumb more
> stuff thru just for this test
> I'm filing jiras for some test failures that are not obvious and need an
> investigation later
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883956#comment-15883956
]
Sergey Shelukhin edited comment on HIVE-14990 at 2/28/17 1:47 AM:
--
Many of the above tests no longer fail or turned out to be expected given the
hacky nature of this testing.
Remaining tests to look at, not covered by other subtasks of the parent:
{panel}
Minimr:
parallel_orderby
CliDriver:
-avro_partitioned-
-extrapolate_part_stats_full,extrapolate_part_stats_partial-
-skewjoin- (filed a jira)
MiniLlapLocal:
-dynpart_sort_opt_vectorization,dynpart_sort_optimization,dynpart_sort_optimization2-
exchgpartition2lel
extrapolate_part_stats_partial_ndv
{panel}
was (Author: sershe):
Many of the above tests no longer fail or turned out to be expected given the
hacky nature of this testing.
Remaining tests to look at, not covered by other subtasks of the parent:
{noformat}
Minimr:
parallel_orderby
CliDriver:
avro_partitioned
extrapolate_part_stats_full,extrapolate_part_stats_partial
skewjoin
MiniLlapLoca:
dynpart_sort_opt_vectorization,dynpart_sort_optimization,dynpart_sort_optimization2
exchgpartition2lel
extrapolate_part_stats_partial_ndv
{noformat}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.12.patch, HIVE-14990.13.patch,
> HIVE-14990.14.patch, HIVE-14990.15.patch, HIVE-14990.16.patch,
> HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff in partition "base file name" (path
> changes).
> 7) TestTxnCommands - all the conversion tests, as they check for bucket count
> using file lists (path changes).
> 8) HBase metastore tests cause methods are not implemented.
> 9) Some load and ExIm tests that export a table and then rely on specific
> path for load (path changes).
> 10) Bucket map join/etc. - diffs; disabled the optimization for MM tables due
> to how it accounts for buckets
> 11) rand - different results due to different sequence of processing.
> 12) many (not all i.e. not the ones with just one insert) tests that have
> stats output, such as file count, for obvious reasons
> 13) materialized views, not handled by design - the test check erroneously
> makes them "mm", no easy way to tell them apart, I don't want to plumb more
> stuff thru just for this test
> I'm filing jiras for some test failures that are not obvious and need an
> investigation later
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668996#comment-15668996
]
Sergey Shelukhin edited comment on HIVE-14990 at 2/10/17 3:53 AM:
--
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
-autoColumnStats_9-
-stats_noscan_2-
-symlink_text_input_format-
-temp_table_precedence-
-offset_limit_global_optimizer-
-rand_partitionpruner2-
-materialized_view_authorization_sqlstd,materialized-
-merge_dynamic_partition, etc-
-orc_vectorization_ppd-
-parquet_join2-
-repl_3_exim_metadata-
-sample6-
-sample_islocalmode_hook-
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
-autoColumnStats_9-
-stats_noscan_2-
-symlink_text_input_format-
-temp_table_precedence-
-offset_limit_global_optimizer-
-rand_partitionpruner2-
materialized_view_authorization_sqlstd,materialized_
-merge_dynamic_partition, etc-
-orc_vectorization_ppd-
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.12.patch, HIVE-14990.13.patch,
> HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff in
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668996#comment-15668996
]
Sergey Shelukhin edited comment on HIVE-14990 at 2/9/17 2:18 AM:
-
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
-autoColumnStats_9-
-stats_noscan_2-
-symlink_text_input_format-
-temp_table_precedence-
-offset_limit_global_optimizer-
-rand_partitionpruner2-
materialized_view_authorization_sqlstd,materialized_
-merge_dynamic_partition, etc-
-orc_vectorization_ppd-
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
-autoColumnStats_9-
-stats_noscan_2-
-symlink_text_input_format-
-temp_table_precedence-
-offset_limit_global_optimizer-
-rand_partitionpruner2-
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.12.patch, HIVE-14990.13.patch,
> HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668996#comment-15668996
]
Sergey Shelukhin edited comment on HIVE-14990 at 2/9/17 2:10 AM:
-
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
-autoColumnStats_9-
-stats_noscan_2-
-symlink_text_input_format-
-temp_table_precedence-
-offset_limit_global_optimizer-
-rand_partitionpruner2-
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
-autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2-
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.12.patch, HIVE-14990.13.patch,
> HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668996#comment-15668996
]
Sergey Shelukhin edited comment on HIVE-14990 at 2/9/17 2:09 AM:
-
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
-autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2-
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.12.patch, HIVE-14990.13.patch,
> HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff in
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668996#comment-15668996
]
Sergey Shelukhin edited comment on HIVE-14990 at 2/8/17 1:12 AM:
-
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
-stats_list_bucket-
-show_tablestatus-
-vector_udf2-
-list_bucket_dml_14-
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
stats_list_bucket
show_tablestatus
-vector_udf2-
list_bucket_dml_14
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.12.patch, HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff in partition "base file name" (path
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668996#comment-15668996
]
Sergey Shelukhin edited comment on HIVE-14990 at 11/17/16 3:02 AM:
---
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
stats_list_bucket
show_tablestatus
-vector_udf2-
list_bucket_dml_14
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{noformat}
TestCliDriver:
stats_list_bucket
show_tablestatus
vector_udf2
list_bucket_dml_14
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{noformat}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff in partition "base file name" (path
> changes).
> 7)
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668996#comment-15668996
]
Sergey Shelukhin edited comment on HIVE-14990 at 11/16/16 1:36 AM:
---
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{noformat}
TestCliDriver:
stats_list_bucket
show_tablestatus
vector_udf2
list_bucket_dml_14
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{noformat}
was (Author: sershe):
Updated test list to fix/declare irrelevant before closing this
{noformat}
TestCliDriver:
stats_list_bucket
show_tablestatus
vector_udf2
list_bucket_dml_14
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{noformat}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff in partition "base file name" (path
> changes).
> 7) TestTxnCommands - all the conversion tests, as they check for bucket
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648980#comment-15648980
]
Sergey Shelukhin edited comment on HIVE-14990 at 11/15/16 3:28 AM:
---
Looked at all the remaining tests. Out of 749 failed tests, about 100 failures
and diffs are (or might be, at least) relevant.
Many of them are similar e.g. are missing stats, but I don't know if they are
missing stats for the same reason.
Many e.g. exim may be due to unsupported/path-dependant scenarios that were not
immediately obvious.
Not sure why TestSparkCliDriver fails. Fails in client init for me with no
useful logs (logs child process exited with 127, then times out).
I think we'll fix that during branch merge, if still broken.
Crossing out ones that are actually irrelevant
{panel}
TestCliDriver:
authorization_insert
create_default_prop
exim_04_evolved_parts
-exim_11_managed_external-
-exim_12_external_location-
-exim_15_external_part-
-exim_18_part_external-
-exim_19_00_part_external_location-
-exim_19_part_external_location-
insert1
-list_bucket_dml_8-
-mm_all-
orc_createas1
ppd_join4
stats_empty_dyn_part
stats_partscan_1_23
temp_table_display_colstats_tbllvl
-temp_table_options1-
vector_udf2
list_bucket_dml_14,list_bucket_*
llap_acid
insert_overwrite_directory2
authorization_load
autoColumnStats_9
create_like
drop_database_removes_partition_dirs
drop_table_removes_partition_dirs
index_auto_update
exim_01_nonpart,exim_02_part,-exim_04_all_part,exim_05_some_part-,exim_06_one_part,exim_20_part_managed_location
exim_16_part_external,exim_17_part_managed
load_overwrite
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_int_type_promotion
orc_vectorization_ppd
parquet_join2
-partition_wise_fileformat,partition_wise_fileformat3-
-repl_1_drop-,repl_3_exim_metadata
sample6
sample_islocalmode_hook
show_tablestatus
smb_bucket_1
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
stats_list_bucket
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Looked at all the remaining tests. Out of 749 failed tests, about 100 failures
and diffs are (or might be, at least) relevant.
Many of them are similar e.g. are missing stats, but I don't know if they are
missing stats for the same reason.
Many e.g. exim may be due to unsupported/path-dependant scenarios that were not
immediately obvious.
Not sure why TestSparkCliDriver fails. Fails in client init for me with no
useful logs (logs child process exited with 127, then times out).
I think we'll fix that during branch merge, if still broken.
Crossing out ones that are actually irrelevant
{panel}
TestCliDriver:
authorization_insert
create_default_prop
exim_04_evolved_parts
-exim_11_managed_external-
-exim_12_external_location-
-exim_15_external_part-
-exim_18_part_external-
-exim_19_00_part_external_location-
-exim_19_part_external_location-
insert1
-list_bucket_dml_8-
-mm_all-
orc_createas1
ppd_join4
stats_empty_dyn_part
stats_partscan_1_23
temp_table_display_colstats_tbllvl
-temp_table_options1-
vector_udf2
list_bucket_dml_14,list_bucket_*
llap_acid
insert_overwrite_directory2
authorization_load
autoColumnStats_9
create_like
drop_database_removes_partition_dirs
drop_table_removes_partition_dirs
index_auto_update
exim_01_nonpart,exim_02_part,-exim_04_all_part,exim_05_some_part-,exim_06_one_part,exim_20_part_managed_location
exim_16_part_external,exim_17_part_managed
load_overwrite
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_int_type_promotion
orc_vectorization_ppd
parquet_join2
partition_wise_fileformat,partition_wise_fileformat3
repl_1_drop,repl_3_exim_metadata
sample6
sample_islocalmode_hook
show_tablestatus
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648980#comment-15648980
]
Sergey Shelukhin edited comment on HIVE-14990 at 11/15/16 2:57 AM:
---
Looked at all the remaining tests. Out of 749 failed tests, about 100 failures
and diffs are (or might be, at least) relevant.
Many of them are similar e.g. are missing stats, but I don't know if they are
missing stats for the same reason.
Many e.g. exim may be due to unsupported/path-dependant scenarios that were not
immediately obvious.
Not sure why TestSparkCliDriver fails. Fails in client init for me with no
useful logs (logs child process exited with 127, then times out).
I think we'll fix that during branch merge, if still broken.
Crossing out ones that are actually irrelevant
{panel}
TestCliDriver:
authorization_insert
create_default_prop
exim_04_evolved_parts
-exim_11_managed_external-
-exim_12_external_location-
-exim_15_external_part-
-exim_18_part_external-
-exim_19_00_part_external_location-
-exim_19_part_external_location-
insert1
-list_bucket_dml_8-
-mm_all-
orc_createas1
ppd_join4
stats_empty_dyn_part
stats_partscan_1_23
temp_table_display_colstats_tbllvl
-temp_table_options1-
vector_udf2
list_bucket_dml_14,list_bucket_*
llap_acid
insert_overwrite_directory2
authorization_load
autoColumnStats_9
create_like
drop_database_removes_partition_dirs
drop_table_removes_partition_dirs
index_auto_update
exim_01_nonpart,exim_02_part,-exim_04_all_part,exim_05_some_part-,exim_06_one_part,exim_20_part_managed_location
exim_16_part_external,exim_17_part_managed
load_overwrite
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_int_type_promotion
orc_vectorization_ppd
parquet_join2
partition_wise_fileformat,partition_wise_fileformat3
repl_1_drop,repl_3_exim_metadata
sample6
sample_islocalmode_hook
show_tablestatus
smb_bucket_1
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
stats_list_bucket
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Looked at all the remaining tests. Out of 749 failed tests, about 100 failures
and diffs are (or might be, at least) relevant.
Many of them are similar e.g. are missing stats, but I don't know if they are
missing stats for the same reason.
Many e.g. exim may be due to unsupported/path-dependant scenarios that were not
immediately obvious.
Not sure why TestSparkCliDriver fails. Fails in client init for me with no
useful logs (logs child process exited with 127, then times out).
I think we'll fix that during branch merge, if still broken.
Crossing out ones that are actually irrelevant
{panel}
TestCliDriver:
authorization_insert
create_default_prop
exim_04_evolved_parts
-exim_11_managed_external-
-exim_12_external_location-
-exim_15_external_part-
-exim_18_part_external-
-exim_19_00_part_external_location-
-exim_19_part_external_location-
insert1
-list_bucket_dml_8-
-mm_all-
orc_createas1
ppd_join4
stats_empty_dyn_part
stats_partscan_1_23
temp_table_display_colstats_tbllvl
temp_table_options1
vector_udf2
list_bucket_dml_14,list_bucket_*
llap_acid
insert_overwrite_directory2
authorization_load
autoColumnStats_9
create_like
drop_database_removes_partition_dirs
drop_table_removes_partition_dirs
index_auto_update
exim_01_nonpart,exim_02_part,exim_04_all_part,exim_05_some_part,exim_06_one_part,exim_16_part_external,exim_17_part_managed,exim_20_part_managed_location
load_overwrite
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_int_type_promotion
orc_vectorization_ppd
parquet_join2
partition_wise_fileformat,partition_wise_fileformat3
repl_1_drop,repl_3_exim_metadata
sample6
sample_islocalmode_hook
show_tablestatus
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648980#comment-15648980
]
Sergey Shelukhin edited comment on HIVE-14990 at 11/10/16 1:58 AM:
---
Looked at all the remaining tests. Out of 749 failed tests, about 100 failures
and diffs are (or might be, at least) relevant.
Many of them are similar e.g. are missing stats, but I don't know if they are
missing stats for the same reason.
Many e.g. exim may be due to unsupported/path-dependant scenarios that were not
immediately obvious.
Not sure why TestSparkCliDriver fails. Fails in client init for me with no
useful logs (logs child process exited with 127, then times out).
I think we'll fix that during branch merge, if still broken.
Crossing out ones that are actually irrelevant
{panel}
TestCliDriver:
authorization_insert
create_default_prop
exim_04_evolved_parts
-exim_11_managed_external-
-exim_12_external_location-
-exim_15_external_part-
-exim_18_part_external-
-exim_19_00_part_external_location-
-exim_19_part_external_location-
insert1
list_bucket_dml_8
mm_all
orc_createas1
ppd_join4
stats_empty_dyn_part
stats_partscan_1_23
temp_table_display_colstats_tbllvl
temp_table_options1
vector_udf2
list_bucket_dml_14,list_bucket_*
llap_acid
insert_overwrite_directory2
authorization_load
autoColumnStats_9
create_like
drop_database_removes_partition_dirs
drop_table_removes_partition_dirs
index_auto_update
exim_01_nonpart,exim_02_part,exim_04_all_part,exim_05_some_part,exim_06_one_part,exim_16_part_external,exim_17_part_managed,exim_20_part_managed_location
load_overwrite
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_int_type_promotion
orc_vectorization_ppd
parquet_join2
partition_wise_fileformat,partition_wise_fileformat3
repl_1_drop,repl_3_exim_metadata
sample6
sample_islocalmode_hook
show_tablestatus
smb_bucket_1
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
stats_list_bucket
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Looked at all the remaining tests. Out of 749 failed tests, about 100 failures
and diffs are (or might be, at least) relevant.
Many of them are similar e.g. are missing stats, but I don't know if they are
missing stats for the same reason.
Many e.g. exim may be due to unsupported/path-dependant scenarios that were not
immediately obvious.
Not sure why TestSparkCliDriver fails. Fails in client init for me with no
useful logs (logs child process exited with 127, then times out).
I think we'll fix that during branch merge, if still broken.
{noformat}
TestCliDriver:
authorization_insert
create_default_prop
exim_04_evolved_parts
exim_11_managed_external
exim_12_external_location
exim_15_external_part
exim_18_part_external
exim_19_00_part_external_location
exim_19_part_external_location
insert1
list_bucket_dml_8
mm_all
orc_createas1
ppd_join4
stats_empty_dyn_part
stats_partscan_1_23
temp_table_display_colstats_tbllvl
temp_table_options1
vector_udf2
list_bucket_dml_14,list_bucket_*
llap_acid
insert_overwrite_directory2
authorization_load
autoColumnStats_9
create_like
drop_database_removes_partition_dirs
drop_table_removes_partition_dirs
index_auto_update
exim_01_nonpart,exim_02_part,exim_04_all_part,exim_05_some_part,exim_06_one_part,exim_16_part_external,exim_17_part_managed,exim_20_part_managed_location
load_overwrite
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_int_type_promotion
orc_vectorization_ppd
parquet_join2
partition_wise_fileformat,partition_wise_fileformat3
repl_1_drop,repl_3_exim_metadata
sample6
sample_islocalmode_hook
show_tablestatus
smb_bucket_1
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
stats_list_bucket
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[ https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631299#comment-15631299 ] Sergey Shelukhin edited comment on HIVE-14990 at 11/3/16 2:31 AM: -- Fixed a bunch more issues. The next one is that bucketmapjoin doesn't work. Overall the bucket usage that is not enforsible, like loading files one by one into bucketed table, won't work. It's already extremely fragile it seems. was (Author: sershe): Fixed a bunch more issues. The next one is that bucketmapjoin doesn't work. Overall the bucket usage that is not enforsible, like loading files one by one into bucketed table, won't work. > run all tests for MM tables and fix the issues that are found > - > > Key: HIVE-14990 > URL: https://issues.apache.org/jira/browse/HIVE-14990 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch, > HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch, > HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch, > HIVE-14990.patch > > > Expected failures (lack of support in MM tables for certain commands) > 1) All HCat tests > 2) Almost all merge tests (alter .. concat is not supported). > 3) Tests that run dfs commands with specific paths. > 4) Truncate column -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610209#comment-15610209
]
Sergey Shelukhin edited comment on HIVE-14990 at 10/27/16 1:00 AM:
---
Hmm, I cannot repro this, but the common failure appears to be
{noformat}
2016-10-26T14:05:40,126 ERROR [1a99f054-7f7d-499e-a675-53046ee84069 main]
CliDriver: Failed with exception java.io.IOException:java.io.IOException: Not a
file:
pfile:/home/hiveptest/104.154.45.228-hiveptest-0/apache-github-source-source/itests/qtest/target/warehouse/ctas_date_1/mm_0
java.io.IOException: java.io.IOException: Not a file:
pfile:/home/hiveptest/104.154.45.228-hiveptest-0/apache-github-source-source/itests/qtest/target/warehouse/ctas_date_1/mm_0
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:522)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:429)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:148)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:2233)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:253)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
at
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1371)
Caused by: java.io.IOException: Not a file:
pfile:/home/hiveptest/104.154.45.228-hiveptest-0/apache-github-source-source/itests/qtest/target/warehouse/ctas_date_1/mm_0
at
org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:322)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:373)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:305)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:460)
... 48 more
{noformat}
Looking...
was (Author: sershe):
Hmm, I cannot repro this, but the common failure appears to be
{noformat}
2016-10-26T14:05:40,126 ERROR [1a99f054-7f7d-499e-a675-53046ee84069 main]
CliDriver: Failed with exception java.io.IOException:java.io.IOException: Not a
file:
pfile:/home/hiveptest/104.154.45.228-hiveptest-0/apache-github-source-source/itests/qtest/target/warehouse/ctas_date_1/mm_0
java.io.IOException: java.io.IOException: Not a file:
pfile:/home/hiveptest/104.154.45.228-hiveptest-0/apache-github-source-source/itests/qtest/target/warehouse/ctas_date_1/mm_0
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:522)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:429)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:148)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:2233)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:253)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
at
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1371)
{noformat}
Looking...
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
