[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307803#comment-16307803 ] Peter Vary commented on HIVE-14146: --- [~trinevo]: This is fixed only in 2.3.0 and not backported to 1.x line, since the solution depends on HIVE-14013 which fix is 2.3.0 only too... > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Fix For: 2.3.0 > > Attachments: HIVE-14146.10.patch, HIVE-14146.11.patch, > HIVE-14146.2.patch, HIVE-14146.3.patch, HIVE-14146.4.patch, > HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.7.patch, > HIVE-14146.8.patch, HIVE-14146.9.patch, HIVE-14146.patch, changes > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16300051#comment-16300051 ] Marcin Łuczyński commented on HIVE-14146: - I got a customer with Hive 1.2.1.2.4 who has the same problem. Was this fix backported into 1.x stream? > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Fix For: 2.3.0 > > Attachments: HIVE-14146.10.patch, HIVE-14146.11.patch, > HIVE-14146.2.patch, HIVE-14146.3.patch, HIVE-14146.4.patch, > HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.7.patch, > HIVE-14146.8.patch, HIVE-14146.9.patch, HIVE-14146.patch, changes > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549184#comment-15549184 ] Peter Vary commented on HIVE-14146: --- [~aihuaxu] I have checked those lines you mentioned. The difference between the new and the original lines is that in the new version there are whitespaces at the end of every cell to make the width of the last column the same as every other column. The new line was there before the patch. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.10.patch, HIVE-14146.11.patch, > HIVE-14146.2.patch, HIVE-14146.3.patch, HIVE-14146.4.patch, > HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.7.patch, > HIVE-14146.8.patch, HIVE-14146.9.patch, HIVE-14146.patch, changes > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546066#comment-15546066 ] Aihua Xu commented on HIVE-14146: - [~pvary] I was about to commit the patch. But I noticed that in some files such as alter_table_invalidate_column_stats.q.out, seems we append some empty new line after {{# col_name data_type comment }}. Is that expected? Can you double check that part? If it's good, I will commit it later. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.10.patch, HIVE-14146.11.patch, > HIVE-14146.2.patch, HIVE-14146.3.patch, HIVE-14146.4.patch, > HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.7.patch, > HIVE-14146.8.patch, HIVE-14146.9.patch, HIVE-14146.patch, changes > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533280#comment-15533280 ] Aihua Xu commented on HIVE-14146: - +1. The patch looks good to me. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.10.patch, HIVE-14146.11.patch, > HIVE-14146.2.patch, HIVE-14146.3.patch, HIVE-14146.4.patch, > HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.7.patch, > HIVE-14146.8.patch, HIVE-14146.9.patch, HIVE-14146.patch, changes > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526568#comment-15526568 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12830464/HIVE-14146.11.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10645 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin] org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char] org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testMetaDataCounts org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.testDelegationTokenSharedStore org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching org.apache.hive.spark.client.TestSparkClient.testJobSubmission {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1314/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1314/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1314/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12830464 - PreCommit-HIVE-Build > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.10.patch, HIVE-14146.11.patch, > HIVE-14146.2.patch, HIVE-14146.3.patch, HIVE-14146.4.patch, > HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.7.patch, > HIVE-14146.8.patch, HIVE-14146.9.patch, HIVE-14146.patch, changes > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523506#comment-15523506 ] Aihua Xu commented on HIVE-14146: - [~pvary] Do you have the latest patch for reviewing? Can you attach a new one to run the precommit test first? > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.10.patch, HIVE-14146.2.patch, > HIVE-14146.3.patch, HIVE-14146.4.patch, HIVE-14146.5.patch, > HIVE-14146.6.patch, HIVE-14146.7.patch, HIVE-14146.8.patch, > HIVE-14146.9.patch, HIVE-14146.patch, changes > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429068#comment-15429068 ] Peter Vary commented on HIVE-14146: --- It might be confusing, but here is what the patch changes: describe escape_comments_tbl1; {noformat:title=Beeline original} +--+---+---+--+ | col_name | data_type |comment| +--+---+---+--+ | col1 | string| a | | b| NULL | NULL | | p1 | string| a | | b| NULL | NULL | | | NULL | NULL | | # Partition Information | NULL | NULL | | # col_name | data_type | comment | | | NULL | NULL | | p1 | string| a | | b| NULL | NULL | +--+---+---+--+ {noformat} {noformat:title=Beeline patch} +--+---+---+--+ | col_name | data_type |comment| +--+---+---+--+ | col1 | string| a\nb | | p1 | string| a\nb | | | NULL | NULL | | # Partition Information | NULL | NULL | | # col_name | data_type | comment | | | NULL | NULL | | p1 | string| a\nb | +--+---+---+--+ {noformat} {noformat:title=Cli original} col1string a b p1 string a b # Partition Information # col_name data_type comment p1 string a b {noformat} {noformat:title=Cli patch} col1string a b p1 string a b # Partition Information # col_name data_type comment p1 string a b {noformat} describe formatted escape_comments_tbl1; {noformat:title=Beeline original} +---++---+--+ | col_name| data_type |comment| +---++---+--+ | # col_name| data_type | comment | | | NULL | NULL | | col1 | string | a | | b | NULL | NULL | | | NULL | NULL | | # Partition Information | NULL | NULL | | # col_name| data_type | comment | | | NULL | NULL | | p1| string | a | | b | NULL | NULL
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406674#comment-15406674 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12821848/HIVE-14146.9.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 10433 tests executed *Failed tests:* {noformat} TestMsgBusConnection - did not produce a TEST-*.xml file TestQueryLifeTimeHook - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_concatenate_indexed_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_invalidate_column_stats org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_part_coltype org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_with_constraints org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auth org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_indexes_edge_cases org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_indexes_syntax org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_concatenate_indexed_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_view_as_select_with_partition org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.stringifyValidTxns org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.testTxnRange org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/753/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/753/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-753/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 16 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12821848 - PreCommit-HIVE-MASTER-Build > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, > HIVE-14146.7.patch, HIVE-14146.8.patch, HIVE-14146.9.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404997#comment-15404997 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12821645/HIVE-14146.8.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10428 tests executed *Failed tests:* {noformat} TestMsgBusConnection - did not produce a TEST-*.xml file TestQueryLifeTimeHook - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_view_as_select org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_like org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view_translate org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_describe_comment_indent org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_view_as_select_with_partition org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.stringifyValidTxns org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.testTxnRange org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/741/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/741/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-741/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12821645 - PreCommit-HIVE-MASTER-Build > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, > HIVE-14146.7.patch, HIVE-14146.8.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403267#comment-15403267 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12821327/HIVE-14146.7.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10420 tests executed *Failed tests:* {noformat} TestMsgBusConnection - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_nullable_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_avro_non_nullable_union org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.stringifyValidTxns org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.testTxnRange {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/728/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/728/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-728/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 9 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12821327 - PreCommit-HIVE-MASTER-Build > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, > HIVE-14146.7.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402337#comment-15402337 ] Aihua Xu commented on HIVE-14146: - I see. Sure. Seems reasonable to make such change. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, > HIVE-14146.7.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402245#comment-15402245 ] Peter Vary commented on HIVE-14146: --- I rebased and run the qtests. It seems ok now - we will see the result not too soon :) What I am not sure about, how to handle the CLI/BeeLine differences here. CLI uses formatted outputting, which specifically displays comments with \n in new lines, but only in column comments, and not in table comments. Probably the nicest solution would be to keep the \n-s as newlines in CLI mode in table comments as well, so at least in CLI every newline would remain a newline, and in BeeLine every newline would be a \n. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, > HIVE-14146.7.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402216#comment-15402216 ] Aihua Xu commented on HIVE-14146: - [~pvary] Yeah. You may need to rebase to the latest change. Do you need to generate the new baseline for your new test case? > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, > HIVE-14146.7.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15401055#comment-15401055 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12820998/HIVE-14146.6.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/706/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/706/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-706/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n /usr/java/jdk1.8.0_25 ]] + export JAVA_HOME=/usr/java/jdk1.8.0_25 + JAVA_HOME=/usr/java/jdk1.8.0_25 + export PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-MASTER-Build-706/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 4f1cd26 HIVE-14363: bucketmap inner join query fails due to NullPointerException in some cases (Hari Subramaniyan, reviewed by Matt McCline) + git clean -f -d Removing ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFStringToMap.java + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 4f1cd26 HIVE-14363: bucketmap inner join query fails due to NullPointerException in some cases (Hari Subramaniyan, reviewed by Matt McCline) + git merge --ff-only origin/master Already up-to-date. + git gc + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0, p1, or p2 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12820998 - PreCommit-HIVE-MASTER-Build > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400036#comment-15400036 ] Peter Vary commented on HIVE-14146: --- [~aihuaxu] Checked, and found the following: - Beeline - when using "describe pretty" the \n is padded and displayed - this is as intended - the original behavior is not changed - Cli - when using "describe pretty" the \n is padded and displayed - this is as intended - the original behavior is not changed What I have missed, that in Cli when using "describe formatted" the following code sets the output to padded: {noformat} package org.apache.hadoop.hive.ql.exec; [..] public class DDLTask extends Task implements Serializable { [..] private int describeTable(Hive db, DescTableDesc descTbl) throws HiveException { [..] // In case the query is served by HiveServer2, don't pad it with spaces, // as HiveServer2 output is consumed by JDBC/ODBC clients. boolean isOutputPadded = !SessionState.get().isHiveServerQuery(); {noformat} So the column comments are printed as a formatted strings with new lines - but not the table comment. (same as in pretty mode) What do you think we should do here? I personally would prefer to remove this code below (pretty comment formatting), so every comment should be handled in the same way (printing \n instead of newline): {noformat} package org.apache.hadoop.hive.ql.metadata.formatting; [..] public final class MetaDataFormatUtils { [..] private static void formatWithIndentation(String colName, String colType, String colComment, [..] // comment indent processing for multi-line comments // comments should be indented the same amount on each line // if the first line comment starts indented by k, // the following line comments should also be indented by k String[] commentSegments = colComment.split("\n|\r|\r\n"); tableInfo.append(String.format("%-" + ALIGNMENT + "s", commentSegments[0])).append(LINE_DELIM); int colNameLength = ALIGNMENT > colName.length() ? ALIGNMENT : colName.length(); int colTypeLength = ALIGNMENT > colType.length() ? ALIGNMENT : colType.length(); for (int i = 1; i < commentSegments.length; i++) { tableInfo.append(String.format("%" + colNameLength + "s" + FIELD_DELIM + "%" + colTypeLength + "s" + FIELD_DELIM + "%s", "", "", commentSegments[i])).append(LINE_DELIM); } {noformat} > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399610#comment-15399610 ] Aihua Xu commented on HIVE-14146: - >From the test case result. seems we still have some comments not escaped. >[~pvary] Can you check? The one in code review seems to be correct. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399532#comment-15399532 ] Sergio Peña commented on HIVE-14146: Latest patch looks good. +1 Wait for HiveQa before commit. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370453#comment-15370453 ] Peter Vary commented on HIVE-14146: --- Review request created: https://reviews.apache.org/r/49882/ > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370422#comment-15370422 ] Peter Vary commented on HIVE-14146: --- Checked the test results and these are not related. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368241#comment-15368241 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12816835/HIVE-14146.5.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10293 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables_compact org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit org.apache.hadoop.hive.cli.TestMinimrCliDriver.org.apache.hadoop.hive.cli.TestMinimrCliDriver {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/428/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/428/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-428/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12816835 - PreCommit-HIVE-MASTER-Build > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, > HIVE-14146.4.patch, HIVE-14146.5.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367757#comment-15367757 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12816825/HIVE-14146.4.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/426/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/426/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-426/ Messages: {noformat} This message was trimmed, see log for full details [copy] Copying 15 files to /data/hive-ptest/working/apache-github-source-source/llap-tez/target/tmp/conf [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-llap-tez --- [INFO] Compiling 2 source files to /data/hive-ptest/working/apache-github-source-source/llap-tez/target/test-classes [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java: /data/hive-ptest/working/apache-github-source-source/llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java uses or overrides a deprecated API. [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java: Recompile with -Xlint:deprecation for details. [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java: /data/hive-ptest/working/apache-github-source-source/llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java uses unchecked or unsafe operations. [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-llap-tez --- [INFO] Tests are skipped. [INFO] [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-llap-tez --- [INFO] Building jar: /data/hive-ptest/working/apache-github-source-source/llap-tez/target/hive-llap-tez-2.2.0-SNAPSHOT.jar [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-llap-tez --- [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-llap-tez --- [INFO] Installing /data/hive-ptest/working/apache-github-source-source/llap-tez/target/hive-llap-tez-2.2.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-llap-tez/2.2.0-SNAPSHOT/hive-llap-tez-2.2.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-github-source-source/llap-tez/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-llap-tez/2.2.0-SNAPSHOT/hive-llap-tez-2.2.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Spark Remote Client 2.2.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ spark-client --- [INFO] Deleting /data/hive-ptest/working/apache-github-source-source/spark-client/target [INFO] Deleting /data/hive-ptest/working/apache-github-source-source/spark-client (includes = [datanucleus.log, derby.log], excludes = []) [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ spark-client --- [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ spark-client --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ spark-client --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-github-source-source/spark-client/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ spark-client --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ spark-client --- [INFO] Compiling 28 source files to /data/hive-ptest/working/apache-github-source-source/spark-client/target/classes [WARNING] /data/hive-ptest/working/apache-github-source-source/spark-client/src/main/java/org/apache/hive/spark/client/SparkClientUtilities.java: /data/hive-ptest/working/apache-github-source-source/spark-client/src/main/java/org/apache/hive/spark/client/SparkClientUtilities.java uses or overrides a deprecated API. [WARNING]
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366154#comment-15366154 ] Aihua Xu commented on HIVE-14146: - [~pvary] I notice that we are calling HiveStringUtils.esapeJava(escapeHiveCommand(string)). Does it make sense to move HiveStringUtils.esapeJava() call inside escapeHiveCommand() so it will control escaping characters in one place? > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.2.patch, HIVE-14146.3.patch, HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364329#comment-15364329 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12816391/HIVE-14146.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 42 failed/errored test(s), 10003 tests executed *Failed tests:* {noformat} TestCliDriver-bool_literal.q-authorization_cli_createtab.q-explain_ddl.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-cbo_rp_join1.q-union_top_level.q-insert_update_delete.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-create_func1.q-bucketmapjoin3.q-enforce_order.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-describe_xpath.q-autogen_colalias.q-udf_named_struct.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-encryption_join_with_different_encryption_keys.q-bucketcontext_3.q-udf_smallint.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-groupby4.q-convert_enum_to_string.q-mapjoin_filter_on_outerjoin.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-index_compact.q-merge_dynamic_partition2.q-cbo_rp_subq_exists.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-nullscript.q-vector_char_mapjoin1.q-load_dyn_part3.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-parquet_ppd_decimal.q-cluster.q-groupby_sort_6.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-ptf_general_queries.q-unionDistinct_1.q-udf_version.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-sample_islocalmode_hook_use_metadata.q-cbo_rp_semijoin.q-udf_when.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-schema_evol_text_vec_mapwork_part_all_complex.q-metadataonly1.q-deleteAnalyze.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-skewjoinopt3.q-rcfile_merge1.q-multigroupby_singlemr.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-stats_publisher_error_1.q-auto_join1.q-cast_to_int.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-tez_joins_explain.q-rename_column.q-varchar_serde.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-tez_smb_empty.q-char_2.q-udf_date_sub.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-udf_to_float.q-decimal_precision2.q-ppd_gby_join.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-unicode_notation.q-gen_udf_example_add10.q-ppd_join4.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-vector_complex_join.q-interval_udf.q-udf_classloader_dynamic_dependency_resolution.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-vector_partition_diff_num_cols.q-stats2.q-union11.q-and-12-more - did not produce a TEST-*.xml file TestMinimrCliDriver-index_bitmap3.q-bucketizedhiveinputformat.q-orc_mr_pathalias.q-and-1-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join0 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_sortmerge_join_14 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cross_join org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_fetchwork_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_aggregate_9 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_coalesce_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_count_distinct org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_precision org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_round org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_multi_insert org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_reduce1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_varchar_simple org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_16 org.apache.hive.spark.client.TestSparkClient.testJobSubmission {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/381/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/381/console Test
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363417#comment-15363417 ] Hive QA commented on HIVE-14146: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12816201/HIVE-14146.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10281 tests executed *Failed tests:* {noformat} TestMiniTezCliDriver-vector_non_string_partition.q-delete_where_non_partitioned.q-auto_sortmerge_join_16.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_concatenate_indexed_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auth org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_indexes_edge_cases org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_indexes_syntax org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_concatenate_indexed_table {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/373/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/373/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-373/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 12 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12816201 - PreCommit-HIVE-MASTER-Build > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362693#comment-15362693 ] Peter Vary commented on HIVE-14146: --- Good catch [~aihuaxu]. I realised that myself, and in the process to updating the patch :) Thanks anyway!!! > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362648#comment-15362648 ] Aihua Xu commented on HIVE-14146: - [~pvary] Seems "show create table" output doesn't escape the "\n" in the test. Is that expected? > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > Attachments: HIVE-14146.patch > > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362599#comment-15362599 ] Peter Vary commented on HIVE-14146: --- Thanks [~niklaus.xiao]. For human use, your solution is perfect, but we had to parse the response by code. My example above is clearly a bug, which should be solved. My proposed solution is to print out an escaped \n for this scenario. > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362446#comment-15362446 ] niklaus xiao commented on HIVE-14146: - You can try this: {code} desc pretty commtest; +--++--+ | col_name | data_type | comment | +--++--+--+ | col_name data_type comment | NULL | NULL | | | NULL | NULL | | first_nm stringIndicates First name | NULL | NULL | |of an individual | NULL | NULL | +--++--+ {code} > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[ https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358959#comment-15358959 ] Peter Vary commented on HIVE-14146: --- Solution depends on the HIVE-14013 patch, since I think it would be a good idea, to have the unicode parameters displayed correctly as well > Column comments with "\n" character "corrupts" table metadata > - > > Key: HIVE-14146 > URL: https://issues.apache.org/jira/browse/HIVE-14146 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary > > Create a table with the following(noting the \n in the COMMENT): > {noformat} > CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an > individual’); > {noformat} > Describe shows that now the metadata is messed up: > {noformat} > beeline> describe commtest; > +---++---+--+ > | col_name | data_type |comment| > +---++---+--+ > | first_nm | string | Indicates First name | > | of an individual | NULL | NULL | > +---++---+--+ > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)