[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119036#comment-16119036 ] Aihua Xu commented on HIVE-14013: - [~hzfeng] Sorry for the late reply. I didn't do any configuration change. What I did is just build hive and create a table with unicode in the comment. > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 2.3.0 > > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108625#comment-16108625 ] hzfeng commented on HIVE-14013: --- Thanks a lot Is it convenient for guiding me how you configured hive-2.3.0? I will be appreciate. > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 2.3.0 > > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107963#comment-16107963 ] Aihua Xu commented on HIVE-14013: - [~fenghaizhu] I can't see your attachment. But I tried simple test and it works fine. {noformat} hive> desc unicode_comments_tbl1; OK col1string 第一列 p1 string 分割 # Partition Information # col_name data_type comment p1 string 分割 {noformat} > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 2.3.0 > > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102958#comment-16102958 ] fenghaizhu commented on HIVE-14013: --- The part marked as red should be Chinese characters. [image: Inline image 1] > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 2.3.0 > > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102953#comment-16102953 ] fenghaizhu commented on HIVE-14013: --- hi Aihua Thanks for your replying; I downloaded the tar ball from http://ftp.cuhk.edu.hk/pub/packages/apache.org/hive/hive-2.3.0/apache-hive-2.3.0-src.tar.gz , extracted and configured it with hadoop-2.7.3. Then I started interacted hive and created a table with comments in Chinese. When I showed create table or desc table, I found the comments displayed as . My hive-site.xml is as bellow: please help to check , I am fresh with hive. = javax.jdo.option.ConnectionURL jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionDriverName com.mysql.jdbc.Driver Driver class name for a JDBC metastore javax.jdo.option.ConnectionUserName hive username to use against metastore database javax.jdo.option.ConnectionPassword hive password to use against metastore database datanucleus.autoCreateSchema true datanucleus.autoCreateTables true datanucleus.autoCreateColumns true hive.metastore.warehouse.dir /hive location of default database for the warehouse hive.downloaded.resources.dir /export/servers/software/hive/tmp/resources Temporary local directory for added resources in the remote file system. hive.exec.dynamic.partition true hive.exec.dynamic.partition.mode nonstrict hive.exec.local.scratchdir /export/servers/software/hive/tmp/HiveJobsLog Local scratch space for Hive jobs hive.querylog.location /export/servers/software/hive/tmp/HiveRunLog Location of Hive run time structured log file hive.server2.logging.operation.log.location /export/servers/software/hive/tmp/OpertitionLog Top level directory where operation tmp are stored if logging functionality is enabled hive.server2.thrift.bind.host 10.186.207.180 hive.server2.thrift.port 1 hive.server2.thrift.http.port 10001 hive.server2.thrift.http.path cliservice hive.server2.webui.host 10.186.207.180 hive.server2.webui.port 10002 hive.scratch.dir.permission 755 hive.server2.enable.doAs false hive.auto.convert.join false spark.dynamicAllocation.enabled true 动态分配资源 spark.driver.extraJavaOptions -XX:PermSize=128M -XX:MaxPermSize=512M > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 2.3.0 > > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100183#comment-16100183 ] Aihua Xu commented on HIVE-14013: - [~hzfeng] Can you share the repro steps? Meantime I will try out a simple case. > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 2.3.0 > > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099771#comment-16099771 ] hzfeng commented on HIVE-14013: --- Hi Aihua According to the status of this bug I think this bug has been resolved in version 2.3.0. However, I downloaded and installed the hive-2.3.0-bin which is released on July 19th 2017, I still hit the problem that 'Describe table doesn't show unicode properly for Chinese comments'. Would you guide me how to check and resolve it ? Expect your reply. > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 2.3.0 > > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353410#comment-15353410 ] Aihua Xu commented on HIVE-14013: - The tests are not related. Pushed to master. Thanks Yongzhi for reviewing. > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 2.2.0 > > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352179#comment-15352179 ] Hive QA commented on HIVE-14013: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12813823/HIVE-14013.4.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10274 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/279/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/279/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-279/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12813823 - PreCommit-HIVE-MASTER-Build > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351762#comment-15351762 ] Yongzhi Chen commented on HIVE-14013: - patch-4 LGTM, +1 pending on tests > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch, HIVE-14013.4.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351613#comment-15351613 ] Yongzhi Chen commented on HIVE-14013: - Should you do something similar to what it does in ? {noformat} /** * Translator object for escaping Java. * * While {@link #escapeJava(String)} is the expected method of use, this * object allows the Java escaping functionality to be used * as the foundation for a custom translator. * * @since 3.0 */ public static final CharSequenceTranslator ESCAPE_JAVA = new LookupTranslator( new String[][] { {"\"", "\\\""}, {"\\", ""}, }).with( new LookupTranslator(EntityArrays.JAVA_CTRL_CHARS_ESCAPE()) ).with( UnicodeEscaper.outsideOf(32, 0x7f) ); {noformat} Just remove last .with(..) ? > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15341697#comment-15341697 ] Aihua Xu commented on HIVE-14013: - The test failures are not related. [~ychena] Can you help review the code? > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338165#comment-15338165 ] Hive QA commented on HIVE-14013: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12811388/HIVE-14013.3.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10236 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_repair org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_table_nonprintable org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testPartitionsCheck org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testTableCheck {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/168/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/168/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-168/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12811388 - PreCommit-HIVE-MASTER-Build > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch, > HIVE-14013.3.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335345#comment-15335345 ] Hive QA commented on HIVE-14013: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12811140/HIVE-14013.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10234 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_repair org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unicode_notation org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_table_nonprintable org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testPartitionsCheck org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testTableCheck {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/145/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/145/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-145/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12811140 - PreCommit-HIVE-MASTER-Build > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334074#comment-15334074 ] Aihua Xu commented on HIVE-14013: - Talked to Peter offline. He is investigating escape-specific issues for comments. Will file a separate jira for that since this jira is to handle unicode issue. > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334071#comment-15334071 ] Aihua Xu commented on HIVE-14013: - Attached patch-2: added unicode support for index comment. > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch, HIVE-14013.2.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15333924#comment-15333924 ] Aihua Xu commented on HIVE-14013: - Thanks Peter for reviewing and suggestion, but we don't want to escape internally while only when we are displaying to UI. I see that I may miss one index comment. I will fix that. > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14013) Describe table doesn't show unicode properly
[ https://issues.apache.org/jira/browse/HIVE-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331761#comment-15331761 ] Peter Vary commented on HIVE-14013: --- Hi, I have stumbled upon your patch, and I think it would be useful to apply your escaping to every comment field, like: +++ ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java (revision ) @@ -231,7 +233,7 @@ appendColumnStatsNoFormatting(colBuffer, "", "", "", "", "", "", "", ""); } } -colBuffer.append(comment == null ? "" : comment); +colBuffer.append(comment == null ? "" : HiveStringUtils.escapeJava(comment)); colBuffer.append(LINE_DELIM); } @@ -266,7 +268,7 @@ IndexType indexType = HiveIndex.getIndexTypeByClassName(indexHandlerClass); indexColumns.add(indexType.getName()); -indexColumns.add(index.getParameters().get("comment")); + indexColumns.add(HiveStringUtils.escapeJava(index.getParameters().get("comment"))); formatOutput(indexColumns.toArray(new String[0]), indexInfo); Regards, Peter > Describe table doesn't show unicode properly > > > Key: HIVE-14013 > URL: https://issues.apache.org/jira/browse/HIVE-14013 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14013.1.patch > > > Describe table output will show comments incorrectly rather than the unicode > itself. > {noformat} > hive> desc formatted t1; > # Detailed Table Information > Table Type: MANAGED_TABLE > Table Parameters: > COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} > comment \u8868\u4E2D\u6587\u6D4B\u8BD5 > numFiles0 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)