[jira] [Commented] (HIVE-13057) Remove duplicate copies of TableDesc property values in PartitionDesc

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154649#comment-15154649
 ] 

Xuefu Zhang commented on HIVE-13057:


+1 LGTM

> Remove duplicate copies of TableDesc property values in PartitionDesc
> -
>
> Key: HIVE-13057
> URL: https://issues.apache.org/jira/browse/HIVE-13057
> Project: Hive
>  Issue Type: Bug
>Reporter: Mohit Sabharwal
>Assignee: Mohit Sabharwal
> Attachments: HIVE-13057.patch
>
>
> For a partitioned table, each PartitionDesc has a copy of corresponding 
> TableDesc.
> While TableDesc is mutable and hence cannot be interned, it's property values 
> can be.
> For a simple select on a table with 100K partitions, this cut total number of 
> String instances by ~65%.
> Most replicated strings were location, serde, input/output format, column, 
> types, table name, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13057) Remove duplicate copies of TableDesc property values in PartitionDesc

[Mohit Sabharwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149543#comment-15149543
 ] 

Mohit Sabharwal commented on HIVE-13057:


Test failures are unrelated. (Also occur in unrelated runs like: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6976/#showFailuresLink)

> Remove duplicate copies of TableDesc property values in PartitionDesc
> -
>
> Key: HIVE-13057
> URL: https://issues.apache.org/jira/browse/HIVE-13057
> Project: Hive
>  Issue Type: Bug
>Reporter: Mohit Sabharwal
>Assignee: Mohit Sabharwal
> Attachments: HIVE-13057.patch
>
>
> For a partitioned table, each PartitionDesc has a copy of corresponding 
> TableDesc.
> While TableDesc is mutable and hence cannot be interned, it's property values 
> can be.
> For a simple select on a table with 100K partitions, this cut total number of 
> String instances by ~65%.
> Most replicated strings were location, serde, input/output format, column, 
> types, table name, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13057) Remove duplicate copies of TableDesc property values in PartitionDesc

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147281#comment-15147281
 ] 

Hive QA commented on HIVE-13057:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787776/HIVE-13057.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 9803 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6989/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6989/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6989/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12787776 - PreCommit-HIVE-TRUNK-Build

> Remove duplicate copies of TableDesc property values in PartitionDesc
> -
>
> Key: HIVE-13057
> URL: https://issues.apache.org/jira/browse/HIVE-13057
> Project: Hive
>  Issue Type: Bug
>Reporter: Mohit Sabharwal
>Assignee: Mohit Sabharwal
> Attachments: HIVE-13057.patch
>
>
> For a partitioned table, each PartitionDesc has a copy of corresponding 
> TableDesc.
> While TableDesc is mutable and hence cannot be interned, it's property values 
> can be.
> For a simple select on a table with 100K partitions, this cut total number of 
> String instances by ~65%.
> Most replicated strings were location, serde, input/output format, column, 
> types, table name, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)