[jira] [Commented] (HIVE-5234) partition name filtering uses suboptimal datastructures
[ https://issues.apache.org/jira/browse/HIVE-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763240#comment-13763240 ] Hudson commented on HIVE-5234: -- FAILURE: Integrated in Hive-trunk-h0.21 #2322 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2322/]) HIVE-5234 : partition name filtering uses suboptimal datastructures (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1521158) * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java partition name filtering uses suboptimal datastructures --- Key: HIVE-5234 URL: https://issues.apache.org/jira/browse/HIVE-5234 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: HIVE-5234.D12777.1.patch Some DSes used in name-based partition filtering, as well as related methods, are suboptimal, which can cost 100-s of ms on large number of partitions. I noticed while perf testing HIVE-4914, but it can also be applied separately given that the patch over there will take some time to get in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5234) partition name filtering uses suboptimal datastructures
[ https://issues.apache.org/jira/browse/HIVE-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763253#comment-13763253 ] Hudson commented on HIVE-5234: -- ABORTED: Integrated in Hive-trunk-hadoop2 #419 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/419/]) HIVE-5234 : partition name filtering uses suboptimal datastructures (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1521158) * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java partition name filtering uses suboptimal datastructures --- Key: HIVE-5234 URL: https://issues.apache.org/jira/browse/HIVE-5234 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: HIVE-5234.D12777.1.patch Some DSes used in name-based partition filtering, as well as related methods, are suboptimal, which can cost 100-s of ms on large number of partitions. I noticed while perf testing HIVE-4914, but it can also be applied separately given that the patch over there will take some time to get in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5234) partition name filtering uses suboptimal datastructures
[ https://issues.apache.org/jira/browse/HIVE-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761983#comment-13761983 ] Hudson commented on HIVE-5234: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #89 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/89/]) HIVE-5234 : partition name filtering uses suboptimal datastructures (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1521158) * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java partition name filtering uses suboptimal datastructures --- Key: HIVE-5234 URL: https://issues.apache.org/jira/browse/HIVE-5234 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: HIVE-5234.D12777.1.patch Some DSes used in name-based partition filtering, as well as related methods, are suboptimal, which can cost 100-s of ms on large number of partitions. I noticed while perf testing HIVE-4914, but it can also be applied separately given that the patch over there will take some time to get in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5234) partition name filtering uses suboptimal datastructures
[ https://issues.apache.org/jira/browse/HIVE-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762001#comment-13762001 ] Hudson commented on HIVE-5234: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #157 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/157/]) HIVE-5234 : partition name filtering uses suboptimal datastructures (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1521158) * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java partition name filtering uses suboptimal datastructures --- Key: HIVE-5234 URL: https://issues.apache.org/jira/browse/HIVE-5234 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.13.0 Attachments: HIVE-5234.D12777.1.patch Some DSes used in name-based partition filtering, as well as related methods, are suboptimal, which can cost 100-s of ms on large number of partitions. I noticed while perf testing HIVE-4914, but it can also be applied separately given that the patch over there will take some time to get in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5234) partition name filtering uses suboptimal datastructures
[
https://issues.apache.org/jira/browse/HIVE-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760213#comment-13760213
]
Hive QA commented on HIVE-5234:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12601751/HIVE-5234.D12777.1.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 3084 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.fileformats.TestOrcDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask
org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteRead
org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteRead
org.apache.hive.hcatalog.pig.TestOrcHCatStorer.testStoreTableMulti
org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteReadMR
org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteReadMR
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/644/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/644/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
partition name filtering uses suboptimal datastructures
---
Key: HIVE-5234
URL: https://issues.apache.org/jira/browse/HIVE-5234
Project: Hive
Issue Type: Improvement
Components: Metastore
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HIVE-5234.D12777.1.patch
Some DSes used in name-based partition filtering, as well as related methods,
are suboptimal, which can cost 100-s of ms on large number of partitions. I
noticed while perf testing HIVE-4914, but it can also be applied separately
given that the patch over there will take some time to get in.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
