[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[ https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760965#comment-13760965 ] Hudson commented on HIVE-5182: -- FAILURE: Integrated in Hive-trunk-h0.21 #2316 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2316/]) HIVE-5182 : log more stuff via PerfLogger (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1520711) * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SortMergeJoinTaskDispatcher.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java log more stuff via PerfLogger - Key: HIVE-5182 URL: https://issues.apache.org/jira/browse/HIVE-5182 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.12.0 Attachments: HIVE-5182.D12639.1.patch, HIVE-5182.D12639.2.patch, HIVE-5182.D12639.3.patch PerfLogger output is useful in understanding perf. There are large gaps in it, however, and it's not clear what is going on during these. Some sections are large and have no breakdown. It would be nice to add more stuff. At this point I'm not certain where exactly, whoever makes the patch (me?) will just need to look at the above gaps and fill them in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[
https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760154#comment-13760154
]
Hive QA commented on HIVE-5182:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12601747/HIVE-5182.D12639.3.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 3084 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.pig.TestOrcHCatLoaderComplexSchema.testTupleInBagInTupleInBag
org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable
org.apache.hive.hcatalog.mapreduce.TestHCatExternalDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/643/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/643/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
log more stuff via PerfLogger
-
Key: HIVE-5182
URL: https://issues.apache.org/jira/browse/HIVE-5182
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HIVE-5182.D12639.1.patch, HIVE-5182.D12639.2.patch,
HIVE-5182.D12639.3.patch
PerfLogger output is useful in understanding perf. There are large gaps in
it, however, and it's not clear what is going on during these. Some sections
are large and have no breakdown. It would be nice to add more stuff. At this
point I'm not certain where exactly, whoever makes the patch (me?) will just
need to look at the above gaps and fill them in.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[ https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760258#comment-13760258 ] Phabricator commented on HIVE-5182: --- ashutoshc has accepted the revision HIVE-5182 [jira] log more stuff via PerfLogger. +1 REVISION DETAIL https://reviews.facebook.net/D12639 BRANCH HIVE-5182 ARCANIST PROJECT hive To: JIRA, ashutoshc, sershe log more stuff via PerfLogger - Key: HIVE-5182 URL: https://issues.apache.org/jira/browse/HIVE-5182 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-5182.D12639.1.patch, HIVE-5182.D12639.2.patch, HIVE-5182.D12639.3.patch PerfLogger output is useful in understanding perf. There are large gaps in it, however, and it's not clear what is going on during these. Some sections are large and have no breakdown. It would be nice to add more stuff. At this point I'm not certain where exactly, whoever makes the patch (me?) will just need to look at the above gaps and fill them in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[ https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760879#comment-13760879 ] Hudson commented on HIVE-5182: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #85 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/85/]) HIVE-5182 : log more stuff via PerfLogger (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1520711) * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SortMergeJoinTaskDispatcher.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java log more stuff via PerfLogger - Key: HIVE-5182 URL: https://issues.apache.org/jira/browse/HIVE-5182 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.12.0 Attachments: HIVE-5182.D12639.1.patch, HIVE-5182.D12639.2.patch, HIVE-5182.D12639.3.patch PerfLogger output is useful in understanding perf. There are large gaps in it, however, and it's not clear what is going on during these. Some sections are large and have no breakdown. It would be nice to add more stuff. At this point I'm not certain where exactly, whoever makes the patch (me?) will just need to look at the above gaps and fill them in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[ https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760926#comment-13760926 ] Hudson commented on HIVE-5182: -- FAILURE: Integrated in Hive-trunk-hadoop2 #413 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/413/]) HIVE-5182 : log more stuff via PerfLogger (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1520711) * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SortMergeJoinTaskDispatcher.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java log more stuff via PerfLogger - Key: HIVE-5182 URL: https://issues.apache.org/jira/browse/HIVE-5182 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.12.0 Attachments: HIVE-5182.D12639.1.patch, HIVE-5182.D12639.2.patch, HIVE-5182.D12639.3.patch PerfLogger output is useful in understanding perf. There are large gaps in it, however, and it's not clear what is going on during these. Some sections are large and have no breakdown. It would be nice to add more stuff. At this point I'm not certain where exactly, whoever makes the patch (me?) will just need to look at the above gaps and fill them in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[
https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13758993#comment-13758993
]
Hive QA commented on HIVE-5182:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12601422/HIVE-5182.D12639.2.patch
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 2909 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteRead
org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteRead
org.apache.hive.hcatalog.pig.TestOrcHCatStorer.testStoreTableMulti
org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testSequenceTableWriteReadMR
org.apache.hive.hcatalog.pig.TestOrcHCatStorer.testStoreBasicTable
org.apache.hive.hcatalog.mapreduce.TestHCatExternalDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask
org.apache.hive.hcatalog.pig.TestHCatStorerMulti.testStoreBasicTable
org.apache.hive.hcatalog.pig.TestHCatStorerMulti.testStorePartitionedTable
org.apache.hive.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable
org.apache.hive.hcatalog.mapreduce.TestSequenceFileReadWrite.testTextTableWriteReadMR
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/627/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/627/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
log more stuff via PerfLogger
-
Key: HIVE-5182
URL: https://issues.apache.org/jira/browse/HIVE-5182
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HIVE-5182.D12639.1.patch, HIVE-5182.D12639.2.patch
PerfLogger output is useful in understanding perf. There are large gaps in
it, however, and it's not clear what is going on during these. Some sections
are large and have no breakdown. It would be nice to add more stuff. At this
point I'm not certain where exactly, whoever makes the patch (me?) will just
need to look at the above gaps and fill them in.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[ https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13759484#comment-13759484 ] Ashutosh Chauhan commented on HIVE-5182: I will suggest to add it inside serialize/deserializeObject as well, since we have caches around these objects. So, if our code is working correctly, this should not be called more than once per process. Further, if we do see them getting printed multiple times, we know we got something to fix. log more stuff via PerfLogger - Key: HIVE-5182 URL: https://issues.apache.org/jira/browse/HIVE-5182 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-5182.D12639.1.patch, HIVE-5182.D12639.2.patch PerfLogger output is useful in understanding perf. There are large gaps in it, however, and it's not clear what is going on during these. Some sections are large and have no breakdown. It would be nice to add more stuff. At this point I'm not certain where exactly, whoever makes the patch (me?) will just need to look at the above gaps and fill them in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[ https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13757594#comment-13757594 ] Ashutosh Chauhan commented on HIVE-5182: Can you add it for * Utilities::getInputSummary() * CombineHiveInputFormat::getSplits() * HiveInputFormat::getSplits() * Utilities::serializeObject() * Utilities::deserializeObject() log more stuff via PerfLogger - Key: HIVE-5182 URL: https://issues.apache.org/jira/browse/HIVE-5182 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-5182.D12639.1.patch PerfLogger output is useful in understanding perf. There are large gaps in it, however, and it's not clear what is going on during these. Some sections are large and have no breakdown. It would be nice to add more stuff. At this point I'm not certain where exactly, whoever makes the patch (me?) will just need to look at the above gaps and fill them in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[ https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756813#comment-13756813 ] Sergey Shelukhin commented on HIVE-5182: This test is known flaky test, seems to be unrelated log more stuff via PerfLogger - Key: HIVE-5182 URL: https://issues.apache.org/jira/browse/HIVE-5182 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-5182.D12639.1.patch PerfLogger output is useful in understanding perf. There are large gaps in it, however, and it's not clear what is going on during these. Some sections are large and have no breakdown. It would be nice to add more stuff. At this point I'm not certain where exactly, whoever makes the patch (me?) will just need to look at the above gaps and fill them in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5182) log more stuff via PerfLogger
[
https://issues.apache.org/jira/browse/HIVE-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755458#comment-13755458
]
Hive QA commented on HIVE-5182:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12600906/HIVE-5182.D12639.1.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 2902 tests executed
*Failed tests:*
{noformat}
org.apache.hcatalog.mapreduce.TestHCatExternalDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/584/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/584/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
log more stuff via PerfLogger
-
Key: HIVE-5182
URL: https://issues.apache.org/jira/browse/HIVE-5182
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HIVE-5182.D12639.1.patch
PerfLogger output is useful in understanding perf. There are large gaps in
it, however, and it's not clear what is going on during these. Some sections
are large and have no breakdown. It would be nice to add more stuff. At this
point I'm not certain where exactly, whoever makes the patch (me?) will just
need to look at the above gaps and fill them in.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
