[
https://issues.apache.org/jira/browse/HIVE-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473752#comment-16473752
]
Vihang Karajgaonkar commented on HIVE-19514:
--------------------------------------------
Ptest server now only copies successful logs. This inprovement shaves off 10
min from the total execution time.
{noformat}
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase
took 64 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF:
ExecutionPhase.TotalRsyncElapsedTime took 8 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took
5 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase
took 0 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase
took 0 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase
took 0 minutes
{noformat}
> Successful test logs should be copied
> -------------------------------------
>
> Key: HIVE-19514
> URL: https://issues.apache.org/jira/browse/HIVE-19514
> Project: Hive
> Issue Type: Sub-task
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Major
>
> Ptest tries to copy ~20G worth of logs from the worker nodes to the server.
> The total aggregate time spent by all the Ptest server in copying these files
> is close to 15 min (although much of that is in parallel threads). Most of
> the times we don't check for the logs. We seldom check for logs of successful
> tests. We should ignore copying successful test logs by default to improve
> runtime.
> {noformat}
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:235 Executed 13568
> tests
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF:
> ExecutionPhase took 71 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF:
> ExecutionPhase.TotalRsyncElapsedTime took 15 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase
> took 5 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF:
> ReportingPhase took 0 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF:
> TestCheckPhase took 0 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase
> took 0 minutes
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
