[
https://issues.apache.org/jira/browse/HIVE-19571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504003#comment-16504003
]
Sahil Takiar commented on HIVE-19571:
-------------------------------------
Moving this to unassigned as I won't be able to work on this anytime soon.
Anyone else feel free to pick this up.
I think the attached patch solves a lot of the issues, but it still needs to be
tested. We might be able to get away with just writing Java unit tests to test
this.
> Ability to run multiple pre-commit jobs on a ptest server
> ---------------------------------------------------------
>
> Key: HIVE-19571
> URL: https://issues.apache.org/jira/browse/HIVE-19571
> Project: Hive
> Issue Type: Sub-task
> Components: Testing Infrastructure
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
> Attachments: HIVE-17317.WIP.1.patch
>
>
> I've been taking a look at the Disk, Network, and CPU usage of the GCE
> instances we run ptest on, and it doesn't look like we are fully utilizing
> the machines. The resource usage is very up and down.
> During each ptest execution, there is a large chunk of time (~20 min) where
> its just the Jenkins job that is doing any work (checking out github repos,
> building code, figuring out test batches, etc.). During this time, the ptest
> nodes are mostly idle - the CPU and Disk I/O are almost zero.
> Even when ptest is running, I think some of resources are under-utilized.
> Network and disk resource spike at the beginning of the job, probably because
> ptest is distributing resources to each machine, each slave is downloading
> jars, etc. However, after that, when the actual tests run, there is almost 0
> network activity (which makes sense since tests runs on a single node). For
> disk usage, there is activity, but not nearly as high as when the setup phase
> was occuring. CPU usage fluctuates between 40-80%.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
