[
https://issues.apache.org/jira/browse/HADOOP-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HADOOP-15549:
---------------------------------
Status: Patch Available (was: Open)
Re-measured on trunk instead of 3.0.0, and also used 'taskset -c 1' to bind
everything to one core to try to reduce variance:
2.8.2:
{code}
535.052533 task-clock (msec) # 0.918 CPUs utilized
( +- 0.88% )
3,055 context-switches # 0.006 M/sec
( +- 1.02% )
1 cpu-migrations # 0.002 K/sec
12,644 page-faults # 0.024 M/sec
( +- 0.06% )
1,953,309,627 cycles # 3.651 GHz
( +- 0.16% )
2,221,327,797 instructions # 1.14 insn per cycle
( +- 0.17% )
417,919,978 branches # 781.082 M/sec
( +- 0.17% )
18,726,810 branch-misses # 4.48% of all branches
( +- 0.19% )
0.582855783 seconds time elapsed
( +- 1.70% )
{code}
3.2 without patch:
{code}
751.038338 task-clock (msec) # 0.927 CPUs utilized
( +- 0.43% )
3,646 context-switches # 0.005 M/sec
( +- 2.26% )
1 cpu-migrations # 0.001 K/sec
( +- 25.00% )
19,233 page-faults # 0.026 M/sec
( +- 0.42% )
2,735,218,817 cycles # 3.642 GHz
( +- 0.62% )
3,218,012,767 instructions # 1.18 insn per cycle
( +- 0.54% )
604,477,739 branches # 804.856 M/sec
( +- 0.48% )
25,664,033 branch-misses # 4.25% of all branches
( +- 0.55% )
0.810230298 seconds time elapsed
( +- 0.98% )
{code}
3.2 with patch:
{code}
3.2 with patch:
679.940626 task-clock (msec) # 0.918 CPUs utilized
( +- 1.14% )
3,302 context-switches # 0.005 M/sec
( +- 1.16% )
1 cpu-migrations # 0.001 K/sec
16,819 page-faults # 0.025 M/sec
( +- 0.06% )
2,375,283,537 cycles # 3.493 GHz
( +- 0.33% )
2,722,724,476 instructions # 1.15 insn per cycle
( +- 0.27% )
511,944,028 branches # 752.925 M/sec
( +- 0.24% )
21,981,131 branch-misses # 4.29% of all branches
( +- 0.33% )
0.740316578 seconds time elapsed
( +- 0.96% )
{code}
> Upgrade to commons-configuration 2.1 regresses task CPU consumption
> -------------------------------------------------------------------
>
> Key: HADOOP-15549
> URL: https://issues.apache.org/jira/browse/HADOOP-15549
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
> Affects Versions: 3.0.2
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Major
> Attachments: hadoop-15549.txt
>
>
> HADOOP-13660 upgraded from commons-configuration 1.x to 2.x.
> commons-configuration is used when parsing the metrics configuration
> properties file. The new builder API used in the new version apparently makes
> use of a bunch of very bloated reflection and classloading nonsense to
> achieve the same goal, and this results in a regression of >100ms of CPU time
> as measured by a program which simply initializes DefaultMetricsSystem.
> This isn't a big deal for long-running daemons, but for MR tasks which might
> only run a few seconds on poorly-tuned jobs, this can be noticeable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
