IMO the timings of the benchmark suite are a bit unstable -- this is not the fault of Intel's setup, I noticed it also when running the suite myself.
On Tue, Jan 26, 2016 at 06:48:54PM +0000, Stewart, David C wrote: > Wow, what happened to Python default to cause such a regression? > > > > > On 1/26/16, 7:31 AM, "lp_benchmark_robot" <lp_benchmark_ro...@intel.com> > wrote: > > >Results for project Python default, build date 2016-01-26 03:07:40 +0000 > >commit: cbd4a6a2657e > >previous commit: f700bc0412bc > >revision date: 2016-01-26 02:54:37 +0000 > >environment: Haswell-EP > > cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, > > stepping 2, LLC 45 MB > > mem: 128 GB > > os: CentOS 7.1 > > kernel: Linux 3.10.0-229.4.2.el7.x86_64 > > > >Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 > >from 2015-02-25 12:15:33+00:00 > > > >---------------------------------------------------------------------------------- > > benchmark relative change since change since current > > rev run > > std_dev* last run baseline > > with PGO > >---------------------------------------------------------------------------------- > >:-) django_v2 0.21% -2.93% 8.95% > >16.19% > >:-| pybench 0.10% 0.05% -1.87% > >5.40% > >:-( regex_v8 2.72% -0.02% -4.67% > >4.57% > >:-| nbody 0.13% -0.92% -1.33% > >7.40% > >:-| json_dump_v2 0.20% 0.87% -1.59% > >11.48% > >:-| normal_startup 0.90% -0.57% 0.10% > >5.35% > >---------------------------------------------------------------------------------- > >* Relative Standard Deviation (Standard Deviation/Average) > > > >If this is not displayed properly please visit our results page here: > >http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-01-26/ > > > >Note: Benchmark results are measured in seconds. > > > >Subject Label Legend: > >Attributes are determined based on the performance evolution of the workloads > >compared to the previous measurement iteration. > >NEUTRAL: performance did not change by more than 1% for any workload > >GOOD: performance improved by more than 1% for at least one workload and > >there > >is no regression greater than 1% > >BAD: performance dropped by more than 1% for at least one workload and there > >is > >no improvement greater than 1% > >UGLY: performance improved by more than 1% for at least one workload and also > >dropped by more than 1% for at least one workload > > > > > >Our lab does a nightly source pull and build of the Python project and > >measures > >performance changes against the previous stable version and the previous > >nightly > >measurement. This is provided as a service to the community so that quality > >issues with current hardware can be identified quickly. > > > >Intel technologies' features and benefits depend on system configuration and > >may > >require enabled hardware, software or service activation. Performance varies > >depending on system configuration. > _______________________________________________ > Python-checkins mailing list > python-check...@python.org > https://mail.python.org/mailman/listinfo/python-checkins _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
