M.-A. Lemburg wrote:
>>>> That's why the timers being used by pybench will become a
>>>> parameter that you can then select to adapt pybench it to
>>>> the OS your running pybench on.
>>> Wasn't that decision a consequence of the problems found during
>>> the sprint?
>> It's a consequence of a discussion I had with Steve Holden
>> and Tim Peters:
>>
>> I believe that using wall-clock timers
>> for benchmarking is not a good approach due to the high
>> noise level. Process time timers typically have a lower
>> resolution, but give a better picture of the actual
>> run-time of your code and also don't exhibit as much noise
>> as the wall-clock timer approach. Of course, you have
>> to run the tests somewhat longer to get reasonable
>> accuracy of the timings.
>>
>> Tim thinks that it's better to use short running tests and
>> an accurate timer, accepting the added noise and counting
>> on the user making sure that the noise level is at a
>> minimum.
>
> I just had an idea: if we could get each test to run
> inside a single time slice assigned by the OS scheduler,
> then we could benefit from the better resolution of the
> hardware timers while still keeping the noise to a
> minimum.
>
> I suppose this could be achieved by:
>
> * making sure that each tests needs less than 10ms to run
>
> * calling time.sleep(0) after each test run
>
> Here's some documentation on the Linux scheduler:
>
> http://www.samspublishing.com/articles/article.asp?p=101760&seqNum=2&rl=1
>
> Table 3.1 has the minimum time slice: 10ms.
>
> What do you think ? Would this work ?
I ran some tests related to this and it appears that provide
the test itself uses less than 1ms, chances are
high that you don't get any forced context switches in your
way while running the test.
It also appears that you have to use time.sleep(10e6) to
get the desired behavior. time.sleep(0) seems to receive
some extra care, so doesn't have the intended effect - at
least not on Linux.
I've checked this on AMD64 and Intel Pentium M. The script is
attached - it will run until you get more than 10 forced
context switches in 100 runs of the test, incrementing the
runtime of the test in each round.
It's also interesting that the difference between max and min
run-time of the tests can be as low as 0.2% on the Pentium,
whereas the AMD64 always stays around 4-5%. On an old AMD Athlon,
the difference rare goes below 50% - this might also have
to do with the kernel version running on that machine which
is 2.4 whereas the AMD64 and Pentium M are running 2.6.
Note that is needs to the resource module, so it won't work
on Windows.
It's interesting that even pressing a key on your keyboard
will cause forced context switches.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jun 02 2006)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
import resource, time
def workload(rounds):
x = 0
for i in range(rounds):
x = x + 1
def microbench():
print 'Microbench'
sleeptime = 10e-6
sleep = time.sleep
timer = time.time
rtype = resource.RUSAGE_SELF
rounds = 100
while 1:
times = []
rstart = resource.getrusage(rtype)
for i in range(100):
# Make sure the test is run at the start of a scheduling time
# slice
sleep(sleeptime)
# Test
start = timer()
workload(rounds)
stop = timer()
times.append(stop - start)
rstop = resource.getrusage(rtype)
volswitches = rstop[-2] - rstart[-2]
forcedswitches = rstop[-1] - rstart[-1]
min_time = min(times)
max_time = max(times)
diff = max_time - min_time
if forcedswitches == 0:
print 'Rounds: %i' % rounds
print ' min time: %f seconds' % min_time
print ' max time: %f seconds' % max_time
print ' diff: %f %% = %f seconds' % (diff / min_time * 100.0,
diff)
print ' context switches: %r %r' % (volswitches, forcedswitches)
print
elif forcedswitches > 10:
break
rounds += 100
microbench()
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
