Re: [SLUG] Error in Time() command
On 24/05/2010, at 8:47 AM, Glen Turner wrote: > On Mon, 2010-05-24 at 09:02 +1000, Peter Chubb wrote: > >> Actually it doesn't give the whole answer. > > Wow, thanks heaps Peter. Thank you all. Particularly Peter. That is truly an amazing depth of information to digest. > > tenzero: so there are 1000 (CONFIG_HZ) samples per second. For each > sample your program is one of: not scheduled, running in user, running > in system, or has yielded the processor due to a blocking event such as > I/O or an explicit sleep(). > > It is possible that all processes yield and you get scheduled twice in > one sample -- I'd note that, and then ignore the possibility. Run an > infinite loop in another process if it worries you. That bastard will > never yield, and so your process will never be scheduled twice in a > tick. If you have multiple CPUs, bind one infinite loop to each CPU. In > reality, unless your results are odd, this is a lot of work to exclude > an unlikely case. > > With luck, your program is such that you can use strace to count the > blocking events on a single run of your program. Then pretend that the > scheduler tick misses every one of these. So if you program has 10 > blocking events and runs for 1.00 second then there result has a bound > of [1.00, 1.01]. Including the reporting error from the API [0.99, > 1.02]. > > You will save yourself a world of statistics if your "better" program's > range falls completely under the "worse" program's range. In this case it is. The hardware device dramatically outperforms the software and the error in the hardware device is half a clock cycle or 20ns while the error in the software is in the ms scale. Mostly for completeness, I simply wanted to discuss the error in the software measurement. > > In your Appendix you acknowledge Peter's contribution with a footnote > (eg, "Thanks to Dr Peter Chubb of UNSW for explaining the sampling > nature of the Linux task accounting"). In general, you don't cite these > sort of e-mail discussions since they are "all care and no > responsibility" discussions rather than a considered opinion ready for > peer review. Of course, where the posting becomes a part of the record > (such Linus's announcement of Linux) then you reference. > > You will see from this discussion the common research hassle that > determining the error of an experiment is usually more work than > determining the result. Indeed, that's exactly how I'm finding this. > > Best of luck with your studies, > Glen Thanks. All things going to plan, I should graduate in about a month. This is for my final year project in Electronic Engineering. Again, thanks for everyones help. Cheers. D. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Error in Time() command
> "Dave" == Dave Kempe writes: >> 250, on my Debian/sid system running 2.6.32-5-openvz-amd64. FWIW. >> >> Daniel Dave> A little off topic, but, I noticed that the CONFIG_HZ variable Dave> on openvz systems is often 250. I found 1000 to be a better Dave> setting for responsiveness on a heavily loaded openvz box. It Dave> felt like being able to schedule more small things more often Dave> made sense when you have many containers. Or is that totally not Dave> how it works? Higher HZ will generally give better responsiveness, lower HZ better throughput. Like all rules of thumb this one doesn't always work, it depends on the workloads. -- Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au http://www.ertos.nicta.com.au ERTOS within National ICT Australia -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Error in Time() command
> 250, on my Debian/sid system running 2.6.32-5-openvz-amd64. FWIW. > > Daniel A little off topic, but, I noticed that the CONFIG_HZ variable on openvz systems is often 250. I found 1000 to be a better setting for responsiveness on a heavily loaded openvz box. It felt like being able to schedule more small things more often made sense when you have many containers. Or is that totally not how it works? Dave -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Suggestions for a monochrome printer.
On 23/05/2010 9:41 AM, Jon wrote: How much do you value your limbs at? We have a Brother HL-5250DN which we are fairly happy with. The current successor seems to be this: We've used the HL-5350DN in our office here and we're very happy with it. It's low cost and so are its consumables. Terry -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Error in Time() command
Glen Turner writes: > On Mon, 2010-05-24 at 09:02 +1000, Peter Chubb wrote: > >> Actually it doesn't give the whole answer. > > Wow, thanks heaps Peter. > > tenzero: so there are 1000 (CONFIG_HZ) samples per second. 250, on my Debian/sid system running 2.6.32-5-openvz-amd64. FWIW. Daniel -- ✣ Daniel Pittman✉ dan...@rimspace.net☎ +61 401 155 707 ♽ made with 100 percent post-consumer electrons -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[SLUG] Nima Talebi has sent you a message on Zoji!
Hi Slug, Nima Talebi has sent a message to guests of the event "PSIGMATECK Mainevent One"! To view and reply to this message, click here (please don't reply to this email!): http://www.zoji.com/qAQgZYTrkhWbeuYZbUrAnv2yCOqNJk4D-9pgHkqZbm3H5bHNmz3bMvOjLystyCgtTNj6eMpybY0_* To view the event "PSIGMATECK Mainevent One", click here: http://www.zoji.com/qAQgZYTrkhXVdtsQgw9wu8PaKnRD*avu4aaEB2907*vvPX8jXMRaNQ__* - From:Nima Talebi To: Guests of the event "PSIGMATECK Mainevent One" Subject: PSIGMATECK Mainevent One - Location changed to: UNSW, Building K17, Room 201B The event venue has been finalised! Here are the details: Room 201B, Building K17, Department of Computer Science and Engineering, UNSW. - Don't want to receive email from this person anymore? To block them, click here: http://www.zoji.com/vw9t3NwFv4bhsDTRDrVOkwg7VDPL4VB0qCqxWK-wWIe9iTYWnWXUMTdew8xrJbaED7uwG-YLAAE_* -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Error in Time() command
On Mon, 2010-05-24 at 09:02 +1000, Peter Chubb wrote: > Actually it doesn't give the whole answer. Wow, thanks heaps Peter. tenzero: so there are 1000 (CONFIG_HZ) samples per second. For each sample your program is one of: not scheduled, running in user, running in system, or has yielded the processor due to a blocking event such as I/O or an explicit sleep(). It is possible that all processes yield and you get scheduled twice in one sample -- I'd note that, and then ignore the possibility. Run an infinite loop in another process if it worries you. That bastard will never yield, and so your process will never be scheduled twice in a tick. If you have multiple CPUs, bind one infinite loop to each CPU. In reality, unless your results are odd, this is a lot of work to exclude an unlikely case. With luck, your program is such that you can use strace to count the blocking events on a single run of your program. Then pretend that the scheduler tick misses every one of these. So if you program has 10 blocking events and runs for 1.00 second then there result has a bound of [1.00, 1.01]. Including the reporting error from the API [0.99, 1.02]. You will save yourself a world of statistics if your "better" program's range falls completely under the "worse" program's range. In your Appendix you acknowledge Peter's contribution with a footnote (eg, "Thanks to Dr Peter Chubb of UNSW for explaining the sampling nature of the Linux task accounting"). In general, you don't cite these sort of e-mail discussions since they are "all care and no responsibility" discussions rather than a considered opinion ready for peer review. Of course, where the posting becomes a part of the record (such Linus's announcement of Linux) then you reference. You will see from this discussion the common research hassle that determining the error of an experiment is usually more work than determining the result. Best of luck with your studies, Glen -- Glen Turner Australia's Academic & Research Network (AARNet) www.gdt.id.au/~gdt -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Error in Time() command
> "Glen" == Glen Turner writes: Glen> On Sun, 2010-05-23 at 17:02 +0800, tenz...@iinet.net.au wrote: >> I'm seeking a preferably citeable reference to the amount of error >> in the returned result from a Time() command. I want to be able to >> quote the level of error in timing the execution speed of my >> project. Glen> "man time" gives the answer. Actually it doesn't give the whole answer. The times reported by the kernel are statistical samples of the process's progress -- at each tick, a sample is taken. the currently process has either its user or system time counter incremented, depending on whether it was running in user or kernel at the time of the interrupt. For processes that do a lot of short sleeps, this can be way out; but processes that are mostly CPU-bound and that run for a long time relative to the tick length can use the times reliably. If you're on a platform that can do CONFIG_VIRT_CPU_ACCOUNTING (Itanium, s390) you can rely on precise results at the cost of about a 3% extra overhead on every context switch and kernel/userspace crossing. Alternatively, the microstate accounting patch can give you precise times on Itanium, x86, x86_64, ARM and MIPS, although the latest version isn't on the sourceforge website yet (it's on my TO-DO LIST) Peter C -- Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au http://www.ertos.nicta.com.au ERTOS within National ICT Australia -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Error in Time() command
On Sun, 2010-05-23 at 17:02 +0800, tenz...@iinet.net.au wrote:
> I'm seeking a preferably citeable reference to the amount of error
> in the returned result from a Time() command. I want to be
> able to quote the level of error in timing the execution speed
> of my project.
"man time" gives the answer.
> These statistics consist of (i) the elapsed real time
> between invocation and termination, (ii) the user CPU
> time (the sum of the tms_utime and tms_cutime values
> in a struct tms as returned by times(2)), and (iii)
> the system CPU time (the sum of the tms_stime and
> tms_cstime values in a struct tms as returned by times(2)).
"man 2 times" says that these data types are clock_t and
> times() returns the number of clock ticks that have
> elapsed since an arbitrary point in the past.
and
> The number of clock ticks per second can be obtained
> using: sysconf(_SC_CLK_TCK);
so
#include
#include
int main(void) {
printf("_SC_CLK_TCK is %ld\n", sysconf(_SC_CLK_TCK));
return 0;
}
which says for my platform
_SC_CLK_TCK is 100
Now the measuring precision may not match the reporting precision. But
the quantum of your kernel's task scheduler is somewhere between 100 and
1000 (see the CONFIG_HZ kernel compilation flag), so it is safe to say
that the reported precision of the tms API is the source of maximum
error.
If you need more precise runtimes and more details of consumed resources
then see the TASKSSTATS system. The documentation which accompanies the
kernel source contains sample code which will print all process exits
and the resources used to a high precision.
You also have control of the error. If you are lacking precision, then
give the program a task which makes it run for 10x or 100x longer.
As for citable you've got Buckley's. Shove the argument and program
above into an appendix and cross-reference it as you would any other
minor experiment or incidental proof. In general, these intermediate
results don't contribute to word count, but do check the local policy.
--
Glen Turner
Re: [SLUG] Error in Time() command
On 23/05/2010, at 5:40 PM, Peter Chubb wrote: >> "tenzero" == tenzero writes: > > tenzero> Hi everyone, I'm seeking a preferably citeable reference to > tenzero> the amount of error in the returned result from a Time() > tenzero> command. I want to be able to quote the level of error in > tenzero> timing the execution speed of my project. > > > > Do you mean time(2) ??? Actually I used time(1). This may have been unswise, I'm not certain. What I am trying to do, is simply determine the execution time of a small app I wrote in Java. Like this: time (java mcoption 104 .05859375 0.001129155 0.09765625 64 4) I am simply trying to record for performance testing, how long my java app takes to execute the Monte Carlo Simulation. I will be using it as a performance baseline to compare against my FPGA implementation. In terms of error, I am simply trying to find out the measurement error in the returned value. eg +- 1 millisecond or whatever. > > Short answer: it depends. > Long answer: time(2) returns the number of seconds since the epoch. > It's accurate for timing in the absence of leap-seconds, and jumps > caused by NTP etc. But it can go backwards, and forwards by more than > one step at a time. > > For details, see time(7). > > > If you want accurate timings, use one of the POSIX monotonic clocks. > man 3 clock_gettime and look for CLOCK_MONOTONIC > > Also, you are aware (I assume) that how long something takes depends > not only on its own processing time, but on how much else is happening > on the system? Yes I accept that I cannot fully control what else is happening across the entire system. Cheers. D. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Suggestions for a monochrome printer.
You're quite right. Computer Choice recently did a survey of colour printers. The one aspect that stood out was the cost of consumables, which, in short order, could easily exceed the initial cost of the printer. Bill Bennett. >> "Jon" == Jon writes: > > Jon> How much do you value your limbs at? We have a Brother HL-5250DN > Jon> which we are fairly happy with. The current successor seems to be > Jon> this: > > Jon> http://www.shopbot.com.au/pp-brother-hl-5340d-price-181171.html > > Jon> which is listed starting at $218. > > We bought outselves a second-hand Lexmark C510 --- which does colour and > duplex, at 30ppm with postscript (it cost around $260 from eBay). > > > It's worth looking for something with native postScript. But also > check consumables, and how many pages you're expecting to get through. > -- > SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ > Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html > -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Error in Time() command
On Sun, 2010-05-23 at 17:02 +0800, tenz...@iinet.net.au wrote: > Hi everyone, > > I'm seeking a preferably citeable reference to the amount of error in > the returned result from a Time() command. I want to be > able to quote the level of error in timing the execution speed of my project. > > While I have been using this command partly on Ubuntu for my project, > I would particularly like to find out the level of this error > on Mac OS X. Google and Google scholar have been disapointing. > > I concede that this is a little off topic and will move it OT if > people are unhappy. I concede I have no idea the actual problem you are having. Can you please explain what the issue is? Subtracting two times or something else, reset on ntpdate. Ta Ken -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Error in Time() command
> "tenzero" == tenzero writes: tenzero> Hi everyone, I'm seeking a preferably citeable reference to tenzero> the amount of error in the returned result from a Time() tenzero> command. I want to be able to quote the level of error in tenzero> timing the execution speed of my project. Do you mean time(2) ??? Short answer: it depends. Long answer: time(2) returns the number of seconds since the epoch. It's accurate for timing in the absence of leap-seconds, and jumps caused by NTP etc. But it can go backwards, and forwards by more than one step at a time. For details, see time(7). If you want accurate timings, use one of the POSIX monotonic clocks. man 3 clock_gettime and look for CLOCK_MONOTONIC Also, you are aware (I assume) that how long something takes depends not only on its own processing time, but on how much else is happening on the system? You can measure just your own process (and its children's) times using the times() system call; if the kernel has been compiled with CONFIG_VIRT_CPU_ACOUNTING time spent in interrupts will not be included in your times. This isn't such an issue on BSD-derived systems such as MacOSX, because they use separately scheduled threads for interrupts. -- Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au http://www.ertos.nicta.com.au ERTOS within National ICT Australia -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Suggestions for a monochrome printer.
> "Jon" == Jon writes: Jon> How much do you value your limbs at? We have a Brother HL-5250DN Jon> which we are fairly happy with. The current successor seems to be Jon> this: Jon> http://www.shopbot.com.au/pp-brother-hl-5340d-price-181171.html Jon> which is listed starting at $218. We bought outselves a second-hand Lexmark C510 --- which does colour and duplex, at 30ppm with postscript (it cost around $260 from eBay). It's worth looking for something with native postScript. But also check consumables, and how many pages you're expecting to get through. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[SLUG] Error in Time() command
Hi everyone, I'm seeking a preferably citeable reference to the amount of error in the returned result from a Time() command. I want to be able to quote the level of error in timing the execution speed of my project. While I have been using this command partly on Ubuntu for my project, I would particularly like to find out the level of this error on Mac OS X. Google and Google scholar have been disapointing. I concede that this is a little off topic and will move it OT if people are unhappy. Cheers. D. :: "Never ascribe to malice that which may adequately be explained by incompetence! -- Napoleon Bonaparte" -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
