total uptime between shutdowns
Hi Guys, Is there a tool that will show me the total uptime (availability) of a machine between reboots? That is, if the machine was up for 24 hours and then shutdown for an hour and then up for 23 hours I want an answer of 47. I need this for charge-back of departments using cloud computer which they sometimes turn off. Thanks, -tom ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: total uptime between shutdowns
2011/2/1 Tom Rosenfeld trosenf...@gmail.com Hi Guys, Is there a tool that will show me the total uptime (availability) of a machine between reboots? That is, if the machine was up for 24 hours and then shutdown for an hour and then up for 23 hours I want an answer of 47. I need this for charge-back of departments using cloud computer which they sometimes turn off. Thanks, -tom Zabbix can do availability reports, based on either an agent running on the machines, or alerts triggering (for example machine stops answering to pings). There may be a better way (TM). :) -- Shimi ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: total uptime between shutdowns
On Tue, Feb 01, 2011 at 10:00:52AM +0200, Tom Rosenfeld wrote: Hi Guys, Is there a tool that will show me the total uptime (availability) of a machine between reboots? That is, if the machine was up for 24 hours and then shutdown for an hour and then up for 23 hours I want an answer of 47. I do not know about anything that does exactly this, but you might want to look at 'ud - Uptime Daemon'. But... I need this for charge-back of departments using cloud computer which they sometimes turn off. I wouldn't use something from inside the guest for this. How do they turn it off? Using some interface you created for them? I'd stuff into this interface the counting. You should also consider what happens when they hibernate (e.g. Xen's 'xm save'/'xm restore'), in case you let them do this. This does not cost you CPU time or RAM - except for the memory image which takes space (and needs to be backed up) it's exactly like shutting down. But the guest will not notice - you'll not see inside it anything that says that it was shutdown and restarted. -- Didi ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: total uptime between shutdowns
On Tue, 2011-02-01 at 10:00 +0200, Tom Rosenfeld wrote: Hi Guys, Is there a tool that will show me the total uptime (availability) of a machine between reboots? That is, if the machine was up for 24 hours and then shutdown for an hour and then up for 23 hours I want an answer of 47. I need this for charge-back of departments using cloud computer which they sometimes turn off. 1. Maybe the cloud computers' provider has the appropriate tool? In this case, if you don't find the tool yourself, you'll have to tell us which cloud provider are you working with. 2. The 'uptime' command gives the uptime since last reboot. You may want to add 'uptime /var/log/my_uptimes_log.txt' to the shutdown script, and rotate process, using a custom Perl script, the /var/log/my_uptime_log.txt file each month to total the uptime in that month. --- Omer -- MCSE - acronym for Minesweeper Consultant Solitaire Expert. (Unknown) My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: total uptime between shutdowns
Hi Omer, Tom,
On Tue, Feb 01, 2011 at 10:18:41AM +0200, Omer Zak wrote:
2. The 'uptime' command gives the uptime since last reboot. You may
want to add 'uptime /var/log/my_uptimes_log.txt' to the shutdown
script, and rotate process, using a custom Perl script,
the /var/log/my_uptime_log.txt file each month to total the uptime in
that month.
Another option: parse the output of last.
baruch
--
~. .~ Tk Open Systems
=}ooO--U--Ooo{=
- bar...@tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: total uptime between shutdowns
Your solution will work, of course, only on a tidy shutdown. For unplanned shutdown (aka - power failure) it will not work. Ez On Tue, Feb 1, 2011 at 10:18 AM, Omer Zak w...@zak.co.il wrote: On Tue, 2011-02-01 at 10:00 +0200, Tom Rosenfeld wrote: Hi Guys, Is there a tool that will show me the total uptime (availability) of a machine between reboots? That is, if the machine was up for 24 hours and then shutdown for an hour and then up for 23 hours I want an answer of 47. I need this for charge-back of departments using cloud computer which they sometimes turn off. 1. Maybe the cloud computers' provider has the appropriate tool? In this case, if you don't find the tool yourself, you'll have to tell us which cloud provider are you working with. 2. The 'uptime' command gives the uptime since last reboot. You may want to add 'uptime /var/log/my_uptimes_log.txt' to the shutdown script, and rotate process, using a custom Perl script, the /var/log/my_uptime_log.txt file each month to total the uptime in that month. --- Omer -- MCSE - acronym for Minesweeper Consultant Solitaire Expert. (Unknown) My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark
On Tue, Feb 01, 2011, Elazar Leibovich wrote about Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark: Long story short, he claims there that modern computers are now highly non-deterministic, he demonstrated 20% running time variation by the same JVM running the same code. I haven't listened to that talk, so I'm just talking now from my own experiences. It depends on what you're benchmarking, and why. Shlomi was benchmarking a single-process, CPU-intensive, program, and garbage collection was not involved. Everything in his program was perfectly deterministic. Indeed, the computer around him is *not* deterministic - other processes might randomly decide to do something - he might get mail in the middle, updatedb might start, he might be doing some interactive work at the same time, and some clock application might be updating the display every second, and something might suddely decide to access the disk. Or whatever. But if he runs his deterministic application 5 times and gets 5 different durations, each duration is composed of the deterministic run-time of the application plus a random delay caused by other things on the system. The *minimum* of these 5 durations is the one that had the minimum random delay, and is thus closest to the true run time. Presumably, if one runs the application on a machine which is as idle as humanly possible, you'd get something close to this minimum. It is true that when the application itself is non-deterministic, or when it closely interacts with other non-deterministic parts of the system (e.g., io-intensive, depends on network delays, etc.) averaging might make more sense. Nadav. -- Nadav Har'El| Tuesday, Feb 1 2011, 27 Shevat 5771 n...@math.technion.ac.il |- Phone +972-523-790466, ICQ 13349191 |Windows-2000/Professional isn't. http://nadav.harel.org.il | ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark
On Tue, Feb 1, 2011 at 12:09 PM, Nadav Har'El n...@math.technion.ac.ilwrote:
But if he runs his deterministic application 5 times and gets 5 different
durations, each duration is composed of the deterministic run-time of
the application plus a random delay caused by other things on the system.
The *minimum* of these 5 durations is the one that had the minimum random
delay, and is thus closest to the true run time. Presumably, if one runs
the application on a machine which is as idle as humanly possible, you'd
get something close to this minimum.
See this paper[2] which is referred in the slides.
Maybe the minimal result you're getting is due to the UNIX environment
variables?
We see that something external and orthogonal
to the program, i.e., changing the size (in bytes) of an unused
environment variable, can dramatically (frequently by about
33% and once by almost 300%) change the performance of
our program. This phenomenon occurs because the UNIX
environment is loaded into memory before the call stack.
Thus, changing the UNIX environment size changes the
location of the call stack which in turn affects the alignment
of local variables in various hardware structures.
This simple example demonstrates that computer systems are sensitive: an
insignificant and seemingly irrelevant change can dramatically affect the
performance of the
system. As a consequence of this sensitivity, we will find
that different experimental setups will produce different outcomes and thus
cause measurement bias.
The program they tested[1] is strictly CPU bound. BTW, standard deviation
wouldn't work as well, but it shows (I think) that there's no such think as
ideal minimal runtime.
[1]
static int i = 0, j = 0, k = 0;
int main() {
int g = 0, inc = 1;
for (; g65536; g++) {
i += inc;
j += inc;
k += inc;
}
return 0;
}
[2] http://www-plan.cs.colorado.edu/diwan/asplos09.pdf
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark
On Tuesday 01 Feb 2011 12:09:17 Nadav Har'El wrote: On Tue, Feb 01, 2011, Elazar Leibovich wrote about Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark: Long story short, he claims there that modern computers are now highly non-deterministic, he demonstrated 20% running time variation by the same JVM running the same code. I have noticed small variations in my testing times for Freecell Solver, but they were not nearly as dramatic as a 20% difference in run-time. At most they were 1 or 2 seconds out of 72-73 total run-time. I haven't listened to that talk, so I'm just talking now from my own experiences. It depends on what you're benchmarking, and why. Shlomi was benchmarking a single-process, CPU-intensive, program, and garbage collection was not involved. Everything in his program was perfectly deterministic. Well, my program is not single-process, but it is multi-threaded (because I noticed using two threads makes it somewhat faster), and I have a multi- process version that used to perform a little better than the multi-threaded version, but no longer does. What I do is solve different deals in different tasks. Indeed, the computer around him is *not* deterministic - other processes might randomly decide to do something - he might get mail in the middle, updatedb might start, he might be doing some interactive work at the same time, and some clock application might be updating the display every second, and something might suddely decide to access the disk. Or whatever. Well, normally, before benchmarking, I exit from X-Windows, kill all stray processes from the X session, and also I run the benchmarked process under the highest possible priority (by using a sudo_renice script). Currently a typical entry in my benchmarks reads: [quote] bash scripts/pgo.bash gcc total r3509 ; trunk ARGS=--worker-step 16 -l eo 2.6.37 (vanilla from kernel.org and a custom .config file). ssh (while not running X). gcc-4.5.1 with -flto and -fwhole-program. google-perftools-1.6 compiled with gcc-4.5.1. Using sudo_renice. ./Tatzer -l p4b Highlight: vanilla 2.6.37 kernel. 72.7685720920563s [/quote] So it's somewhat more deterministic. But if he runs his deterministic application 5 times and gets 5 different durations, each duration is composed of the deterministic run-time of the application plus a random delay caused by other things on the system. The *minimum* of these 5 durations is the one that had the minimum random delay, and is thus closest to the true run time. Presumably, if one runs the application on a machine which is as idle as humanly possible, you'd get something close to this minimum. Right. Regards, Shlomi Fish It is true that when the application itself is non-deterministic, or when it closely interacts with other non-deterministic parts of the system (e.g., io-intensive, depends on network delays, etc.) averaging might make more sense. Nadav. -- - Shlomi Fish http://www.shlomifish.org/ Understand what Open Source is - http://shlom.in/oss-fs Chuck Norris can make the statement This statement is false a true one. Please reply to list if it's a mailing list post - http://shlom.in/reply . ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark
2011/2/1 Elazar Leibovich elaz...@gmail.com See this paper[2] which is referred in the slides. [2] http://www-plan.cs.colorado.edu/diwan/asplos09.pdf This morning I took part in a meeting here at work where something related was discussed, so I am hot for the topic. Apologies all around. The paper you reference focuses on the so-called measurement bias. The authors refer to this term as well known to medical and other sciences. On another page they mention social sciences, too. The real meaning of measurement bias is sloppy experimentation where external factors are not properly taken into account or neutralized, rendering results irreproducible. Real experimental sciences like physics know the phenomenon very well and do not consider it a necessary evil. From my point of view, the title of the paper should have been When you do obviously wrong things your results should not surprise you. I am not saying the paper is worthless, on the contrary, it is certainly useful to draw attention to the fact that you need to know what you measure and what affects your measurement, especially in a culture that lacks such awareness. It's a far cry from saying that computers are not deterministic though. If you want to measure A and in fact you measure something else and that something else depends on various factors that you are unable to control, it is *not* a law of Nature. Uncontrolled external factors may affect the correctness of your program and not just performance. The meeting I took part in this morning touched on the following issue: the details of the environment and the setup of the build machine are not a part of the code that is built on it, are not controlled in the same manner, etc. As a result, different builds of exactly the same code need to be carefully checked by QA because they may behave differently (and their performance may differ). Big operational problem that intuitively very few people expect (same code - why should it be re-tested?). The problem is, however, not an intrinsic lack of determinism but the presence of factors that are (erroneously) considered external and are therefore poorly controlled. The presentation that you sent highlights another issue: you cannot control complicated 3rd party software stacks. You wrote some java code that needs a JVM to run, and the latter has service threads that do things like GC, JIT compilation, serialization/deserialization, marshalling/unmarshalling, etc. The JVM itself is scheduled, these service threads are scheduled, they happen at different times from run to run, affecting optimization, concurrency, all sorts of other things. From the point of view of a typical java programmer these are uncontrollable factors. Are they unavoidable? Not at all. If you need a certain level of control choose technologies that allow it. Say, you do a lot of messaging. You should realize that java, for instance, does not allow you much control of memory allocation and as a result serialization/deserialization is unavoidable (of course, the marketoids only stress that java does it automatically, and do not dwell of the question why it needs serialization in the first place). This will likely hit your performance and predictability. If you choose, e.g., C/C++ and good networking technologies you can control memory allocation, and therefore serialization, much better (and even avoid serialization altogether). -- Oleg Goldshmidt | o...@goldshmidt.org ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark
On Tue, Feb 01, 2011 at 12:49:00PM +0200, Elazar Leibovich wrote:
The program they tested[1] is strictly CPU bound. BTW, standard deviation
wouldn't work as well, but it shows (I think) that there's no such think as
ideal minimal runtime.
[1]
static int i = 0, j = 0, k = 0;
int main() {
int g = 0, inc = 1;
for (; g65536; g++) {
i += inc;
j += inc;
k += inc;
}
return 0;
}
[2] http://www-plan.cs.colorado.edu/diwan/asplos09.pdf
That program is CPU-bound, but the time it runs is short enough for the
size of the environment to actually mean something.
--
Tzafrir Cohen | tzaf...@jabber.org | VIM is
http://tzafrir.org.il || a Mutt's
tzaf...@cohens.org.il || best
tzaf...@debian.org|| friend
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Hamakor General Assembly on Thu, 17-Feb-2011 in Shenkar
Hi all, Hamakor, the Israeli Non-profit organisation for Free Software and Open Source Code will hold a general assembly on Thursday, 17 February, 2011 in the Shenkar College in Ramat Gan. The assembly will be held at 18:30 in room 304 (in the old building of Shenkar). On the agenda: 1. Approving the verbal report for the year 2007 (complement for the proper administration of the NPO.) 2. Approving the financial and verbal reports for 2010. 3. Elections for the Board and the Comptrollers' Committee. 4. Approving the authorised signers in the NPO (as a continuation of the of the board elections results .). One can propose further items for the agenda until a week before the meeting by sending an E-mail to the board ( bo...@hamakor.org.il ). Hamakor members that are interested in proposing themselves for the board and/or the comptrollers committee are requested to announce their intention to the discussion list ( http://www.iglu.org.il/mailing-lists/hamakor-discussions.html ). Hope to see you there. Regards, Shlomi Fish -- - Shlomi Fish http://www.shlomifish.org/ Freecell Solver - http://fc-solve.berlios.de/ Chuck Norris can make the statement This statement is false a true one. Please reply to list if it's a mailing list post - http://shlom.in/reply . ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: total uptime between shutdowns
Thanks for all your tips!
I found that the information is basically available in 'last -x'.
Below is a rough script I put together to estimate total times rounded to
hours:
Thanks,
-tom
#!/bin/bash
#
# Compute total hours machine has been running, minus time it was turned
off.
# Based on output from 'last -x'
#
STOP_F=/tmp/stop
START_F=/tmp/start
last -x |awk '/shutdown/ {print $5, $6, $7, $8}' |tac $STOP_F
last -x |awk '/reboot/ {print $5, $6, $7, $8}' |tac $START_F
# Start times (should be 1 more than shutdown times):
# get rid of 1st start line since machine is still up
sed -i '1d' $START_F
#
# Read 1 line from each file and compare
# (see redirection at end of while loop
#
while read STOP 8
do
read START 7
# compute time machine was stopped
start_s=$(date -d $START +%s)
stop_s=$(date -d $STOP +%s)
((duration = start_s - stop_s))
((tot_hours = duration / (60*60) ))
echo total hours stopped = $tot_hours
((saved_hours+=tot_hours))
done 7$START_F 8$STOP_F
#
# Compute total uptime since first boot
#
# seconds at 1st boot
first_s=$( date -d $(last |tail -1 | awk '{$1=; $2= ;print }') +%s)
now_s=$(date +%s)
((elapsed_hours= (now_s - first_s) /(60*60) ))
echo Total elapsed hours = $elapsed_hours
echo Saved hours = $saved_hours
echo Total charged hours = $((elapsed_hours - saved_hours))
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark
On Tue, Feb 01, 2011, Oleg Goldshmidt wrote about Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark: awareness. It's a far cry from saying that computers are not deterministic though. If you want to measure A and in fact you measure something else and that something else depends on various factors that you are unable to control, it is *not* a law of Nature. I wholeheartedly agree that this is the case today, and we can still make meaningful, deterministic, measurements. But thinking about this, I wonder how long into the future this will remain so. Computer software is getting more and more complicated and complex as the years grow by, and it is getting harder and harder to control, or even be aware of everything that happens under the hood. Today, I think it's still possible (though probably taking years of study) to be aware of *everything* that happens while your program is running - in the program, kernel, libraries, processor, peripherals, background processes, remote servers, and so on - but I wonder if in 20 years this will still be possible. I wonder if in 20 years somebody can actually say ok, I'll run this benchmark without X windows like Sholomi did, or will X be so intertwined with everything that you can hardly remove it (just like you can hardly remove the UI from MS Windows). Even today, can you imagine running a program *without a kernel* or even without glibc, just so you don't run into the the unknown slowdowns that those things might cause your problems? Uncontrolled external factors may affect the correctness of your program and not just performance. The meeting I took part in this morning touched on the following issue: the details of the environment and the setup of the build machine are not a part of the code that is built on it, are not controlled in the same manner, etc. As a result, different builds of exactly the same code need to be carefully checked by QA because they may behave differently (and their performance may differ). Right, this reminds me of the intricate relationship between genetics and embryology in biology. Most people think that the DNA specifies exactly what the animal will look like. But DNA is like software, it cooperates with preexisting hardware, and changing the hardware might change what the software does, even if the software is untouched. There are science-fiction stories about finding dinosaur DNA and creating a dinosaur. But even if we have dino DNA, how do we recreate dio embryology, i.e., a dino mom with a dino ovum (egg cell), the dino egg, and so on? -- Nadav Har'El| Wednesday, Feb 2 2011, 28 Shevat 5771 n...@math.technion.ac.il |- Phone +972-523-790466, ICQ 13349191 |Vi-Vi-Vi is the editor of the beast -- http://nadav.harel.org.il |Richard Stallman (author of Emacs) ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
