join bug
Hello, Is there a size limit for the input files for join? I want to do it with large files, but even files of 1000 lines fail Thanks, Martin ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
date +%s ignores TZ
Hi, this is probably all correct behavior as it is right now (coreutils 6.9): $ date +%s 120433 $ TZ=GMT date +%s 120433 $ TZ=PDT date +%s 120433 but is there actually a way to do $ TZ=anything date +%s -d `date '+%Y-%m-%d %H:%M:%S'`; without invoking date twice? thanks, Jan ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Shred 5.2.1 VS. Shred 6.10
We've been utilizing an internal tool/userspace/stack/thing here for a number of years that, along with performing other maintenance tasks, uses shred version 5.2.1 (which was packaged with an old version of Ubuntu, I believe) to wipe various drives in different circumstances. Depending on the speed and size of the drive, this process can take anywhere from 2 to 5 hours, and in our experience this is very reasonable for a 7-pass wipe. Recently, we decided to update our tool/userspace/stack/thing (what this REALLY is is a PXE-booted Linux image of about 50 megs containing lots of different scripts, system probing/gathering utilities, etc.) and along with this shred was upgraded to 6.10. What we've noticed is that shred is approximately 5-6x slower than before in every case, and I'm not quite sure what has changed. I've checked the ChangeLog and I see that some changes were made with regards to the ISAAC code being divorced into a separate file, but I'm not entirely sure I have the familiarity to really dive into the code and track down what is going on, nor can I imagine any reason this change would have any effect on the speed of shred. My question is: is there anything that has changed substantially with the way shred works between theses two versions (which I admit is a 4year span) to result in such a difference? Is it conceivable that the old version was bugged, and working too quickly? Or perhaps the new version is bugged, and working too slowly? Or perhaps the new version is simply more sophisticated, generates better random numbers, and this is just the way it is? :) The difference, I'm sure, has to do with the way random numbers are being generated in each version, as the uniform passes run at equal speeds. Some data below: A real case, using a full /dev/sda wipe: 40GB PATA/SATA Hybrid Drive / 7 passes - Shred 5.2.1: 2.6hrs - Shred 6.10: 13.1hrs A contrived case, using a small 20MB dd'd image: 20MB (dd if=/dev/zero of=foo bs=1M count=20) / 7 passes: - Shred 5.2.1: 4.9s - Shred 6.10: 22.2s ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: date +%s ignores TZ
Jan Engelhardt wrote: $ date +%s 120433 %s seconds since 1970-01-01 00:00:00 UTC $ TZ=GMT date +%s 120433 $ TZ=PDT date +%s 120433 Right. I assume you were *very fast* typing in that data and that seconds did not move on while you were doing it. :-) I get the point though. That value is a timezone independent value. but is there actually a way to do $ TZ=anything date +%s -d `date '+%Y-%m-%d %H:%M:%S'`; without invoking date twice? I think something was lost in translation. date '+%Y-%m-%d %H:%M:%S' That will always produce the current time. That means that date +%s -d `date '+%Y-%m-%d %H:%M:%S'` is always the same as date +%s There is no need to call date twice to get that result. Please say a few more words about what you are trying to do. I think with a little more understanding it will make better sense. I see that you are trying to produce a unix seconds epoch based upon some time but creating it with date doesn't do it. You would need some other time. date +%s -d 'last thursday' 1204182000 Bob ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: date +%s ignores TZ
On Feb 29 2008 14:20, Bob Proulx wrote: Right. I assume you were *very fast* typing in that data and that seconds did not move on while you were doing it. :-) I get the point though. That value is a timezone independent value. but is there actually a way to do $ TZ=anything date +%s -d `date '+%Y-%m-%d %H:%M:%S'`; without invoking date twice? I think something was lost in translation. date '+%Y-%m-%d %H:%M:%S' That will always produce the current time. That means that date +%s -d `date '+%Y-%m-%d %H:%M:%S'` is always the same as date +%s There is no need to call date twice to get that result. There is (my default zone is /etc/localtime - /usr/share/zoneinfo/Europe/Berlin): $ TZ=GMT date +%s -d `date '+%Y-%m-%d %H:%M:%S'` 1204325194 $ date +%s 1204321595 (now with not-so-fast typing! :) Please say a few more words about what you are trying to do. I think with a little more understanding it will make better sense. I wanted to get the number of seconds since the start of the day. echo $[`date +%s` % 86400]; unfortunately does not do the right thing — it would show 82800 instead of 0 when it is (local) midnight. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: date +%s ignores TZ
Jan Engelhardt [EMAIL PROTECTED] writes: this is probably all correct behavior as it is right now (coreutils 6.9): $ date +%s 120433 $ TZ=GMT date +%s 120433 $ TZ=PDT date +%s 120433 %s is defined as seconds since 1970-01-01 00:00:00 UTC which obviously is a constant at any given time throughout the world. but is there actually a way to do $ TZ=anything date +%s -d `date '+%Y-%m-%d %H:%M:%S'`; without invoking date twice? Please explain what you are trying to achieve. Andreas. -- Andreas Schwab, SuSE Labs, [EMAIL PROTECTED] SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 And now for something completely different. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: date +%s ignores TZ
Jan Engelhardt wrote: I wanted to get the number of seconds since the start of the day. echo $[`date +%s` % 86400]; How about: echo $[$(date +%s) - $(date -d '' +%s)] Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: date +%s ignores TZ
Jan Engelhardt wrote: There is (my default zone is /etc/localtime - /usr/share/zoneinfo/Europe/Berlin): $ TZ=GMT date +%s -d `date '+%Y-%m-%d %H:%M:%S'` 1204325194 $ date +%s 1204321595 (now with not-so-fast typing! :) :-) I wanted to get the number of seconds since the start of the day. echo $[`date +%s` % 86400]; Note that the $[expression] syntax is deprecated and is scheduled for removal from a future version of the shell. Please convert to using the now standard $((expression)) syntax. echo $(( $(date +%s) % 86400 )); unfortunately does not do the right thing — it would show 82800 instead of 0 when it is (local) midnight. Midnight and noon are neither a.m. nor p.m. but midnight is considered the start of the day. Therefore normal convention would use a seconds range of 0-86399 seconds in any given day similar to 0-59 seconds in a minute. Do I understand that you want to use 1-86400? That would be like using 1-60 seconds in a minute, right? In which case I would simply special case the zero case and convert it to the max value specially. But I probably wouldn't do it. I can't think of any totally race free way to do this without invoking date multiple times. The problem is right around midnight we want to avoid having one invocation from before and one from after. My technique is to get the time once and then shape that single point in time as I need it so as to avoid any possibility of problems at midnight. But I don't see that as being too inefficient so I would simply invoke date several times and not worry about it. Perhaps someone else will have a better optimized strategy. nowseconds=$(date +%s) dateday=$(date -d $nowseconds +%F) # e.g. 2008-02-29 secondsatdaystart=$(date -d $dateday +%s) secondssincedaystart=$(( $nowseconds - $secondsatdaystart )) echo $secondssincedaystart That still treats midnight as 0 seconds since day start. I think that is the right thing to do but if you want it the other way then you could add a case. nowseconds=$(date +%s) dateday=$(date -d $nowseconds +%F) # e.g. 2008-02-29 secondsatdaystart=$(date -d $dateday +%s) secondssincedaystart=$(( $nowseconds - $secondsatdaystart )) case $secondssincedaystart in (0) secondssincedaystart=86400 ;; esac echo $secondssincedaystart I am also sure there are ways to optimize this further but this seemed good enough. I didn't test that very much but the numbers seemed reasonable to me at brief glance at them. Bob ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: date +%s ignores TZ
Brian Dessent wrote: Jan Engelhardt wrote: I wanted to get the number of seconds since the start of the day. echo $[`date +%s` % 86400]; How about: echo $[$(date +%s) - $(date -d '' +%s)] That works most of the time and if I were never to run this at midnight I would do just what you suggest here and wouldn't worry about it further. But Jan specifically mentioned wanting midnight. That made me avoid suggesting that case. If this needs to be robust at midnight then it needs one date invocation which needs more work. The problem is that the first date might execute at midnight minus one second. The second date might execute at midnight, which is the next day. If that happens then a negative value would result because the start of the day subtracted off would be the start of the next day. Therefore the need to get the time once only. Bob ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
