Re: difference in seconds between two formatted dates ...

2023-12-29 Thread Albretch Mueller
 at the end of the day I had my cake and ate it, too:

site="www.debian.com"
site="www.google.fr"
###
dt=$(date +%Y%m%d%H%M%S.%N);
whois > "${site}_${dt}_whois.log" 2>&1
sudo strace --output "${site}_${dt}_strace_ping.log" ping "${site}" -c
4 > "${site}_${dt}_ping.log" 2>&1
ls -l "${site}_${dt}"*".log"; wc -l "${site}_${dt}"*".log"

 Once again, thank you Greg,
 lbrtchx



When strace breaks process or is blocked (was: Re: difference in seconds between two formatted dates ...)

2023-12-27 Thread Max Nikulin

On 27/12/2023 07:45, Greg Wooledge wrote:

Yeah, even on Debian systems where ping isn't setuid root, it still seems
to need special capabilities that strace interferes with, or isn't
allowed to attach to, or something.


$ /usr/sbin/getcap /usr/bin/ping
/usr/bin/ping cap_net_raw=ep

It is still elevated privileges and they are dropped for traced 
processes to avoid malicious actions by the controlling process. SETGID 
may be a trick namely to avoid tracing:


$ ls -l /usr/bin/ssh-agent
-rwxr-sr-x 1 root _ssh 481664 Dec 19 21:51 /usr/bin/ssh-agent


So, pick something other than ping, or run the strace as root.


Attaching by unprivileged processes to trace another process (strace -p 
PID) may be disabled using the following sysctl:


kernel.yama.ptrace_scope = 1

For details see
https://www.kernel.org/doc/html/latest/admin-guide/LSM/Yama.html

In Ubuntu this security measure is enabled out of the box. In Debian 
higher priority was given to developers who may need to attach debugger 
to a running process.


So it is not uncommon when tracing is blocked.



Re: difference in seconds between two formatted dates ...

2023-12-26 Thread Charlie Gibbs

On 2023-12-25 18:05, Jeffrey Walton wrote:


On Mon, Dec 25, 2023 at 8:43 PM Charlie Gibbs  wrote:


On Mon Dec 25 12:01:59 2023 "Andrew M.A. Cater"  wrote:


Yes - that's the obvious way. I set my machines to /etc/UTC (or
/etc/GMT) and leave them there. No daylight saving time, no offsets -
all logs unambiguous. That's why (worldwide) radio logkeeping is/was
in UTC. If you're travelling in an aircraft, you don't _need_ to know
ground time but you do need to know flight time against a reference
time. The Royal Air Force keep to UTC wherever they are in the world
for just this reason.


Not just the RAF.  All aviation works in UTC, to avoid problems when
flights cross time zone boundaries, and to keep wide-area weather
forecasts sane.  Your average airline passenger never sees UTC,
since airlines use it behind the scenes and convert it to local time
for display purposes.  (That's why you can see some strange intervals
between departure and arrival times.)


The US airlines I worked for in the late 1980s and 1990s used Zulu
time. If I recall correctly, flight arrivals and departures were
specified like 10:34Z or 23:10Z.

I don't know why Z was used instead of UTC or GMT. Probably to save
space, and save some ink if a schedule was printed.


Not to mention time, back in the days when weather data was broadcast
across networks of Baudot Teletypes running at 45 baud.


I don't know if that is still the case.


It is.  In fact, even though weather data has evolved somewhat, it is
still in a highly compressed form which, once you learn to read it,
enables you to scan a lot of data very quickly.  Here's a copy of
this hour's METARs (weather observations) and TAF (terminal area
forecast) for Vancouver, B.C.  Note the Z at the end of many times
(ddhhmmZ), although the Z is omitted if this wouldn't be ambiguous.

METAR CYVR 270300Z 08010KT 20SM SCT140 OVC160 08/05 A2998 RMK AC3AS5 SLP155=
METAR CYVR 270200Z 09007KT 20SM OVC160 08/05 A3000 RMK AC8 SLP161=
METAR CYVR 270100Z CCA 05010G15KT 25SM BKN150 BKN170 09/05 A3002 RMK 
AC5AC3 SLP168=
METAR CYVR 270100Z 05010G15KT 25SM BKN150 BKN170 09/05 A3002 RMK AC4AC3 
SLP168=


TAF CYVR 270304Z 2703/2806 09008KT P6SM FEW080 OVC160
FM271100 09012G22KT P6SM SCT040 OVC100 TEMPO 2711/2715 P6SM
-SHRA FEW020 BKN040 OVC080
FM271500 09012G22KT P6SM BKN040 OVC120
FM271800 10012G22KT P6SM BKN080 OVC150
FM272200 10012G22KT P6SM -SHRA BKN020 OVC040
FM280400 11015G25KT P6SM FEW040 SCT120 BKN200
RMK NXT FCST BY 270600Z=

--
/~\  Charlie Gibbs  |  "Some of you may die,
\ /|  but it's a sacrifice
 X   I'm really at ac.dekanfrus |  I'm willing to make."
/ \  if you read it the right way.  |-- Lord Farquaad (Shrek)



Re: difference in seconds between two formatted dates ...

2023-12-26 Thread Greg Wooledge
On Wed, Dec 27, 2023 at 12:39:26AM +, Albretch Mueller wrote:
>  I am getting an "Operation not permitted" error while strace tries to
> attach to that pid:
> 
>  "strace: attach: ptrace(PTRACE_SEIZE, 52527): Operation not permitted"
> 
> 
> $   ping www.google.fr -c 4 &
> pid=$!
> strace -p "$pid"
> wait "$pid"

Yeah, even on Debian systems where ping isn't setuid root, it still seems
to need special capabilities that strace interferes with, or isn't
allowed to attach to, or something.

unicorn:~$ strace -o log ping www.debian.com -c 4
ping: socktype: SOCK_RAW
ping: socket: Operation not permitted
ping: => missing cap_net_raw+p capability or setuid?

So, pick something other than ping, or run the strace as root.



Re: difference in seconds between two formatted dates ...

2023-12-26 Thread Albretch Mueller
On 12/26/23, Greg Wooledge  wrote:
> If you want to launch a SIMPLE BACKGROUND PROCESS (note the SIMPLE here,
> this is IMPORTANT), and then strace it while it runs, you'd do it like
> this:
>
> ping www.google.fr -c 4 &
> pid=$!
> strace -p "$pid"
> wait "$pid"

 I am getting an "Operation not permitted" error while strace tries to
attach to that pid:

 "strace: attach: ptrace(PTRACE_SEIZE, 52527): Operation not permitted"


$   ping www.google.fr -c 4 &
pid=$!
strace -p "$pid"
wait "$pid"
[1] 52527
strace: attach: ptrace(PTRACE_SEIZE, 52527): Operation not permitted
PING www.google.fr (64.233.185.94) 56(84) bytes of data.
64 bytes from yb-in-f94.1e100.net (64.233.185.94): icmp_seq=1 ttl=55
time=54.2 ms
64 bytes from yb-in-f94.1e100.net (64.233.185.94): icmp_seq=2 ttl=55
time=61.3 ms
64 bytes from yb-in-f94.1e100.net (64.233.185.94): icmp_seq=3 ttl=55
time=60.5 ms
64 bytes from yb-in-f94.1e100.net (64.233.185.94): icmp_seq=4 ttl=55 time=125 ms

--- www.google.fr ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 54.240/75.219/124.823/28.769 ms
[1]+  Doneping www.google.fr -c 4
$



Re: difference in seconds between two formatted dates ...

2023-12-26 Thread Andy Smith
Hello,

On Sun, Dec 24, 2023 at 11:05:53PM +, Albretch Mueller wrote:
> Why would %S be in the range second (00..60), instead of
> (00..59)?:

Remember what seems like years ago in this thread when I said that
if we're being pedantic, this is more complicated than you think,
and you scoffed that it was obvious even to a schoolchild?

Welcome to the classroom, Albretch.

Thanks,m
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: difference in seconds between two formatted dates ...

2023-12-26 Thread Greg Wooledge
On Tue, Dec 26, 2023 at 06:31:19PM +, Albretch Mueller wrote:
>  I think I am getting close to where I need to with this (am I?):
> 
> $ bash -c 'ping www.google.fr -c 4 &'; echo "Caller PID: $$"; strace -p $$

I don't understand what you're trying to do here.  Is this an
oversimplified example that has been so twisted and distorted that we
can't tell what the point is?

If you want to strace a ping process, just use:

strace ping www.google.fr -c 4

There is no reason to invoke a second shell, nor to run the ping process
in the background, nor to use strace on an already-started ping
process.

Even if you DID for some reason want to launch a BACKGROUND PROCESS and
the strace it, you're doing it wrong.  The $$ you reference in your
strace command is the PID of the script.  Your bash -c is a child of
the script, and your ping is a child of the bash -c (thus, a grandchild
of the script).

If you want to launch a SIMPLE BACKGROUND PROCESS (note the SIMPLE here,
this is IMPORTANT), and then strace it while it runs, you'd do it like
this:

ping www.google.fr -c 4 &
pid=$!
strace -p "$pid"
wait "$pid"

Note that there is no "bash -c".  This is IMPORTANT.  The ping process
is a direct child of the script.  That's why we can get its PID with
the $! special parameter, after launching it with & into the background.

If the thing you're trying to strace is actually a COMPLEX DAEMON that
does its own forking and shit, then this MAY NOT WORK.  You could
attempt to attach strace to the daemon's original PID, but if the daemon
forks and then commits suicide (a VERY OUTDATED design), then your
strace may be too late.  It might be trying to attach to a process
that's already dead.

If the thing you're trying to strace is a long-running daemon that
forks child process but DOES NOT kill itself, then it MIGHT work.
You'd want the "-f" option to trace all child processes as well as
the main PID.

Now, you see, we cannot tell from your "ping" or "bash -c ping" example
what it is that you're ACTUALLY trying to attach to.  What is it
REALLY?  How does it behave?  What information are you trying to
capture in your strace?

What PROBLEM are you even trying to solve?

Why are you writing a SCRIPT to solve your problem instead of running
an strace command by hand to get a one-off trace log so that you can
analyze it?

(Oh, also, just to add more bruises on the dead horse, ping is sometimes
a setuid root program.  Depends on the OS.  If it's setuid root on your
system, you might not be able to strace it without being root to start
with.)



Re: difference in seconds between two formatted dates ...

2023-12-26 Thread Albretch Mueller
On 12/26/23, Greg Wooledge  wrote:
> On Tue, Dec 26, 2023 at 02:57:54PM +, Albretch Mueller wrote:
>>  1) how do you set up the process to be straced as a parameter? Something
>> like:
>> prx="echo hello"
>> logfile="file.txt"
>> #
>> strace "${prx}" 2>"${logfile}"
>> ls -l "${logfile}"; wc -l "${logfile}"; cat "${logfile}"
>
> Why?  This does not appear to have any usefulness.  You're just making
> your life harder for no reason.

 Well, the way I see such my options, you could:
 1.1) parametrize a call to a function, or
 1.2) use named files for each type of strace run.
 I would rather use §1.1

> See also .

 Thank you. As you have suggested to me. I will have to use a
highlevel language if I want to take care of my "coblesome" cases. I
tend to entangle myself in corner cases for which scripts aren't so
useful.

>>  2) how do you know for sure that you are stracing the same process
>> that you are running and is logging some data?
>
> You are EXTREMELY confused about something, and we NEED to address this
> immediately.
>
> If you run "strace ./foo", a new instance of ./foo is launched right
> then and there, and you get the system call trace of THAT instance.

 Based on:

 
https://askubuntu.com/questions/137233/how-to-command-ping-display-time-and-date-of-ping/867500

 I think I am getting close to where I need to with this (am I?):

$ bash -c 'ping www.google.fr -c 4 &'; echo "Caller PID: $$"; strace -p $$
Caller PID: 45588
strace: Process 45588 attached
wait4(-1, PING www.google.fr (142.250.190.131) 56(84) bytes of data.
64 bytes from ord37s36-in-f3.1e100.net (142.250.190.131): icmp_seq=1
ttl=54 time=50.0 ms
64 bytes from ord37s36-in-f3.1e100.net (142.250.190.131): icmp_seq=2
ttl=54 time=34.7 ms
64 bytes from ord37s36-in-f3.1e100.net (142.250.190.131): icmp_seq=3
ttl=54 time=38.7 ms
64 bytes from ord37s36-in-f3.1e100.net (142.250.190.131): icmp_seq=4
ttl=54 time=40.3 ms

--- www.google.fr ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 34.735/40.952/50.048/5.628 ms



^Cstrace: Process 45588 detached
 
$

 However, strace doesn't end gracefully even though it seems to attach
to the running process  before it starts and based on what strace
itself logs, the process seems to end just fine.



Re: difference in seconds between two formatted dates ...

2023-12-26 Thread Greg Wooledge
On Tue, Dec 26, 2023 at 02:57:54PM +, Albretch Mueller wrote:
>  1) how do you set up the process to be straced as a parameter? Something 
> like:
> prx="echo hello"
> logfile="file.txt"
> #
> strace "${prx}" 2>"${logfile}"
> ls -l "${logfile}"; wc -l "${logfile}"; cat "${logfile}"

Why?  This does not appear to have any usefulness.  You're just making
your life harder for no reason.

See also .

>  2) how do you know for sure that you are stracing the same process
> that you are running and is logging some data?

You are EXTREMELY confused about something, and we NEED to address this
immediately.

If you run "strace ./foo", a new instance of ./foo is launched right
then and there, and you get the system call trace of THAT instance.

If another instance of ./foo was already running, you do NOT get the
system call trace of that first instance.

If you WANT the system call trace of an ALREADY-RUNNING PROCESS, you
need to get its process ID (PID) and use strace's "-p pid" option to
CONNECT to the existing process and begin tracing it.

That will not give you past system calls from the already-running
process.  What's done is done.  It will only give you system calls
that are made from that point forward.

>  The way I understand the PID options of strace:
> 
>  it seems to be attaching to some already running deamon or server process.

Yes.  If that's what you want.

>  How do you make it swallow a process that would go in one step/moment?

Don't use -p.  -p connects to an existing PID.  Without -p you create a
whole new process.

>  3) I have noticed that strace output is not totally predictable, so
> "parsing" is not that safe, straightforward

It's not meant to be parsed.  It's meant to be read by YOU, as part of
your debugging or diagnostics.  You use this tool when you want information
that nothing else will give you.

The strace output is long, detailed, and difficult to understand.  It
will take some practice to learn how to interpret it.

> $ strace wget --help  2>&1 | tail -n 5
> Email bug reports, questions, discussions to 
> and/or open issues at https://savannah.gnu.org/bugs/?func=additem=wget.
> ) = 956
> exit_group(0)   = ?
> +++ exited with 0 +++
> $

You're talking about timing and buffering issues here.  If these are
a concern, don't use stderr.  Use strace's -o option to log the trace
to a logfile.  That way it will not intermix with the program's standard
output and standard error.

Remember that standard output, when NOT going to a terminal, is usually
buffered.  In your example there, you've got wget's output going to a pipe
(which is not a terminal), so it gets buffered.  This can cause weird
artifacts, if you were expecting each line to be visible immediately for
example.  Even worse, you've got wget's stdout and stderr, and strace's
stderr, all mixed together into a single stream, with who-knows-what
kind of buffering on each component.



Re: difference in seconds between two formatted dates ...

2023-12-26 Thread Albretch Mueller
On 12/26/23, Thomas Schmitt  wrote:
> man strace says:
>   -o filename Write the trace output to the file  filename  rather
>   than  to  stderr.
>
> So strace normally directs its output to stderr. That's file descriptor 2.
> You can redirect it by the "2>file" gesture of the shell:
>
>   $ strace echo hello 2>file
>   hello
>   $ cat file
>   execve("/bin/echo", ["echo"], [/* 35 vars */]) = 0
>   brk(0)  = 0x13ba000
>   ...
>   exit_group(0)   = ?
>   +++ exited with 0 +++
>
> Or you may use the mentioned -o option which will keep the tracee's stderr
> out of the file:
>
>   $ strace -o file echo hello
>   hello
>   $ cat file
>   ...

 three questions:

 1) how do you set up the process to be straced as a parameter? Something like:
prx="echo hello"
logfile="file.txt"
#
strace "${prx}" 2>"${logfile}"
ls -l "${logfile}"; wc -l "${logfile}"; cat "${logfile}"

 2) how do you know for sure that you are stracing the same process
that you are running and is logging some data? I had read up from the
links that I included that there are ways to make the shell spit the
process number before a process is run, but how do you then insert the
stracing segment in the one liner in order the use that process id
before the process is actually run?

 The way I understand the PID options of strace:

strace --help ...
 -p PID, --attach=PID: trace process with process id PID, may be repeated
 --tips[=[[id:]ID][,[format:]FORMAT]]: show strace tips, tricks, and
tweaks on exit
 id: non-negative integer or random; default is random
 format: none, compact, full; default is compact

 it seems to be attaching to some already running deamon or server process.

 How do you make it swallow a process that would go in one step/moment?

 3) I have noticed that strace output is not totally predictable, so
"parsing" is not that safe, straightforward

 prx="echo hello"
logfile="file.txt"
#
strace "${prx}" 2>"${logfile}"
ls -l "${logfile}"; wc -l "${logfile}"; cat "${logfile}"


$ strace file strace.wlog.txt 2>&1 | tail -n 5
write(1, "strace.wlog.txt: ASCII text, wit"..., 56strace.wlog.txt:
ASCII text, with very long lines (348)
) = 56
munmap(0x7f127f083000, 8281024) = 0
exit_group(0)   = ?
+++ exited with 0 +++

$ strace wget --help  2>&1 | tail -n 5
Email bug reports, questions, discussions to 
and/or open issues at https://savannah.gnu.org/bugs/?func=additem=wget.
) = 956
exit_group(0)   = ?
+++ exited with 0 +++
$

Notice the diffing "munmap(0x7f127f083000, 8281024) = 0" line
which I think comes from strace.

 lbrtchc



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread Thomas Schmitt
Hi,

Albretch Mueller wrote:
> But how do you strace a program saving the output (of the stracing)
> in a logfile while you also save that program's output without making
> it part of the stracing?

man strace says:

  -o filename Write the trace output to the file  filename  rather
  than  to  stderr.

So strace normally directs its output to stderr. That's file descriptor 2.
You can redirect it by the "2>file" gesture of the shell:

  $ strace echo hello 2>file
  hello
  $ cat file
  execve("/bin/echo", ["echo"], [/* 35 vars */]) = 0
  brk(0)  = 0x13ba000
  ...
  exit_group(0)   = ?
  +++ exited with 0 +++

Or you may use the mentioned -o option which will keep the tracee's stderr
out of the file:

  $ strace -o file echo hello
  hello
  $ cat file
  ...


Have a nice day :)

Thomas



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread Albretch Mueller
On 12/25/23, Greg Wooledge  wrote:
> If you want to see what a process is doing, there's strace.  It can
> even be told to follow all the children of a process (strace -f).

 But how do you strace a program saving the output (of the stracing)
in a logfile while you also save that program's output without making
it part of the stracing? Say you go:

$ strace -f wget --help

You can clearly see the output of "wget --help" tailgated as part of
the stracing (which, of course, you can parse out), but I want two
separate log files. One for the stracing and the other for the actual
output of that program you ran.

I found some posts suggesting that to be possible, but I couldn't get it right:

https://serverfault.com/questions/205498/how-to-get-pid-of-just-started-process
https://askubuntu.com/questions/137233/how-to-command-ping-display-time-and-date-of-ping/867500#867500

lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread jeremy ardley


On 26/12/23 10:05, Jeffrey Walton wrote:

I don't know why Z was used instead of UTC or GMT. Probably to save
space, and save some ink if a schedule was printed.



ZULU time is military, primarily NATO. The world is divided up into 
alphabetic time zones using the NATO phonetic alphabet with a few 
exceptions like India (the letter)


Being extremely pedantic, GMT and UT and UTC are all slightly different, 
UT and UTC particularly. GMT is now tied to UT. UTC is more precise than 
UT/GMT


Re: difference in seconds between two formatted dates ...

2023-12-25 Thread John Hasler
Jeff writes:
> I don't know why Z was used instead of UTC or GMT.

https://en.wikipedia.org/wiki/Coordinated_Universal_Time#Time_zones
-- 
John Hasler 
j...@sugarbit.com
Elmwood, WI USA



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread Jeffrey Walton
On Mon, Dec 25, 2023 at 8:43 PM Charlie Gibbs  wrote:
>
> On Mon Dec 25 12:01:59 2023 "Andrew M.A. Cater"  wrote:
>
>  > Yes - that's the obvious way. I set my machines to /etc/UTC (or
>  > /etc/GMT) and leave them there. No daylight saving time, no offsets -
>  > all logs unambiguous. That's why (worldwide) radio logkeeping is/was
>  > in UTC. If you're travelling in an aircraft, you don't _need_ to know
>  > ground time but you do need to know flight time against a reference
>  > time. The Royal Air Force keep to UTC wherever they are in the world
>  > for just this reason.
>
> Not just the RAF.  All aviation works in UTC, to avoid problems when
> flights cross time zone boundaries, and to keep wide-area weather
> forecasts sane.  Your average airline passenger never sees UTC,
> since airlines use it behind the scenes and convert it to local time
> for display purposes.  (That's why you can see some strange intervals
> between departure and arrival times.)

The US airlines I worked for in the late 1980s and 1990s used Zulu
time. If I recall correctly, flight arrivals and departures were
specified like 10:34Z or 23:10Z.

I don't know why Z was used instead of UTC or GMT. Probably to save
space, and save some ink if a schedule was printed.

I don't know if that is still the case.

Jeff



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread Charlie Gibbs

On Mon Dec 25 12:01:59 2023 "Andrew M.A. Cater"  wrote:

> Yes - that's the obvious way. I set my machines to /etc/UTC (or
> /etc/GMT) and leave them there. No daylight saving time, no offsets -
> all logs unambiguous. That's why (worldwide) radio logkeeping is/was
> in UTC. If you're travelling in an aircraft, you don't _need_ to know
> ground time but you do need to know flight time against a reference
> time. The Royal Air Force keep to UTC wherever they are in the world
> for just this reason.

Not just the RAF.  All aviation works in UTC, to avoid problems when
flights cross time zone boundaries, and to keep wide-area weather
forecasts sane.  Your average airline passenger never sees UTC,
since airlines use it behind the scenes and convert it to local time
for display purposes.  (That's why you can see some strange intervals
between departure and arrival times.)

As a side note, a similar dichotomy applies to airport designators;
passengers and baggage handlers only see the three-letter IATA codes
(e.g. YYZ for Toronto), while flight plans are filed using the 4-letter
ICAO codes (e.g. CYYZ for Toronto).  For the most part, Canadian ICAO
codes are the IATA code with a C in front, and American ICAO codes are
the IATA code with a K in front - but there are exceptions.  And ICAO
codes cover all registered airports, not just those with scheduled
airline service.

--
/~\  Charlie Gibbs  |  Life is perverse.
\ /|  It can be beautiful -
 X   I'm really at ac.dekanfrus |  but it won't.
/ \  if you read it the right way.  |-- Lily Tomlin



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread Greg Wooledge
On Mon, Dec 25, 2023 at 03:35:04PM +, Albretch Mueller wrote:
> On 12/21/23, Greg Wooledge  wrote:
> > So... this is interesting.  Apparently timedatectl doesn't simply look
> > at the target of /etc/localtime.  There's a DELAY before the value is
> > correctly reported.  This tells me that timedatectl is in communication
> > with some process (perhaps PID 1, I don't know), and this other process
> > only discovers that /etc/localtime has changed after some time has passed.
> > Is it *polling*?  I have no idea, but that's what it looks like.
> 
>  This thread has taken a life of its own and I have learned quite a
> bit from our back and forth. This is not how I intuitively thought it
> worked. I thought you had to actively ask the OS to update itself ...
> Now I am interested in learning all there is to be learned from this
> whole time keeping methodology and how it relates to systemd and the
> boot process.

Be sure to read the other responses to that message.  There's a weird
timing issue caused by the fact that running "timedatectl" triggers
a service process to run, unless one is already running.  This service
process only stays running for 30 seconds, and it only reads the
/etc/localtime symlink when it starts up.

So, if you've already got one running, it won't pick up a changed
/etc/localtime.  But if you wait until the current one dies, then the
*next* one will.

I have no idea why this subsystem was designed to work this way.  It
seems awkward to me, but I'll give them the benefit of the doubt
for now -- there's probably *some* reason to do it this way, even if
it's not immediately clear to me.

>  Is there a way to start the Linux kernel of a Debian Live running
> instance enabling you to log the whole process (in a more in depth way
> than dmesg) and then go "follow tcp" for each listed process in dmesg
> as you do with wireshark?

If you want to see what a process is doing, there's strace.  It can
even be told to follow all the children of a process (strace -f).



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread Albretch Mueller
On 12/21/23, Greg Wooledge  wrote:
> So... this is interesting.  Apparently timedatectl doesn't simply look
> at the target of /etc/localtime.  There's a DELAY before the value is
> correctly reported.  This tells me that timedatectl is in communication
> with some process (perhaps PID 1, I don't know), and this other process
> only discovers that /etc/localtime has changed after some time has passed.
> Is it *polling*?  I have no idea, but that's what it looks like.

 This thread has taken a life of its own and I have learned quite a
bit from our back and forth. This is not how I intuitively thought it
worked. I thought you had to actively ask the OS to update itself ...
Now I am interested in learning all there is to be learned from this
whole time keeping methodology and how it relates to systemd and the
boot process.

 Is there a way to start the Linux kernel of a Debian Live running
instance enabling you to log the whole process (in a more in depth way
than dmesg) and then go "follow tcp" for each listed process in dmesg
as you do with wireshark?

 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread Andrew M.A. Cater
On Mon, Dec 25, 2023 at 12:24:55AM +, Albretch Mueller wrote:
> On 12/25/23, David Wright  wrote:
> > On Sun 24 Dec 2023 at 23:05:53 (+), Albretch Mueller wrote:
> >> On 12/18/23, Max Nikulin  wrote:
> ...
> >> Why would %S be in the range
> >> second (00..60), instead of (00..59)?:
> >
> > Leap seconds—see the example already in the thread:
> >   https://lists.debian.org/debian-user/2023/12/msg00976.html
> 
>  So, a possible (the only?) solution to those kinds of problems would
> be to always and explicitly use UTC, right? Or, using the longitude 20
> West (just crossing Iceland, which is 60+ North) or 170 West (too
> close to "Vladimir Putin") where so few people live that I don't think
> that anyone would care about day time savings or any of that.
> 

Yes - that's the obvious way. I set my machines to /etc/UTC (or /etc/GMT)
and leave them there. No daylight saving time, no offsets - all logs
unambiguous. That's why (worldwide) radio logkeeping is/was in UTC.
If you're travelling in an aircraft, you don't _need_ to know ground time
but you do need to know flight time against a reference time. The Royal
Air Force keep to UTC wherever they are in the world for just this reason.

Anything else is an offset against the reference: if all your Linux
boxes have timestamps against the epoch - which can also be related
to a human time if you *have* to - you have a reference there..

The problem comes when someone gives you logs that are taken against a 
different reference. (See also mapping against Greenwhich meridian (UK)
and Paris meridian (France) for many years - two sets of maps that aren't
*hugely* different on a world scale but locally very different).

>  All kinds of software keep time diffs. I am trying to use it in an
> obvious human readable way right in the file names.
> 
>  lbrtchx
> 



Re: difference in seconds between two formatted dates ...

2023-12-25 Thread Tixy
On Sun, 2023-12-24 at 23:05 +, Albretch Mueller wrote:
[...]
> Why would %S be in the range second (00..60), instead of (00..59)?:

To support leap seconds [1].

[1] https://en.wikipedia.org/wiki/Leap_second

-- 
Tixy



Re: difference in seconds between two formatted dates ...

2023-12-24 Thread Greg Wooledge
On Mon, Dec 25, 2023 at 12:24:55AM +, Albretch Mueller wrote:
>  I am trying to use it in an
> obvious human readable way right in the file names.

With that restriction, "always use UTC" is going to be your best path
forward.  It should be possible to convert a well-chosen human-readable
time/date format in UTC to epoch time, and vice versa.

How human-readable a UTC time/date string actually is will depend on the
human, of course.  But every other choice is going to be more difficult,
or more error-prone, or both.



Re: difference in seconds between two formatted dates ...

2023-12-24 Thread Albretch Mueller
On 12/25/23, David Wright  wrote:
> On Sun 24 Dec 2023 at 23:05:53 (+), Albretch Mueller wrote:
>> On 12/18/23, Max Nikulin  wrote:
...
>> Why would %S be in the range
>> second (00..60), instead of (00..59)?:
>
> Leap seconds—see the example already in the thread:
>   https://lists.debian.org/debian-user/2023/12/msg00976.html

 So, a possible (the only?) solution to those kinds of problems would
be to always and explicitly use UTC, right? Or, using the longitude 20
West (just crossing Iceland, which is 60+ North) or 170 West (too
close to "Vladimir Putin") where so few people live that I don't think
that anyone would care about day time savings or any of that.

 All kinds of software keep time diffs. I am trying to use it in an
obvious human readable way right in the file names.

 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-24 Thread David Wright
On Sun 24 Dec 2023 at 23:05:53 (+), Albretch Mueller wrote:
> On 12/18/23, Max Nikulin  wrote:
> > Timestamp format you have chosen is ambiguous.
> >
> > TZ=Europe/Berlin date -d '@1698539400' '+%Y%m%d%H%M%S'
> > 20231029023000
> >
> > TZ=Europe/Berlin date -d '@1698543000' '+%Y%m%d%H%M%S'
> > 20231029023000
> >
> > You had issues with setting time and timezone, so '+%s' may give
> > incorrect results.
> 
> Hmm! and one hour difference is not detected, because of the way is
> being parsed. Why would that happen?

Spring forward and fall back—the clocks change, skipping an hour in
spring and repeating an hour in autumn (Northern hemisphere).

> Why would %S be in the range
> second (00..60), instead of (00..59)?:

Leap seconds—see the example already in the thread:

  https://lists.debian.org/debian-user/2023/12/msg00976.html

> seks00=1698539400; TZ=Europe/Berlin date -d '@1698539400' '+%Y%m%d%H%M%S'
> 
> seks02=1698543000; TZ=Europe/Berlin date -d '@1698543000' '+%Y%m%d%H%M%S'
> 
> diff_seks=$(( seks02 - seks00 ))
> echo "// __ \$diff_seks: |$diff_seks|"
> 
> https://man7.org/linux/man-pages/man1/date.1.html
> %Y year
> %m month (01..12)
> %d day of month (e.g., 01)
> %H hour (00..23)
> %M minute (00..59)
> %S second (00..60)

Cheers,
David.



Re: difference in seconds between two formatted dates ...

2023-12-24 Thread Albretch Mueller
On 12/18/23, Max Nikulin  wrote:
> Timestamp format you have chosen is ambiguous.
>
> TZ=Europe/Berlin date -d '@1698539400' '+%Y%m%d%H%M%S'
> 20231029023000
>
> TZ=Europe/Berlin date -d '@1698543000' '+%Y%m%d%H%M%S'
> 20231029023000
>
> You had issues with setting time and timezone, so '+%s' may give
> incorrect results.

Hmm! and one hour difference is not detected, because of the way is
being parsed. Why would that happen? Why would %S be in the range
second (00..60), instead of (00..59)?:

seks00=1698539400; TZ=Europe/Berlin date -d '@1698539400' '+%Y%m%d%H%M%S'

seks02=1698543000; TZ=Europe/Berlin date -d '@1698543000' '+%Y%m%d%H%M%S'

diff_seks=$(( seks02 - seks00 ))
echo "// __ \$diff_seks: |$diff_seks|"

https://man7.org/linux/man-pages/man1/date.1.html
%Y year
%m month (01..12)
%d day of month (e.g., 01)
%H hour (00..23)
%M minute (00..59)
%S second (00..60)
~
 C



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread tomas
On Fri, Dec 22, 2023 at 09:15:27AM +0700, Max Nikulin wrote:
> On 20/12/2023 22:04, Greg Wooledge wrote:
> > The key point here is that you don't STORE these human-readable time
> > strings anywhere.  You simply *produce*  them on demand, using the
> > epoch time values that you *do*  store.
> 
> Greg, I agree to almost everything you write, however I believe that text
> timestamp representation is perfectly valid. To be clear, I consider UTC
> formatted time as human readable, despite some people are not comfortable
> with it:
> 
> date --utc --rfc-3339=seconds
> 2023-12-22 01:38:50+00:00

As long as the time offset is there, you are my guest :-)

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-21 Thread Greg Wooledge
On Thu, Dec 21, 2023 at 07:31:31PM -0600, David Wright wrote:
> Bear in mind that I was explaining my use of "all-UTC machine".
> Were you to construct such a beast, I think the first thing you
> might set, actively, is the RTC. You wouldn't just assume that
> it was already set to UTC.
> 
> What would be a better term for such a machine in the state described?

A non-networked machine with its default time zone set to UTC.  The
admin would need to keep setting the clock by hand as it drifts, but
otherwise, this is not a special or unusual setup... if this were 1990.

> Anyway, having followed these actions, someone could now write and
> test scripts without worrying about timezones, and then find out that
> they fail when someone in the real world runs them.

A cynic might observe that nobody else in the world is ever going to
run those scripts.  But yeah, this is a hive of bugs being hatched.
We've pointed out the obvious ones, and that's all we can do for now.



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread Max Nikulin

On 20/12/2023 22:04, Greg Wooledge wrote:

The key point here is that you don't STORE these human-readable time
strings anywhere.  You simply *produce*  them on demand, using the
epoch time values that you *do*  store.


Greg, I agree to almost everything you write, however I believe that 
text timestamp representation is perfectly valid. To be clear, I 
consider UTC formatted time as human readable, despite some people are 
not comfortable with it:


date --utc --rfc-3339=seconds
2023-12-22 01:38:50+00:00

or

date --utc --iso-8601=seconds
2023-12-22T01:39:07+00:00

from my point of view, it is no worse than seconds since epoch as 
integer. In the commands above "--utc" may be omitted since timezone 
offset is explicitly specified.


Latest activity related to text timestamp representation (known to me) 
is the following draft

https://datatracker.ietf.org/doc/draft-ietf-sedate-datetime-extended/
"Date and Time on the Internet: Timestamps with additional information"

I would try to avoid storing timestamp in file name. If it is absolutely 
necessary and dates are limited to 4-digit years then I would still prefer


date --utc '+%Y%m%d%H%M%S'
20231222014904

to %s seconds (assuming that 60 produced by %S does not cause parser 
error). I still consider this format as human readable despite some 
inconvenience. I would consider adding "Z" suffix to make it clear that 
it is UTC time rather than local one.


Seconds since epoch are suitable in most cases when binary storage is 
available. For text storage formatted date may be better. Extra care is 
required to properly store formatted local time.


Back to the code posted by Albretch.

I would avoid BASH for any sufficiently complex problem, so I agree with 
suggestions posted earlier. On the other hand parser implementation in 
date(1) may be more reliable that date-time libraries for some 
programming languages.


Invoking date without "--utc" I consider as a call for a trouble due to 
absence of time zone offset:


date '+%Y%m%d%H%M%S'  # Do not do it

The code may be accidentally run inside an environment with configured 
timezone.


I am not sure that UTC is really UTC on that particular machine due to 
the following thread, so bizarre results might be expected:

https://lists.debian.org/msgid-search/CAFakBwhvEThCnmbfSA1uZJw1BTTznY+J=gfegq1nqrrkzbm...@mail.gmail.com
differences between hwclock <-> date due to time zone issues? ...
Thu, 23 Mar 2023 21:41:40 +



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread David Wright
On Thu 21 Dec 2023 at 07:15:12 (-0500), Greg Wooledge wrote:
> On Wed, Dec 20, 2023 at 10:52:33PM -0600, David Wright wrote:
> > Sorry for the synecdoche, but I think it expresses the comprehensive
> > setting of UTC across the entirety of the computer and its operating
> > system, from the RTC, through /etc/timezone and /etc/localhost, to
> > the users' sessions. By this active (not just default) means, users
> > can remain blissfully unaware of the effects of setting timezones
> > other than UTC, just as the OP appeared to be, until reminded.
> 
> I'm not even sure what you're trying to say here.

I'm trying to explain something I thought was a simple concept,
an "all-UTC machine". So I suggested how you might make one.
Take a PC, turn it on, and set the CMOS clock to UTC. Boot it up,
run dpkg-reconfigure tzdata and set UTC as the timezone.

> "Active"?  Do you
> think /etc/timezone and /etc/localhost somehow have agency?  That
> they have intent?

No, I just meant that /you/ actively set everything that you could
to UTC, as if your universe was set in London on a wintry day.

> They're just settings.

Yes, and I just meant that you'd have to set the machine's System
Timetone to UTC. I'll hazard a guess that most people here won't
have that already set as their system timezone.

What would "passive" settings be? I don't know, but I've just
seen the term used in this thread:

  https://lists.debian.org/debian-user/2023/12/msg01131.html

> As far as the RTC (real time clock) goes, that just exists to
> bootstrap the system clock at boot time, before NTP takes over.
> If the system isn't connected to a network with a time server
> available, then of course NTP never takes over, and the system clock
> tries its best to keep up with time based on the initial RTC value,
> unless/until a sysadmin decides to run a date command to set the
> system clock more accurately.

Exactly, so the RTC is the primary source of time for a system in
the so-called "unexposed" mode of operation.

> Again, there isn't any agency here.  The RTC is just a resource that
> the system can use, once per boot, to get things started.  It could
> be set correctly, or incorrectly.  It could be set to local time, as
> was common when dual-booting with Windows, or to UTC.  On systems
> that run NTP, the RTC is mostly vestigial.  Its setting has very
> little effect on anything -- perhaps some early logfile timestamps.

Bear in mind that I was explaining my use of "all-UTC machine".
Were you to construct such a beast, I think the first thing you
might set, actively, is the RTC. You wouldn't just assume that
it was already set to UTC.

What would be a better term for such a machine in the state described?

Anyway, having followed these actions, someone could now write and
test scripts without worrying about timezones, and then find out that
they fail when someone in the real world runs them. Which is what
I posted in:

  https://lists.debian.org/debian-user/2023/12/msg00915.html

on my "exposed" "America/Chicago machine".

Cheers,
David.



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread David Wright
On Thu 21 Dec 2023 at 06:38:55 (+0100), to...@tuxteam.de wrote:
> On Wed, Dec 20, 2023 at 10:52:33PM -0600, David Wright wrote:
> > On Wed 20 Dec 2023 at 08:37:46 (+0100), to...@tuxteam.de wrote:
> > > On Wed, Dec 20, 2023 at 12:00:29AM -0600, David Wright wrote:
> > > 
> > > [...]
> > > 
> > > > Yes, I'm guessing that the OP is in my timezone, as just a few of
> > > > their previous posts have -5/-6 offsets. But most are +0, and
> > > > I wonder whether the OP ran this code on an all-UTC machine.
> > > > (IDK whether their using gmail is relevant.)
> > > 
> > > Nitpick and reminder: in UNIX and cousins, the "machine" has no
> > > timezone. It's the executable (and its children, if they don't
> > > change it). See:
> > > 
> > >   tomas@trotzki:~$ date
> > >   Wed Dec 20 08:24:32 CET 2023
> > >   tomas@trotzki:~$ TZ=Asia/Singapore bash
> > >   tomas@trotzki:~$ date
> > >   Wed Dec 20 15:24:47 +08 2023
> > >   tomas@trotzki:~$ exit
> > > 
> > > What is /etc/timezone for, then? you may ask.
> > > 
> > > It's just the default for when you don't pick any.
> > 
> > Sorry for the synecdoche, but I think it expresses the comprehensive
> > setting of UTC across the entirety of the computer and its operating
> > system, from the RTC, through /etc/timezone and /etc/localhost, to
> > the users' sessions.
> 
> Now I'm confused. The timezone is just a (pointer to a) set of rules
> stating how to translate UNIX time into a human readable form. So it
> just touches applications intending to show times to a human.
> 
> What is the RTC doing here, then?

I was under the impression that the OP had a mode called "unexposed".
I assumed that meant that the machine was isolated from other
machines. Perhaps I've overlooked some other method of initialising
System Time after booting up in unexposed mode, other than the RTC.

Cheers,
David.



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread gene heskett

On 12/21/23 16:31, Greg Wooledge wrote:

On Thu, Dec 21, 2023 at 03:34:49PM -0500, gene heskett wrote:

unforch it does not create them and in my recent experience it may run but
does not work without being able to log.


That would be a bug, given that this stats directory is apparently
optional.  (It logs through syslog just fine without the stats dir.)

What evidence did you see that makes you think it wasn't working?

.
A nominal +12 minute error was not corrected despite several restarts of 
ntpsec. Reading the .conf it finally dawned that I did not have that 
directory, so I made it, chowned it to ntpsec:ntpsec and by the time I 
had looked to see the that it was using that dir, I found the clock was 
by then about .0017 microseconds off.  Serendipity? DarnedifIknow.


Take care, stay warm and well, Greg.

Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis



Re: RTC and (old) Windows [was: difference in seconds between two formatted dates ...]

2023-12-21 Thread David Christensen

On 12/21/23 04:22, to...@tuxteam.de wrote:


I used to work in a shop Back Then (TM) (roughly Windows 3.1). We did
C programs for a living and had a mix of Windows boxes and Linux boxes.

Windows boxes were "naive" and had local time. We had a time zone
with summer and winter time.

On time transitions, all hell broke loose with Makefiles, which look
at file time stamps :-)

We ended up setting the Windows boxes to Monrovia/Liberia: no time
jumps *and* (more or less) GMT. No more hassles...



That is a great idea -- thank you!  :-)


David




Re: systemd and timezone (was: Re: difference in seconds between two formatted dates ...)

2023-12-21 Thread Nicolas George
to...@tuxteam.de (12023-12-21):
> I've sometimes the impression that desktop environments are losing
> the concept pf multi-user operating systems and are regreding to
> something like Windows 95.

Desktop environment and the “modern” applications designed for them had
already lost the ability to put back everything in place by quitting and
restarting. Now they are losing the concept of multiple users, and they
are also losing the ability to run several independent instances of the
same program.

Desktop environment suck.

Regards,

-- 
  Nicolas George



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread Greg Wooledge
On Thu, Dec 21, 2023 at 03:34:49PM -0500, gene heskett wrote:
> unforch it does not create them and in my recent experience it may run but
> does not work without being able to log.

That would be a bug, given that this stats directory is apparently
optional.  (It logs through syslog just fine without the stats dir.)

What evidence did you see that makes you think it wasn't working?



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread gene heskett

On 12/21/23 15:04, Greg Wooledge wrote:

On Thu, Dec 21, 2023 at 02:51:50PM -0500, gene heskett wrote:

can us see your /etc/ntpsec/ntp.conf?  And, do you have a
/var/log/ntpsec subdir ownwd by ntpsec:ntpsec?


unicorn:~$ ls -ld /var/log/ntpsec /etc/ntpsec/ntp.conf
ls: cannot access '/var/log/ntpsec': No such file or directory
-rw-r--r-- 1 root root 1922 Jan 16  2023 /etc/ntpsec/ntp.conf

My ntp.conf file was migrated from a Debian 11 /etc/ntp.conf
file, with whatever adjustments the ntp -> ntpsec transition scripts
did to it.  Should be very generic, but here you go.

==
# /etc/ntpsec/ntp.conf, configuration for ntpd; see ntp.conf(5) for help

driftfile /var/lib/ntpsec/ntp.drift
leapfile /usr/share/zoneinfo/leap-seconds.list

# To enable Network Time Security support as a server, obtain a certificate
# (e.g. with Let's Encrypt), configure the paths below, and uncomment:
# nts cert CERT_FILE
# nts key KEY_FILE
# nts enable

# You must create /var/log/ntpsec (owned by ntpsec:ntpsec) to enable logging.
#statsdir /var/log/ntpsec/
#statistics loopstats peerstats clockstats
#filegen loopstats file loopstats type day enable
#filegen peerstats file peerstats type day enable
#filegen clockstats file clockstats type day enable

# This should be maxclock 7, but the pool entries count towards maxclock.
tos maxclock 11

# Comment this out if you have a refclock and want it to be able to discipline
# the clock by itself (e.g. if the system is not connected to the network).
tos minclock 4 minsane 3

# Specify one or more NTP servers.

# Public NTP servers supporting Network Time Security:
# server time.cloudflare.com nts

# pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
# pick a different set every time it starts up.  Please consider joining the
# pool: 
pool 0.debian.pool.ntp.org iburst
pool 1.debian.pool.ntp.org iburst
pool 2.debian.pool.ntp.org iburst
pool 3.debian.pool.ntp.org iburst

# Access control configuration; see /usr/share/doc/ntpsec-doc/html/accopt.html
# for details.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
restrict default kod nomodify nopeer noquery limited

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1
==

Now let's look for logs.

unicorn:/var/log$ sudo grep ntpsec *
[...]
syslog.1:2023-12-16T15:01:52.641110-05:00 unicorn ntpd[815]: statistics 
directory /var/log/ntpsec/ does not exist or is unwriteable, error No such file 
or directory

Well, look at that.  I wonder why the ntpsec package didn't create that.
Let's take a look at  and see if there's
already a report for it.

Here we go: 
"ntpsec: Missing /var/log/ntpsec is logged as an error"

So I guess one's expected to create this themselves, but only if they
care enough to do it...?  Weird.

.
unforch it does not create them and in my recent experience it may run 
but does not work without being able to log. I created it that subdir 
and chowned it,  then I went around to my other machines doing likewise 
and the other machines now use this one as a stratum 2 server. by 
removing the pool 0,1,2,3 entries from their ntp.conf is removing 6 
other machines from the traffic into debian's ntp server pool.


Those of us with goodly sized private networks hiding behind a NATing 
router should do that to reduce the load on debians ntp server pool.


Even with a stratum 2 rating, you are still within a microsecond of the 
cesium beam clock in Boulder Colorado USA. When I first set it up a week 
ago, I quickly found I was part of a pool and had external to my local 
network clients, but I finally found how to stop that.


Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread Greg Wooledge
On Thu, Dec 21, 2023 at 02:51:50PM -0500, gene heskett wrote:
> can us see your /etc/ntpsec/ntp.conf?  And, do you have a
> /var/log/ntpsec subdir ownwd by ntpsec:ntpsec?

unicorn:~$ ls -ld /var/log/ntpsec /etc/ntpsec/ntp.conf 
ls: cannot access '/var/log/ntpsec': No such file or directory
-rw-r--r-- 1 root root 1922 Jan 16  2023 /etc/ntpsec/ntp.conf

My ntp.conf file was migrated from a Debian 11 /etc/ntp.conf
file, with whatever adjustments the ntp -> ntpsec transition scripts
did to it.  Should be very generic, but here you go.

==
# /etc/ntpsec/ntp.conf, configuration for ntpd; see ntp.conf(5) for help

driftfile /var/lib/ntpsec/ntp.drift
leapfile /usr/share/zoneinfo/leap-seconds.list

# To enable Network Time Security support as a server, obtain a certificate
# (e.g. with Let's Encrypt), configure the paths below, and uncomment:
# nts cert CERT_FILE
# nts key KEY_FILE
# nts enable

# You must create /var/log/ntpsec (owned by ntpsec:ntpsec) to enable logging.
#statsdir /var/log/ntpsec/
#statistics loopstats peerstats clockstats
#filegen loopstats file loopstats type day enable
#filegen peerstats file peerstats type day enable
#filegen clockstats file clockstats type day enable

# This should be maxclock 7, but the pool entries count towards maxclock.
tos maxclock 11

# Comment this out if you have a refclock and want it to be able to discipline
# the clock by itself (e.g. if the system is not connected to the network).
tos minclock 4 minsane 3

# Specify one or more NTP servers.

# Public NTP servers supporting Network Time Security:
# server time.cloudflare.com nts

# pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
# pick a different set every time it starts up.  Please consider joining the
# pool: 
pool 0.debian.pool.ntp.org iburst
pool 1.debian.pool.ntp.org iburst
pool 2.debian.pool.ntp.org iburst
pool 3.debian.pool.ntp.org iburst

# Access control configuration; see /usr/share/doc/ntpsec-doc/html/accopt.html
# for details.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
restrict default kod nomodify nopeer noquery limited

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1
==

Now let's look for logs.

unicorn:/var/log$ sudo grep ntpsec *
[...]
syslog.1:2023-12-16T15:01:52.641110-05:00 unicorn ntpd[815]: statistics 
directory /var/log/ntpsec/ does not exist or is unwriteable, error No such file 
or directory

Well, look at that.  I wonder why the ntpsec package didn't create that.
Let's take a look at  and see if there's
already a report for it.

Here we go: 
"ntpsec: Missing /var/log/ntpsec is logged as an error"

So I guess one's expected to create this themselves, but only if they
care enough to do it...?  Weird.



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread gene heskett

On 12/21/23 07:38, Greg Wooledge wrote:

On Thu, Dec 21, 2023 at 06:08:26AM +, Albretch Mueller wrote:

  and what would the systemd way to synch the RTC (Real Time Clock) and
UTC?


I don't understand this question at all.  The system clock value is
normally written to the RTC as a backup when the system shuts down.
Then, the RTC value is read at boot time to initialize the system clock.


Why is it I am noticing a 14 seconds difference on my computer
(booted with a Debian Live DVD)?


Because you're not networked?  If the system has no time sources to draw
upon, other than its own battery-backed RTC, then it will continue to
drift farther and farther from the correct time.


$ timedatectl
Local time: Thu 2023-12-21 00:52:20 UTC
Universal time: Thu 2023-12-21 00:52:20 UTC
  RTC time: Thu 2023-12-21 00:52:06
 Time zone: Etc/UTC (UTC, +)
System clock synchronized: no
   NTP service: n/a
   RTC in local TZ: no


I don't think this command's output is accurate for systems using NTP
services that *aren't* systemd's.  I'm running ntpsec on mine, and I
also get that same "NTP service: n/a" line.

However, I also get "System clock synchronized: yes".  I'm honestly
not sure what those two lines mean.  I don't know how far I would
trust this command, on systems that are not fully invested in the
systemd takeover.

Hmm... let's try a brief experiment.

unicorn:~$ sudo ln -sf /usr/share/zoneinfo/America/Chicago /etc/localtime
unicorn:~$ timedatectl | grep -m1 zone
 Time zone: America/Chicago (CST, -0600)
unicorn:~$ sudo ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime
unicorn:~$ timedatectl | grep -m1 zone
 Time zone: America/Chicago (CST, -0600)
unicorn:~$ timedatectl | grep -m1 zone
 Time zone: America/New_York (EST, -0500)

There was a fair bit of time elapsed between those last two commands,
as I was busy pasting things into this email.  I don't know how long,
exactly.  More than a second, but less than two minutes.

So... this is interesting.  Apparently timedatectl doesn't simply look
at the target of /etc/localtime.  There's a DELAY before the value is
correctly reported.  This tells me that timedatectl is in communication
with some process (perhaps PID 1, I don't know), and this other process
only discovers that /etc/localtime has changed after some time has passed.
Is it *polling*?  I have no idea, but that's what it looks like.

More and more reasons not to let systemd touch my clock.  Not that I
needed more of them, but... here we stand.

.

can us see your /etc/ntpsec/ntp.conf?  And, do you have a
/var/log/ntpsec subdir ownwd by ntpsec:ntpsec?

Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread Greg Wooledge
On Thu, Dec 21, 2023 at 10:36:06PM +0700, Max Nikulin wrote:
> I have another guess. systemd-timedated is activated on demand and reads
> /etc/localtime. It exits a half of a minute later. Perhaps second command
> caused start of new process since the old one was dead already.

Hmm.  OK, logs do seem to support that this could be the case.

unicorn:~$ sudo systemctl status systemd-timedated
[...]
Dec 21 07:18:14 unicorn systemd[1]: Starting systemd-timedated.service - Time &>
Dec 21 07:18:14 unicorn systemd[1]: Started systemd-timedated.service - Time & >
Dec 21 07:18:44 unicorn systemd[1]: systemd-timedated.service: Deactivated succ>
Dec 21 07:25:47 unicorn systemd[1]: Starting systemd-timedated.service - Time &>
Dec 21 07:25:47 unicorn systemd[1]: Started systemd-timedated.service - Time & >
Dec 21 07:26:39 unicorn systemd[1]: systemd-timedated.service: Deactivated succ>
Dec 21 07:26:59 unicorn systemd[1]: Starting systemd-timedated.service - Time &>
Dec 21 07:26:59 unicorn systemd[1]: Started systemd-timedated.service - Time & >
Dec 21 07:27:29 unicorn systemd[1]: systemd-timedated.service: Deactivated succ>

> I do not think that it expect that something changes /etc/localtime behind
> the scene. I admit inotify might be implemented, but expected way is to call
> "timedatectl set-timezone ZONE".

"Expected" by... well, not by *me*, that's for sure.  Maybe expected by
systemd developers?  Now I'm really curious what that command does.
Let's find out.

unicorn:~$ sudo timedatectl set-timezone America/Chicago
unicorn:~$ ls -ld /etc/timezone /etc/localtime
lrwxrwxrwx 1 root root 37 Dec 21 12:18 /etc/localtime -> 
../usr/share/zoneinfo/America/Chicago
-rw-r--r-- 1 root root 17 Dec  9 07:33 /etc/timezone

Looks like it does NOT know about Debian's legacy /etc/timezone file, and
does not update it.  I therefore cannot recommend that anyone on a Debian
system use this command to change their time zone, unless they follow it
up by manually editing /etc/timezone.



Re: systemd and timezone (was: Re: difference in seconds between two formatted dates ...)

2023-12-21 Thread tomas
On Thu, Dec 21, 2023 at 11:04:35AM -0500, Jeffrey Walton wrote:

[...]

> I think you will find a fair number of Unix & Linux servers set a
> default timezone. I sometimes have to set TZ in my bashrc because of
> an unexpected default timezone. Or that's been my experience at the
> GCC Compile Farm, .

But it's not the "server's timezone". It is the default timezone for
applications which haven't set one themselves (if they care about it
at all).

A "server's timezone" makes so much sense as a "server's $PATH setting".
Or its $LANG.

But I'm out of it.

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: systemd and timezone (was: Re: difference in seconds between two formatted dates ...)

2023-12-21 Thread Jeffrey Walton
On Thu, Dec 21, 2023 at 12:51 AM  wrote:
>
> On Thu, Dec 21, 2023 at 10:30:42AM +0700, Max Nikulin wrote:
>
> [...]
>
> > See systemd-timedated.service(8) and org.freedesktop.timedate1(5)
> >
> > busctl introspect org.freedesktop.timedate1 /org/freedesktop/timedate1
> > # Values are stripped
> > org.freedesktop.DBus.Properties interface -
>
> [...]
>
> > Desktop environments use this interface.
>
> Ugh.
>
> [...]
>
> > I would not be surprised to find an "Automatic time zone" checkbox in GUI
> > settings similar to e.g. Android.
>
> Double ugh.
>
> UNIX got that right from the start. Now this crazy notion "the computer
> HAS to have a timezone of its own" is creeping in.
>
> Glad I stay clear from that "Desktop" craze. Thanks for giving me yet
> another reason :-)

I think you will find a fair number of Unix & Linux servers set a
default timezone. I sometimes have to set TZ in my bashrc because of
an unexpected default timezone. Or that's been my experience at the
GCC Compile Farm, .

Jeff



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread Max Nikulin

On 21/12/2023 19:38, Greg Wooledge wrote:

On Thu, Dec 21, 2023 at 06:08:26AM +, Albretch Mueller wrote:



Why is it I am noticing a 14 seconds difference on my computer
(booted with a Debian Live DVD)?


Have you executed any commands setting time since boot? Does the 
difference remain after reboot?



$ timedatectl
Local time: Thu 2023-12-21 00:52:20 UTC
Universal time: Thu 2023-12-21 00:52:20 UTC
  RTC time: Thu 2023-12-21 00:52:06
 Time zone: Etc/UTC (UTC, +)
System clock synchronized: no
   NTP service: n/a
   RTC in local TZ: no


I don't think this command's output is accurate for systems using NTP
services that *aren't* systemd's.  I'm running ntpsec on mine, and I
also get that same "NTP service: n/a" line.


You may try to pull more NTP-related info from timedatectl. I am unsure 
if ntpsec is supported.



However, I also get "System clock synchronized: yes".  I'm honestly
not sure what those two lines mean.


"NTPSynchronized shows whether the kernel reports the time as 
synchronized (c.f.  adjtimex(3))."



Hmm... let's try a brief experiment.

[...]

unicorn:~$ timedatectl | grep -m1 zone
 Time zone: America/Chicago (CST, -0600)
unicorn:~$ timedatectl | grep -m1 zone
 Time zone: America/New_York (EST, -0500)

[...]

So... this is interesting.  Apparently timedatectl doesn't simply look
at the target of /etc/localtime.  There's a DELAY before the value is
correctly reported.  This tells me that timedatectl is in communication
with some process (perhaps PID 1, I don't know), and this other process
only discovers that /etc/localtime has changed after some time has passed.


I have another guess. systemd-timedated is activated on demand and reads 
/etc/localtime. It exits a half of a minute later. Perhaps second 
command caused start of new process since the old one was dead already.


I do not think that it expect that something changes /etc/localtime 
behind the scene. I admit inotify might be implemented, but expected way 
is to call "timedatectl set-timezone ZONE".




Re: systemd and timezone (was: Re: difference in seconds between two formatted dates ...)

2023-12-21 Thread tomas
On Thu, Dec 21, 2023 at 09:08:09AM -0500, Dan Ritter wrote:
> Max Nikulin wrote: 
> > I am not going to discuss code posted by Albretch, despite it has serious
> > issues from my point of view. This is a response to Greg.
> > 
> > On 20/12/2023 22:04, Greg Wooledge wrote:
> > > The default time zone has nothing to do with systemd, nor with any other
> > > init system that may be in place.  Systemd does not know or care about
> > > the system's default time zone.
> > 
> > See systemd-timedated.service(8) and org.freedesktop.timedate1(5)
> > 
> > busctl introspect org.freedesktop.timedate1 /org/freedesktop/timedate1
> > # Values are stripped
> > org.freedesktop.DBus.Properties interface -
> > .PropertiesChanged  signalsa{sv}as
> > org.freedesktop.timedate1   interface -
> > .SetTimezonemethodsb
> > .Timezone   property  s
> > 
> > Desktop environments use this interface.
> 
> Is this set per-user? Because I certainly have multiple users on
> the same computer at the same time from different timezones. And
> it is quite possible on a few of those machines to have multiple
> desktop users, each from a different TZ.

I've sometimes the impression that desktop environments are losing
the concept pf multi-user operating systems and are regreding to
something like Windows 95.

But hey. I'm just an old fart ;-)

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: systemd and timezone (was: Re: difference in seconds between two formatted dates ...)

2023-12-21 Thread Dan Ritter
Max Nikulin wrote: 
> I am not going to discuss code posted by Albretch, despite it has serious
> issues from my point of view. This is a response to Greg.
> 
> On 20/12/2023 22:04, Greg Wooledge wrote:
> > The default time zone has nothing to do with systemd, nor with any other
> > init system that may be in place.  Systemd does not know or care about
> > the system's default time zone.
> 
> See systemd-timedated.service(8) and org.freedesktop.timedate1(5)
> 
> busctl introspect org.freedesktop.timedate1 /org/freedesktop/timedate1
> # Values are stripped
> org.freedesktop.DBus.Properties interface -
> .PropertiesChanged  signalsa{sv}as
> org.freedesktop.timedate1   interface -
> .SetTimezonemethodsb
> .Timezone   property  s
> 
> Desktop environments use this interface.

Is this set per-user? Because I certainly have multiple users on
the same computer at the same time from different timezones. And
it is quite possible on a few of those machines to have multiple
desktop users, each from a different TZ.

-dsr-



Re: difference in seconds between two formatted dates ...

2023-12-21 Thread Greg Wooledge
On Thu, Dec 21, 2023 at 06:08:26AM +, Albretch Mueller wrote:
>  and what would the systemd way to synch the RTC (Real Time Clock) and
> UTC?

I don't understand this question at all.  The system clock value is
normally written to the RTC as a backup when the system shuts down.
Then, the RTC value is read at boot time to initialize the system clock.

> Why is it I am noticing a 14 seconds difference on my computer
> (booted with a Debian Live DVD)?

Because you're not networked?  If the system has no time sources to draw
upon, other than its own battery-backed RTC, then it will continue to
drift farther and farther from the correct time.

> $ timedatectl
>Local time: Thu 2023-12-21 00:52:20 UTC
>Universal time: Thu 2023-12-21 00:52:20 UTC
>  RTC time: Thu 2023-12-21 00:52:06
> Time zone: Etc/UTC (UTC, +)
> System clock synchronized: no
>   NTP service: n/a
>   RTC in local TZ: no

I don't think this command's output is accurate for systems using NTP
services that *aren't* systemd's.  I'm running ntpsec on mine, and I
also get that same "NTP service: n/a" line.

However, I also get "System clock synchronized: yes".  I'm honestly
not sure what those two lines mean.  I don't know how far I would
trust this command, on systems that are not fully invested in the
systemd takeover.

Hmm... let's try a brief experiment.

unicorn:~$ sudo ln -sf /usr/share/zoneinfo/America/Chicago /etc/localtime
unicorn:~$ timedatectl | grep -m1 zone
Time zone: America/Chicago (CST, -0600)
unicorn:~$ sudo ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime
unicorn:~$ timedatectl | grep -m1 zone
Time zone: America/Chicago (CST, -0600)
unicorn:~$ timedatectl | grep -m1 zone
Time zone: America/New_York (EST, -0500)

There was a fair bit of time elapsed between those last two commands,
as I was busy pasting things into this email.  I don't know how long,
exactly.  More than a second, but less than two minutes.

So... this is interesting.  Apparently timedatectl doesn't simply look
at the target of /etc/localtime.  There's a DELAY before the value is
correctly reported.  This tells me that timedatectl is in communication
with some process (perhaps PID 1, I don't know), and this other process
only discovers that /etc/localtime has changed after some time has passed.
Is it *polling*?  I have no idea, but that's what it looks like.

More and more reasons not to let systemd touch my clock.  Not that I
needed more of them, but... here we stand.



RTC and (old) Windows [was: difference in seconds between two formatted dates ...]

2023-12-21 Thread tomas
On Thu, Dec 21, 2023 at 07:15:12AM -0500, Greg Wooledge wrote:

[...]

> Again, there isn't any agency here.  The RTC is just a resource that
> the system can use, once per boot, to get things started.  It could
> be set correctly, or incorrectly.  It could be set to local time, as
> was common when dual-booting with Windows, or to UTC.  On systems
> that run NTP, the RTC is mostly vestigial.  Its setting has very
> little effect on anything -- perhaps some early logfile timestamps.

Anecdote time:

I used to work in a shop Back Then (TM) (roughly Windows 3.1). We did
C programs for a living and had a mix of Windows boxes and Linux boxes.

Windows boxes were "naive" and had local time. We had a time zone
with summer and winter time.

On time transitions, all hell broke loose with Makefiles, which look
at file time stamps :-)

We ended up setting the Windows boxes to Monrovia/Liberia: no time
jumps *and* (more or less) GMT. No more hassles...

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-21 Thread Greg Wooledge
On Wed, Dec 20, 2023 at 10:52:33PM -0600, David Wright wrote:
> Sorry for the synecdoche, but I think it expresses the comprehensive
> setting of UTC across the entirety of the computer and its operating
> system, from the RTC, through /etc/timezone and /etc/localhost, to
> the users' sessions. By this active (not just default) means, users
> can remain blissfully unaware of the effects of setting timezones
> other than UTC, just as the OP appeared to be, until reminded.

I'm not even sure what you're trying to say here.  "Active"?  Do you
think /etc/timezone and /etc/localhost somehow have agency?  That
they have intent?

They're just settings.  When an application wants to convert an
epoch time to a date/time string, it looks for the TZ environment
variable, and if that's not present, it looks for either /etc/localtime
or /etc/timezone, depending on how it was programmed.

As far as the RTC (real time clock) goes, that just exists to
bootstrap the system clock at boot time, before NTP takes over.
If the system isn't connected to a network with a time server
available, then of course NTP never takes over, and the system clock
tries its best to keep up with time based on the initial RTC value,
unless/until a sysadmin decides to run a date command to set the
system clock more accurately.

Again, there isn't any agency here.  The RTC is just a resource that
the system can use, once per boot, to get things started.  It could
be set correctly, or incorrectly.  It could be set to local time, as
was common when dual-booting with Windows, or to UTC.  On systems
that run NTP, the RTC is mostly vestigial.  Its setting has very
little effect on anything -- perhaps some early logfile timestamps.



Re: difference in seconds between two formatted dates ...

2023-12-20 Thread tomas
On Thu, Dec 21, 2023 at 06:08:26AM +, Albretch Mueller wrote:
> On 12/21/23, Max Nikulin  wrote:
> > as a primary source. For majority using systemd
> 
>  and what would the systemd way to synch the RTC

  https://en.wikipedia.org/wiki/Network_Time_Protocol

> (Real Time Clock)

The "real time clock" is only relevant at boot time, when you
don't have access to an NTP server.

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-20 Thread Albretch Mueller
On 12/21/23, Max Nikulin  wrote:
> as a primary source. For majority using systemd

 and what would the systemd way to synch the RTC (Real Time Clock) and
UTC? Why is it I am noticing a 14 seconds difference on my computer
(booted with a Debian Live DVD)?

$ readlink /etc/localtime
/usr/share/zoneinfo/Etc/UTC

$ ls -l /etc/localtime
lrwxrwxrwx 1 root root 27 Dec 14 03:14 /etc/localtime ->
/usr/share/zoneinfo/Etc/UTC

$ timedatectl
   Local time: Thu 2023-12-21 00:52:20 UTC
   Universal time: Thu 2023-12-21 00:52:20 UTC
 RTC time: Thu 2023-12-21 00:52:06
Time zone: Etc/UTC (UTC, +)
System clock synchronized: no
  NTP service: n/a
  RTC in local TZ: no
$

 lbrtchx



Re: systemd and timezone (was: Re: difference in seconds between two formatted dates ...)

2023-12-20 Thread tomas
On Thu, Dec 21, 2023 at 10:30:42AM +0700, Max Nikulin wrote:

[...]

> See systemd-timedated.service(8) and org.freedesktop.timedate1(5)
> 
> busctl introspect org.freedesktop.timedate1 /org/freedesktop/timedate1
> # Values are stripped
> org.freedesktop.DBus.Properties interface -

[...]

> Desktop environments use this interface.

Ugh.

[...]

> I would not be surprised to find an "Automatic time zone" checkbox in GUI
> settings similar to e.g. Android.

Double ugh.

UNIX got that right from the start. Now this crazy notion "the computer
HAS to have a timezone of its own" is creeping in.

Glad I stay clear from that "Desktop" craze. Thanks for giving me yet
another reason :-)

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-20 Thread Max Nikulin

On 21/12/2023 11:37, David Wright wrote:

On Wed 20 Dec 2023 at 07:43:51 (+), Albretch Mueller wrote:

$ cat  /etc/localtime
TZif2UTCTZif2UTC
UTC0

Take care. That's a binary file. /etc/timezone is the text one.


That is why

readlink /etc/localtime

or

ls -l /etc/localtime

as a primary source. For majority using systemd, more detailed report 
may be get from


timedatectl

Consider /etc/timezone is a Debian-specific legacy. File a bug against 
packages that still use it. Check this file only to confirm that such 
legacy applications use consistent configuration


cat /etc/timezone



Re: difference in seconds between two formatted dates ...

2023-12-20 Thread tomas
On Wed, Dec 20, 2023 at 10:52:33PM -0600, David Wright wrote:
> On Wed 20 Dec 2023 at 08:37:46 (+0100), to...@tuxteam.de wrote:
> > On Wed, Dec 20, 2023 at 12:00:29AM -0600, David Wright wrote:
> > 
> > [...]
> > 
> > > Yes, I'm guessing that the OP is in my timezone, as just a few of
> > > their previous posts have -5/-6 offsets. But most are +0, and
> > > I wonder whether the OP ran this code on an all-UTC machine.
> > > (IDK whether their using gmail is relevant.)
> > 
> > Nitpick and reminder: in UNIX and cousins, the "machine" has no
> > timezone. It's the executable (and its children, if they don't
> > change it). See:
> > 
> >   tomas@trotzki:~$ date
> >   Wed Dec 20 08:24:32 CET 2023
> >   tomas@trotzki:~$ TZ=Asia/Singapore bash
> >   tomas@trotzki:~$ date
> >   Wed Dec 20 15:24:47 +08 2023
> >   tomas@trotzki:~$ exit
> > 
> > What is /etc/timezone for, then? you may ask.
> > 
> > It's just the default for when you don't pick any.
> 
> Sorry for the synecdoche, but I think it expresses the comprehensive
> setting of UTC across the entirety of the computer and its operating
> system, from the RTC, through /etc/timezone and /etc/localhost, to
> the users' sessions.

Now I'm confused. The timezone is just a (pointer to a) set of rules
stating how to translate UNIX time into a human readable form. So it
just touches applications intending to show times to a human.

What is the RTC doing here, then?

(Now, yes, there is a provision for telling the system that the RTC is
broken, because it is shared with another, crippled, operating system,
but this is another kettle of fish and has nothing to do with /etc/timezone,
and luckily is slowly fading away).

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-20 Thread David Wright
On Wed 20 Dec 2023 at 08:37:46 (+0100), to...@tuxteam.de wrote:
> On Wed, Dec 20, 2023 at 12:00:29AM -0600, David Wright wrote:
> 
> [...]
> 
> > Yes, I'm guessing that the OP is in my timezone, as just a few of
> > their previous posts have -5/-6 offsets. But most are +0, and
> > I wonder whether the OP ran this code on an all-UTC machine.
> > (IDK whether their using gmail is relevant.)
> 
> Nitpick and reminder: in UNIX and cousins, the "machine" has no
> timezone. It's the executable (and its children, if they don't
> change it). See:
> 
>   tomas@trotzki:~$ date
>   Wed Dec 20 08:24:32 CET 2023
>   tomas@trotzki:~$ TZ=Asia/Singapore bash
>   tomas@trotzki:~$ date
>   Wed Dec 20 15:24:47 +08 2023
>   tomas@trotzki:~$ exit
> 
> What is /etc/timezone for, then? you may ask.
> 
> It's just the default for when you don't pick any.

Sorry for the synecdoche, but I think it expresses the comprehensive
setting of UTC across the entirety of the computer and its operating
system, from the RTC, through /etc/timezone and /etc/localhost, to
the users' sessions. By this active (not just default) means, users
can remain blissfully unaware of the effects of setting timezones
other than UTC, just as the OP appeared to be, until reminded.

Obviously I have no idea what the OP's actual machine is set up for,
and I don't have any clue what their scheme is supposed to achieve.
Hence "I wonder whether …" quoted above.

Cheers,
David.



Re: difference in seconds between two formatted dates ...

2023-12-20 Thread David Wright
On Wed 20 Dec 2023 at 07:43:51 (+), Albretch Mueller wrote:
> On 12/20/23, David Wright  wrote:
> > To be fair to the OP, there was no official "script", but just some code:
> >   https://lists.debian.org/debian-user/2023/12/msg00894.html
> > which I pasted into /tmp/lbrtchx.sh. The filename suffix was a mere
> > convenience to make emacs colour the code and tidy the indentation,
> > speeding up finding lines broken by the MUA. I then ran it with:
> >   $ bash /tmp/lbrtchx.sh
> >
> >> It *looks* like this command is trying to take a date/time string in
> >> one format, and convert it to a different format, and then append a
> >> +00:00 time zone offset even though that's not the correct offset for
> >> the author's time zone (as far as I know).
> >
> > Yes, I'm guessing that the OP is in my timezone, as just a few of
> > their previous posts have -5/-6 offsets. But most are +0, and
> > I wonder whether the OP ran this code on an all-UTC machine.
> > (IDK whether their using gmail is relevant.)
> 
>  If I understand what you seem to not understand about my work around
> to get a time difference in seconds out of my "cobblesome" date
> formatting for file names, this is my time zone
> 
> $ date +%z
> +

I guessed that. I didn't "understand" it because you never stated it.

> (remember I am using a
> Debian Live DVD, on "exposed mode" ;-))

What's that got to do with the price of fish?

> $ cat  /etc/localtime
> TZif2UTCTZif2UTC
> UTC0

Take care. That's a binary file. /etc/timezone is the text one.

>  At the end of the day, all I need is a time difference in seconds
> (and yes, milliseconds would be better), which would be the same
> regardless of your time zone.

A precise clock might be better, then. Computers deliberately play
tricks with time to make themselves more useful to the rest of us.

>  I do see the good in what you are suggesting to me and I will have to
> include time zones in the file names as well and deal with the
> possible cases (someone working at Charles de Gaulle Airport in
> Paris/France boards a plane to Boston Logan/MA/USA ...). But the
> actual question I will have to deal with is how to check and possibly
> reset the time zone and the time via a network in a reliable way once
> a ToG booted computer gains access to the Internet for which I will
> have to use systemd-timesyncd when it boots and shuts down/ when it
> changes modes.
> 
>  Am I clearer now?

I was only concerned with the date command's abilities and its
interaction with timezones. I'm certainly not going to dive into
what lies behind all this stuff, with your talk of "baselining
the system's state" and so on.

Cheers,
David.



systemd and timezone (was: Re: difference in seconds between two formatted dates ...)

2023-12-20 Thread Max Nikulin
I am not going to discuss code posted by Albretch, despite it has 
serious issues from my point of view. This is a response to Greg.


On 20/12/2023 22:04, Greg Wooledge wrote:


The selection of the computer's default time zone by its owner is not
in ANY way related to the computer's geographic location.

The default time zone has nothing to do with systemd, nor with any other
init system that may be in place.  Systemd does not know or care about
the system's default time zone.


See systemd-timedated.service(8) and org.freedesktop.timedate1(5)

busctl introspect org.freedesktop.timedate1 /org/freedesktop/timedate1
# Values are stripped
org.freedesktop.DBus.Properties interface -
.PropertiesChanged  signalsa{sv}as
org.freedesktop.timedate1   interface -
.ListTimezones  method-
.SetLocalRTCmethodbbb
.SetNTP methodbb
.SetTimemethodxbb
.SetTimezonemethodsb
.CanNTP property  b
.LocalRTC   property  b
.NTPproperty  b
.NTPSynchronizedproperty  b
.RTCTimeUSecproperty  t
.TimeUSec   property  t
.Timezone   property  s

Desktop environments use this interface.


Again, you're not making any sense.  How does this computer know where
it is geographically located?  Does it have GPS hardware inside it?
And even knowing a user's geographic location is *not* enough to know
which time zone the user wishes to use. Time zones are poliitical.


I would not be surprised to find an "Automatic time zone" checkbox in 
GUI settings similar to e.g. Android. I am unsure however if it has been 
implemented by any Debian package. Ubuntu installer is able to guess 
time zone. Likely it is based on a GeoIP database and an external server 
to determine outgoing IP address.


List of WiFi networks around is enough to get location without GPS. 
$LANG may improve this guess to resolve political (ethnic) ambiguity.


My impression is that there are enough users who do not know their 
timezone and enough users would be happy to get correct timezone 
automatically. Some of them choose a timezone having slight differences 
and later complain that the TZ DB is not precise.


To configure default timezone system-wide I still recommend

dpkg-reconfigure tzdata

since it handles the case of obsolete application. I would not neglect 
systemd though since it is widespread now. So output of the following 
commands maybe useful to diagnose issues related to localtime


timedatectl
set | grep -E '^(TZ|LANG|LC_.*)='




Re: difference in seconds between two formatted dates ...

2023-12-20 Thread Nicolas George
Greg Wooledge (12023-12-20):
> Huh?  None of this makes any sense.

I salute you for your patience and excellent message.

Just a few remarks:

> Learn how time works.  Then rewrite everything.

Fortunately, we do not have to handle relativistic issues à la
“meanwhile at the other side of the galaxy”…

> Any piece of software that wants to calculate durations needs to work
> with epoch times, or something equivalent to epoch times.

A small nitpick here NOT RELEVANT TO THE CURRENT DISCUSSION but
important if wannabe developers learn things reading you.

This is true for long durations, not short durations, with the
difference between the two being: is the system allowed to reboot in the
middle?

If the system might reboot between the beginning and the end, then you
are absolutely right, the “wall clock” of Unix systems is the right
choice.

But if the system cannot reboot unless it makes the duration irrelevant
(think: timeout for a network connection), then the right choice is not
the wall clock but the monotonic clock.

See:
https://pubs.opengroup.org/onlinepubs/95399/functions/clock_gettime.html

(For shell, the monotinic time is in /proc/uptime)

> So, to be completely blunt here, what you want to do is store ALL
> timestamps in epoch time format.  This could be a string like
> "1703082118", or it could be the raw 64-bit integer which this string
> represents.  Either way's fine, depending on your programming language
> and tool set.

A format that can be reliably converted with a rather simple algorithm
over the contemporary era, like the ISO time syntax, would be acceptable
too.

Last: a long time ago, I wrote this for French Usenet (and it somehow
found its way on this website; I should publish it on my own):

https://www.generation-nt.com/reponses/la-gestion-de-l-heure-sous-linux-entraide-500251.html

I think it might be of interest, and nowadays LLMs can translate it
correctly I guess.

Regards,

-- 
  Nicolas George



Re: difference in seconds between two formatted dates ...

2023-12-20 Thread Greg Wooledge
On Wed, Dec 20, 2023 at 01:12:35PM +, Albretch Mueller wrote:
>  The only way I see is for the running computer on "exposed mode" to
> check via systemd if the time zone has been changed

Huh?  None of this makes any sense.  First of all, the system's default
time zone only changes if a user with root privileges decides to change
it.  This is done by running "dpkg-reconfigure tzdata", or by manually
editing one file and redirecting one symbolic link, if you're stubborn.

The selection of the computer's default time zone by its owner is not
in ANY way related to the computer's geographic location.

The default time zone has nothing to do with systemd, nor with any other
init system that may be in place.  Systemd does not know or care about
the system's default time zone.  "Checking via systemd" is a phrase
with no meaning, in this case.

> and reset it

What?  You want your software to *undo* the choices made by the computer's
owner?

> in
> that case before you use date for file naming for all data that is
> kept in a measured way.

So... no, you didn't actually mean "reset the system's default time zone
to a previous value".  You mean something like "use the currently
selected default time zone" -- but that's what everything does all
the time, unless the TZ environment variable is set.

I think you need to brush up on the fundamentals here.

>  In the case of the person flying from Paris to Boston he will find
> out that he flew away from continental Europe once he reboots his
> computer in "exposed mode" (pun intended).

Again, you're not making any sense.  How does this computer know where
it is geographically located?  Does it have GPS hardware inside it?
And even knowing a user's geographic location is *not* enough to know
which time zone the user wishes to use.  Time zones are poliitical.
There are places on earth where two groups of people in the same
geographic location use two different time zones.  Because politics.

> He will notified of the
> time zone he had used before and all such deltas will be kept.

Stop this.  Stop this IMMEDIATELY.

Learn how time works.  Then rewrite everything.

The system clock stores time as an offset from a fixed point in time
known as "the epoch" -- which is midnight, 1 January 1970, UTC.
This storage form is known as "epoch time" or "Unix time".

When displaying the current time to a user, the epoch time value is
converted into a human-readable time string, using the user's chosen
locale and time zone.  Users select these things by setting environment
variables.  If the environment variables are not set, the system-wide
default values are used instead.

Right now, as I write this, the epoch time is 1703082118 which is
1.703... billion seconds after the epoch.  Using the standard Gregorian
calendar, in the America/New_York time zone, this epoch time value
can be displayed as "Wed Dec 20 09:21:58 EST 2023" which is a string
that makes sense to me, a human being, who lives in this time zone.
It could also be displayed as "Wed Dec 20 09:21:58 AM EST 2023" and
this still makes sense to me.  The difference between these two strings
is the locale definition that I, the end user, have chosen for my
environment.  It is a personal preference.

Any piece of software that wants to calculate durations needs to work
with epoch times, or something equivalent to epoch times.  Two moments
in time must be encoded in a way that they can be subtracted from each
other to determine how much time elapsed between moment one and moment
two.

Since we're on Unix systems which work with epoch time internally, there's
no reason to make up an equivalent, or to reinvent any wheels.  Just use
the epoch time values that the system clock is working with already.

So, to be completely blunt here, what you want to do is store ALL
timestamps in epoch time format.  This could be a string like
"1703082118", or it could be the raw 64-bit integer which this string
represents.  Either way's fine, depending on your programming language
and tool set.

Because you have all your timestamps in epoch format, calculating
intervals is easy -- you just subtract, and that's how many seconds
have elapsed.  You can convert this number of seconds to an interval
expressed in other units, like "1 hour, 23 minutes and 17 seconds"
using a bit of arithmetic.  Some programming languages may have tools
to do this for you.

If you want to display human-readable time strings corresponding to
your stored epoch times, then you use a tool which takes an epoch
time value as input, and produces a human-readable time string as
output.  This is *incredibly complicated* so you do *not* do this
yourself using stone knives and bear skins.  You use a *tool* that
has been developed and honed and debugged for decades.

Since your chosen language seems to be bash, you have two good choices
for the tool to do this: bash's builtin printf, or GNU date.

GNU date is easy enough:

unicorn:~$ date -d @1703082118
Wed Dec 20 09:21:58 

Re: difference in seconds between two formatted dates ...

2023-12-20 Thread tomas
On Wed, Dec 20, 2023 at 01:12:35PM +, Albretch Mueller wrote:
> On 12/20/23, to...@tuxteam.de  wrote:
> > On Wed, Dec 20, 2023 at 07:15:20AM -0500, Greg Wooledge wrote:
> >> On Wed, Dec 20, 2023 at 07:43:51AM +, Albretch Mueller wrote:
> >> >  I do see the good in what you are suggesting to me and I will have to
> >> > include time zones in the file names as well and deal with the
> >> > possible cases (someone working at Charles de Gaulle Airport in
> >> > Paris/France boards a plane to Boston Logan/MA/USA ...).
> >
> > How would this person's computer notice?
> 
>  The only way I see is for the running computer on "exposed mode" to
> check via systemd if the time zone has been changed and reset it in
> that case before you use date for file naming for all data that is
> kept in a measured way.

Lucky me, I'm still on SysV init

You still don't know how this timezone stuff works, do you?

Time zone is a *user* concept, not a *system* concept. The system
only knows about UNIX time, which is very close to UTC (but not
equal).

The only thing which might change your time "from the outside" is
ntp, to synchronize your clock (including the occasional leap second).

>  In the case of the person flying from Paris to Boston he will find
> out that he flew away from continental Europe once he reboots his
> computer in "exposed mode" (pun intended). He will notified of the
> time zone he had used before and all such deltas will be kept. The
> only measured data that will be deleted is the one which remains the
> same.

This is not how it works.

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-20 Thread Albretch Mueller
On 12/20/23, to...@tuxteam.de  wrote:
> On Wed, Dec 20, 2023 at 07:15:20AM -0500, Greg Wooledge wrote:
>> On Wed, Dec 20, 2023 at 07:43:51AM +, Albretch Mueller wrote:
>> >  I do see the good in what you are suggesting to me and I will have to
>> > include time zones in the file names as well and deal with the
>> > possible cases (someone working at Charles de Gaulle Airport in
>> > Paris/France boards a plane to Boston Logan/MA/USA ...).
>
> How would this person's computer notice?

 The only way I see is for the running computer on "exposed mode" to
check via systemd if the time zone has been changed and reset it in
that case before you use date for file naming for all data that is
kept in a measured way.

 In the case of the person flying from Paris to Boston he will find
out that he flew away from continental Europe once he reboots his
computer in "exposed mode" (pun intended). He will notified of the
time zone he had used before and all such deltas will be kept. The
only measured data that will be deleted is the one which remains the
same.

 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-20 Thread tomas
On Wed, Dec 20, 2023 at 07:15:20AM -0500, Greg Wooledge wrote:
> On Wed, Dec 20, 2023 at 07:43:51AM +, Albretch Mueller wrote:
> >  I do see the good in what you are suggesting to me and I will have to
> > include time zones in the file names as well and deal with the
> > possible cases (someone working at Charles de Gaulle Airport in
> > Paris/France boards a plane to Boston Logan/MA/USA ...).

How would this person's computer notice?

> You're writing code that's intended to be used by OTHER PEOPLE?
> 
> Do it CORRECTLY, then!

Oh, oh. I'd wish more in our profession heeded this. Then, I'd
wish my managers gave me the time to actually do it (instead of
me having to wrestle that time out of them, time and again).

:-)

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-20 Thread Greg Wooledge
On Wed, Dec 20, 2023 at 07:43:51AM +, Albretch Mueller wrote:
>  I do see the good in what you are suggesting to me and I will have to
> include time zones in the file names as well and deal with the
> possible cases (someone working at Charles de Gaulle Airport in
> Paris/France boards a plane to Boston Logan/MA/USA ...).

You're writing code that's intended to be used by OTHER PEOPLE?

Do it CORRECTLY, then!



Re: difference in seconds between two formatted dates ...

2023-12-19 Thread Albretch Mueller
On 12/20/23, David Wright  wrote:
> To be fair to the OP, there was no official "script", but just some code:
>   https://lists.debian.org/debian-user/2023/12/msg00894.html
> which I pasted into /tmp/lbrtchx.sh. The filename suffix was a mere
> convenience to make emacs colour the code and tidy the indentation,
> speeding up finding lines broken by the MUA. I then ran it with:
>   $ bash /tmp/lbrtchx.sh
>
>> It *looks* like this command is trying to take a date/time string in
>> one format, and convert it to a different format, and then append a
>> +00:00 time zone offset even though that's not the correct offset for
>> the author's time zone (as far as I know).
>
> Yes, I'm guessing that the OP is in my timezone, as just a few of
> their previous posts have -5/-6 offsets. But most are +0, and
> I wonder whether the OP ran this code on an all-UTC machine.
> (IDK whether their using gmail is relevant.)

 If I understand what you seem to not understand about my work around
to get a time difference in seconds out of my "cobblesome" date
formatting for file names, this is my time zone (remember I am using a
Debian Live DVD, on "exposed mode" ;-))

$ date +%z
+

$ cat  /etc/localtime
TZif2UTCTZif2UTC
UTC0

 At the end of the day, all I need is a time difference in seconds
(and yes, milliseconds would be better), which would be the same
regardless of your time zone.

 I do see the good in what you are suggesting to me and I will have to
include time zones in the file names as well and deal with the
possible cases (someone working at Charles de Gaulle Airport in
Paris/France boards a plane to Boston Logan/MA/USA ...). But the
actual question I will have to deal with is how to check and possibly
reset the time zone and the time via a network in a reliable way once
a ToG booted computer gains access to the Internet for which I will
have to use systemd-timesyncd when it boots and shuts down/ when it
changes modes.

 Am I clearer now?

 Thank you,
 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-19 Thread tomas
On Wed, Dec 20, 2023 at 12:00:29AM -0600, David Wright wrote:

[...]

> Yes, I'm guessing that the OP is in my timezone, as just a few of
> their previous posts have -5/-6 offsets. But most are +0, and
> I wonder whether the OP ran this code on an all-UTC machine.
> (IDK whether their using gmail is relevant.)

Nitpick and reminder: in UNIX and cousins, the "machine" has no
timezone. It's the executable (and its children, if they don't
change it). See:

  tomas@trotzki:~$ date
  Wed Dec 20 08:24:32 CET 2023
  tomas@trotzki:~$ TZ=Asia/Singapore bash
  tomas@trotzki:~$ date
  Wed Dec 20 15:24:47 +08 2023
  tomas@trotzki:~$ exit

What is /etc/timezone for, then? you may ask.

It's just the default for when you don't pick any.

As to gmail... I rather not think about that (still aching from
my cognitive dissonance to see someone so much trying to protect
himself from intrusion to entrust his communication to the biggest
vacuum cleaner for personal data and human behaviour of known
history, but I disgress).

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-19 Thread David Wright
On Mon 18 Dec 2023 at 14:12:12 (-0500), Greg Wooledge wrote:
> On Mon, Dec 18, 2023 at 12:35:29PM -0600, David Wright wrote:
> > OK, I tried running it (attached). What should it show?
> 
> That the OP is confused about many things.
> 
> > # date --help
> 
> No shebang.  But the script uses bash syntax.  When executed FROM BASH,
> the script will "work" because bash will intercept the Exec format error
> from the kernel and try again under a child bash shell.
> 
> When executing the "script" from any other parent, it will simply fail.
> Either the kernel's Exec format error will be treated as a final, fatal
> error, or it'll be retried under a child sh shell, which won't be able
> to process the bashisms.

To be fair to the OP, there was no official "script", but just some code:
  https://lists.debian.org/debian-user/2023/12/msg00894.html
which I pasted into /tmp/lbrtchx.sh. The filename suffix was a mere
convenience to make emacs colour the code and tidy the indentation,
speeding up finding lines broken by the MUA. I then ran it with:
  $ bash /tmp/lbrtchx.sh

> It *looks* like this command is trying to take a date/time string in
> one format, and convert it to a different format, and then append a
> +00:00 time zone offset even though that's not the correct offset for
> the author's time zone (as far as I know).

Yes, I'm guessing that the OP is in my timezone, as just a few of
their previous posts have -5/-6 offsets. But most are +0, and
I wonder whether the OP ran this code on an all-UTC machine.
(IDK whether their using gmail is relevant.)

I ran it on my machine (America/Chicago) to demonstrate to myself,
and also the OP, that the code couldn't be correct.

> If the conversion is correct (apart from the tz offset), but the epoch
> time that results is not correct, then it's very likely the +00:00
> that's causing the failure.
> 
> I'm in America/New_York, and if I were to try something like this:
> 
> read -r s dt < <(date '+%s %Y-%m-%dT%H:%M:%S+00:00')
> echo "$s"
> date +%s -d"$dt"
> 
> I would expect the two numbers printed to be *different* from each other.
> A date/time string generated in my time zone but with a forced +00:00
> tz offset is simply incorrect, as it would be in most time zones in
> the world.
> 
> Leaving off the +00:00 would allow it to work *most* of the time, with
> the exceptions being the times during a daylight saving transition.
> 
> Skipping the date/time string and only using the +%s part would allow
> it to work *all* of the time, possibly except for leap seconds (I'm
> not up to speed on those).

AIUI with POSIX timezones, you don't have to worry about leap seconds
with computations of this sort: they effectively don't exist.

$ TZ=posix/UTC date -d '1972-06-30T23:59:59' '+%s'
78796799
$ TZ=posix/UTC date -d '1972-06-30T23:59:60' '+%s'
date: invalid date ‘1972-06-30T23:59:60’
$ TZ=posix/UTC date -d '1972-07-01T00:00:00' '+%s'
78796800
$ TZ=posix/UTC date -d '1972-07-01T00:00:01' '+%s'
78796801
$ 

as opposed to:

$ TZ=right/UTC date -d '1972-06-30T23:59:59' '+%s'
78796799
$ TZ=right/UTC date -d '1972-06-30T23:59:60' '+%s'
78796800
$ TZ=right/UTC date -d '1972-07-01T00:00:00' '+%s'
78796801
$ TZ=right/UTC date -d '1972-07-01T00:00:01' '+%s'
78796802
$ 

Cheers,
David.



Re: difference in seconds between two formatted dates ...

2023-12-18 Thread Greg Wooledge
On Mon, Dec 18, 2023 at 12:35:29PM -0600, David Wright wrote:
> OK, I tried running it (attached). What should it show?

That the OP is confused about many things.

> # date --help

No shebang.  But the script uses bash syntax.  When executed FROM BASH,
the script will "work" because bash will intercept the Exec format error
from the kernel and try again under a child bash shell.

When executing the "script" from any other parent, it will simply fail.
Either the kernel's Exec format error will be treated as a final, fatal
error, or it'll be retried under a child sh shell, which won't be able
to process the bashisms.

> if [[ $s00 -eq $ISeks00 ]]; then

This line is bash syntax, not sh.  Here's one of the places it'll fail
if not invoked from bash.

Also, this command is treating the contents of the s00 and ISeks00
variables as strings containing numbers, converting both to integers,
and doing an integer comparison on them.  That's fine since they both
came from date +%s, in theory, but it's not a habit one should generally
embrace when comparing command outputs.

Usually one would want a string comparison, which would be done with =
instead of -eq.

> dttm02ISeks="${dt02:0:4}-${dt02:4:2}-${dt02:6:2}T${dt02:8:2}:${dt02:10:2}:${dt02:12:2}+00:00"

More bash syntax.  Those string slicing expansions like ${dt02:0:4} are
not valid in sh.

It *looks* like this command is trying to take a date/time string in
one format, and convert it to a different format, and then append a
+00:00 time zone offset even though that's not the correct offset for
the author's time zone (as far as I know).

If the conversion is correct (apart from the tz offset), but the epoch
time that results is not correct, then it's very likely the +00:00
that's causing the failure.

I'm in America/New_York, and if I were to try something like this:

read -r s dt < <(date '+%s %Y-%m-%dT%H:%M:%S+00:00')
echo "$s"
date +%s -d"$dt"

I would expect the two numbers printed to be *different* from each other.
A date/time string generated in my time zone but with a forced +00:00
tz offset is simply incorrect, as it would be in most time zones in
the world.

Leaving off the +00:00 would allow it to work *most* of the time, with
the exceptions being the times during a daylight saving transition.

Skipping the date/time string and only using the +%s part would allow
it to work *all* of the time, possibly except for leap seconds (I'm
not up to speed on those).



Re: difference in seconds between two formatted dates ...

2023-12-18 Thread David Wright
On Mon 18 Dec 2023 at 06:02:48 (+), Albretch Mueller wrote:
> On 12/18/23, David Wright  wrote:
> > Another problem in what you posted is that you sometimes run date
> > in your local timezone (generally for the "now" times), but you
> > append +00:00 as the timezone for those --date strings that you
> > construct from several substrings. You need to use UT throughout.
> 
>  What difference would that make when all I need is a time difference?
> The way I understand such time format issues is that it needs to just
> be the same in the two dates.
> 
> >> ANyway, my hack around it wasn't effortful at
> >> all.
> >
> > I don't know whether you're referring to something already posted,
> > or some new script written in the wake of what you've read here.
> 
>  That silly hack which lousy bash script I posted within its test case

OK, I tried running it (attached). What should it show?

Cheers,
David.
# date --help
set -x
date --version 2>&1 | head -n 1

s00=$(date +%s)
dt00=$(date +%Y%m%d%H%M%S)

dttm00ISeks="${dt00:0:4}-${dt00:4:2}-${dt00:6:2}T${dt00:8:2}:${dt00:10:2}:${dt00:12:2}+00:00"
ISeks00=$(date -d"${dttm00ISeks}" +%s)

if [[ $s00 -eq $ISeks00 ]]; then
echo "// __ \$ISeks00: |$ISeks00|, \$dttm00ISeks: |${dttm00ISeks}| 
conversion OK!"
else
echo "// __ \$s00: |$s00|, \$dt00: |${dt00}|"
echo "// __ \$ISeks00: |$ISeks00|, \$dttm00ISeks: |${dttm00ISeks}| 
conversion NOT OK!"
fi

# 63 weeks 5 days 11 hours 0 minutes et 43 seconds later
s02=$(( s00 + 63*7*24*60*60 + 5*24*60*60 + 11*60*60 + 0*60 + 43))

dt02=$(date --date @${s02} +%Y%m%d%H%M%S)
echo "// __ \$s02: |$s02|, \$dt02: |${dt02}|"

dttm02ISeks="${dt02:0:4}-${dt02:4:2}-${dt02:6:2}T${dt02:8:2}:${dt02:10:2}:${dt02:12:2}+00:00"
ISeks02=$(date -d"${dttm02ISeks}" +%s)

if [[ $s02 -eq $ISeks02 ]]; then
echo "// __ \$ISeks02: |$ISeks02|, \$dttm02ISeks: |${dttm02ISeks}| 
conversion OK!"
_diffs=$(( s02 - s00 ))
_diffISeks=$(( ISeks02 - ISeks00 ))
if [[ $_diffs -eq $_diffISeks ]]; then
echo "// __ differences also OK!"
else
echo "// __  differences NOT OK! \$_diffs: |$_diffs|, \$_diffISeks: 
|${_diffISeks}|"
fi
else
echo "// __ \$s02: |$s02|, \$dt02: |${dt02}|"
echo "// __ \$ISeks02: |$ISeks02|, \$dttm02ISeks: |${dttm02ISeks}| 
conversion NOT OK!"
fi

set +x
date (GNU coreutils) 8.32
// __ $s00: |1702924186|, $dt00: |20231218122946|
// __ $ISeks00: |1702902586|, $dttm00ISeks: |2023-12-18T12:29:46+00:00| 
conversion NOT OK!
// __ $s02: |1741498229|, $dt02: |20250308233029|
// __ $s02: |1741498229|, $dt02: |20250308233029|
// __ $ISeks02: |1741476629|, $dttm02ISeks: |2025-03-08T23:30:29+00:00| 
conversion NOT OK!
+ date --version
+ head -n 1
++ date +%s
+ s00=1702924186
++ date +%Y%m%d%H%M%S
+ dt00=20231218122946
+ dttm00ISeks=2023-12-18T12:29:46+00:00
++ date -d2023-12-18T12:29:46+00:00 +%s
+ ISeks00=1702902586
+ [[ 1702924186 -eq 1702902586 ]]
+ echo '// __ $s00: |1702924186|, $dt00: |20231218122946|'
+ echo '// __ $ISeks00: |1702902586|, $dttm00ISeks: |2023-12-18T12:29:46+00:00| 
conversion NOT OK!'
+ s02=1741498229
++ date --date @1741498229 +%Y%m%d%H%M%S
+ dt02=20250308233029
+ echo '// __ $s02: |1741498229|, $dt02: |20250308233029|'
+ dttm02ISeks=2025-03-08T23:30:29+00:00
++ date -d2025-03-08T23:30:29+00:00 +%s
+ ISeks02=1741476629
+ [[ 1741498229 -eq 1741476629 ]]
+ echo '// __ $s02: |1741498229|, $dt02: |20250308233029|'
+ echo '// __ $ISeks02: |1741476629|, $dttm02ISeks: |2025-03-08T23:30:29+00:00| 
conversion NOT OK!'
+ set +x


date can't parse its own output was: difference in seconds between two formatted dates ...)

2023-12-18 Thread tomas
On Mon, Dec 18, 2023 at 08:17:21AM -0500, Greg Wooledge wrote:
> On Mon, Dec 18, 2023 at 02:07:14PM +0100, to...@tuxteam.de wrote:
> > I'm still amazed the OP hasn't understood that "date" can output
> > custom formats -- and that it's not always possible to parse back
> > a date in some custom format into a meaningful timestamp.
> 
> unicorn:~$ date +"On this the %dth day of the %mth month of the year %Y 
> (Common Era), at %M minutes past the %Hth hour"
> On this the 18th day of the 12th month of the year 2023 (Common Era), at 15 
> minutes past the 08th hour
> unicorn:~$ date -d "$(date +"On this the %dth day of the %mth month of the 
> year %Y (Common Era), at %M minutes past the %Hth hour")"
> date: invalid date ‘On this the 18th day of the 12th month of the year 2023 
> (Common Era), at 15 minutes past the 08th hour’

How 'bout

  date -d $(date +"1582-10-08T12:31:19")

...then you'd have to have a stern talk with old Gregor ;-D

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-18 Thread Greg Wooledge
On Mon, Dec 18, 2023 at 02:07:14PM +0100, to...@tuxteam.de wrote:
> I'm still amazed the OP hasn't understood that "date" can output
> custom formats -- and that it's not always possible to parse back
> a date in some custom format into a meaningful timestamp.

unicorn:~$ date +"On this the %dth day of the %mth month of the year %Y (Common 
Era), at %M minutes past the %Hth hour"
On this the 18th day of the 12th month of the year 2023 (Common Era), at 15 
minutes past the 08th hour
unicorn:~$ date -d "$(date +"On this the %dth day of the %mth month of the year 
%Y (Common Era), at %M minutes past the %Hth hour")"
date: invalid date ‘On this the 18th day of the 12th month of the year 2023 
(Common Era), at 15 minutes past the 08th hour’

WHAT DO YOU MEAN INVALID DATE, YOU JUST PRODUCED IT!!

Yeah, I know, it's ridiculous... but this seems to be the level of
discourse required to get through to Albretch.



Re: difference in seconds between two formatted dates ...

2023-12-18 Thread tomas
On Mon, Dec 18, 2023 at 07:16:25AM -0500, Greg Wooledge wrote:

[...]

> If you wreck this by putting the wrong timezone offset on your
> human-readable times [...]

If you lie about your time zone you might come in too late for
your train :-)

> In addition to that, a case has already been shown where your chosen
> format, with or without a fixed timezone offset, is ambiguous -- it
> could refer to two different times.  This is an issue everywhere
> daylight saving time is used.  It's rare, but it's real.

I'm still amazed the OP hasn't understood that "date" can output
custom formats -- and that it's not always possible to parse back
a date in some custom format into a meaningful timestamp.

For an extreme case, just try

  date +"This is not a date"

(with a tip o' the hat to René Magritte). For a less extreme example

  date +"%H:%M:%S"

(I use that every minute in my digital clock display).

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-18 Thread Greg Wooledge
On Mon, Dec 18, 2023 at 06:02:48AM +, Albretch Mueller wrote:
> On 12/18/23, David Wright  wrote:
> > Another problem in what you posted is that you sometimes run date
> > in your local timezone (generally for the "now" times), but you
> > append +00:00 as the timezone for those --date strings that you
> > construct from several substrings. You need to use UT throughout.
> 
>  What difference would that make when all I need is a time difference?
> The way I understand such time format issues is that it needs to just
> be the same in the two dates.

The only advantage of storing timestamps in a human-readable format
would be that you, the human, can read them and know what they mean.

If you wreck this by putting the wrong timezone offset on your
human-readable times, then they're no longer human-readable.  So there
is literally NO point in doing all this stupidly difficult formatting and
parsing work, when you could just use seconds-since-epoch format instead.

In addition to that, a case has already been shown where your chosen
format, with or without a fixed timezone offset, is ambiguous -- it
could refer to two different times.  This is an issue everywhere
daylight saving time is used.  It's rare, but it's real.



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Albretch Mueller
On 12/18/23, David Wright  wrote:
> Another problem in what you posted is that you sometimes run date
> in your local timezone (generally for the "now" times), but you
> append +00:00 as the timezone for those --date strings that you
> construct from several substrings. You need to use UT throughout.

 What difference would that make when all I need is a time difference?
The way I understand such time format issues is that it needs to just
be the same in the two dates.

>> ANyway, my hack around it wasn't effortful at
>> all.
>
> I don't know whether you're referring to something already posted,
> or some new script written in the wake of what you've read here.

 That silly hack which lousy bash script I posted within its test case

 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread David Wright
On Mon 18 Dec 2023 at 01:11:26 (+), Albretch Mueller wrote:
> On 12/18/23, David Wright  wrote:
> > When you write   dt00=$(date +%Y%m%d%H%M%S)
> > that's /your/ format, not coreutils'.
> 
>  I (erroneously?) thought coreutils was maintaining Linux date, so if
> they tell you on their --help instructions to use certain options
> (including cobbling them together to your heart's content) and you are
> telling them you are using their format and how, they should be able
> to parse that string.

You can prettyprint a timedate with date any way you care to, but
scanning a timedate with --date requires care. It's safest if you
give it a string in one of its own built-in formats. You appear
to be expecting your second date command to somehow guess the format
you used in the first command, which obviously can't happen.

Another problem in what you posted is that you sometimes run date
in your local timezone (generally for the "now" times), but you
append +00:00 as the timezone for those --date strings that you
construct from several substrings. You need to use UT throughout.

> ANyway, my hack around it wasn't effortful at
> all.

I don't know whether you're referring to something already posted,
or some new script written in the wake of what you've read here.

Cheers,
David.



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Max Nikulin

On 18/12/2023 06:00, Albretch Mueller wrote:

On 12/17/23, Andy Smith  wrote:

how on earth would that not always produce an accurate duration?

All this paranoia, but in computer time you trust? 
 Falsehoods programmers believe about time
 https://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b923ca

  and how does my paranoia relate to that wall of itemized statements
which could be reduced to just a few?


Sources are given below that wall and they may be more pleasant to read. 
There are a lot of blog posts describing various pitfalls related to 
date and time.


Timestamp format you have chosen is ambiguous.

TZ=Europe/Berlin date -d '@1698539400' '+%Y%m%d%H%M%S'
20231029023000

TZ=Europe/Berlin date -d '@1698543000' '+%Y%m%d%H%M%S'
20231029023000

You had issues with setting time and timezone, so '+%s' may give 
incorrect results.




Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Albretch Mueller
On 12/18/23, David Wright  wrote:
> When you write   dt00=$(date +%Y%m%d%H%M%S)
> that's /your/ format, not coreutils'.

 I (erroneously?) thought coreutils was maintaining Linux date, so if
they tell you on their --help instructions to use certain options
(including cobbling them together to your heart's content) and you are
telling them you are using their format and how, they should be able
to parse that string. ANyway, my hack around it wasn't effortful at
all.

 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread David Wright
On Sun 17 Dec 2023 at 23:00:37 (+), Albretch Mueller wrote:
> On 12/17/23, Andy Smith  wrote:
> > All this paranoia, but in computer time you trust? 
> > Falsehoods programmers believe about time
> > https://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b923ca
> 
>  and how does my paranoia relate to that wall of itemized statements
> which could be reduced to just a few?

I understood the paranoia related to your earlier post, outlining
"exposed" and "private" modes and why you require all this date
processing to support them.

>  2) coreutils date is not quite true to itself when it comes to en-
> and decoding its own formatting options

Which of its own formatting options does it not support?

$ date -Ins | tee /tmp/d-ins
2023-12-17T18:29:38,909648062-06:00
$ date -d "$(cat /tmp/d-ins)"
Sun Dec 17 18:29:38 CST 2023
$ date --rfc-email | tee /tmp/d-ema
Sun, 17 Dec 2023 18:29:47 -0600
$ date -d "$(cat /tmp/d-ema)"
Sun Dec 17 18:29:47 CST 2023
$ date --rfc-3339=ns | tee /tmp/d-rns
2023-12-17 18:29:54.724220949-06:00
$ date -d "$(cat /tmp/d-rns)"
Sun Dec 17 18:29:54 CST 2023
$ date | tee /tmp/d-dat
Sun Dec 17 18:30:01 CST 2023
$ date -d "$(cat /tmp/d-dat)"
Sun Dec 17 18:30:01 CST 2023
$ 

All correct.

When you write   dt00=$(date +%Y%m%d%H%M%S)
that's /your/ format, not coreutils'.

Cheers,
David.



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Albretch Mueller
On 12/17/23, Andy Smith  wrote:
>> how on earth would that not always produce an accurate duration?
>
> All this paranoia, but in computer time you trust? 
> Falsehoods programmers believe about time
> https://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b923ca

 and how does my paranoia relate to that wall of itemized statements
which could be reduced to just a few? No the earth doesn't revolve
around the sun in exactly 356 days, ... and the adoption of the
Gregorian calendar already caused quite a fuss in the history of
humankind, which, believe me, didn't have to do at all with my
paranoia.

 All my paranoia is talking about is that:
 1) for historical reasons going back to the Sumerian/Babylonian
civilization, the date format we use is not 10-based
 2) coreutils date is not quite true to itself when it comes to en-
and decoding its own formatting options
 3) since time is a scalar magnitude, a time difference will be
constant regardless of my paranoia and the statements on that link you
posted.
 4) contrary to what happens with floating point numbers (which, in
general, you can't represent truthfully as binary), a time difference
in seconds is whole number which you can exactly represent as binary.

 Even people who are not paranoid and know their sh!t would easily see
my points. I would say it is primary school Arithmetic.

 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Albretch Mueller
 Well, yes, "date --date= ..." doesn't work in the way I would wish
("logically" think about it):

 
https://www.gnu.org/software/coreutils/manual/html_node/General-date-syntax.html

 parsing any format allowed by date:

 
https://www.gnu.org/software/coreutils/manual/html_node/Date-conversion-specifiers.html

 which made me think of how/why did I start using my "coblesome"
formatting, but there is nothing really that homely about it. Those
formatting options are listed right on:

 date --help

$ date --version 2>&1 | head -n 1
date (GNU coreutils) 8.32

 Based on to...@tuxteam.de's suggestions I worked around a downright
simple hack around those formatting problematics with coreutils date:

s00=$(date +%s)
dt00=$(date +%Y%m%d%H%M%S)

dttm00ISeks="${dt00:0:4}-${dt00:4:2}-${dt00:6:2}T${dt00:8:2}:${dt00:10:2}:${dt00:12:2}+00:00"
ISeks00=$(date -d"${dttm00ISeks}" +%s)

if [[ $s00 -eq $ISeks00 ]]; then
 echo "// __ \$ISeks00: |$ISeks00|, \$dttm00ISeks: |${dttm00ISeks}|
conversion OK!"
else
 echo "// __ \$s00: |$s00|, \$dt00: |${dt00}|"
 echo "// __ \$ISeks00: |$ISeks00|, \$dttm00ISeks: |${dttm00ISeks}|
conversion NOT OK!"
fi

# 63 weeks 5 days 11 hours 0 minutes et 43 seconds later
s02=$(( s00 + 63*7*24*60*60 + 5*24*60*60 + 11*60*60 + 0*60 + 43))

dt02=$(date --date @${s02} +%Y%m%d%H%M%S)
echo "// __ \$s02: |$s02|, \$dt02: |${dt02}|"

dttm02ISeks="${dt02:0:4}-${dt02:4:2}-${dt02:6:2}T${dt02:8:2}:${dt02:10:2}:${dt02:12:2}+00:00"
ISeks02=$(date -d"${dttm02ISeks}" +%s)

if [[ $s02 -eq $ISeks02 ]]; then
 echo "// __ \$ISeks02: |$ISeks02|, \$dttm02ISeks: |${dttm02ISeks}|
conversion OK!"
 _diffs=$(( s02 - s00 ))
 _diffISeks=$(( ISeks02 - ISeks00 ))
 if [[ $_diffs -eq $_diffISeks ]]; then
  echo "// __ differences also OK!"
 else
  echo "// __  differences NOT OK! \$_diffs: |$_diffs|, \$_diffISeks:
|${_diffISeks}|"
 fi
else
 echo "// __ \$s02: |$s02|, \$dt02: |${dt02}|"
 echo "// __ \$ISeks02: |$ISeks02|, \$dttm02ISeks: |${dttm02ISeks}|
conversion NOT OK!"
fi

 thank you,
 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread David Wright
On Sun 17 Dec 2023 at 15:28:58 (+), Albretch Mueller wrote:
> On 12/17/23, to...@tuxteam.de  wrote:
> > On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
> >> dt00=$(date +%Y%m%d%H%M%S)

>  Actually, my basic idea is if you can encode a date using formatting
> options this utility provides, then you should be able to decode it
> using the same options that same utility provides, no? ... and then
> get "seconds since 1970-01-01" which wouldn't matter when you are
> computing the difference between two 10-based whole numbers which can
> be represented exactly in binary Arithmetic.

date -d expects "mostly free format human readable", which means
it won't work with arbitrary fixed formats, unless they conform
to certain standards …

> > If you could live with a datetime format which date can understand
> > (instead of your cobbled-up pseudo numeric monster above) ...
> 
>  Thank you. I learned something out of it, but I used such "cobbled-up
> pseudo numeric monster" because, in kind of a poor-man's "measured"
> way, I use size, line count, last mod and sha256 metadata in file
> names baselining the system's state. I avoid spaces and any other
> characters you can't use in most fs' and/or may be interpreted by
> shells or OSs in their own ways.

Perhaps add five characters (- - T : :) to your poor-man's format,
and stick to UTC. If the colons are an embarrassment, sed them to
hyphens for filenames and back again for date -d. If that's too
much lengthening, just use sed alone to insert/remove those five
characters appropriately. Something like:

  $ LC_ALL=C TZ=UT date -d "$(sed -E 
's/()(..)(..)(..)(..)(..)/\1-\2-\3T\4:\5:\6/' <<<'19631122183000')" +'%s'
  -192778200
  $ LC_ALL=C TZ=UT date -d '@-192778200' +'%Y%m%d%H%M%S'
  19631122183000
  $ 

Cheers,
David.



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Greg Wooledge
On Sun, Dec 17, 2023 at 03:28:58PM +, Albretch Mueller wrote:
> On 12/17/23, Greg Wooledge  wrote:
> > On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
> >> ... after some long processing for which seconds would be exact
> >> enough, then I would like to get the seconds elapsed since dt00
> >
> > Are you working in bash, or sh?
> 
>  bash
> 
> > Bash offers some alternatives, to avoid having to fork two date(1)
> > processes, light as those are.  The first is:
> >
> > unicorn:~$ printf -v dt00 '%(%s)T' -1
> > unicorn:~$ echo "$dt00"
> > 1702821082
> 
>  but this is not what I meant and I am sure you can once again impress
> everyone here with your bash skills/wisdom. Is there such a thing as:
> 
>  dt00=$(date +%Y%m%d%H%M%S)
>  printf -(whatever the formatting for seconds would be) "${dt00}%Y%m%d%H%M%S"

No, there is nothing in bash which scans date/time strings.  You'd need
to use an external tool for this.  GNU date is one such tool.

>  I dealing with my "exposed mode" kinds of states ;-). I use a
> computer which initial state is kept as part of the file name as I
> explained, then as part of another ("exposed" or "private") reboot I
> want to check that file, so I would not be able to then simply do $((
> dt02 - dt00 ))

You've moved the goalposts.  Now the state is not being kept WITHIN
a script, but in a FILE that's used by two separate scripts (one that
writes it, and one that has to interpret it).

This is a very different kind of problem.

You'll need to choose a tool or a language that can parse your chosen
storage format.  This might be GNU date.  This might be Perl, Python,
Tcl, or some other language that has appropriate facilities for parsing
date/time strings.

You'll also be strongly encouraged to choose a storage format that
facilitates such parsing.  You might even want to work backwards --
figure out how you're going to parse the string, then choose the
format that works best with that specific parser.



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Andy Smith
Hello,

On Sun, Dec 17, 2023 at 03:28:58PM +, Albretch Mueller wrote:
> On 12/17/23, Andy Smith  wrote:
> > subtracting one epoch time from another doesn't always produce
> > an accurate duration.

[…]

> how on earth would that not always produce an accurate duration?

All this paranoia, but in computer time you trust? 

Falsehoods programmers believe about time
https://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b923ca

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread tomas
On Sun, Dec 17, 2023 at 03:28:58PM +, Albretch Mueller wrote:
> On 12/17/23, to...@tuxteam.de  wrote:

[...]

> > You wouldn't expect bash to intuit such a crooky arithmetic as
> > Gregorian datetime [...]

>  Actually, my basic idea is if you can encode a date using formatting
> options this utility provides, then you should be able to decode it
> using the same options that same utility provides, no?

Template expansion (printf is kind of that) is usually much easier
than parsing. Parsing might even lead to ambiguities unless you have
been super careful when "designing" your "language".

Just imagine variable-length items (there are none in your case,
but how should the parser know that?)

This is also why "scanf" is much less useful than "printf".

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Andrew M.A. Cater
On Sun, Dec 17, 2023 at 03:28:58PM +, Albretch Mueller wrote:
> On 12/17/23, to...@tuxteam.de  wrote:
> > On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
> >> dt00=$(date +%Y%m%d%H%M%S)
> >> echo "// __ \$dt00: |${dt00}|"
> >>
> >> ... after some long processing for which seconds would be exact
> >> enough, then I would like to get the seconds elapsed since dt00, like
> >> this:
> >>
> >> dt02=$(date +%Y%m%d%H%M%S)
> >> echo "// __ \$dt02: |${dt02}|"
> >
> > [...]
> >>  because bash Arithmetic is 10-based.
> >
> > You wouldn't expect bash to intuit such a crooky arithmetic as
> > Gregorian datetime (and then, based on what? The number beginning
> > with 2023? Sounds kind of exciting).
> 
>  Actually, my basic idea is if you can encode a date using formatting
> options this utility provides, then you should be able to decode it
> using the same options that same utility provides, no? ... and then
> get "seconds since 1970-01-01" which wouldn't matter when you are
> computing the difference between two 10-based whole numbers which can
> be represented exactly in binary Arithmetic.
> 
> > If you could live with a datetime format which date can understand
> > (instead of your cobbled-up pseudo numeric monster above) ...
> 
>  Thank you. I learned something out of it, but I used such "cobbled-up
> pseudo numeric monster" because, in kind of a poor-man's "measured"
> way, I use size, line count, last mod and sha256 metadata in file
> names baselining the system's state. I avoid spaces and any other
> characters you can't use in most fs' and/or may be interpreted by
> shells or OSs in their own ways.
> 
> > น Pedants at this point may feel the need to launch launch into a
> > sub-thread about how subtracting one epoch time from another doesn't
> > always produce an accurate duration. I can't stop you, but it won't
> > be news to me.
> 
>  No calendar issue here. All is needed is turning dates into seconds
> to get the time diff in seconds. I am not trying to start a
> sub-thread, but how on earth would that not always produce an accurate
> duration?
> 
>  lbrtchx
>

https://xkcd.com/2867/ refers,

Andy

(amaca...@debian.org) 



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Albretch Mueller
On 12/17/23, to...@tuxteam.de  wrote:
> On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
>> dt00=$(date +%Y%m%d%H%M%S)
>> echo "// __ \$dt00: |${dt00}|"
>>
>> ... after some long processing for which seconds would be exact
>> enough, then I would like to get the seconds elapsed since dt00, like
>> this:
>>
>> dt02=$(date +%Y%m%d%H%M%S)
>> echo "// __ \$dt02: |${dt02}|"
>
> [...]
>>  because bash Arithmetic is 10-based.
>
> You wouldn't expect bash to intuit such a crooky arithmetic as
> Gregorian datetime (and then, based on what? The number beginning
> with 2023? Sounds kind of exciting).

 Actually, my basic idea is if you can encode a date using formatting
options this utility provides, then you should be able to decode it
using the same options that same utility provides, no? ... and then
get "seconds since 1970-01-01" which wouldn't matter when you are
computing the difference between two 10-based whole numbers which can
be represented exactly in binary Arithmetic.

> If you could live with a datetime format which date can understand
> (instead of your cobbled-up pseudo numeric monster above) ...

 Thank you. I learned something out of it, but I used such "cobbled-up
pseudo numeric monster" because, in kind of a poor-man's "measured"
way, I use size, line count, last mod and sha256 metadata in file
names baselining the system's state. I avoid spaces and any other
characters you can't use in most fs' and/or may be interpreted by
shells or OSs in their own ways.

On 12/17/23, Greg Wooledge  wrote:
> On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
>> ... after some long processing for which seconds would be exact
>> enough, then I would like to get the seconds elapsed since dt00
>
> Are you working in bash, or sh?

 bash

> Bash offers some alternatives, to avoid having to fork two date(1)
> processes, light as those are.  The first is:
>
> unicorn:~$ printf -v dt00 '%(%s)T' -1
> unicorn:~$ echo "$dt00"
> 1702821082

 but this is not what I meant and I am sure you can once again impress
everyone here with your bash skills/wisdom. Is there such a thing as:

 dt00=$(date +%Y%m%d%H%M%S)
 printf -(whatever the formatting for seconds would be) "${dt00}%Y%m%d%H%M%S"

 I dealing with my "exposed mode" kinds of states ;-). I use a
computer which initial state is kept as part of the file name as I
explained, then as part of another ("exposed" or "private") reboot I
want to check that file, so I would not be able to then simply do $((
dt02 - dt00 ))

 After rebooting (separately for "exposed" or "private" mode):
 a) those kinds of name measured files are checked,
 b) if everything is fine you would keep some basic statistics (last
difference; as well as, incrementally, average, stddev, skewness and
kurtosis of differences so far)
 c) delete the file(s) previous to the latest one
 ...

 and, no, I am not expecting for bash to do that kind of statistics
for me, but I would love to see once again how wrong I am.

On 12/17/23, Andy Smith  wrote:
> น Pedants at this point may feel the need to launch launch into a
> sub-thread about how subtracting one epoch time from another doesn't
> always produce an accurate duration. I can't stop you, but it won't
> be news to me.

 No calendar issue here. All is needed is turning dates into seconds
to get the time diff in seconds. I am not trying to start a
sub-thread, but how on earth would that not always produce an accurate
duration?

 lbrtchx



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Greg Wooledge
On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
> ... after some long processing for which seconds would be exact
> enough, then I would like to get the seconds elapsed since dt00

Are you working in bash, or sh?  It makes a difference here.

Others have already mentioned using "date +%s" (a GNU extension)
to get epoch time, twice, and then subtracting those.  That should
be good enough for most purposes, and it'll work in sh, as long as
you have GNU date.  (This is the *only* way that works in sh, apart
from using the "time" builtin on a function call, and parsing its
output, which I would not recommend.)

Bash offers some alternatives, to avoid having to fork two date(1)
processes, light as those are.  The first is:

unicorn:~$ printf -v dt00 '%(%s)T' -1
unicorn:~$ echo "$dt00"
1702821082

This requires GNU libc(*) and bash 4.2 or higher; as an added bonus, if
you use bash 4.3 or higher, you can omit the -1 argument.

Another poster mentioned EPOCHSECONDS.  This is available in bash 5.0
or higher.  You can grab copies of this twice, and then subtract your
copies.  This has the advantage of working on systems without GNU libc
or GNU date.

Finally, there's the SECONDS variable.  You can use this in two different
ways.  The first way is to treat it like EPOCHSECONDS -- capture copies
of it twice, and subtract them.

The second way is to set it to 0 and the start of your interval, and
then capture its value at the end of your interval.

SECONDS=0
sleep 5
echo "$SECONDS sec elapsed"

This works in every version of bash I've got available, so let's call
it 2.05b or higher just to be safe.  It also works without needing
GNU date or GNU libc.  If you're using bash, this is *the* most portable
option.

(*) It requires GNU libc because bash uses the system's strftime(3)
for interpretation of the format specifier.  Therefore, %s only
works on systems where libc has that, which means GNU and BSD are
good, but typically not legacy Unixes.



Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Teemu Likonen
* 2023-12-17 10:58:28+, Andy Smith wrote:

> On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
>> you get seconds in dt00 and dt02 and then the difference. You can't
>> go: $(( dt02 - dt00 )) because bash Arithmetic is 10-based.
>
> Why don't you just get the time as epoch time e.g. "date +%s" and
> then you can subtract one from the other¹.

Also, Bash has variable EPOCHSECONDS which expands to the current Unix
epoch time.

$ echo $EPOCHSECONDS; sleep 5; echo $EPOCHSECONDS 
1702812641
1702812646

-- 
/// Teemu Likonen - .-.. https://www.iki.fi/tlikonen/
// OpenPGP: 6965F03973F0D4CA22B9410F0F2CAE0E07608462


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-17 Thread tomas
On Sun, Dec 17, 2023 at 10:58:28AM +, Andy Smith wrote:

[...]

> ¹ Pedants at this point may feel the need to launch launch into a
> sub-thread about how subtracting one epoch time from another doesn't
> always produce an accurate duration. I can't stop you, but it won't
> be news to me.

They can now rant at two :-)

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-17 Thread tomas
On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
> dt00=$(date +%Y%m%d%H%M%S)
> echo "// __ \$dt00: |${dt00}|"
> 
> ... after some long processing for which seconds would be exact
> enough, then I would like to get the seconds elapsed since dt00, like
> this:
> 
> dt02=$(date +%Y%m%d%H%M%S)
> echo "// __ \$dt02: |${dt02}|"

[...]
>  because bash Arithmetic is 10-based.

You wouldn't expect bash to intuit such a crooky arithmetic as
Gregorian datetime (and then, based on what? The number beginning
with 2023? Sounds kind of exciting).

If you could live with a datetime format which date can understand
(instead of your cobbled-up pseudo numeric monster above), you might
harness date to do the work for you.

To wit

  date -I
  => 2023-12-17

(alternatively, if you want a finer resolution):

  date -Iseconds
  => 2023-12-17T11:48:28+01:00

Now date can "read" those (option -d):

  date -d $(date -Iseconds)
  => Sun Dec 17 11:49:46 CET 2023

With all of this, you can do:

  d1=$(date -Iseconds) ; sleep 5 ; d2=$(date -Iseconds)
  echo "$d1 $d2"
  => 2023-12-17T11:51:43+01:00 2023-12-17T11:51:48+01:0

and now:

  echo $(( $(date -d"$d2" +"%s") - $(date -d"$d1" +"%s") ))
  => 5

... the "%s" format indicator telling date to print "seconds since
1970-01-01". Instead of -Iseconds you can do -Ins. See the date man
page for all the gory details.

Me? I'd keep internal results always as "seconds since 1970-01-01"
and only transform on display.

Cheers
-- 
t


signature.asc
Description: PGP signature


Re: difference in seconds between two formatted dates ...

2023-12-17 Thread Andy Smith
Hello,

On Sun, Dec 17, 2023 at 10:12:11AM +, Albretch Mueller wrote:
> dt00=$(date +%Y%m%d%H%M%S)
> echo "// __ \$dt00: |${dt00}|"
> 
> ... after some long processing for which seconds would be exact
> enough, then I would like to get the seconds elapsed since dt00, like
> this:
> 
> dt02=$(date +%Y%m%d%H%M%S)
> echo "// __ \$dt02: |${dt02}|"
> 
> you get seconds in dt00 and dt02 and then the difference. You can't
> go: $(( dt02 - dt00 )) because bash Arithmetic is 10-based.

Why don't you just get the time as epoch time e.g. "date +%s" and
then you can subtract one from the other¹.

You can also take a copy of the contents of /proc/uptime twice and
subtract one from the other.

Thanks,
Andy

¹ Pedants at this point may feel the need to launch launch into a
sub-thread about how subtracting one epoch time from another doesn't
always produce an accurate duration. I can't stop you, but it won't
be news to me.

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



difference in seconds between two formatted dates ...

2023-12-17 Thread Albretch Mueller
dt00=$(date +%Y%m%d%H%M%S)
echo "// __ \$dt00: |${dt00}|"

... after some long processing for which seconds would be exact
enough, then I would like to get the seconds elapsed since dt00, like
this:

dt02=$(date +%Y%m%d%H%M%S)
echo "// __ \$dt02: |${dt02}|"

you get seconds in dt00 and dt02 and then the difference. You can't
go: $(( dt02 - dt00 )) because bash Arithmetic is 10-based.

I am pretty sure there is such a thing in bash, but I can't find it in
my scripts.

lbrtchx