Send inn-workers mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/inn-workers
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of inn-workers digest..."
Today's Topics:
1. Re: Fwd: 64-bit time_t transition for 32-bit archs: a
proposal (Julien ?LIE)
2. Re: Fwd: 64-bit time_t transition for 32-bit archs: a
proposal (Julien ?LIE)
3. Re: Fwd: 64-bit time_t transition for 32-bit archs: a
proposal (Russ Allbery)
----------------------------------------------------------------------
Message: 1
Date: Tue, 27 Jun 2023 20:17:49 +0200
From: Julien ?LIE <[email protected]>
To: [email protected], Olaf Titz <[email protected]>
Subject: Re: Fwd: 64-bit time_t transition for 32-bit archs: a
proposal
Message-ID: <[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed
About the news spool:
> @02nnaabbccddyyyy00000000000000000000@
> "aabbccdd" is the arrival time in hexadecimal, and "yyyy" a sequence
> number.
With timehash, articles are stored in files named:
<patharticles>/time-nn/bb/cc/yyyy-aadd
(Time read as 0xaabbccdd in 32-bit.)
Supposing 0xeeffgghhaabbccdd in 64-bit, the token could become
a/ @02nneeffgghhaabbccddyyyy000000000000@
(full rewrite)
or
b/ @02nnaabbccddyyyyeeffgghh000000000000@
And the file names could become:
a/ <patharticles>/time-nn/bb/cc/yyyy-aadd-eeffgghh
with fixed length (articles will be named yyyy-aadd-00000000 until 2038).
b/ <patharticles>/time-nn/bb/cc/yyyy-aadd[-hh][gg][ff][ee]
with a bit of complexity: articles will be named yyyy-aadd until 2038
(no change from the current naming), and then yyyy-aadd-hh until epoch
reaches 2^40, and yyyy-aadd-hhgg until epoch reaches 2^48, and so on.
It will mean extra complexity when parsing filenames and tokens. It
seems manageable, though.
Of course, any other combination could be discussed, like yyyy-aaddhh or
yyyy-hhaadd.
But well, is it really worth doing something right now to handle these
future names?
0xaabbccdd will overflow when epoch reaches 2^32, which is in year 2106...
It gives us time to implement that new parsing, if these storage methods
still exist in 2100s. No need for an history or overview rebuild.
We could maybe just fix the parsing for a 64-bit time_t, if there's
really something to fix, and leave the names and tokens unchanged?
> The storage tokens in history should also be changed accordingly.
> They contain the same "aabbcc": > @04nn00aabbccyyyyxxxx0000000000000000@
With timecaf, articles are stored in files named:
<patharticles>/timecaf-nn/bb/aacc.CF
Changes similar to timehash could be done for it.
The remaining issue is for the CAF file header containing a time_t
LastCleaned field.
Hmm, as time_t may already be 32-bit or 64-bit depending on the system,
couldn't it be switched to uint64_t LastCleaned?
And we'll have to ship a tool which converts the CAF files appropriately.
Shouldn't we also change the size_t and off_t variables in the CAF file
header to uint64_t at the same time?
As space is allotted for 262144 (2^18) articles in current CAF files and
from time to time people complain with that limit (when reinjecting at
high speed articles in their news spool, more than 262144 can arrive in
a time frame of 4 minutes, which leads to storage failures).
At the same time we do the change to uint64_t LastCleaned, couldn't we
increase the allotted space to handle for instance 4,194,304 (2^22)
articles?
Handling 64-bit time_t would then be an opportunity to improve timecaf :)
Other ideas welcome of course.
--
Julien ?LIE
??? Ouvre l'?il, et le bon?!
? L'autre, je peux pas encore l'ouvrir, je risque pas de me
tromper?!?? (Ast?rix)
------------------------------
Message: 2
Date: Tue, 27 Jun 2023 20:38:37 +0200
From: Julien ?LIE <[email protected]>
To: [email protected]
Subject: Re: Fwd: 64-bit time_t transition for 32-bit archs: a
proposal
Message-ID: <[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed
Hi Russ,
>> Aren't there caveats with printf()-like calls or atol()-like calls?
>
>> We often have casts of time_t variables to "%lu" or "%ld" in printf(). Do
>> they work fine on both 32-bit and 64-bit architectures with a 64-bit
>> time_t after 2038?
>> Example in innd/site.c:
>> snprintf(pbuff, sizeof(pbuff), "%ld", (long) Data->Arrived);
>
>> In expire/makehistory.c, we have:
>> arrived = (time_t) atol(p);
>> Wouldn't atoll(p) be needed in some cases?
>
>> (We're also reading the epoch times in active.times with an atol() call in
>> nnrpd/commands.c.)
>
> Yes, I think you're right; those need to become long long (although
> unsigned long buys another decade or so).
Using unsigned long allows to reach epoch 2^32 in 2106, so maybe we
should just fix the %ld format to use %lu instead.
Using long long seems to be more complicated as it is not available
everywhere. Checking HAVE_UNSIGNED_LONG_LONG_INT and changing all
printf() calls to use %lu or %llu looks tedious and decreases code
readability. Unless that's not the right approach?
> It's not yet clear whether Debian will try to convert the i386
> architecture to 64-bit time_t, drop it, or do something else with it.
> There's a large debate right now in the Linux world over whether i386 is
> more useful as a legacy build to run old software (which will never be
> able to use 64-bit time_t), or as a platform for running current software.
> It's fairly hard to find new 32-bit-only x86 processors these days, and
> even the large numbers of legacy systems out there are getting pretty old.
>
> But this will come up for other architectures even if i386 doesn't convert
> (ARM, for instance, and there's probably at least one person out there
> running INN on an ARM Linux system).
Some people might also use "-m32" for their builds, and indeed on ARM
Linux systems, long is 32-bit. %lu will work until 2106 for them.
atol(), strtol() and like should then also be changed to atoul() and
strtoul().
Would these changes be OK to do right now or should we wait a bit, for
possible other wide-spread best practices to deal with the 2038 bug?
--
Julien ?LIE
??Tant qu'il y a des marmites, il y a de l'espoir?!?? (Ast?rix)
------------------------------
Message: 3
Date: Tue, 27 Jun 2023 11:50:55 -0700
From: Russ Allbery <[email protected]>
To: [email protected]
Subject: Re: Fwd: 64-bit time_t transition for 32-bit archs: a
proposal
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8
Julien ?LIE <[email protected]> writes:
> Using long long seems to be more complicated as it is not available
> everywhere.
I'm not sure if this is really true any more, although Autoconf doesn't
provide much in the way of guidance. But I would be surprised if one
encountered a system without long long these days. I think one would have
to be compiling with a proprietary UNIX compiler from some old commercial
UNIX.
> Some people might also use "-m32" for their builds, and indeed on ARM
> Linux systems, long is 32-bit. %lu will work until 2106 for them.
> atol(), strtol() and like should then also be changed to atoul() and
> strtoul().
> Would these changes be OK to do right now or should we wait a bit, for
> possible other wide-spread best practices to deal with the 2038 bug?
I think changing to use %lu for time_t is completely harmless and there's
no reason not to do it. (We don't use negative times to represent times
before the start of epoch, and I can't imagine a situation where that
would be meaningful for a Usenet news server, given that Usenet didn't
exist until well after the start of epoch.) The only possible problem I
can think of is if we have a -1 time_t somewhere, but that would probably
already be a bug and I'm not sure this would make it any worse.
--
Russ Allbery ([email protected]) <https://www.eyrie.org/~eagle/>
Please send questions to the list rather than mailing me directly.
<https://www.eyrie.org/~eagle/faqs/questions.html> explains why.
------------------------------
Subject: Digest Footer
_______________________________________________
inn-workers mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/inn-workers
------------------------------
End of inn-workers Digest, Vol 151, Issue 2
*******************************************
