Re: Accurately serializing and deserializing a SysTime in binary format
On Tuesday, 21 July 2020 at 12:21:16 UTC, Steven Schveighoffer
wrote:
On 7/21/20 7:44 AM, Ecstatic Coder wrote:
On Tuesday, 21 July 2020 at 11:01:20 UTC, drug wrote:
On 7/20/20 10:04 PM, Ecstatic Coder wrote:
I'm currently implementing a small open source backup tool
(dub), and therefore I need to accurately store the file
modification SysTime in binary format, so that I can later
load this SysTime from the snapshot file to compare it with
the current file modification SysTime.
Having unfortunately not understood how to do this from the
SysTime documentation, in despair, I've tried to directly
serialize the 16 bytes of the SysTime value. This worked
fine until I call the ".toISOString()" on the deserialized
SysTime, which inevitably crashes the executable ;)
That is probably a bug. I serialize SysTime as long by means
msgpack for exchanging between C++ client and D server and it
works pretty nice.
Ah thanks for telling me :)
The loaded byte array in the union type was indeed the same as
the saved one, so I immediately thought it was crashing
because of some hidden pointer for timezone or something which
was then pointing to garbage at reloading, causing the crash
of the ".toISOString" call.
Not a bug.
8 of those 16 bytes is a pointer to the timezone, which is
going to be different on different processes.
What you should do I think is serialize the stdTime [1], and
set the time zone to whatever you want:
long serialize(SysTime st) { return st.stdTime; }
SysTime deserialize(long st) { return SysTime(st, UTC()); }
The stdTime is always stored as UTC to make math a lot easier.
The time zone is only used for display.
-Steve
[1]
https://dlang.org/phobos/std_datetime_systime.html#.SysTime.stdTime
Smart :)
Now I understand my mistake was to try to directly serialize a
SysTime as provided by the "getTimes" function, instead of
converting it to a StdTime, which is more versatile...
Re: Accurately serializing and deserializing a SysTime in binary format
On 7/21/20 7:44 AM, Ecstatic Coder wrote:
On Tuesday, 21 July 2020 at 11:01:20 UTC, drug wrote:
On 7/20/20 10:04 PM, Ecstatic Coder wrote:
I'm currently implementing a small open source backup tool (dub), and
therefore I need to accurately store the file modification SysTime in
binary format, so that I can later load this SysTime from the
snapshot file to compare it with the current file modification SysTime.
Having unfortunately not understood how to do this from the SysTime
documentation, in despair, I've tried to directly serialize the 16
bytes of the SysTime value. This worked fine until I call the
".toISOString()" on the deserialized SysTime, which inevitably
crashes the executable ;)
That is probably a bug. I serialize SysTime as long by means msgpack
for exchanging between C++ client and D server and it works pretty nice.
Ah thanks for telling me :)
The loaded byte array in the union type was indeed the same as the saved
one, so I immediately thought it was crashing because of some hidden
pointer for timezone or something which was then pointing to garbage at
reloading, causing the crash of the ".toISOString" call.
Not a bug.
8 of those 16 bytes is a pointer to the timezone, which is going to be
different on different processes.
What you should do I think is serialize the stdTime [1], and set the
time zone to whatever you want:
long serialize(SysTime st) { return st.stdTime; }
SysTime deserialize(long st) { return SysTime(st, UTC()); }
The stdTime is always stored as UTC to make math a lot easier. The time
zone is only used for display.
-Steve
[1] https://dlang.org/phobos/std_datetime_systime.html#.SysTime.stdTime
Re: Accurately serializing and deserializing a SysTime in binary format
On 7/21/20 2:44 PM, Ecstatic Coder wrote:
Ah thanks for telling me :)
The loaded byte array in the union type was indeed the same as the saved
one, so I immediately thought it was crashing because of some hidden
pointer for timezone or something which was then pointing to garbage at
reloading, causing the crash of the ".toISOString" call.
Ah, sorry, I serialize exactly long value and use it as SysTime, for
example:
```D
struct Foo
{
long value;
void toString(Writer)(ref Writer w) const
if (isOutputRange!(Writer, char))
{
import std.datetime: SysTime;
value.SysTime.toUTC.toISOExtString(w);
}
}
```
So it is not exactly what you said
Re: Accurately serializing and deserializing a SysTime in binary format
On Tuesday, 21 July 2020 at 11:01:20 UTC, drug wrote: On 7/20/20 10:04 PM, Ecstatic Coder wrote: I'm currently implementing a small open source backup tool (dub), and therefore I need to accurately store the file modification SysTime in binary format, so that I can later load this SysTime from the snapshot file to compare it with the current file modification SysTime. Having unfortunately not understood how to do this from the SysTime documentation, in despair, I've tried to directly serialize the 16 bytes of the SysTime value. This worked fine until I call the ".toISOString()" on the deserialized SysTime, which inevitably crashes the executable ;) That is probably a bug. I serialize SysTime as long by means msgpack for exchanging between C++ client and D server and it works pretty nice. Anyway, that's not really want I intended to do, as in practice a "ulong" already has enough resolution for that purpose. So sorry for my ignorance, but I would definitely need some help on how to : - convert a file modification SysTime to a serializable number, for instance the number of hectonanoseconds since 1/1/1970 in UTC; - convert that number back into a SysTime that I can compare to the modification SysTime of the same file. Eric Ah thanks for telling me :) The loaded byte array in the union type was indeed the same as the saved one, so I immediately thought it was crashing because of some hidden pointer for timezone or something which was then pointing to garbage at reloading, causing the crash of the ".toISOString" call.
Re: Accurately serializing and deserializing a SysTime in binary format
On 7/20/20 10:04 PM, Ecstatic Coder wrote: I'm currently implementing a small open source backup tool (dub), and therefore I need to accurately store the file modification SysTime in binary format, so that I can later load this SysTime from the snapshot file to compare it with the current file modification SysTime. Having unfortunately not understood how to do this from the SysTime documentation, in despair, I've tried to directly serialize the 16 bytes of the SysTime value. This worked fine until I call the ".toISOString()" on the deserialized SysTime, which inevitably crashes the executable ;) That is probably a bug. I serialize SysTime as long by means msgpack for exchanging between C++ client and D server and it works pretty nice. Anyway, that's not really want I intended to do, as in practice a "ulong" already has enough resolution for that purpose. So sorry for my ignorance, but I would definitely need some help on how to : - convert a file modification SysTime to a serializable number, for instance the number of hectonanoseconds since 1/1/1970 in UTC; - convert that number back into a SysTime that I can compare to the modification SysTime of the same file. Eric
Re: Accurately serializing and deserializing a SysTime in binary format
As my question obviously didn't interest any expert, I took advantage of my lunch break to do some more research ;) Maybe I'm wrong, but to my knowledge, there is no function to get the number of hectonanoseconds since January 1, 1970. Fortunately I can get the number of seconds since the same date, and the number of remaining hectonanoseconds, and then use them in conjunction to create a new "SysTime". With that I've got everything needed to fix my problem, and as I can store those values as two independent "uint", it's easy to compress them in the snapshot file, so no regrets :)
Accurately serializing and deserializing a SysTime in binary format
I'm currently implementing a small open source backup tool (dub), and therefore I need to accurately store the file modification SysTime in binary format, so that I can later load this SysTime from the snapshot file to compare it with the current file modification SysTime. Having unfortunately not understood how to do this from the SysTime documentation, in despair, I've tried to directly serialize the 16 bytes of the SysTime value. This worked fine until I call the ".toISOString()" on the deserialized SysTime, which inevitably crashes the executable ;) Anyway, that's not really want I intended to do, as in practice a "ulong" already has enough resolution for that purpose. So sorry for my ignorance, but I would definitely need some help on how to : - convert a file modification SysTime to a serializable number, for instance the number of hectonanoseconds since 1/1/1970 in UTC; - convert that number back into a SysTime that I can compare to the modification SysTime of the same file. Eric
