Re: Accurately serializing and deserializing a SysTime in binary format
On Tuesday, 21 July 2020 at 12:21:16 UTC, Steven Schveighoffer
wrote:
On 7/21/20 7:44 AM, Ecstatic Coder wrote:
On Tuesday, 21 July 2020 at 11:01:20 UTC, drug wrote:
On 7/20/20 10:04 PM, Ecstatic Coder wrote:
I'm currently implementing a small open source backup tool
(dub), and therefore I need to accurately store the file
modification SysTime in binary format, so that I can later
load this SysTime from the snapshot file to compare it with
the current file modification SysTime.
Having unfortunately not understood how to do this from the
SysTime documentation, in despair, I've tried to directly
serialize the 16 bytes of the SysTime value. This worked
fine until I call the ".toISOString()" on the deserialized
SysTime, which inevitably crashes the executable ;)
That is probably a bug. I serialize SysTime as long by means
msgpack for exchanging between C++ client and D server and it
works pretty nice.
Ah thanks for telling me :)
The loaded byte array in the union type was indeed the same as
the saved one, so I immediately thought it was crashing
because of some hidden pointer for timezone or something which
was then pointing to garbage at reloading, causing the crash
of the ".toISOString" call.
Not a bug.
8 of those 16 bytes is a pointer to the timezone, which is
going to be different on different processes.
What you should do I think is serialize the stdTime [1], and
set the time zone to whatever you want:
long serialize(SysTime st) { return st.stdTime; }
SysTime deserialize(long st) { return SysTime(st, UTC()); }
The stdTime is always stored as UTC to make math a lot easier.
The time zone is only used for display.
-Steve
[1]
https://dlang.org/phobos/std_datetime_systime.html#.SysTime.stdTime
Smart :)
Now I understand my mistake was to try to directly serialize a
SysTime as provided by the "getTimes" function, instead of
converting it to a StdTime, which is more versatile...
Re: Accurately serializing and deserializing a SysTime in binary format
On Tuesday, 21 July 2020 at 11:01:20 UTC, drug wrote: On 7/20/20 10:04 PM, Ecstatic Coder wrote: I'm currently implementing a small open source backup tool (dub), and therefore I need to accurately store the file modification SysTime in binary format, so that I can later load this SysTime from the snapshot file to compare it with the current file modification SysTime. Having unfortunately not understood how to do this from the SysTime documentation, in despair, I've tried to directly serialize the 16 bytes of the SysTime value. This worked fine until I call the ".toISOString()" on the deserialized SysTime, which inevitably crashes the executable ;) That is probably a bug. I serialize SysTime as long by means msgpack for exchanging between C++ client and D server and it works pretty nice. Anyway, that's not really want I intended to do, as in practice a "ulong" already has enough resolution for that purpose. So sorry for my ignorance, but I would definitely need some help on how to : - convert a file modification SysTime to a serializable number, for instance the number of hectonanoseconds since 1/1/1970 in UTC; - convert that number back into a SysTime that I can compare to the modification SysTime of the same file. Eric Ah thanks for telling me :) The loaded byte array in the union type was indeed the same as the saved one, so I immediately thought it was crashing because of some hidden pointer for timezone or something which was then pointing to garbage at reloading, causing the crash of the ".toISOString" call.
Re: Accurately serializing and deserializing a SysTime in binary format
As my question obviously didn't interest any expert, I took advantage of my lunch break to do some more research ;) Maybe I'm wrong, but to my knowledge, there is no function to get the number of hectonanoseconds since January 1, 1970. Fortunately I can get the number of seconds since the same date, and the number of remaining hectonanoseconds, and then use them in conjunction to create a new "SysTime". With that I've got everything needed to fix my problem, and as I can store those values as two independent "uint", it's easy to compress them in the snapshot file, so no regrets :)
Accurately serializing and deserializing a SysTime in binary format
I'm currently implementing a small open source backup tool (dub), and therefore I need to accurately store the file modification SysTime in binary format, so that I can later load this SysTime from the snapshot file to compare it with the current file modification SysTime. Having unfortunately not understood how to do this from the SysTime documentation, in despair, I've tried to directly serialize the 16 bytes of the SysTime value. This worked fine until I call the ".toISOString()" on the deserialized SysTime, which inevitably crashes the executable ;) Anyway, that's not really want I intended to do, as in practice a "ulong" already has enough resolution for that purpose. So sorry for my ignorance, but I would definitely need some help on how to : - convert a file modification SysTime to a serializable number, for instance the number of hectonanoseconds since 1/1/1970 in UTC; - convert that number back into a SysTime that I can compare to the modification SysTime of the same file. Eric
Re: Full precision double to string conversion
On Saturday, 3 November 2018 at 18:04:07 UTC, Stanislav Blinov
wrote:
On Saturday, 3 November 2018 at 17:26:19 UTC, Ecstatic Coder
wrote:
void main() {
double value = -12.000123456;
int precision = 50;
import std.stdio;
writefln("%.*g", precision, value);
import std.format;
string str = format("%.*g", precision, value);
writeln(str);
}
Prints:
-12.00012345600743415512260980904102325439453125
-12.00012345600743415512260980904102325439453125
That's not quite the -12.000123456 that you'd get from C#'s
ToString().
Unfortunately, but that's still better though, thanks :)
I don't think you understood what I meant. Neither C# nor D
attempt to exhaust the precision when converting, given default
arguments. It's merely a matter of those defaults. The snippet
above obviously provides *more* digits that the default
.ToString() in C# would.
But indeed what I really need is a D function which gives a
better decimal approximation to the provided double constant,
exactly in the same way those in Dart and C# do.
Is there really no such function in D ?
When you call .ToString() in C# with no arguments, it assumes
the "G" format specifier.
https://docs.microsoft.com/en-us/dotnet/standard/base-types/standard-numeric-format-strings?view=netframework-4.7.2#the-general-g-format-specifier
So for a double, it will use 15-digit precision. D's to!string
simply uses lower default. If you want the exact same behavior
as in C#, you can do this:
string toStringLikeInCSharp(double value) {
import std.format : format;
return format("%.15G", value);
}
void main() {
double value = -12.000123456;
import std.stdio;
writeln(value.toStringLikeInCSharp); // prints:
-12.000123456
}
This version perfectly gets the job done!
Thanks a lot for your help :)
Re: Full precision double to string conversion
Actually, what I need is the D equivalent of the default
ToString() function we have in Dart and C#.
I don't think it means what you think it means:
void main() {
double value = -12.000123456;
int precision = 50;
import std.stdio;
writefln("%.*g", precision, value);
import std.format;
string str = format("%.*g", precision, value);
writeln(str);
}
Prints:
-12.00012345600743415512260980904102325439453125
-12.00012345600743415512260980904102325439453125
That's not quite the -12.000123456 that you'd get from C#'s
ToString().
Unfortunately, but that's still better though, thanks :)
All of them? Most implementations of conversion algorithms
actually stop when it's "good enough". AFAIR, D doesn't even
have it's own implementation and forwards to C, unless that
changed in recent years.
What I meant was that getting too many significant digits would
still be a better solution than not having them.
But indeed what I really need is a D function which gives a
better decimal approximation to the provided double constant,
exactly in the same way those in Dart and C# do.
Is there really no such function in D ?
Re: Full precision double to string conversion
On Saturday, 3 November 2018 at 12:45:03 UTC, Danny Arends wrote:
On Saturday, 3 November 2018 at 12:27:19 UTC, Ecstatic Coder
wrote:
import std.conv;
import std.stdio;
void main()
{
double value = -12.000123456;
writeln( value.sizeof );
writeln( value );
writeln( value.to!string() );
writeln( value.to!dstring() );
}
/*
8
-12.0001
-12.0001
-12.0001
*/
In Dart, value.toString() returns "-12.000123456".
In C#, value.ToString() returns "-12.000123456".
In D, value.to!string() returns "-12.0001" :(
How can I convert a double value -12.000123456 to its string
value "-12.000123456", i.e. without loosing double-precision
digits ?
Specify how many digits you want with writefln:
writefln("%.8f", value);
Actually, what I need is the D equivalent of the default
ToString() function we have in Dart and C#.
I mean a dumb double-to-string standard library conversion
function which returns a string including all the double
precision digits stored in the 52 significant bits of the value,
preferably with the trailing zeroes removed.
For an unknown reason, D's default double-to-string conversion
function only expose the single-precision significant digits :(
Full precision double to string conversion
import std.conv;
import std.stdio;
void main()
{
double value = -12.000123456;
writeln( value.sizeof );
writeln( value );
writeln( value.to!string() );
writeln( value.to!dstring() );
}
/*
8
-12.0001
-12.0001
-12.0001
*/
In Dart, value.toString() returns "-12.000123456".
In C#, value.ToString() returns "-12.000123456".
In D, value.to!string() returns "-12.0001" :(
How can I convert a double value -12.000123456 to its string
value "-12.000123456", i.e. without loosing double-precision
digits ?
Re: Is there a pragma or command line option to ask DMD to show a warning when it implicitly casts from long to ulong ?
On Friday, 15 December 2017 at 21:55:12 UTC, Michael wrote:
On Friday, 15 December 2017 at 21:29:10 UTC, Jonathan M Davis
wrote:
On Friday, December 15, 2017 20:40:10 Ecstatic Coder via
Digitalmars-d-learn wrote:
It's taken me some time to find an implicit cast bug ("if
(my_sometimes_negative_index >= this_array.length)"), while a
simple C++-like implicit-cast warning would have allowed me
to find this nasty bug WAY sooner...
No.
https://issues.dlang.org/show_bug.cgi?id=259
- Jonathan M Davis
How does something like this get left for so long...
+1
Is there a pragma or command line option to ask DMD to show a warning when it implicitly casts from long to ulong ?
It's taken me some time to find an implicit cast bug ("if
(my_sometimes_negative_index >= this_array.length)"), while a
simple C++-like implicit-cast warning would have allowed me to
find this nasty bug WAY sooner...
Re: Panda Antivirus puts the latest dmd installer in quarantine
Why is it put in quarantine? Posting the reason and some information about it would help a lot to identify the issue. If I remember well, it has detected a "W32/exploit.gen". The file was put in quarantine in the second I tried to execute the installer. Here is the current VirusTotal analysis : https://www.virustotal.com/fr/file/d12e9521ab0ad6a9c0babadeb789692b625fa3535ff406f27a7f47a096430a99/analysis/ The following malware signatures were detected a few hours ago : Baidu Win32.Trojan.WisdomEyes.16070401.9500.9674 20170706 CAT-QuickHeal Trojan.Generic 20170706 Comodo Heur.Packed.Unknown 20170706 TrendMicro-HouseCallSuspicious_GEN.F47V0606 20170706 To be sure, I've just submitted the file for a new analysis, from a file downloaded on my linux system. The Comodo detection has disappeared, but the others remain : Baidu Win32.Trojan.WisdomEyes.16070401.9500.9674 20170706 CAT-QuickHeal Trojan.Generic 20170706 TrendMicro-HouseCallSuspicious_GEN.F47V0606 20170706
Panda Antivirus puts the latest dmd installer in quarantine
But no problem with any file stored inside the current .7z archive file. So I guess the problem comes from the installer executable itself. Please try to fix this as soon as possible, as this immediately drives people away from D before they even got a chance to install it...
