Re: [fpc-devel] I get duplicate GUIDs under Linux
Why can't FPC automatically call randomize() in the RTL. Put it in some initialization section. That way, at application startup, randomize() is already called and only Random() needs to be used? Please not! This would hamper all kinds of scientific, statistical and financial modelling. For all those (and more) it is sometimes necessary to have a *reproducable* random distribution. There's more to random than random ;-) ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On 31/05/2008, Vinzent Höfler [EMAIL PROTECTED] wrote: Note that random() will always return the same sequence if you don't call randomize() once in the program. Yes. But don't make the mistake to call it more than once. Why can't FPC automatically call randomize() in the RTL. Put it in some initialization section. That way, at application startup, randomize() is already called and only Random() needs to be used? I could never figure out why this wasn't done in Delphi either? After all, if you use Random(), you want something random, yet many developers make the common mistakes of not calling Randomize() or calling it to often. If FPC handled that for us, nobody would every make those mistakes again! Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
[ Charset ISO-8859-1 unsupported, converting... ] On 31/05/2008, Vinzent H?fler [EMAIL PROTECTED] wrote: Note that random() will always return the same sequence if you don't call randomize() once in the program. Yes. But don't make the mistake to call it more than once. Why can't FPC automatically call randomize() in the RTL. Put it in some initialization section. That way, at application startup, randomize() is already called and only Random() needs to be used? Then the whole random system is linked in, even when not used. Also when you would want to set a different randseed, you'd call it twice (though I can't imagine that being such a problem nowadays) ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Graeme Geldenhuys schrieb: On 31/05/2008, Vinzent Höfler [EMAIL PROTECTED] wrote: Note that random() will always return the same sequence if you don't call randomize() once in the program. Yes. But don't make the mistake to call it more than once. Why can't FPC automatically call randomize() in the RTL. Put it in some initialization section. That way, at application startup, randomize() is already called and only Random() needs to be used? I could never figure out why this wasn't done in Delphi either? After all, if you use Random(), you want something random, yet many developers make the common mistakes of not calling Randomize() or calling it to often. If FPC handled that for us, nobody would every make those mistakes again! People might want to start with a defined randseed to reproduce behaviour. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On 02 Jun 2008, at 09:49, Marco van de Voort wrote: Why can't FPC automatically call randomize() in the RTL. Put it in some initialization section. That way, at application startup, randomize() is already called and only Random() needs to be used? Then the whole random system is linked in, even when not used. Randomize only causes the randseed variable to be initialised/used. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On 02/06/2008, Marco van de Voort [EMAIL PROTECTED] wrote: Then the whole random system is linked in, even when not used. I would have thought this is already linked in as part of the RTL. Simply not initialized until the first Randomize() call. Also when you would want to set a different randseed, you'd call it twice (though I can't imagine that being such a problem nowadays) Ok, valid point. Even so, I still think many people use Randomize() and Random() incorrectly which causes more harm (unexpected results) compared to *maybe* having to set a different randseed and calling randomize() twice. Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Daniël Mantione schreef: Op Mon, 2 Jun 2008, schreef Florian Klaempfl: all, if you use Random(), you want something random, yet many developers make the common mistakes of not calling Randomize() or calling it to often. If FPC handled that for us, nobody would every make those mistakes again! People might want to start with a defined randseed to reproduce behaviour. This implies createguid should not call randomize automatically either, it prevents you having deterministic behaviour, especially in a program where guids and normal use of the randomizer are combined. It is not a problem anymore. Call CreateGuid once, set the randseed and you get predictable guids afterwards. Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Op Mon, 2 Jun 2008, schreef Florian Klaempfl: all, if you use Random(), you want something random, yet many developers make the common mistakes of not calling Randomize() or calling it to often. If FPC handled that for us, nobody would every make those mistakes again! People might want to start with a defined randseed to reproduce behaviour. This implies createguid should not call randomize automatically either, it prevents you having deterministic behaviour, especially in a program where guids and normal use of the randomizer are combined. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On Mon, 2 Jun 2008, Daniël Mantione wrote: Op Mon, 2 Jun 2008, schreef Florian Klaempfl: all, if you use Random(), you want something random, yet many developers make the common mistakes of not calling Randomize() or calling it to often. If FPC handled that for us, nobody would every make those mistakes again! People might want to start with a defined randseed to reproduce behaviour. This implies createguid should not call randomize automatically either, it prevents you having deterministic behaviour, especially in a program where guids and normal use of the randomizer are combined. Creating GUIDs should never be deterministic.. ? Michael.___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Op Mon, 2 Jun 2008, schreef Michael Van Canneyt: People might want to start with a defined randseed to reproduce behaviour. This implies createguid should not call randomize automatically either, it prevents you having deterministic behaviour, especially in a program where guids and normal use of the randomizer are combined. Creating GUIDs should never be deterministic.. ? For debugging it can be usefull to have deterministic behaviour. But that was not the point. The point was, that a call to randomize disrupts the current state of the random generator. That means that if you use random outside guid creation, creating a guid can have unintended side effects on your random generation. I agree it is nitpicking, but strictly speaking, it is a bad idea to automatically initialize the random generator. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Klaus Hartnegg schreef: a call to randomize disrupts the current state of the random generator. That means that if you use random outside guid creation, creating a guid can have unintended side effects on your random generation. Maybe the GUID generator should have an automatic call to randomize, but use its own randseed (or preserve the original value): randseed_old := randseed; if not guid_rnd_initialized then begin randomize; guid_rnd_initialized := true; end else randseed := guid_randseed; result := rnd2guid(random); guid_randseed := randseed; randseed := randseed_old; It is not that easy to restore the state of random number generator. Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On Mon, 2 Jun 2008, Daniël Mantione wrote: Op Mon, 2 Jun 2008, schreef Michael Van Canneyt: Creating GUIDs should never be deterministic.. ? a call to randomize disrupts the current state of the random generator. That means that if you use random outside guid creation, creating a guid can have unintended side effects on your random generation. Maybe the GUID generator should have an automatic call to randomize, but use its own randseed (or preserve the original value): randseed_old := randseed; if not guid_rnd_initialized then begin randomize; guid_rnd_initialized := true; end else randseed := guid_randseed; result := rnd2guid(random); guid_randseed := randseed; randseed := randseed_old;___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Vincent Snijders wrote: Klaus Hartnegg schreef: a call to randomize disrupts the current state of the random generator. That means that if you use random outside guid creation, creating a guid can have unintended side effects on your random generation. Maybe the GUID generator should have an automatic call to randomize, but use its own randseed (or preserve the original value): randseed_old := randseed; if not guid_rnd_initialized then begin randomize; guid_rnd_initialized := true; end else randseed := guid_randseed; result := rnd2guid(random); guid_randseed := randseed; randseed := randseed_old; It is not that easy to restore the state of random number generator. Not that easy is quite an understatement here. ;) It's very close to impossible without changing the System-unit. That was one issue I had. The other one was simple that System's RNG is not thread-safe, so I finally decided to use an RNG on a per object basis here. This makes the necessary locking of a system wide RNG in threading context much easier and additionally I can still use a deterministic RNG in contexts where I need it, because now I can have as many dedicated RNGs as I like. :) Maybe, such Random class should be incorporated in the FCL, too? Vinzent. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Vinzent Höfler schrieb: If Randomize() were truly and more random, one wouldn't need Random() after all, would one? Randomize is rather slow compared with random. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Zitat von Graeme Geldenhuys [EMAIL PROTECTED]:
Hi,
I'm trying to do performance testing in Firebird RDBMS using GUID's
and primary keys. I constantly got PK violations. I then wrote the
code show below to create 50,000 GUID's and add them to a StringList
(no duplicates allowed). Every time I run this I get a duplicate GUID
at iteration 1021. My original test is supposed to create 5mil
records, but I can't get past 1000! :-(
Sounds like the Debian openssl key fiasko.
Add it to the list:
http://xkcd.com/424/
Mattias
Actual program output:
[EMAIL PROTECTED]:index_performance$ ./index_performance.elf
Duplicate value detected: {FD3DF19D-A878-F8BA-F9D1-108658451E05}
List.Count = 1021
GUID's are supposed to be very unique! Wikipedia says something like:
Create a billion every second for 10 years or something before you
deplete the range.
So what is wrong with my code or maybe the Free Pascal implementation?
[ code ]--
procedure TMainForm.btnTestClicked(Sender: TObject);
var
sl: TStringList;
s: string;
i: integer;
d: integer;
guid: TGUID;
begin
sl := TStringList.Create;
try
try
sl.Sorted := True;
sl.Duplicates := dupError;
for i := 1 to 5 do
begin
CreateGUID(guid);
s := GUIDToString(guid);
sl.Add(s);
end;
writeln('Created 50,000 GUIDs');
except
on e: EStringListError do
begin
writeln('Duplicate value detected: ' + s);
writeln(' List.Count = ', sl.Count);
end;
on e: Exception do
writeln('Unknown error: ', e.Message);
end;
finally
sl.Free;
end;
end;
[ end ]
Changing the line that adds the GUID to the StringList by prefixing
the counter, all is fine. So I can't see a problem in my code.
sl.Add(IntToStr(i) + s);
Regards,
- Graeme -
___
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
--
Powered by NetMail
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On Thu, 29 May 2008, Graeme Geldenhuys wrote: Hi, I'm trying to do performance testing in Firebird RDBMS using GUID's and primary keys. I constantly got PK violations. I then wrote the code show below to create 50,000 GUID's and add them to a StringList (no duplicates allowed). Every time I run this I get a duplicate GUID at iteration 1021. My original test is supposed to create 5mil records, but I can't get past 1000! :-( As a short workaround: The default implementation on Linux uses a random() call to generate a guid, combined with a timestamp, if I recall correctly. You can use the kernel mechanism by inluding libuuid in your uses clause. That should ensure uniqueness. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On Thu, 29 May 2008, Graeme Geldenhuys wrote: I'm trying to do performance testing in Firebird RDBMS using GUID's and primary keys. I constantly got PK violations. I then wrote the code show below to create 50,000 GUID's and add them to a StringList (no duplicates allowed). Every time I run this I get a duplicate GUID at iteration 1021. My original test is supposed to create 5mil records, but I can't get past 1000! :-( As a short workaround: The default implementation on Linux uses a random() call to generate a guid, combined with a timestamp, if I recall correctly. rtl/linux/suuid.inc reads from '/proc/sys/kernel/random/uuid' Maybe it is not used though. You can use the kernel mechanism by inluding libuuid in your uses clause. That should ensure uniqueness. It's unportable though. It is wiser to try to find out why the default (and portable call) doesn't work on Linux, using a system generator. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On 29 May 2008, at 13:26, Michael Van Canneyt wrote: The default implementation on Linux uses a random() call to generate a guid, combined with a timestamp, if I recall correctly. Note that random() will always return the same sequence if you don't call randomize() once in the program. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
2008/5/29 Mattias Gärtner [EMAIL PROTECTED]: Sounds like the Debian openssl key fiasko. I saw Ubuntu had some updates to resolve this issue on my system. Got the updates, rebooted, recompiled my application and still 1021 is my limit before I get duplicates. What is really weird is that once I hit the 1021 limit, I can't run that method again. I get a duplicate after 1 add! It seems to behave exactly like random() without calling randomize(). It seems to remember the last position it it's seed or something and tries to continue where it left off. The same pattern, over and over. Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
2008/5/29 Michael Van Canneyt [EMAIL PROTECTED]: The default implementation on Linux uses a random() call to generate a guid, combined with a timestamp, if I recall correctly. You can use the kernel mechanism by inluding libuuid in your uses clause. That should ensure uniqueness. Yup, that works fine, but like Marco said... The underlying problem still persists. Thanks for the workaround though. Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Zitat von Graeme Geldenhuys [EMAIL PROTECTED]: 2008/5/29 Mattias Gärtner [EMAIL PROTECTED]: Sounds like the Debian openssl key fiasko. I saw Ubuntu had some updates to resolve this issue on my system. Got the updates, rebooted, recompiled my application and still 1021 is my limit before I get duplicates. Sorry. Openssl has *nothing* to do with the CreateGUID function. But the CreateGUID calls the kernel function and if this does not work it calls Randomize and calls 16 times random(256). But Randomize only work from time to time, so don't call it often - Bug in GetRandomBytes. IMO GetRandomBytes needs to check if Randomize has been called. Or even better: Randomize should not reset the sequence if no random number is available. What is really weird is that once I hit the 1021 limit, I can't run that method again. I get a duplicate after 1 add! It seems to behave exactly like random() without calling randomize(). It seems to remember the last position it it's seed or something and tries to continue where it left off. The same pattern, over and over. Maybe the 1021 is the limit of /dev/random. Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
2008/5/29 Marco van de Voort [EMAIL PROTECTED]: rtl/linux/suuid.inc reads from '/proc/sys/kernel/random/uuid' Maybe it is not used though. How do I know if it's using Random or '/proc/sys/kernel/random/uuid'? Browsing the RTL source code with Lazarus, I believe I am using the latter, but not 100% sure. Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Graeme Geldenhuys schrieb: 2008/5/29 Marco van de Voort [EMAIL PROTECTED]: rtl/linux/suuid.inc reads from '/proc/sys/kernel/random/uuid' Maybe it is not used though. How do I know if it's using Random or '/proc/sys/kernel/random/uuid'? Browsing the RTL source code with Lazarus, I believe I am using the latter, but not 100% sure. If it uses /proc/sys/kernel/random/uuid, strace shows it. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
2008/5/29 Mattias Gärtner [EMAIL PROTECTED]: Sorry. Openssl has *nothing* to do with the CreateGUID function. I thought that much, but then thought it wouldn't hurt installing the patch anyway. What is really weird is that once I hit the 1021 limit, I can't run that method again. I get a duplicate after 1 add! It seems to behave exactly like random() without calling randomize(). It seems to remember the last position it it's seed or something and tries to continue where it left off. The same pattern, over and over. Maybe the 1021 is the limit of /dev/random. I gather that you can reproduce the issue and also get stuck on 1021? Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
2008/5/29 Florian Klaempfl [EMAIL PROTECTED]:
How do I know if it's using Random or '/proc/sys/kernel/random/uuid'?
Browsing the RTL source code with Lazarus, I believe I am using the
latter, but not 100% sure.
If it uses /proc/sys/kernel/random/uuid, strace shows it.
Thanks Florian. I am using the '.../random/uuid'.
nanosleep({0, 7}, {3082747920, 3219878756}) = 0
open(/proc/sys/kernel/random/uuid, O_RDONLY|O_LARGEFILE) = 872
read(872, 2c4bd506-c58f-4b6d-b88c-eec01a05..., 36) = 36
ioctl(3, FIONREAD, [0]) = 0
nanosleep({0, 7}, {3082747920, 3219878756}) = 0
open(/proc/sys/kernel/random/uuid, O_RDONLY|O_LARGEFILE) = 873
read(873, ea14fbdc-8d6b-45de-ae7a-5e6e1908..., 36) = 36
ioctl(3, FIONREAD, [0]) = 0
nanosleep({0, 7}, unfinished ...
Process 6582 detached
I'm not putting a 700ms sleep between each CreateGUID call, to see
what happens. I noticed that once I hit the 1021 limit at normal speed
(no sleep). Every time I click the button quickly (2-3 times per
second) it instantly generates the same GUID. If I wait a second
between clicks, I get a duplicate on the 2nd add to stringlist. Weird,
but then, I got no idea what this will prove! :-)
Regards,
- Graeme -
___
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Graeme Geldenhuys schreef: 2008/5/29 Florian Klaempfl [EMAIL PROTECTED]: How do I know if it's using Random or '/proc/sys/kernel/random/uuid'? Browsing the RTL source code with Lazarus, I believe I am using the latter, but not 100% sure. If it uses /proc/sys/kernel/random/uuid, strace shows it. Thanks Florian. I am using the '.../random/uuid'. See also: http://forum.soft32.com/linux/PATCH-Time-based-RFC-4122-UUID-generator-ftopict467475.html random/uuid doesn't seem to support reading lot's of UUID is a short period. Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Zitat von Graeme Geldenhuys [EMAIL PROTECTED]: 2008/5/29 Mattias Gärtner [EMAIL PROTECTED]: Sorry. Openssl has *nothing* to do with the CreateGUID function. I thought that much, but then thought it wouldn't hurt installing the patch anyway. It would hurt a lot *not* installing the patch. The bigger problem is that the other distributions including windows, Mac OS X, BSD, ... server must now install the blacklist too. I guess non debian systems now have bigger problems than debian. What is really weird is that once I hit the 1021 limit, I can't run that method again. I get a duplicate after 1 add! It seems to behave exactly like random() without calling randomize(). It seems to remember the last position it it's seed or something and tries to continue where it left off. The same pattern, over and over. Maybe the 1021 is the limit of /dev/random. I gather that you can reproduce the issue and also get stuck on 1021? Yes. 1019 And then pango gave: (testproject:27449): Pango-WARNING **: error opening config file '/etc/pango/pangorc': Too many open files Apparently CreateGUID is buggy and should not be used under Linux. Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
GUID's are supposed to be very unique! Wikipedia says something like: Create a billion every second for 10 years or something before you deplete the range. So what is wrong with my code or maybe the Free Pascal implementation? I quicktested the FreeBSD implementation, verified it used the syscall, and the first half million are no problem under either 6.3 or (the very recent) 7.0 ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On Thu, 29 May 2008, Mattias Gärtner wrote: Zitat von Graeme Geldenhuys [EMAIL PROTECTED]: 2008/5/29 Mattias Gärtner [EMAIL PROTECTED]: Sorry. Openssl has *nothing* to do with the CreateGUID function. I thought that much, but then thought it wouldn't hurt installing the patch anyway. It would hurt a lot *not* installing the patch. The bigger problem is that the other distributions including windows, Mac OS X, BSD, ... server must now install the blacklist too. I guess non debian systems now have bigger problems than debian. What is really weird is that once I hit the 1021 limit, I can't run that method again. I get a duplicate after 1 add! It seems to behave exactly like random() without calling randomize(). It seems to remember the last position it it's seed or something and tries to continue where it left off. The same pattern, over and over. Maybe the 1021 is the limit of /dev/random. I gather that you can reproduce the issue and also get stuck on 1021? Yes. 1019 And then pango gave: (testproject:27449): Pango-WARNING **: error opening config file '/etc/pango/pangorc': Too many open files Apparently CreateGUID is buggy and should not be used under Linux. Fixed the file handle leak. Merging to fixes now. Michael.___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
2008/5/29 Mattias Gärtner [EMAIL PROTECTED]: It would hurt a lot *not* installing the patch. The bigger problem is that the other distributions including windows, Mac OS X, BSD, ... server must now install the blacklist too. I guess non debian systems now have bigger problems than debian. Yeah, I google'd the debian fiasco to get a better understanding. It's a *huge* screw-up! I gather that you can reproduce the issue and also get stuck on 1021? Yes. 1019 And then pango gave: (testproject:27449): Pango-WARNING **: error opening config file '/etc/pango/pangorc': Too many open files Apparently CreateGUID is buggy and should not be used under Linux. I need to convert existing data to our new system, so I am in need of a lot of GUID's quickly. At least in the interim I have a workaround via the 'libuuid' unit, otherwise I would be in big trouble. How can I force my fpc to use the 'random' option, instead of '.../random/uuid'? Or must I copy and paste the code from RTL into my own function? Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On Thu, 29 May 2008, Graeme Geldenhuys wrote: 2008/5/29 Mattias Gärtner [EMAIL PROTECTED]: Sorry. Openssl has *nothing* to do with the CreateGUID function. I thought that much, but then thought it wouldn't hurt installing the patch anyway. What is really weird is that once I hit the 1021 limit, I can't run that method again. I get a duplicate after 1 add! It seems to behave exactly like random() without calling randomize(). It seems to remember the last position it it's seed or something and tries to continue where it left off. The same pattern, over and over. Maybe the 1021 is the limit of /dev/random. I gather that you can reproduce the issue and also get stuck on 1021? Graeme, can you send me your test program, so I can test my fixes ? Michael.___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
2008/5/29 Michael Van Canneyt [EMAIL PROTECTED]:
can you send me your test program, so I can test my fixes ?
Here you go. It is pretty much what I sent before, except now I made
it a console application instead of a fpGUI one. Below it tries to
create 50,000 GUID's. My SQL version tries 5mil.
-[ project1.pas ]--
program Project1;
{$mode objfpc}{$H+}
uses
{$IFDEF UNIX}{$IFDEF UseCThreads}
cthreads,
{$ENDIF}{$ENDIF}
Classes, SysUtils;
procedure GUIDTest;
var
sl: TStringList;
s: string;
i: integer;
d: integer;
guid: TGUID;
begin
sl := TStringList.Create;
try
try
sl.Sorted := True;
sl.Duplicates := dupError;
for i := 1 to 5 do
begin
CreateGUID(guid);
s := GUIDToString(guid);
sl.Add(s);
//sl.Add(IntToStr(i) + s);
end;
writeln('Created ', sl.Count, ' GUIDs');
except
on e: EStringListError do
begin
writeln('Duplicate value detected: ' + s);
writeln(' List.Count = ', sl.Count);
end;
on e: Exception do
writeln('Unknown error: ', e.Message);
end;
finally
sl.Free;
end;
end;
begin
GUIDTest;
end.
--[ end ]---
Regards,
- Graeme -
___
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On Thu, 29 May 2008, Graeme Geldenhuys wrote: 2008/5/29 Michael Van Canneyt [EMAIL PROTECTED]: can you send me your test program, so I can test my fixes ? Here you go. It is pretty much what I sent before, except now I made it a console application instead of a fpGUI one. Below it tries to create 50,000 GUID's. My SQL version tries 5mil. Well, after my patch, it creates 500.000 as well, no errors. I had to modify your algoritm a bit to make it more efficient, and I'm too lazy to wait for 5 million. So I guess the bug is fixed. It's merged to the fixbranch. If you want to use your own GUID creation routine, you can always set the OnCreateGUID handler in sysutils. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Michael Van Canneyt schreef: On Thu, 29 May 2008, Graeme Geldenhuys wrote: 2008/5/29 Michael Van Canneyt [EMAIL PROTECTED]: can you send me your test program, so I can test my fixes ? Here you go. It is pretty much what I sent before, except now I made it a console application instead of a fpGUI one. Below it tries to create 50,000 GUID's. My SQL version tries 5mil. Well, after my patch, it creates 500.000 as well, no errors. I had to modify your algoritm a bit to make it more efficient, and I'm too lazy to wait for 5 million. So I guess the bug is fixed. It's merged to the fixbranch. The bug was probably there, because GetRandomBytes (which was called if the kernel function didn't work) return duplicates if it is called within the same tick count. Can you fix that too, as suggested by Mattias? Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
On Thu, 29 May 2008, Vincent Snijders wrote: Michael Van Canneyt schreef: On Thu, 29 May 2008, Graeme Geldenhuys wrote: 2008/5/29 Michael Van Canneyt [EMAIL PROTECTED]: can you send me your test program, so I can test my fixes ? Here you go. It is pretty much what I sent before, except now I made it a console application instead of a fpGUI one. Below it tries to create 50,000 GUID's. My SQL version tries 5mil. Well, after my patch, it creates 500.000 as well, no errors. I had to modify your algoritm a bit to make it more efficient, and I'm too lazy to wait for 5 million. So I guess the bug is fixed. It's merged to the fixbranch. The bug was probably there, because GetRandomBytes (which was called if the kernel function didn't work) return duplicates if it is called within the same tick count. Can you fix that too, as suggested by Mattias? I patched it so Randomize is called only once. Merged to fixes. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Zitat von Graeme Geldenhuys [EMAIL PROTECTED]: 2008/5/29 Mattias Gärtner [EMAIL PROTECTED]: It would hurt a lot *not* installing the patch. The bigger problem is that the other distributions including windows, Mac OS X, BSD, ... server must now install the blacklist too. I guess non debian systems now have bigger problems than debian. Yeah, I google'd the debian fiasco to get a better understanding. It's a *huge* screw-up! I gather that you can reproduce the issue and also get stuck on 1021? Yes. 1019 And then pango gave: (testproject:27449): Pango-WARNING **: error opening config file '/etc/pango/pangorc': Too many open files Apparently CreateGUID is buggy and should not be used under Linux. I need to convert existing data to our new system, so I am in need of a lot of GUID's quickly. At least in the interim I have a workaround via the 'libuuid' unit, otherwise I would be in big trouble. How can I force my fpc to use the 'random' option, instead of '.../random/uuid'? Or must I copy and paste the code from RTL into my own function? If the GUID are used in security contexts, then you *must* wait and gather some real random numbers. Otherwise you can simply call Randomize once at the start of the program and use: Procedure GetRandomBytes(Var Buf; NBytes : Integer); Var I : Integer; P : PByte; begin P:[EMAIL PROTECTED]; For I:=0 to NBytes-1 do P[i]:=Random(256); end; Randomize; // call this only once every few seconds for i:=0 to 500 do begin GetRandomBytes(GUID,SizeOf(Guid)); This gave me 5 mil unique guids. Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] I get duplicate GUIDs under Linux
Vincent Snijders schrieb: Michael Van Canneyt schreef: I patched it so Randomize is called only once. Merged to fixes. Thanks. BTW, the log message made me think you introduced a new bug :-) GetRandomBytes calls random only once instead of GetRandomBytes calls randomize only once ... and it is not MT safe ;) because the variable is set before the actual work is done. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
