Re: [lazarus] Making GTK Thread Safe
Sam Liddicott wrote: If you want to fake a older kernel, try: LD_ASSUME_KERNEL=2.2.5 In the environmen before ou run. I know it affects the threading model among other things. Sam Ooh thats is a very nice feature :-) I am also interested in the C version of threadbug. regards Wim -Original Message- From: Giuliano Colla [EMAIL PROTECTED] Sent: 07 January 2008 00:00 To: lazarus@miraclec.com Subject: Re: [lazarus] Making GTK Thread Safe willem ha scritto: Giuliano Colla wrote: willem ha scritto: Please send your modified threadbug program , so that i can compare the results on my system. Please find it here attached. Giuliano Well I have the same result. the virtual memory has increased. But I am using kernel 2.6.22 but the latest kernel is 2.6.23.12 I have libc 2.6.1 but the latest version is 2.7 I am using an older kernel (2.6.20 ) and an older glibc (2.4.11). AFAIK linux thread handling shouldn't have changed in this span. Al doesn't see the same problem with fpc 2.0.2, but from previous posting I seem to remember he's also using an older kernel. What I'd like to do is to make the same test with a C program, just to have a reference, but it'll take some time, because I'm not so good with C ;-) I'll let you know the results as soon as I have them. Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote:
BTW could you refresh my memory? What kernel and libc are you running
with?
linux2.6.22/libc2.3.1
Here is threadbug.c which doesn't show any leaks:
// gcc -lpthread threadbug.c -o threadbug
#include pthread.h
void *threadbug() {
printf(.);
}
main(){
int i;
pthread_attr_t attr;
pthread_attr_init(attr);
pthread_t threads[999];
while (1) {
printf(\nstarting threads...\n);
sleep(3);
for (i=0;i1000;i++)
if ( pthread_create(threads[i],attr,threadbug,NULL) )
perror(pthread_create);
printf(\nfreeing threads...\n);
sleep(3);
for (i=0;i1000;i++) {
pthread_join(threads[i], NULL);
printf(.);
}
}
}
Thanks!
--
Al
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Al Boldi ha scritto:
Giuliano Colla wrote:
BTW could you refresh my memory? What kernel and libc are you running
with?
linux2.6.22/libc2.3.1
Here is threadbug.c which doesn't show any leaks:
Surprise!
I've slightly modified threadbug.c - threadbug1.c (attached), in order
to have stops at the proper points, and verify leakage more precisely.
Scenario for today platform: kernel 2.4.21 heavily patched by RedHat,
with a lot of 2.6 features - glibc 2.3.2 (patched accordingly), only
512Mb RAM. On this platform I'm forced to use fpc 2.0.4 because of glibc
incompatibility.
It turns out that LD_ASSUME_KERNEL 2.2.5 makes each thread allocate two
blocks, one of 2044k, and one of 4k, while this kernel (like the 2.6.20
I've been using yesterday) allocates one block of 10240k and one of 4k.
Therefore with 999 threads I had to use LD_ASSUME, to remain within
memory limits. No leakage. Virtual memory: 4136k-2052136k-4136k one
run 4116k-2052116k-4116k.
Changed to 199 threads. 2.2.5 no leakage: 4128-411680-4128.
kernel 2.4.21 -- LEAKAGE OF THE SAME AMOUNT AS WITH FPC:
3844k-2042400k-34576k. It failed to deallocate 3 blocks of 10240k and
3 blocks of 4k. Removing printf(.) doesn't make any difference.
To be more precise, comparing with fpc multithread behavior (same
scenario), fpc 2.0.4 seems to leak slightly more: it fails to free one
4k block with kernel 2.2.5 (2752-618356-2756) , and 3*10240k + 13*4k
with 2.4.21 (2472-3076876-33244)
But if I remove the Terminate and WaitFor, and use just Free, then with
2.2.5 the leakage disappears (2760k-xxx-2760k) while with 2.4.21 the
leakage is unchanged.
My tentative conclusions. Either the test is wrong, because the
deallocation is somehow deferred, and an I/O operation also suspends the
deallocation, or Linux threads have a bug. But, considering the results
from the original threadbug.c compared with threadbug1.c, I'm beginning
to be in favor of the first of the two.
More to come, stay tuned!
Giuliano
--
Giuliano Colla
Whenever people agree with me, I always feel I must be wrong (O. Wilde)
// gcc -lpthread -lreadline -ltermcap threadbug1.c -o threadbug1
#include pthread.h
#include stdio.h
#include readline/readline.h
#define NUM_THREADS 199
char *line;
int wait = 0;
int running;
void *threadbug() {
printf(.);
}
main(){
int i;
pthread_attr_t attr;
pthread_attr_init(attr);
pthread_t threads[NUM_THREADS];
line = readline(Interactive?);
if (line[0] == 'y') wait = 1; else wait = 0;
running = 1;
while (running) {
printf(\nstarting threads...\n);
sleep(3);
for (i=0;iNUM_THREADS;i++)
if ( pthread_create(threads[i],attr,threadbug,NULL) )
perror(pthread_create);
if (wait) {
line = readline(\nNow?);
if (line[0] == 'h') running = 0;
}
printf(\nfreeing threads...\n);
sleep(3);
for (i=0;iNUM_THREADS;i++) {
pthread_join(threads[i], NULL);
//printf(.);
}
}
line = readline(\nEnter to terminate);
}
Re: [lazarus] Making GTK Thread Safe
Al Boldi wrote: willem wrote: well I made 10 copy's of threadbug and run them in parallel. In my case libpthread does not allocate a 10 Mb memory block, because the 10 processes should allocate 100Mb of memory Top does show only a sligth increase in memory . On my kubuntu there are 145 processes allocated . So i can not reproduce your problem. I just tried on another distribution, and without WaitFor the leak is still there, but a lot smaller, only ~12kb per thread. Did you run with or without WaitFor? How big is the thread overhead? Thanks! -- Al well I modified my threadbug program in a endless repeat until loop. I cannot observe memery leaks on my my kubuntu system. here is is my top output: top - 08:47:47 up 8 min, 1 user, load average: 2.13, 1.36, 0.71 Tasks: 147 total, 2 running, 145 sleeping, 0 stopped, 0 zombie Cpu(s): 56.7%us, 16.6%sy, 0.0%ni, 26.5%id, 0.0%wa, 0.1%hi, 0.0%si, 0.0%st Mem: 1027068k total, 608716k used, 418352k free,16276k buffers Swap: 3004112k total,0k used, 3004112k free, 295140k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 7246 wli22225 0 34160 14m 11m R 78 1.5 3:00.55 konsole 7264 wli22225 0 10600 880 604 S 44 0.1 1:29.93 threadbug 5278 root 15 0 350m 13m 4064 S 18 1.3 0:32.88 Xorg 7044 wli22215 0 35520 17m 13m S2 1.7 0:02.41 kicker 7137 wli22215 0 34260 15m 11m S1 1.5 0:01.17 konsole 7040 wli22215 0 30584 11m 9020 S1 1.1 0:00.89 kwin 7027 wli22215 0 25980 3156 1764 S0 0.3 0:00.23 dcopserver 7244 wli22215 0 2364 1188 884 R0 0.1 0:00.37 top 2114 root 10 -5 000 S0 0.0 0:00.07 ata/0 2314 root 10 -5 000 S0 0.0 0:00.11 scsi_eh_6 5293 haldaemo 16 0 3260 1184 1036 S0 0.1 0:00.08 hald-addon-stor 6235 root 15 0 18188 1060 720 S0 0.1 0:00.02 webserver 7032 wli22215 0 37980 18m 14m S0 1.8 0:01.09 kded 7042 wli22215 0 32704 13m 10m S0 1.4 0:02.44 kdesktop 7086 wli22215 0 46808 25m 18m S0 2.5 0:05.06 ktorrent here is my pmap output: 7264: ./threadbug 08048000300K r-x-- /home/wli222/thread5/threadbug 08093000116K rwx-- /home/wli222/thread5/threadbug 080b140K rwx--[ anon ] b753d000 40K r-x-- /lib/libgcc_s.so.1 b7547000 4K rwx-- /lib/libgcc_s.so.1 b755b000 4K rwx--[ anon ] b755c000 4K -[ anon ] b755d000 8288K rwx--[ anon ] b7d75000 80K r-x-- /lib/tls/i686/cmov/libpthread-2.6.1.so b7d89000 8K rwx-- /lib/tls/i686/cmov/libpthread-2.6.1.so b7d8b000 12K rwx--[ anon ] b7d8e000 1296K r-x-- /lib/tls/i686/cmov/libc-2.6.1.so b7ed2000 4K r-x-- /lib/tls/i686/cmov/libc-2.6.1.so b7ed3000 8K rwx-- /lib/tls/i686/cmov/libc-2.6.1.so b7ed5000 12K rwx--[ anon ] b7ed8000 8K r-x-- /lib/tls/i686/cmov/libdl-2.6.1.so b7eda000 8K rwx-- /lib/tls/i686/cmov/libdl-2.6.1.so b7edf000 76K rwx--[ anon ] b7ef2000104K r-x-- /lib/ld-2.6.1.so b7f0c000 8K rwx-- /lib/ld-2.6.1.so bf9c5000 84K rwx--[ stack ] e000 4K r-x--[ anon ] total10608K [EMAIL PROTECTED]:~$ regards Wim _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
willem ha scritto: I cannot observe memery leaks on my my kubuntu system. Which kernel version and libc are you running? Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote: willem ha scritto: I cannot observe memery leaks on my my kubuntu system. Which kernel version and libc are you running? Giuliano I am using kernel 2.6.22 and libc6 2.6.1-ubuntu10 regards Wim _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote: willem ha scritto: well I made 10 copy's of threadbug and run them in parallel. In my case libpthread does not allocate a 10 Mb memory block, because the 10 processes should allocate 100Mb of memory Top does show only a sligth increase in memory . On my kubuntu there are 145 processes allocated . So i can not reproduce your problem. Well, virtual memory administration is a bit more elaborate than Top totals can show. I suggest you reading: http://www.redhat.com/magazine/001nov04/features/vm/ to have a general idea. A request for memory allocation just provides an entry in the virtual memory table, without any physical counterpart. That's why you may enable the kernel to over commit (see overcommit_memory and overcommit_ratio in the above article), i.e. to accept allocation requests for more memory than available (RAM size + swap size). That can be done if you know for sure that many of your processes allocate more memory than they'll actually use. Allocated memory becomes physical memory only when an attempt to access it is performed, care of MMU unit. And only at this stage Top totals shows it. But unless you've set (temerarily) your kernel parameters otherwise, when the total of allocation requests exceeds the available physical space, further requests are rejected. So you may have Top reporting 800Mb free, and your application terminated with an out of memory error, when requesting the allocation of 10 Mb. I've modified threadbug to activate many threads instead if just one. I show you the results with 300 threads. Thats Top report when threadbug is started: top - 11:56:33 up 37 min, 6 users, load average: 0.33, 0.16, 0.11 Tasks: 127 total, 1 running, 126 sleeping, 0 stopped, 0 zombie Cpu(s): 0.8%us, 0.2%sy, 0.0%ni, 98.8%id, 0.0%wa, 0.2%hi, 0.0%si, 0.0%st Mem: 1024940k total, 496500k used, 528440k free,22416k buffers Swap: 1028152k total,0k used, 1028152k free, 312576k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 4227 colla 17 0 2548 788 488 S0 0.1 0:00.00 threadbug That's the output of Top after 300 threads have been activated: top - 12:00:48 up 42 min, 6 users, load average: 0.30, 0.26, 0.16 Tasks: 127 total, 1 running, 126 sleeping, 0 stopped, 0 zombie Cpu(s): 1.5%us, 0.2%sy, 0.0%ni, 98.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1024940k total, 501224k used, 523716k free,22664k buffers Swap: 1028152k total,0k used, 1028152k free, 312708k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 4227 colla 24 0 3004m 3300 520 S0 0.3 0:00.08 threadbug You can see that memory usage has minimally changed, from 496500k to 501224, but you can also see that the virtual memory allocated to threadbug has jumped from 2548k to 3004m (i.e. 3Gb!). That's Top output after the 300 threads have been destroyed: top - 12:15:55 up 57 min, 6 users, load average: 0.55, 2.65, 2.12 Tasks: 127 total, 2 running, 125 sleeping, 0 stopped, 0 zombie Cpu(s): 0.5%us, 0.2%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1024940k total, 535096k used, 489844k free,24288k buffers Swap: 1028152k total,0k used, 1028152k free, 334308k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 4227 colla 15 0 33364 988 576 S0 0.1 0:01.20 threadbug You can see that total memory usage has increased instead of decreasing, because some allocated memory has been actually used, but what's more important, you can see that virtual memory allocated to threadbug is 33364 kB instead of the initial 2254. Do you obtain different results? Giuliano Please send your modified threadbug program , so that i can compare the results on my system. regards Wim _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
willem ha scritto:
Please send your modified threadbug program , so that i can compare
the results on my system.
Please find it here attached.
Giuliano
program multithread;
{$mode objfpc}{$H+}
uses
{$IFDEF UNIX}
cthreads,
{$ENDIF}
Classes,sysutils
{ add your units here };
type
{ TThreadBug }
TThreadBug=class(TThread)
protected
procedure Execute; override;
public
constructor Create(CreateSuspended: Boolean);
end;
var
Ft: Boolean;
BugArray: array [0..299] of TThreadBug;
{ TThreadBug }
procedure TThreadBug.Execute;
var
I,J:Integer;
begin
write('.');
repeat
sleep(10);
until Terminated;
end;
constructor TThreadBug.Create(CreateSuspended: Boolean);
begin
inherited Create (CreateSuspended);
FreeOnTerminate:= Ft;
end;
var t1:TThreadBug; ans:string; I: Integer;
begin
Ft := False; // set FreeOnTerminate
// Here we create a lot of threads
writeln('Type Enter to start threads');
readln(ans);
for I := 0 to high(BugArray) do begin
BugArray[I]:=TThreadBug.Create(false);
end;
WriteLn('-');
writeln('Type Enter to Destroy threads');
readln(ans);
for I := 0 to high(BugArray) do begin
t1 := BugArray[I];
t1.Terminate;
t1.WaitFor;
write ('.');
if not Ft then begin
sleep(50);
t1.Free;
end;
end;
WriteLn('-');
// Here we create just one
writeln('Type Enter to start thread');
readln(ans);
t1:=TThreadBug.Create(false);
writeln('WaitFor? [y/N]');
readln(ans);
t1.Terminate;
if ans='y' then begin
if not Ft then begin
t1.WaitFor;
writeln('OS thread memory released.');
end;
end else
writeln('OS thread memory left dangling.');
if not Ft then t1.Free;
writeln('Confirm with top now!');
readln;
end.
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote: willem ha scritto: Please send your modified threadbug program , so that i can compare the results on my system. Please find it here attached. Giuliano Well I have the same result. the virtual memory has increased. But I am using kernel 2.6.22 but the latest kernel is 2.6.23.12 I have libc 2.6.1 but the latest version is 2.7 regards Wim _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote:
willem ha scritto:
well I made 10 copy's of threadbug and run them in parallel. In my
case libpthread does not allocate a 10 Mb memory block, because
the 10 processes should allocate 100Mb of memory Top does show only a
sligth increase in memory . On my kubuntu there are
145 processes allocated . So i can not reproduce your problem.
:
:
I've modified threadbug to activate many threads instead if just one.
I show you the results with 300 threads.
Thats Top report when threadbug is started:
Mem: 1024940k total, 496500k used, 528440k free,22416k buffers
Swap: 1028152k total,0k used, 1028152k free, 312576k cached
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
4227 colla 17 0 2548 788 488 S0 0.1 0:00.00 threadbug
That's the output of Top after 300 threads have been activated:
Mem: 1024940k total, 501224k used, 523716k free,22664k buffers
Swap: 1028152k total,0k used, 1028152k free, 312708k cached
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
4227 colla 24 0 3004m 3300 520 S0 0.3 0:00.08 threadbug
You can see that memory usage has minimally changed, from 496500k to
501224, but you can also see that the virtual memory allocated to
threadbug has jumped from 2548k to 3004m (i.e. 3Gb!).
That's Top output after the 300 threads have been destroyed:
Mem: 1024940k total, 535096k used, 489844k free,24288k buffers
Swap: 1028152k total,0k used, 1028152k free, 334308k cached
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
4227 colla 15 0 33364 988 576 S0 0.1 0:01.20 threadbug
You can see that total memory usage has increased instead of decreasing,
because some allocated memory has been actually used, but what's more
important, you can see that virtual memory allocated to threadbug is
33364 kB instead of the initial 2254.
Great analysis!
So if you see this problem on fpc2.2.0 even with WaitFor, then this is a
regression on top of the bug, because your multithread.lpr runs without
leaks using fpc2.0.2.
To get the leak on fpc2.0.2 you have to change your code like this:
for I := 0 to high(BugArray) do begin
t1 := BugArray[I];
t1.Terminate;
-t1.WaitFor;
write ('.'); // this causes intermittent idles causing the leak
if not Ft then begin
sleep(50);
t1.Free;
end;
end;
...and...
writeln('WaitFor? [y/N]');
+ t1.Terminate;
readln(ans); // you need to idle after terminate to see the leak
- t1.Terminate;
if ans='y' then begin
if not Ft then begin
t1.WaitFor;
writeln('OS thread memory released.');
end;
end else
writeln('OS thread memory left dangling.');
Also note that when you TThread.Free a running thread it will do an implied
Terminate and WaitFor, and therefore works without leaking on fpc2.0.2.
Thanks!
--
Al
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
willem ha scritto: Giuliano Colla wrote: willem ha scritto: Please send your modified threadbug program , so that i can compare the results on my system. Please find it here attached. Giuliano Well I have the same result. the virtual memory has increased. But I am using kernel 2.6.22 but the latest kernel is 2.6.23.12 I have libc 2.6.1 but the latest version is 2.7 I am using an older kernel (2.6.20 ) and an older glibc (2.4.11). AFAIK linux thread handling shouldn't have changed in this span. Al doesn't see the same problem with fpc 2.0.2, but from previous posting I seem to remember he's also using an older kernel. What I'd like to do is to make the same test with a C program, just to have a reference, but it'll take some time, because I'm not so good with C ;-) I'll let you know the results as soon as I have them. Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Al Boldi ha scritto:
To get the leak on fpc2.0.2 you have to change your code like this:
for I := 0 to high(BugArray) do begin
t1 := BugArray[I];
t1.Terminate;
-t1.WaitFor;
write ('.'); // this causes intermittent idles causing the leak
if not Ft then begin
sleep(50);
t1.Free;
end;
end;
...and...
writeln('WaitFor? [y/N]');
+ t1.Terminate;
readln(ans); // you need to idle after terminate to see the leak
- t1.Terminate;
if ans='y' then begin
if not Ft then begin
t1.WaitFor;
writeln('OS thread memory released.');
end;
end else
writeln('OS thread memory left dangling.');
Also note that when you TThread.Free a running thread it will do an implied
Terminate and WaitFor, and therefore works without leaking on fpc2.0.2.
Well, the advantage of fpc 2.2.0 is that it's more consistent. :-)
Whatever you do (write or no write, sleep(x) or no sleep,
FreeOnTerminate or not, just Free without Terminate and WaitFor, or
Terminate, WaitFor and Free) the leakage is always the same :-)
But as I've found some old postings about thread memory leaks in Linux,
I'd like to make some comparative tests to ascertain if the problem
comes from fpc or from linux kernel.
BTW could you refresh my memory? What kernel and libc are you running with?
Regards,
Giuliano
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
RE: [lazarus] Making GTK Thread Safe
If you want to fake a older kernel, try: LD_ASSUME_KERNEL=2.2.5 In the environmen before ou run. I know it affects the threading model among other things. Sam -Original Message- From: Giuliano Colla [EMAIL PROTECTED] Sent: 07 January 2008 00:00 To: lazarus@miraclec.com Subject: Re: [lazarus] Making GTK Thread Safe willem ha scritto: Giuliano Colla wrote: willem ha scritto: Please send your modified threadbug program , so that i can compare the results on my system. Please find it here attached. Giuliano Well I have the same result. the virtual memory has increased. But I am using kernel 2.6.22 but the latest kernel is 2.6.23.12 I have libc 2.6.1 but the latest version is 2.7 I am using an older kernel (2.6.20 ) and an older glibc (2.4.11). AFAIK linux thread handling shouldn't have changed in this span. Al doesn't see the same problem with fpc 2.0.2, but from previous posting I seem to remember he's also using an older kernel. What I'd like to do is to make the same test with a C program, just to have a reference, but it'll take some time, because I'm not so good with C ;-) I'll let you know the results as soon as I have them. Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote:
Al Boldi ha scritto:
For now, when you start a thread non-suspended, TThread.Execute(false),
can you see the memory increase for your project1 app when checking with
top? Then, when you TThread.WaitFor/TThread.Free, does it decrease? And
by how much?
Below is the simplified console app.
Thanks for the feedback!
Feedback:
I couldn't see any change with top, so I slightly modified your console
app, by adding:
writeln('Type Enter to start thread');
readln(ans);
before
t1:=TThreadBug.Create(false);
Ok, much better!
With Lazarus 0.9.4.1, fpc 2.2.0 the result is the following:
before starting the thread TOP says:
4732 colla 16 0 2564 808 488 S0 0.1 0:00.00 threadbug
after the thread is started it becomes:
4732 colla 15 0 12860 916 576 S0 0.1 0:00.00 threadbug
That's a huge increase; almost 12mb. On fpc2.0.2/linux2.6.22 it increases by
~2mb, a lot less, but still large.
Can somebody check what the gcc thread overhead is?
and it doesn't change anymore until program termination, either typing y
or n.
If this is with v2.2.0, then this means a regression on top of a bug.
I tried to set true FreeOnTerminate of your thread (with proper
adjustments, because WaitFor and Free don't work anymore, of course) but
there's no change.
For sake of completeness, I made the same with my test program, where I
can terminate, free and restart the thread multiple times, and I see the
memory usage increasing each time: it started with 12Kb
You mean 12mb, right?
and I stopped
when it had become 122Mb! However all memory is released before closing
the program, because heaptrc correctly reports an increasing number of
blocks allocated and freed, as a function of the number of times I
started my thread.
I don't think heaptrc detects RTL associated OS leaks. It only detects RTL
object leaks, and it seems those are freed correctly.
As soon as I can, I'd like to perform the same tests with Kylix, to see
if it makes a difference.
Thanks a lot!
--
Al
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote:
Al Boldi ha scritto:
For now, when you start a thread non-suspended,
TThread.Execute(false), can you see the memory increase for your
project1 app when checking with top? Then, when you
TThread.WaitFor/TThread.Free, does it decrease? And by how much?
Below is the simplified console app.
Thanks for the feedback!
Feedback:
I couldn't see any change with top, so I slightly modified your
console app, by adding:
writeln('Type Enter to start thread');
readln(ans);
before
t1:=TThreadBug.Create(false);
With Lazarus 0.9.4.1, fpc 2.2.0 the result is the following:
before starting the thread TOP says:
4732 colla 16 0 2564 808 488 S0 0.1 0:00.00 threadbug
after the thread is started it becomes:
4732 colla 15 0 12860 916 576 S0 0.1 0:00.00 threadbug
and it doesn't change anymore until program termination, either typing
y or n.
I tried to set true FreeOnTerminate of your thread (with proper
adjustments, because WaitFor and Free don't work anymore, of course)
but there's no change.
For sake of completeness, I made the same with my test program, where
I can terminate, free and restart the thread multiple times, and I see
the memory usage increasing each time: it started with 12Kb and I
stopped when it had become 122Mb! However all memory is released
before closing the program, because heaptrc correctly reports an
increasing number of blocks allocated and freed, as a function of the
number of times I started my thread.
As soon as I can, I'd like to perform the same tests with Kylix, to
see if it makes a difference.
Regards,
Giuliano
I do not think this a fpc problem.
In the threat ring benchmark Free Pascal performs almost as well as C.
I did run this benchmark without problems.
Regards Wim
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
willem wrote:
Giuliano Colla wrote:
Al Boldi ha scritto:
For now, when you start a thread non-suspended,
TThread.Execute(false), can you see the memory increase for your
project1 app when checking with top? Then, when you
TThread.WaitFor/TThread.Free, does it decrease? And by how much?
Below is the simplified console app.
Thanks for the feedback!
Feedback:
I couldn't see any change with top, so I slightly modified your
console app, by adding:
writeln('Type Enter to start thread');
readln(ans);
before
t1:=TThreadBug.Create(false);
With Lazarus 0.9.4.1, fpc 2.2.0 the result is the following:
before starting the thread TOP says:
4732 colla 16 0 2564 808 488 S0 0.1 0:00.00 threadbug
after the thread is started it becomes:
4732 colla 15 0 12860 916 576 S0 0.1 0:00.00 threadbug
and it doesn't change anymore until program termination, either
typing y or n.
I tried to set true FreeOnTerminate of your thread (with proper
adjustments, because WaitFor and Free don't work anymore, of course)
but there's no change.
For sake of completeness, I made the same with my test program, where
I can terminate, free and restart the thread multiple times, and I
see the memory usage increasing each time: it started with 12Kb and I
stopped when it had become 122Mb! However all memory is released
before closing the program, because heaptrc correctly reports an
increasing number of blocks allocated and freed, as a function of the
number of times I started my thread.
As soon as I can, I'd like to perform the same tests with Kylix, to
see if it makes a difference.
Regards,
Giuliano
I do not think this a fpc problem.
In the threat ring benchmark Free Pascal performs almost as well as C.
I did run this benchmark without problems.
Regards Wim
I am using kubuntu gutsy.
lazarus 0.9.24 -386-linux-gtk
fpc 2.2.0
and I run the threatbug with no memory leaks.
i did remove the readln 's and added a for loop.
threadbug ran then 10 000 times without memory leaks on my system.
regards Wim
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
willem ha scritto:
willem wrote:
Giuliano Colla wrote:
Al Boldi ha scritto:
For now, when you start a thread non-suspended,
TThread.Execute(false), can you see the memory increase for your
project1 app when checking with top? Then, when you
TThread.WaitFor/TThread.Free, does it decrease? And by how much?
Below is the simplified console app.
Thanks for the feedback!
Feedback:
I couldn't see any change with top, so I slightly modified your
console app, by adding:
writeln('Type Enter to start thread');
readln(ans);
before
t1:=TThreadBug.Create(false);
With Lazarus 0.9.4.1, fpc 2.2.0 the result is the following:
before starting the thread TOP says:
4732 colla 16 0 2564 808 488 S0 0.1 0:00.00 threadbug
after the thread is started it becomes:
4732 colla 15 0 12860 916 576 S0 0.1 0:00.00 threadbug
and it doesn't change anymore until program termination, either
typing y or n.
I tried to set true FreeOnTerminate of your thread (with proper
adjustments, because WaitFor and Free don't work anymore, of course)
but there's no change.
For sake of completeness, I made the same with my test program,
where I can terminate, free and restart the thread multiple times,
and I see the memory usage increasing each time: it started with
12Kb and I stopped when it had become 122Mb! However all memory is
released before closing the program, because heaptrc correctly
reports an increasing number of blocks allocated and freed, as a
function of the number of times I started my thread.
As soon as I can, I'd like to perform the same tests with Kylix, to
see if it makes a difference.
Regards,
Giuliano
I do not think this a fpc problem.
In the threat ring benchmark Free Pascal performs almost as well as C.
I did run this benchmark without problems.
Regards Wim
I am using kubuntu gutsy.
lazarus 0.9.24 -386-linux-gtk
fpc 2.2.0
and I run the threatbug with no memory leaks.
i did remove the readln 's and added a for loop.
threadbug ran then 10 000 times without memory leaks on my system.
I've realized that information from top is not detailed enough, as it
shows only cumulative memory usage. So I've made a more careful analysis
using pmap, which provides the memory map of a process. (Usage: pmap
[-x] | [-d] PID )
It turns out is that libpthread allocates a 10MB memory chunk (10240Kb
to be exact) when a thread is created, and doesn't release it when the
thread is destroyed. But it reuses the same chunk when the thread is
created again.
So the leakage occurs just once.
This as far as fpc 2.2.0 is concerned, in a simple context as that of
threadbug.
With Lazarus, in a slightly more complex test, using the synchronize
method you may find as many 10240 kb blocks allocated as the times you
started and freed the thread, unless you add an extra WaitFor after
teminating the thread, which shouldn't be necessary with FreeOnTerminate.
IMHO it's a combination of an fpc bug (there's no reason to keep a 10MB
memory block when there's no thread using it) with some obscure Lazarus
bug which doesn't allow the thread to fully terminate.
Attached is a portion the output of pmap (before starting the thread,
after running, and after destroy) for a test where I'd added a large
array to the threadbug code: this accounts for an extra 62528 Kb which
are correctly allocated and set free.
Regards,
Giuliano
Before Starting the thread:
00a94000 64 - - - r-x-- libpthread-2.4.so
00aa4000 4 - - - r-x-- libpthread-2.4.so
00aa5000 4 - - - rwx-- libpthread-2.4.so
00aa6000 8 - - - rwx--[ anon ]
00b7f000 256 - - - rwx--[ anon ]
00d7b000 4 - - - r-x--[ anon ]
08048000 308 - - - r-x-- threadbug
08095000 116 - - - rwx-- threadbug
080b2000 12 - - - rwx--[ anon ]
09e33000 132 - - - rwx--[ anon ]
bfc8e000 88 - - - rwx--[ stack ]
--- --- --- ---
total kB2548 - - -
Thread has run
00a94000 64 - - - r-x-- libpthread-2.4.so
00aa4000 4 - - - r-x-- libpthread-2.4.so
00aa5000 4 - - - rwx-- libpthread-2.4.so
00aa6000 8 - - - rwx--[ anon ]
00b7f000 256 - - - rwx--[ anon ]
00be9000 4 - - - rwx--[ anon ]
00d7b000 4 - - - r-x--[ anon ]
00d7c000 62528 - - - rwx--[ anon ]
04a8c000 4 - - - -[ anon ]
04a8d000 10240 - - - rwx--[ anon ]
08048000 308 - - - r-x-- threadbug
08095000 116 - - - rwx-- threadbug
080b2000 12 - - - rwx--[ anon
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote:
willem ha scritto:
willem wrote:
Giuliano Colla wrote:
Al Boldi ha scritto:
For now, when you start a thread non-suspended,
TThread.Execute(false), can you see the memory increase for your
project1 app when checking with top? Then, when you
TThread.WaitFor/TThread.Free, does it decrease? And by how much?
Below is the simplified console app.
Thanks for the feedback!
Feedback:
I couldn't see any change with top, so I slightly modified your
console app, by adding:
writeln('Type Enter to start thread');
readln(ans);
before
t1:=TThreadBug.Create(false);
With Lazarus 0.9.4.1, fpc 2.2.0 the result is the following:
before starting the thread TOP says:
4732 colla 16 0 2564 808 488 S0 0.1 0:00.00 threadbug
after the thread is started it becomes:
4732 colla 15 0 12860 916 576 S0 0.1 0:00.00 threadbug
and it doesn't change anymore until program termination, either
typing y or n.
I tried to set true FreeOnTerminate of your thread (with proper
adjustments, because WaitFor and Free don't work anymore, of
course) but there's no change.
For sake of completeness, I made the same with my test program,
where I can terminate, free and restart the thread multiple times,
and I see the memory usage increasing each time: it started with
12Kb and I stopped when it had become 122Mb! However all memory is
released before closing the program, because heaptrc correctly
reports an increasing number of blocks allocated and freed, as a
function of the number of times I started my thread.
As soon as I can, I'd like to perform the same tests with Kylix, to
see if it makes a difference.
Regards,
Giuliano
I do not think this a fpc problem.
In the threat ring benchmark Free Pascal performs almost as well as C.
I did run this benchmark without problems.
Regards Wim
I am using kubuntu gutsy.
lazarus 0.9.24 -386-linux-gtk
fpc 2.2.0
and I run the threatbug with no memory leaks.
i did remove the readln 's and added a for loop.
threadbug ran then 10 000 times without memory leaks on my system.
I've realized that information from top is not detailed enough, as it
shows only cumulative memory usage. So I've made a more careful
analysis using pmap, which provides the memory map of a process.
(Usage: pmap [-x] | [-d] PID )
It turns out is that libpthread allocates a 10MB memory chunk (10240Kb
to be exact) when a thread is created, and doesn't release it when the
thread is destroyed. But it reuses the same chunk when the thread is
created again.
So the leakage occurs just once.
This as far as fpc 2.2.0 is concerned, in a simple context as that of
threadbug.
With Lazarus, in a slightly more complex test, using the synchronize
method you may find as many 10240 kb blocks allocated as the times you
started and freed the thread, unless you add an extra WaitFor after
teminating the thread, which shouldn't be necessary with FreeOnTerminate.
IMHO it's a combination of an fpc bug (there's no reason to keep a
10MB memory block when there's no thread using it) with some obscure
Lazarus bug which doesn't allow the thread to fully terminate.
Attached is a portion the output of pmap (before starting the thread,
after running, and after destroy) for a test where I'd added a large
array to the threadbug code: this accounts for an extra 62528 Kb which
are correctly allocated and set free.
Regards,
Giuliano
well I made 10 copy's of threadbug and run them in parallel. In my case
libpthread does not allocate a 10 Mb memory block, because
the 10 processes should allocate 100Mb of memory Top does show only a
sligth increase in memory . On my kubuntu there are
145 processes allocated . So i can not reproduce your problem.
Regards Wim.
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
willem wrote: well I made 10 copy's of threadbug and run them in parallel. In my case libpthread does not allocate a 10 Mb memory block, because the 10 processes should allocate 100Mb of memory Top does show only a sligth increase in memory . On my kubuntu there are 145 processes allocated . So i can not reproduce your problem. I just tried on another distribution, and without WaitFor the leak is still there, but a lot smaller, only ~12kb per thread. Did you run with or without WaitFor? How big is the thread overhead? Thanks! -- Al _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote:
Al Boldi ha scritto:
I played with this some more, and it turns out that not only is
FreeOnTerminate completely broken but Free only frees the OS-thread
memory if the thread is still running or is preceded by a WaitFor. This
is on fpc2.0.2/linux. Maybe this has been fixed in 2.2.0?
Below is the test-code.
Can somebody confirm this on other platforms/fpcVersions.
I've made some quick tests, with what I have ready at hand and with my
test app. Not so different from yours, except that it allows me more
options. Platform Linux, fpc 2.0.4 and 2.2.0, Lazarus.fixes and Lazarus
trunk latest svn. Widgetset gtk1.
With fpc 2.0.4, FreeOnTerminate leaks memory whatever I've tried.
Ok, same here.
With fpc 2.2.0, using Synchronize, I've been unable to get rid of memory
leaks if the thread is terminated within an OnClose event.
If the thread is terminated before closing the form, it behaves properly.
With fpc 2.2.0, if I don't use Synchronize, but rather a message queue
processed by the OnIdle handler, then I don't experience any memory
leak, whatever is the thread termination.
My conclusion: fpc 2.0.4 has almost certainly some bug on
FreeOnTerminate. In many cases it segfaults, meaning, I guess, that
there's some NilAndFree in place of FreeAndNil :-)
With fpc 2.2.0 it's harder to tell at first glance, but one would say
that the problem is on Lazarus side, because the fpc rtl Syncronize
provide just a thread safe frame, and leaves all the work to the main
thread loop.
Synchronize is heavily widgetset dependent. That's why I only report
gtk1 results. With gtk2 I've met inconsistent results from one revision
to the next, with qt synchronize for me doesn't work at all.
Ok, synchronize is another problem, which I would like to deal with
separately.
For now, when you start a thread non-suspended, TThread.Execute(false), can
you see the memory increase for your project1 app when checking with top?
Then, when you TThread.WaitFor/TThread.Free, does it decrease? And by how
much?
Below is the simplified console app.
Thanks for the feedback!
--
Al
---
program threadbug;
{$mode objfpc}{$H+}
uses
{$IFDEF UNIX}
cthreads,
{$ENDIF}
Classes
{ add your units here };
type
{ TThreadBug }
TThreadBug=class(TThread)
procedure Execute; override;
end;
{ TThreadBug }
procedure TThreadBug.Execute;
begin
writeln('just ran a thread... done!');
end;
var t1:TThreadBug; ans:string;
begin
t1:=TThreadBug.Create(false);
writeln('WaitFor? [y/N]');
readln(ans);
if ans='y' then begin
t1.WaitFor;
writeln('OS thread memory released.');
end else
writeln('OS thread memory left dangling.');
t1.Free;
writeln('Confirm with top now!');
readln;
end.
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Al Boldi ha scritto:
For now, when you start a thread non-suspended, TThread.Execute(false), can
you see the memory increase for your project1 app when checking with top?
Then, when you TThread.WaitFor/TThread.Free, does it decrease? And by how
much?
Below is the simplified console app.
Thanks for the feedback!
Feedback:
I couldn't see any change with top, so I slightly modified your console
app, by adding:
writeln('Type Enter to start thread');
readln(ans);
before
t1:=TThreadBug.Create(false);
With Lazarus 0.9.4.1, fpc 2.2.0 the result is the following:
before starting the thread TOP says:
4732 colla 16 0 2564 808 488 S0 0.1 0:00.00 threadbug
after the thread is started it becomes:
4732 colla 15 0 12860 916 576 S0 0.1 0:00.00 threadbug
and it doesn't change anymore until program termination, either typing y
or n.
I tried to set true FreeOnTerminate of your thread (with proper
adjustments, because WaitFor and Free don't work anymore, of course) but
there's no change.
For sake of completeness, I made the same with my test program, where I
can terminate, free and restart the thread multiple times, and I see the
memory usage increasing each time: it started with 12Kb and I stopped
when it had become 122Mb! However all memory is released before closing
the program, because heaptrc correctly reports an increasing number of
blocks allocated and freed, as a function of the number of times I
started my thread.
As soon as I can, I'd like to perform the same tests with Kylix, to see
if it makes a difference.
Regards,
Giuliano
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Al Boldi ha scritto: Al Boldi wrote: Micha Nelissen wrote: Al Boldi wrote: I think the race comes in when you have two competing threads wanting to synchronize the the main thread. It just hangs. Are you sure? This should work properly. Can you provide example (in a bugreport)? Well, it's not easily repeatable, and seems to be very OS dependent. On linux, I work around this problem by inserting a sleep(0) just before the synchronize. But there is another problem, it seems FreeOnTerminate doesn't actually free the memory on exit, unless the main thread actually waits for it to terminate, and this is on linux again. Can you confirm this? I played with this some more, and it turns out that not only is FreeOnTerminate completely broken but Free only frees the OS-thread memory if the thread is still running or is preceded by a WaitFor. This is on fpc2.0.2/linux. Maybe this has been fixed in 2.2.0? Below is the test-code. Can somebody confirm this on other platforms/fpcVersions. I've made some quick tests, with what I have ready at hand and with my test app. Not so different from yours, except that it allows me more options. Platform Linux, fpc 2.0.4 and 2.2.0, Lazarus.fixes and Lazarus trunk latest svn. Widgetset gtk1. With fpc 2.0.4, FreeOnTerminate leaks memory whatever I've tried. With fpc 2.2.0, using Synchronize, I've been unable to get rid of memory leaks if the thread is terminated within an OnClose event. If the thread is terminated before closing the form, it behaves properly. With fpc 2.2.0, if I don't use Synchronize, but rather a message queue processed by the OnIdle handler, then I don't experience any memory leak, whatever is the thread termination. My conclusion: fpc 2.0.4 has almost certainly some bug on FreeOnTerminate. In many cases it segfaults, meaning, I guess, that there's some NilAndFree in place of FreeAndNil :-) With fpc 2.2.0 it's harder to tell at first glance, but one would say that the problem is on Lazarus side, because the fpc rtl Syncronize provide just a thread safe frame, and leaves all the work to the main thread loop. Synchronize is heavily widgetset dependent. That's why I only report gtk1 results. With gtk2 I've met inconsistent results from one revision to the next, with qt synchronize for me doesn't work at all. Giuliano -- Giuliano Colla Whenever people agree with me, I always feel I must be wrong (O. Wilde) _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla schrieb: Isn't that where the dead lock occurs? Suppose Thr1 has just called Synchronize and is waiting until the main thread has executed the synchronized method, it won't terminate, because you have suppended the main thread by waiting for Thr1 to terminate. I'm afraid you're right. A really bullet proof way to terminate an application when even a single thread using synchronize is running is far from straightforward, and hard to generalize. For this reason, and many others, I consider the synchronize method just a hack. Terminating threads is always not easy and hard to generalize. _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
RE: [lazarus] Making GTK Thread Safe
Gmane came to mind, but it has the same problem, you can't use your nntp newsreader if port 119 is blocked. Sam -Original Message- From: Micha Nelissen [EMAIL PROTECTED] Sent: 31 December 2007 21:34 To: lazarus@miraclec.com Subject: Re: [lazarus] Making GTK Thread Safe Sam Liddicott wrote: I can smell an NNTP over http proxy. For syncronous commands and with HTTP keep-alive it should be nearly as eficient as regular NNTP Simply use gmane? Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Sam Liddicott wrote: Gmane came to mind, but it has the same problem, you can't use your nntp newsreader if port 119 is blocked. Gmane has a web interface also, e.g.: http://news.gmane.org/gmane.comp.ide.lazarus.general Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
RE: [lazarus] Making GTK Thread Safe
Yes But you can't use your newsreader, you have to use a web browser. Sam -Original Message- From: Micha Nelissen [EMAIL PROTECTED] Sent: 01 January 2008 11:27 To: lazarus@miraclec.com Subject: Re: [lazarus] Making GTK Thread Safe Sam Liddicott wrote: Gmane came to mind, but it has the same problem, you can't use your nntp newsreader if port 119 is blocked. Gmane has a web interface also, e.g.: http://news.gmane.org/gmane.comp.ide.lazarus.general Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Sam Liddicott wrote: Yes But you can't use your newsreader, you have to use a web browser. So? You're also not allowed to use a web browser? Btw, your email client is broken: it doesn't have the In-Reply-To and References email headers. Please consider using another one. Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Micha Nelissen wrote: Sam Liddicott wrote: Yes But you can't use your newsreader, you have to use a web browser. So? You're also not allowed to use a web browser? I think the thread got lost... so to refresh where I came in: On Thu, Dec 13, 2007 at 04:35:24PM +0300, Al Boldi wrote: Graeme Geldenhuys wrote: On 13/12/2007, Martin Schreiber [EMAIL PROTECTED] wrote: Is there web gateway for this? No, use a news client, Mozilla Thunderbird for win32/Linux or KNode for Linux for example. You will probably like the conversation by NNTP, much more convenient than mailing lists or web forums IMHO. I can't agree more. NNTP is a lot more convenient and preferred by me, that mailing list. The problem is that sometimes NNTP ports are blocked. And I also prefer NNTP access. So yes there may well be NNTP gmane access but I still smell a NNTP-HTTP gateway. Btw, your email client is broken: it doesn't have the In-Reply-To and References email headers. Please consider using another one. Darn top-quoting head-lacking pocket-outlook is a load of kack. Progress has been made on Linux for the HTC Universal, but it's not ready yet and Neo1973 still waiting... Sam _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Florian Klaempfl ha scritto: Giuliano Colla schrieb: Isn't that where the dead lock occurs? Suppose Thr1 has just called Synchronize and is waiting until the main thread has executed the synchronized method, it won't terminate, because you have suppended the main thread by waiting for Thr1 to terminate. I'm afraid you're right. A really bullet proof way to terminate an application when even a single thread using synchronize is running is far from straightforward, and hard to generalize. For this reason, and many others, I consider the synchronize method just a hack. Terminating threads is always not easy and hard to generalize. You're right, but the available tools can make it easier or harder. I come from a long experience in real-time applications, with many concurrent interrelated tasks, and if the design isn't sound enough you end up with deadlocks at each corner. What I don't like is a mechanism which may create deadlocks you're unaware of, because they're dependent on GUI implementation. With a simple test application I've found different behaviors on Kylix 1, Kylix 3, different versions of Lazarus, and even in the same Lazarus version with different widgetsets. I've chosen a different approach. Threads which must use GUI append messages to a queue, which is processed by the OnIdle event handler. That way all GUI related code resides in the main thread, and threads relationships are much simpler to handle. Of course it works also the other way around. The main thread can append messages to queues which are processed by threads. Implementing a thread-safe queue is not so difficult: even a simple ring buffer can do. BTW are the stack and queue components in the contnrs unit intended to be thread-safe? The only thing which currently I miss is a clean way to wake up the main thread. On Kylix I've found that a Timer event in itself isn't sufficient: if the event doesn't perform some operation which modify some widget, then the Application doesn't come out of the idle state, and therefore the OnIdle event handler isn't activated. It appears that the IdleTimer of Lazarus behaves much better, but I've not yet gathered enough data to be sure. Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote: The only thing which currently I miss is a clean way to wake up the main thread. http://www.freepascal.org/docs-html/rtl/classes/wakemainthread.html Should work in a LCL app. Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Micha Nelissen ha scritto: Giuliano Colla wrote: The only thing which currently I miss is a clean way to wake up the main thread. http://www.freepascal.org/docs-html/rtl/classes/wakemainthread.html Should work in a LCL app. Thanks a lot. I had missed that. Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Al Boldi wrote:
Micha Nelissen wrote:
Al Boldi wrote:
I think the race comes in when you have two competing threads wanting
to synchronize the the main thread. It just hangs.
Are you sure? This should work properly. Can you provide example (in a
bugreport)?
Well, it's not easily repeatable, and seems to be very OS dependent. On
linux, I work around this problem by inserting a sleep(0) just before the
synchronize.
But there is another problem, it seems FreeOnTerminate doesn't actually
free the memory on exit, unless the main thread actually waits for it to
terminate, and this is on linux again. Can you confirm this?
I played with this some more, and it turns out that not only is
FreeOnTerminate completely broken but Free only frees the OS-thread memory
if the thread is still running or is preceded by a WaitFor. This is on
fpc2.0.2/linux. Maybe this has been fixed in 2.2.0?
Below is the test-code.
Can somebody confirm this on other platforms/fpcVersions.
Thanks!
--
Al
---
type
{ TThreadBug }
TThreadBug = class(TThread)
public
procedure Execute; override;
end;
{ TForm1 }
TForm1 = class(TForm)
Button1: TButton;
Button2: TButton;
Button3: TButton;
procedure Button1Click(Sender: TObject);
procedure Button2Click(Sender: TObject);
procedure Button3Click(Sender: TObject);
private
{ private declarations }
public
{ public declarations }
t1:TThreadBug;
end;
var
Form1: TForm1;
implementation
{ TThreadBug }
procedure TThreadBug.Execute;
begin
writeln('just ran a thread... done!');
end;
{ TForm1 }
procedure TForm1.Button1Click(Sender: TObject);
begin
t1:=TThreadBug.Create(false);
end;
procedure TForm1.Button2Click(Sender: TObject);
begin
t1.Free; // Only frees RTL object memory
// OS-thread memory left dangling
end;
procedure TForm1.Button3Click(Sender: TObject);
begin
t1.WaitFor; // Correctly frees OS-thread memory
t1.Free; // Frees RTL object memory
end;
_
To unsubscribe: mail [EMAIL PROTECTED] with
unsubscribe as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
RE: [lazarus] Making GTK Thread Safe
I can smell an NNTP over http proxy. For syncronous commands and with HTTP keep-alive it should be nearly as eficient as regular NNTP Sam -Original Message- From: Marco van de Voort [EMAIL PROTECTED] Sent: 30 December 2007 13:15 To: lazarus@miraclec.com Subject: Re: [lazarus] Making GTK Thread Safe On Thu, Dec 13, 2007 at 04:35:24PM +0300, Al Boldi wrote: Graeme Geldenhuys wrote: On 13/12/2007, Martin Schreiber [EMAIL PROTECTED] wrote: Is there web gateway for this? No, use a news client, Mozilla Thunderbird for win32/Linux or KNode for Linux for example. You will probably like the conversation by NNTP, much more convenient than mailing lists or web forums IMHO. I can't agree more. NNTP is a lot more convenient and preferred by me, that mailing list. The problem is that sometimes NNTP ports are blocked. A possible solution to that is spending a few bucks an year for a shell acocunt with NNTP access. (only requirement is that it is pwd based, not key based) Then use a web SSH client to connect to it if a normal ssh is not available. (the web ssh is the reason for the keyless account). I do this for years now. Other advantage is that you always have access to your own lastread pointers, and have to memorize/carry very little. _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Sam Liddicott wrote: I can smell an NNTP over http proxy. For syncronous commands and with HTTP keep-alive it should be nearly as eficient as regular NNTP Simply use gmane? Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On Sunday 30 December 2007 07.39:25 Micha Nelissen wrote: Martin Schreiber wrote: You are right, there seems to be no race condition if one uses WakeMainThread correct. There can be a deadlock if the main thread tries to destroy a thread which is waiting for Synchronize. Hmm I think this is a very stupid idea to do. The thread will continue to execute something so I don't think this is supported. Like destroying a component in an event handler. TThread.Destroy calls Waitfor. What is the supported method to terminate an application which has threads waiting for Synchronize? Martin _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Martin Schreiber ha scritto: On Sunday 30 December 2007 07.39:25 Micha Nelissen wrote: Martin Schreiber wrote: You are right, there seems to be no race condition if one uses WakeMainThread correct. There can be a deadlock if the main thread tries to destroy a thread which is waiting for Synchronize. Hmm I think this is a very stupid idea to do. The thread will continue to execute something so I don't think this is supported. Like destroying a component in an event handler. TThread.Destroy calls Waitfor. What is the supported method to terminate an application which has threads waiting for Synchronize? I don't know it it's officially supported, but for me it works. I've added a Running property to my thread, which is set true when the thread begins execution, and false jut before terminating. In my main form, the OnClose event includes: if assigned(Thr1) then begin if Thr1.Running then begin Thr1.Terminate; Thr1.WaitFor; end; end; By setting true FreeOnTerminate, the thread destroys cleanly itself. Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla schreef: Martin Schreiber ha scritto: On Sunday 30 December 2007 07.39:25 Micha Nelissen wrote: Martin Schreiber wrote: You are right, there seems to be no race condition if one uses WakeMainThread correct. There can be a deadlock if the main thread tries to destroy a thread which is waiting for Synchronize. Hmm I think this is a very stupid idea to do. The thread will continue to execute something so I don't think this is supported. Like destroying a component in an event handler. TThread.Destroy calls Waitfor. What is the supported method to terminate an application which has threads waiting for Synchronize? I don't know it it's officially supported, but for me it works. I've added a Running property to my thread, which is set true when the thread begins execution, and false jut before terminating. In my main form, the OnClose event includes: if assigned(Thr1) then begin if Thr1.Running then begin Thr1.Terminate; Thr1.WaitFor; end; end; Isn't that where the dead lock occurs? Suppose Thr1 has just called Synchronize and is waiting until the main thread has executed the synchronized method, it won't terminate, because you have suppended the main thread by waiting for Thr1 to terminate. Vincent _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On Thu, Dec 13, 2007 at 04:35:24PM +0300, Al Boldi wrote: Graeme Geldenhuys wrote: On 13/12/2007, Martin Schreiber [EMAIL PROTECTED] wrote: Is there web gateway for this? No, use a news client, Mozilla Thunderbird for win32/Linux or KNode for Linux for example. You will probably like the conversation by NNTP, much more convenient than mailing lists or web forums IMHO. I can't agree more. NNTP is a lot more convenient and preferred by me, that mailing list. The problem is that sometimes NNTP ports are blocked. A possible solution to that is spending a few bucks an year for a shell acocunt with NNTP access. (only requirement is that it is pwd based, not key based) Then use a web SSH client to connect to it if a normal ssh is not available. (the web ssh is the reason for the keyless account). I do this for years now. Other advantage is that you always have access to your own lastread pointers, and have to memorize/carry very little. _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Vincent Snijders wrote: Giuliano Colla schreef: Martin Schreiber ha scritto: On Sunday 30 December 2007 07.39:25 Micha Nelissen wrote: Martin Schreiber wrote: You are right, there seems to be no race condition if one uses WakeMainThread correct. There can be a deadlock if the main thread tries to destroy a thread which is waiting for Synchronize. Hmm I think this is a very stupid idea to do. The thread will continue to execute something so I don't think this is supported. Like destroying a component in an event handler. TThread.Destroy calls Waitfor. What is the supported method to terminate an application which has threads waiting for Synchronize? I don't know it it's officially supported, but for me it works. I've added a Running property to my thread, which is set true when the thread begins execution, and false jut before terminating. In my main form, the OnClose event includes: if assigned(Thr1) then begin if Thr1.Running then begin Thr1.Terminate; Thr1.WaitFor; end; end; Isn't that where the dead lock occurs? Suppose Thr1 has just called Synchronize and is waiting until the main thread has executed the synchronized method, it won't terminate, because you have suppended the main thread by waiting for Thr1 to terminate. IIRC WaitFor also handles the main loop (at leasst on delphi) Marc _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Micha Nelissen wrote: Al Boldi wrote: I think the race comes in when you have two competing threads wanting to synchronize the the main thread. It just hangs. Are you sure? This should work properly. Can you provide example (in a bugreport)? Well, it's not easily repeatable, and seems to be very OS dependent. On linux, I work around this problem by inserting a sleep(0) just before the synchronize. But there is another problem, it seems FreeOnTerminate doesn't actually free the memory on exit, unless the main thread actually waits for it to terminate, and this is on linux again. Can you confirm this? Thanks! -- Al _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Vincent Snijders ha scritto: Giuliano Colla schreef: Martin Schreiber ha scritto: On Sunday 30 December 2007 07.39:25 Micha Nelissen wrote: Martin Schreiber wrote: You are right, there seems to be no race condition if one uses WakeMainThread correct. There can be a deadlock if the main thread tries to destroy a thread which is waiting for Synchronize. Hmm I think this is a very stupid idea to do. The thread will continue to execute something so I don't think this is supported. Like destroying a component in an event handler. TThread.Destroy calls Waitfor. What is the supported method to terminate an application which has threads waiting for Synchronize? I don't know it it's officially supported, but for me it works. I've added a Running property to my thread, which is set true when the thread begins execution, and false jut before terminating. In my main form, the OnClose event includes: if assigned(Thr1) then begin if Thr1.Running then begin Thr1.Terminate; Thr1.WaitFor; end; end; Isn't that where the dead lock occurs? Suppose Thr1 has just called Synchronize and is waiting until the main thread has executed the synchronized method, it won't terminate, because you have suppended the main thread by waiting for Thr1 to terminate. I'm afraid you're right. A really bullet proof way to terminate an application when even a single thread using synchronize is running is far from straightforward, and hard to generalize. For this reason, and many others, I consider the synchronize method just a hack. Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Al Boldi wrote: I think the race comes in when you have two competing threads wanting to synchronize the the main thread. It just hangs. Are you sure? This should work properly. Can you provide example (in a bugreport)? Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Martin Schreiber wrote: You are right, there seems to be no race condition if one uses WakeMainThread correct. There can be a deadlock if the main thread tries to destroy a thread which is waiting for Synchronize. Hmm I think this is a very stupid idea to do. The thread will continue to execute something so I don't think this is supported. Like destroying a component in an event handler. Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Michael Van Canneyt wrote: On Wed, 12 Dec 2007, Razvan Adrian Bogdan wrote: The synchronize method is simple but inefficient, it pauses your thread until the main thread has time to execute your code and could take some time until it does, i personally also dislike the fact that one cannot call any procedure/function with any number of parameters with synchronize. That's not possible in pascal without hacking the stack. Why? I think mark proposed once the solution of using a List (StringList or something else) in a synchronised manner with CriticalSections protecting access to the list and this way creating a message system similar to the OS one but this implementation is also not always comfortable but it isn't blocking as synchronize, the problem is not with the windows widgetset which is message based and quite thread safe and one can easily use CriticalSections with Controls, Borland started to make the VCL threadsafe but they didn't complete this task, maybe LCL would be more popular if it was threadsafe, i do realise it's a lot of work but it might be worth it. This may work for windows, but X is not thread safe, so you would have maybe a slightly less blocking system, but it would be blocking nonetheless. I think the gain would be minimal. This is basically the problem with threading: it's not enough for your code to be thread-safe, you also need to make sure all linked libs are thread-safe, which makes threading only useful for code you control. So, making the LCL thread-safe is really the easy part, but more importantly you need to make sure that all libs the LCL links to are thread-safe. One easy work-around may be to run your app multi-processed instead of multi-threaded, thereby conveniently circumventing any threading issues in the foreign libs. Could the LCL support such an architecture? Thanks! -- Al _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Swen Heinig wrote: Hello Al Boldi, you are able to send emails from future? I would suggest to check the date settings on your computer. Thanks for pointing this out! This is a linux suspend-to-disk/ram problem. The bios clock is correct, it's the linux-system clock that gets sometimes advanced exactly 24h when using s2ram/disk. Thanks again! -- Al _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Graeme Geldenhuys ha scritto: On 13/12/2007, Giuliano Colla [EMAIL PROTECTED] wrote: Delphi relied on Windows buggy behavior of sending frequent useless messages around to wake up the main thread frequently. That does sound weird. It is, but it's Microsoft's approach. Just compare what happens on a network connection using Unix's nfs with one using Microsoft's netbios or smb or whatever they call it now. On nfs if there are no client requests, there's no activity. You may disconnect your ethernet cable, and connect it again when you need to access a remote server, and it will just work fine. On smb, there's a a lot of activity (all packets set to high priority, just to annoy the network) asking silly things to all connected clients and servers, which generate a lot of network errors, invisible to users, but reported in the log, if you have a Linux box. Idle, so that the OnIdle handler is never called. My kludge has been to have the Timer update a Label in a hidden corner of the screen, with a So if the corner of the screen is hidden, how does that look? What's there, a black hole. ;-) I'm just teasing! Pretty much yes. A small black hole. :-) Actually it's visible, otherwise the kludge wouldn't work, But it's a black character on a dark background so it goes unnoticed! must be drawn and what not, etc.), and passes that to a graphic thread which is the only one which has access to the display, then you have a real multithread GUI, which could fill a void in what's currently available. It could be a challenging project, don't you think? Challenging to say the least, but sounds interesting! Well, I've thrown the stone in the pond. I'll try to investigate further on what could be done. I don't like to start too big projects without knowing where one will end, but if it turns out that it's possible to set up a feasibility test with minimal functionalities then one could give a thought on doing real work. Giuliano -- Giuliano Colla Whenever people agree with me, I always feel I must be wrong (O. Wilde) _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On 13/12/2007, Giuliano Colla [EMAIL PROTECTED] wrote: Actually it's visible, otherwise the kludge wouldn't work, But it's a black character on a dark background so it goes unnoticed! I hope you have that well documented for a future developer taking over from you. :-) Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Martin Schreiber ha scritto: On Thursday 13 December 2007 01.28, Giuliano Colla wrote: [...] The screen is a shared resource, so it must be granted exclusive access. If one thinks of one main thread which besides doing its job receives messages from all the worker threads, creates a clean queue of what needs to be done (sorting out what's visible and what not, what must be drawn and what not, etc.), and passes that to a graphic thread which is the only one which has access to the display, then you have a real multithread GUI, which could fill a void in what's currently available. It could be a challenging project, don't you think? It just requires a not too large GUI system already existing (such as fpGui, not Lazarus which is too big to play with) and some enthusiasm (and also a lot of time, but for my company it could be important, so some resources could be found). Don't forget MSEide+MSEgui: http://sourceforge.net/projects/mseide-msegui MSEgui has several possibilities to synchronize worker threads with the main GUI thread. First there is application.lock/unlok which works on a mutex. The main eventloop acquires the mutex on start of a new round of the main eventloop and releases it after it gets idle. Secondly there is application.synchronize which runs a callback in a try/finally block with application.lock/unlock. Thirdly there is application.postevent in order to send messages to MSEgui components. The posted events are feed into the main event queue and are delivered to the component in context of the main thread. application.postevent is thread safe and does not block. tmsecomponent.postcomponentevent and asyncevent can be used for convenience, tmseform tmsedatamodule have onevent and onasyncevent where the messages can be handled in context of the main thread. Fourthly there is FPC synchronize with the problem that checksynchronize is called once in onidle of then main event loop. One can call application.wakeupguithread to enable processing but there is a race condition. Worth to mention is teventthread with its own message queue. The graphic subsystem has its own locking mechanism so it is possible to draw onto off screen canvas (pixmap or printer) from worker threads. It appears that a lot of what I was thinking is available in MsEide/MseGui. I will look into it. Thanks a lot. Giuliano -- Giuliano Colla Whenever people agree with me, I always feel I must be wrong (O. Wilde) _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Al Boldi ha scritto: Giuliano Colla wrote: To discover that if your timer event doesn't change something visual, the smart thing decides that there's nothing important to do, and stays Idle, so that the OnIdle handler is never called. My kludge has been to have the Timer update a Label in a hidden corner of the screen, with a sequence of - \ | / - . This wakes up the main thread, which then goes Idle, triggers the OnIdle event, and the queue can be processed. You may see that this is far from satisfactory. Not sure what you mean? OnTimer does trigger OnIdle even if there is no visual update. And even if it wouldn't, you could trivially trigger the OnIdle inside the OnTimer handler yourself. Can't tell in detail what happens, because I was more interested in the result than tracing Kylix bugs/deficiencies, and I forgot the details. I seem to remember that OnIdle is executed just once. Even if the OnIdle event performs some screen updates, no further OnIdle event is generated, thus losing whatever happens from that moment on. Triggering the OnIdle event from Timer in Kylix is not advisable, because it doesn't appear to be properly protected from reentrancy, and you end up with a lock. The final result is that if the timer event doesn't change something visual, the screen isn't updated properly. Up to now I've not yet tested Lazarus with a real demanding threaded app, but from what I've read about the OnIdle events from Felipe and others, I'm afraid that the result won't be much different. I think it is. Try it. I hope you're right, because getting rid of kludges is my hope. I'm anxious to try, but setting up thing in order to test properly is far from trivial, and requires an amount of time not available right now. Simpler cases work well under Lazarus, but they worked well under Kylix too. Giuliano -- Giuliano Colla Whenever people agree with me, I always feel I must be wrong (O. Wilde) _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On Friday 14 December 2007 11.43, Al Boldi wrote: TFrame, visual form inheritance and datamodules with cross form/datamodule design time component linking are implemented for example, Wow, I didn't get that far. I had to disable db-support, because I compiled the MSEide with FPC2.0.2. MSEide+MSEgui 1.6 needs FPC 2.2. BTW, why isn't there a makefile or lpr file to compile the MSEide directly with FPC/laz without the MSEide bin? In MSEide+MSEgui source directory: ppc386 apps/ide/mseide.pas -Fulib/common/* -B -Fulib/common/kernel/i386-linux -Fi/lib/common/kernel (Linux) or ppc386.exe apps\ide\mseide.pas -Fulib\common\* -B -Fulib\common\kernel\i386-win32 -Fi\lib\common\kernel (win32) There are scripts in SVN trunk to do a checkout and compiling MSEide, updatefromsvn on Linux and udatefromsvn.bat for win32. You can get them with viewvc: http://mseide-msegui.svn.sourceforge.net/viewvc/mseide-msegui/trunk/ The script files are at the bottom of the list. Lazarus can not be used to design MSEgui forms. That's sad. MSEgui looks rather professional, but forcing a developer to change the ide is rather drastic. There is a NNTP newsgroup for questions about MSEide+MSEgui: news://news.grid-sky.com/public.mseide-msegui.talk Is there web gateway for this? No, use a news client, Mozilla Thunderbird for win32/Linux or KNode for Linux for example. You will probably like the conversation by NNTP, much more convenient than mailing lists or web forums IMHO. Martin _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Graeme Geldenhuys ha scritto: On 13/12/2007, Giuliano Colla [EMAIL PROTECTED] wrote: Actually it's visible, otherwise the kludge wouldn't work, But it's a black character on a dark background so it goes unnoticed! I hope you have that well documented for a future developer taking over from you. :-) No, I keep this for myself, not to be replaced too easily ;-) Seriously, this sort of things are not only hard to document, but also hard to explain. And you never know if they'll work with next release. Luckily (or sadly) there's no next release for Kylix. Giuliano -- Giuliano Colla Whenever people agree with me, I always feel I must be wrong (O. Wilde) _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On 13/12/2007, Martin Schreiber [EMAIL PROTECTED] wrote: Is there web gateway for this? No, use a news client, Mozilla Thunderbird for win32/Linux or KNode for Linux for example. You will probably like the conversation by NNTP, much more convenient than mailing lists or web forums IMHO. I can't agree more. NNTP is a lot more convenient and preferred by me, that mailing list. Though Gmail does make mailing list conversations much easier to track (and search). Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Graeme Geldenhuys wrote: On 13/12/2007, Martin Schreiber [EMAIL PROTECTED] wrote: Is there web gateway for this? No, use a news client, Mozilla Thunderbird for win32/Linux or KNode for Linux for example. You will probably like the conversation by NNTP, much more convenient than mailing lists or web forums IMHO. I can't agree more. NNTP is a lot more convenient and preferred by me, that mailing list. The problem is that sometimes NNTP ports are blocked. Thanks! -- Al _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On 13/12/2007, Al Boldi [EMAIL PROTECTED] wrote: The problem is that sometimes NNTP ports are blocked. By what? A company firewall? If so, it's very easy to use one of quite a few freely available web interfaces - for the select few that are blocked. For example: I have a WebNews interface to my NNTP server used by tiOPF and fpGUI. WebNews is implemented in Perl and talks to the NNTP server as if it was a normal GUI News Client. WebNews itself is quite fast but my Server, a Pentium 233 with 128MB RAM isn't (who said old computers can't be used for something) so causes a little bit of lag, but still quite acceptable. Here's the link: http://opensoft.homeip.net/webnews Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Martin Schreiber wrote: Fourthly there is FPC synchronize with the problem that checksynchronize is called once in onidle of then main event loop. One can call application.wakeupguithread to enable processing but there is a race condition. AFAICS, these are all variants on the TThread.Synchronize hack. There is no race condition in TThread.Synchronize, so why not use this then? Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Micha Nelissen wrote: Martin Schreiber wrote: Fourthly there is FPC synchronize with the problem that checksynchronize is called once in onidle of then main event loop. One can call application.wakeupguithread to enable processing but there is a race condition. AFAICS, these are all variants on the TThread.Synchronize hack. There is no race condition in TThread.Synchronize, so why not use this then? I think the race comes in when you have two competing threads wanting to synchronize the the main thread. It just hangs. Thanks! -- Al _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On Thursday 13 December 2007 18.16, Micha Nelissen wrote: Martin Schreiber wrote: Fourthly there is FPC synchronize with the problem that checksynchronize is called once in onidle of then main event loop. One can call application.wakeupguithread to enable processing but there is a race condition. AFAICS, these are all variants on the TThread.Synchronize hack. There is no race condition in TThread.Synchronize, so why not use this then? You are right, there seems to be no race condition if one uses WakeMainThread correct. There can be a deadlock if the main thread tries to destroy a thread which is waiting for Synchronize. Martin _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
[lazarus] Making GTK Thread Safe
I've read something interesting in the GTK FAQ, it should be possible to make GTK apps thread safe if they are setup properly, assuming there are many multithreaded apps out there, i think this is really useful, instead of relying on synchronize or simulating Messages Queue with StringList, one could use CriticalSections with Controls and it would be really useful if LCL was as thread safe as possible at least the low level API part such as GTK, of course a full ThreadSafe LCL would be even better but that would probably require too much work. http://www.gtk.org/faq/#AEN482 Razvan _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Razvan Adrian Bogdan wrote: I've read something interesting in the GTK FAQ, it should be possible to make GTK apps thread safe if they are setup properly, assuming there are many multithreaded apps out there, i think this is really useful, instead of relying on synchronize or simulating Messages Queue with StringList, one could use CriticalSections with Controls and it would be really useful if LCL was as thread safe as possible at least the low level API part such as GTK, of course a full ThreadSafe LCL would be even better but that would probably require too much work. The LCL won't be threadsafe, the overhead isn't worth the gain. Marc _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Razvan Adrian Bogdan wrote: On Dec 12, 2007 12:13 PM, Marc Weustink [EMAIL PROTECTED] wrote: The LCL won't be threadsafe, the overhead isn't worth the gain. Do you think it's an overhead to use the internal locking mechanism from GTK ? Not only overhead in gtk widgetset, also in LCL, and aslo in all other widgetsets. I tend to think it's a very light mechanism that you don't even feel in singlethreaded applications, having a threadsafe LCL means one can freely code without any headaches from synchronization making the code look nicer and dramatically cutting developing time, If you start to think your design first it won't cut much. but i do think it's a lot of code. I tend to think the LCLs overhead comes from using classes too much in the lower level API abstraction part. In all my years of delphi threadded app development I never needed threadded updates of my gui. Marc _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Marc Weustink ha scritto: Razvan Adrian Bogdan wrote: On Dec 12, 2007 12:13 PM, Marc Weustink [EMAIL PROTECTED] wrote: The LCL won't be threadsafe, the overhead isn't worth the gain. Do you think it's an overhead to use the internal locking mechanism from GTK ? Not only overhead in gtk widgetset, also in LCL, and aslo in all other widgetsets. I tend to think it's a very light mechanism that you don't even feel in singlethreaded applications, having a threadsafe LCL means one can freely code without any headaches from synchronization making the code look nicer and dramatically cutting developing time, If you start to think your design first it won't cut much. You may think as long as you want, but if the main loop is Idle, the synchronize method will just suspend your thread for a time which can be unacceptable. Therefore you must create a number of kludges to overcome the problem. This takes time, and makes the code unclean and harder to maintain. but i do think it's a lot of code. I tend to think the LCLs overhead comes from using classes too much in the lower level API abstraction part. In all my years of delphi threadded app development I never needed threadded updates of my gui. This doesn't make you really qualified to discuss the topic doesn't it? ;-) If your thread performs a side task which takes data from your main thread, and then processes them in parallel, then of course you don't need to deal with GUI, other than for error messages, which can be handled quite easily. But if your threads do receive data which must be fed to the main thread, and which must trigger visual events, then you become aware of the strong limitations of current implementation. Giuliano -- Giuliano Colla Whenever people agree with me, I always feel I must be wrong (O. Wilde) _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On 12/12/2007, Giuliano Colla [EMAIL PROTECTED] wrote: If your thread performs a side task which takes data from your main thread, and then processes them in parallel, then of course you don't need to deal with GUI, other than for error messages, which can be handled quite easily. But if your threads do receive data which must be fed to the main thread, and which must trigger visual events, then you become aware of the strong limitations of current implementation. I'm no Threads expert, but have been studying code that uses threads heavily. A few such examples are the Borland Multi-Threading competition [1] that was held a few years back. There all the entries were GUI apps and the threads managed to update the GUI just fine (via Synchronized methods I believe). So is it really a issue, or am I not understanding the underlying problem? [1] - I can supply the URL if anybody is interested. Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On 12/12/2007, Razvan Adrian Bogdan [EMAIL PROTECTED] wrote: The synchronize method is simple but inefficient, it pauses your thread until the main thread has time to execute your code and could take some time until it does, i personally also dislike the fact that Ah okay I see what you mean. If the thread doesn't need to communicate with the main GUI thread, it will run perfectly. As soon as it has to talk to the main GUI thread it has to use Synchronize, which might be delayed by a very busy GUI thread. So what if the worker thread posts a custom message to say the windows message queue. Surely that wouldn't cause the thread to be suspended. The GUI control then simply needs to know about the custom message, to process it correctly. Could this work, well on Windows at least? Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On Dec 12, 2007 9:34 PM, Graeme Geldenhuys [EMAIL PROTECTED] wrote: So what if the worker thread posts a custom message to say the windows message queue. Surely that wouldn't cause the thread to be suspended. The GUI control then simply needs to know about the custom message, to process it correctly. Could this work, well on Windows at least? Using messages works on windows, but GTK is the most thread unsafe of all widgetsets, when i tried to port an app that worked fine on windows to GTK1/2 it was quite unstable, even now with a StringList based messages queue there are times when the app simply crashes with strange gtk resize warning but this is probably a GTK stability issue. Being able to do GUI calls in secondary threads is nice to have. Razvan _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
The synchronize method is simple but inefficient, it pauses your thread until the main thread has time to execute your code and could take some time until it does, i personally also dislike the fact that one cannot call any procedure/function with any number of parameters with synchronize. I think mark proposed once the solution of using a List (StringList or something else) in a synchronised manner with CriticalSections protecting access to the list and this way creating a message system similar to the OS one but this implementation is also not always comfortable but it isn't blocking as synchronize, the problem is not with the windows widgetset which is message based and quite thread safe and one can easily use CriticalSections with Controls, Borland started to make the VCL threadsafe but they didn't complete this task, maybe LCL would be more popular if it was threadsafe, i do realise it's a lot of work but it might be worth it. Razvan _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Giuliano Colla wrote: However making Lazarus LCL thread safe can be a lot of work, but would be an enormous bonus which would enable to use it in all fields where a real efficient thread support is needed and currently you have no RAD tools. Multi-thread safe LCL would also mean locking, and that means slowdown also when you don't need it. 'real efficient' is therefore a *very* *Very* simplistic view of the situation. Besides, the GUI thread is meant to be mostly idle; so that it can respond to user's action quickly. You have to synchronize your background tasks anyway if you run them, and want to stop them; so multi-thread safety doesn't gain much on this point. My personal opinion it's only nice for QuickSort vs. Bubble Sort demoes. Micha _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
So what if the worker thread posts a custom message to say the windows message queue. Surely that wouldn't cause the thread to be suspended. The GUI control then simply needs to know about the custom message, to process it correctly. Could this work, well on Windows at least? Using messages works on windows, but GTK is the most thread unsafe of all widgetsets, when i tried to port an app that worked fine on windows to GTK1/2 it was quite unstable, even now with a StringList based messages queue there are times when the app simply crashes with strange gtk resize warning but this is probably a GTK stability issue. Being able to do GUI calls in secondary threads is nice to have. I must confess I don't know too much of GTK1/2 but, for example if fpGUI is a framework from scratch indipendent from GTK, simply sending asynchronous messages to the main thread should work well ? It can elaborate the messages when the main loop will intercept them. In Windows it should work well; with Linux or other operating system I have no idea but I suppose yes. -- Antonio Sanguigni alias slapshot -- GioveLUG (Linux User Group) - http://www.giovelug.org Edupup (Educational distro) - http://www.edupup.org
Re: [lazarus] Making GTK Thread Safe
Graeme Geldenhuys ha scritto: On 12/12/2007, Razvan Adrian Bogdan [EMAIL PROTECTED] wrote: The synchronize method is simple but inefficient, it pauses your thread until the main thread has time to execute your code and could take some time until it does, i personally also dislike the fact that Ah okay I see what you mean. If the thread doesn't need to communicate with the main GUI thread, it will run perfectly. As soon as it has to talk to the main GUI thread it has to use Synchronize, which might be delayed by a very busy GUI thread. Well, on Kylix it was much worse than that. It could be delayed ad libitum by an Idle GUI thread. Delphi relied on Windows buggy behavior of sending frequent useless messages around to wake up the main thread frequently. But an honest to God OS, which doesn't send messages when it has nothing to say, defeats a design based on bugs. Of course we may overcome that, by creating your own queue of events from the thread, and try to process them with an OnIdle event handler, and wake up periodically the main thread with a timer event. To discover that if your timer event doesn't change something visual, the smart thing decides that there's nothing important to do, and stays Idle, so that the OnIdle handler is never called. My kludge has been to have the Timer update a Label in a hidden corner of the screen, with a sequence of - \ | / - . This wakes up the main thread, which then goes Idle, triggers the OnIdle event, and the queue can be processed. You may see that this is far from satisfactory. Up to now I've not yet tested Lazarus with a real demanding threaded app, but from what I've read about the OnIdle events from Felipe and others, I'm afraid that the result won't be much different. So what if the worker thread posts a custom message to say the windows message queue. Surely that wouldn't cause the thread to be suspended. The GUI control then simply needs to know about the custom message, to process it correctly. Could this work, well on Windows at least? See above. One should use a different design. It's useless to attempt to make single graphic components thread safe. It enormously increases the overhead, and the number of possible bugs. What if a widget handled by thread A is partially covered by a widget handled by thread B, and both reside on a window handled by thread C? Just think to the amount of communication between different threads in order to sort out how to redraw in the proper order. The screen is a shared resource, so it must be granted exclusive access. If one thinks of one main thread which besides doing its job receives messages from all the worker threads, creates a clean queue of what needs to be done (sorting out what's visible and what not, what must be drawn and what not, etc.), and passes that to a graphic thread which is the only one which has access to the display, then you have a real multithread GUI, which could fill a void in what's currently available. It could be a challenging project, don't you think? It just requires a not too large GUI system already existing (such as fpGui, not Lazarus which is too big to play with) and some enthusiasm (and also a lot of time, but for my company it could be important, so some resources could be found). Regards, Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
Micha Nelissen ha scritto: Giuliano Colla wrote: However making Lazarus LCL thread safe can be a lot of work, but would be an enormous bonus which would enable to use it in all fields where a real efficient thread support is needed and currently you have no RAD tools. Multi-thread safe LCL would also mean locking, and that means slowdown also when you don't need it. 'real efficient' is therefore a *very* *Very* simplistic view of the situation. Locking is the easy hacking way. Is basically the reverse of what it's done now with the blocking mechanism of synchronize. You don't block the main thread, but rather the working thread. Making LCL thread safe mean to separate the logical handling of what has to be done, from the physical handling of writing to the screen. If it's well implemented it can be faster than now even for applications which don't need multithreading. If you think, any web browser is multithreaded, thread safe, and fills up your screen faster than Lazarus. Besides, the GUI thread is meant to be mostly idle; so that it can respond to user's action quickly. You have to synchronize your background tasks anyway if you run them, and want to stop them; so multi-thread safety doesn't gain much on this point. The reaction to *user* action is quick (mouse and keyboard), the reaction to events coming from outside is far from satisfactory. My personal opinion it's only nice for QuickSort vs. Bubble Sort demoes. Ever thought of applications which receive data from a number of external sources and should display them and react accordingly? They outnumber the applications where a user just sits there and plays with mouse. And they're commercial application which need a good solid software, and a pleasant look. I'm talking of application where the input devices are a combination of network connection plus barcode readers, badge readers, electronic scales, access controls, etc. etc. You find them in POS, in hotels, in company stores, in cash dispensers, in taxi companies, in airports, wherever. The multiple sources require multithreading, and fast response to external events, combined with fast update of display, and with human interaction. There's no RAD worth mentioning today to provide the GUI for all those applications. Giuliano _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On 13/12/2007, Antonio Sanguigni [EMAIL PROTECTED] wrote: I must confess I don't know too much of GTK1/2 but, for example if fpGUI is a framework from scratch indipendent from GTK, simply sending asynchronous messages to the main thread should work well ? It can elaborate the messages This is exactly why I asked about using messages. :) fpGUI has it's own message system implemented (something like Windows), but for all supported platforms. I haven't tested threading with fpGUI yet, but it's on the top of my todo list. Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On 13/12/2007, Giuliano Colla [EMAIL PROTECTED] wrote: Delphi relied on Windows buggy behavior of sending frequent useless messages around to wake up the main thread frequently. That does sound weird. Idle, so that the OnIdle handler is never called. My kludge has been to have the Timer update a Label in a hidden corner of the screen, with a So if the corner of the screen is hidden, how does that look? What's there, a black hole. ;-) I'm just teasing! must be drawn and what not, etc.), and passes that to a graphic thread which is the only one which has access to the display, then you have a real multithread GUI, which could fill a void in what's currently available. It could be a challenging project, don't you think? Challenging to say the least, but sounds interesting! Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
Re: [lazarus] Making GTK Thread Safe
On Thursday 13 December 2007 01.28, Giuliano Colla wrote: [...] The screen is a shared resource, so it must be granted exclusive access. If one thinks of one main thread which besides doing its job receives messages from all the worker threads, creates a clean queue of what needs to be done (sorting out what's visible and what not, what must be drawn and what not, etc.), and passes that to a graphic thread which is the only one which has access to the display, then you have a real multithread GUI, which could fill a void in what's currently available. It could be a challenging project, don't you think? It just requires a not too large GUI system already existing (such as fpGui, not Lazarus which is too big to play with) and some enthusiasm (and also a lot of time, but for my company it could be important, so some resources could be found). Don't forget MSEide+MSEgui: http://sourceforge.net/projects/mseide-msegui MSEgui has several possibilities to synchronize worker threads with the main GUI thread. First there is application.lock/unlok which works on a mutex. The main eventloop acquires the mutex on start of a new round of the main eventloop and releases it after it gets idle. Secondly there is application.synchronize which runs a callback in a try/finally block with application.lock/unlock. Thirdly there is application.postevent in order to send messages to MSEgui components. The posted events are feed into the main event queue and are delivered to the component in context of the main thread. application.postevent is thread safe and does not block. tmsecomponent.postcomponentevent and asyncevent can be used for convenience, tmseform tmsedatamodule have onevent and onasyncevent where the messages can be handled in context of the main thread. Fourthly there is FPC synchronize with the problem that checksynchronize is called once in onidle of then main event loop. One can call application.wakeupguithread to enable processing but there is a race condition. Worth to mention is teventthread with its own message queue. The graphic subsystem has its own locking mechanism so it is possible to draw onto off screen canvas (pixmap or printer) from worker threads. Martin _ To unsubscribe: mail [EMAIL PROTECTED] with unsubscribe as the Subject archives at http://www.lazarus.freepascal.org/mailarchives
