Hi Bill, Greg and Joe,
Threads
More testing, this time on Fedora 21 ARM released Dec 2014.
Kernel: 3.17.4-301.fc21.armv7hl
A user process can do 379 "threads" that's all.
See the "th.c" test file below.
Note, I used the word "do". Even if they (the threads) terminate cleanly,
the user process can only do 379 of them.
So, WSPR will "stop" after 379 x 2mins, or about 11 hours.
This is an ARM Linux problem.
Previous email:
Hi Bill, Greg and Joe,
On reading:
http://www.thegeekstuff.com/2012/04/terminate-c-thread/
they use a function call "pthread_exit(ret)"
Which is not there doing a "grep" on the WSPR code.
How many threads do we have:
alanb@bananapi:~$ cat /proc/sys/kernel/threads-max 13988
x86_64 bash-4.2$ cat /proc/sys/kernel/threads-max 59148
But, per process, about 380 on the Banana Pi.
828 on my PC x86_64 Fedora 20
See this C code:
----------------------cut-------------------
#include<stdio.h>
#include<string.h>
#include<pthread.h>
#include<stdlib.h>
#include<unistd.h>
/* Test how many threads we can do */
// gcc -std=c99 th.c -lpthread
#define MAXT 300
pthread_t thrd[MAXT];
int ret1;
void * thread(void* i)
{
sleep(1);//make the thread stay alive 1 sec
ret1 = 0;
pthread_exit(&ret1);
}
int main()
{
for(int i=0;i<MAXT;i++)
{
int err=pthread_create(&thrd[i],NULL,thread,(void*)i);
if(err!=0)
printf( "thread creation failed: %d", err );
}
printf( "threads created MAXT ");
sleep(20);//wait for 1st group to end, then create more
for(int i=0;i<MAXT;i++)
{
int err=pthread_create(&thrd[i],NULL,thread,(void*)i);
if(err!=0)
printf( "thread creation failed: %d %d", err,i );
}
return 0;
}
-----------------------------cut-----------------------
My ARM problem is, we get to MAX_THREAD_COUNT in about 10 hours.
So, we are not clearing them.
10hrs = 600 mins = 300 2min Rx sessions, processing with 100 threads => 30.000
threads.
But, the Rx process fails at about the 300 to 400 2min sessions.
My "test" code uses the function pthread_exit() but it doesn't help.
So, it's a program design issue on the ARM.
So, I tried on the PC, AMD A4-5000, 8gb ram. 828 max simultaneous threads, But
doing 800
with pthread_exit() call, wait until they finish, we can do another 800.
6146B
Alan VK2ZIW
On Tue, 23 Dec 2014 14:55:43 +1000, Alan VK2ZIW wrote
> Hi Bill and all,
> ps -eLfu alanb =>
> UID PID PPID LWP C NLWP STIME TTY TIME CMD
> alanb 3778 2254 3778 0 1 14:24 pts/0 00:00:00 /bin/sh
> /usr/local/bin/wspr40
> alanb 3780 3778 3780 3 5 14:24 pts/0 00:01:52 python3
> /downloads/hamradio/digital/wsjt/4795wsprB/wspr/wspr.py
> alanb 3780 3778 3781 0 5 14:24 pts/0 00:00:04 python3
> /downloads/hamradio/digital/wsjt/4795wsprB/wspr/wspr.py
> alanb 3780 3778 3783 0 5 14:24 pts/0 00:00:03 python3
> /downloads/hamradio/digital/wsjt/4795wsprB/wspr/wspr.py
> alanb 3780 3778 3875 99 5 15:17 pts/0 00:00:10 python3
> /downloads/hamradio/digital/wsjt/4795wsprB/wspr/wspr.py
> alanb 3780 3778 3876 3 5 15:17 pts/0 00:00:00 python3
> /downloads/hamradio/digital/wsjt/4795wsprB/wspr/wspr.py
>
> Why the 99?
>
> Then on a 2nd ps -eLfu alanb, the 99 is back to 0. Another time, 80.
>
> Puzzled, seems to jump in the "Decode" phase.
> cat /proc/sys/kernel/threads-max
> 13988
> cat /proc/sys/vm/max_map_count
> 65530
> ulimit -a
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 6994
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) unlimited
> open files (-n) 1024
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 6994
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
>
> Alan VK2ZIW
>
> On Sun, 21 Dec 2014 12:24:14 +0000, Bill Somerville wrote
> > On 21/12/2014 11:01, Alan VK2ZIW wrote:
> > Hi Greg and all,
> > Hi Alan,
> >
> > Can we please focus on the "crash " problem?
> >
> > WSPR runs perfectly fine for 10+ hours then crashes:
> >
> > spawning new thread: Resourcetemporarily unavailable
> > Error starting rx thread 11
> > I don't know the WSPR code base apart from acursory skim through it but
> > that error sounds like the o/s has runout of process/thread slots. Clearly
> > WSPR doesn't spawn largenumbers of concurrent threads or processes so I
> > would guesssomething is starting threads or processes and they are
> > notterminating when they should. A simple 'ps' listing now and againwhile
> > WSPR is running should reveal the issue.
> >
> >
> > <snip>
> > Keep Smiling
> >
> > Alan VK2ZIW
> >
>
> Alan
>
> Man's greatest waste of time: Worshipping the wrong God.
> Consider Jesus.
> ---------------------------------------------------------------------------
> Alan Beard Unix Support Technician from 1984 to today
> 70 Wedmore Rd. Sun Solaris, AIX, HP/UX, Linux, SCO OpenServer 5.0.X
> Emu Heights N.S.W. 2750 Routers, terminal servers, printers, terminals etc..
> +61 2 47353013 (h) Support Programming, shell scripting, "C", assembler
> 0414 353013 (mobile) After uni, electr
>
Alan
Man's greatest waste of time: Worshipping the wrong God.
Consider Jesus.
---------------------------------------------------------------------------
Alan Beard Unix Support Technician from 1984 to today
70 Wedmore Rd. Sun Solaris, AIX, HP/UX, Linux, SCO OpenServer 5.0.X
Emu Heights N.S.W. 2750 Routers, terminal servers, printers, terminals etc..
+61 2 47353013 (h) Support Programming, shell scripting, "C", assembler
0414 353013 (mobile) After uni, electr
------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
_______________________________________________
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
