I cannot answer your question about performance, but you have prompted me
to create a simple example with POSIX shared memory. Attached are two
files, producer.c and consumer.scm. The producer (a C program) writes
strings to a POSIX shared memory file, and the consumer (a Scheme program)
reads them, prints them and then unlinks the buffer. The producer uses
mmap, as Dan mentioned in his reply. This may be clunky, but there is no
need to worry about synchronization.
Using e.g. Linux shmget for shared memory between threads _might_ give you
higher performance, but then you have to mess around with semaphores and so
on. I will not have time to adapt the code you link to, but I can help if
you are having trouble with a particular construct in posix-shm.
-Ivan
On Sun, May 5, 2013 at 8:07 AM, Jörg F. Wittenberger
joerg.wittenber...@softeyes.net wrote:
On May 4 2013, Ivan Raikov wrote:
I think you can try to have native threads by running a separate instance
of the Chicken runtime for each thread, but are you sure that you will
really achieve a significant speedup over Unix processes and/or MPI?
It's not the early nineties anymore... I suggest prototyping code and
running real-world benchmarks first.
Hm. I dunno, I'm old school.
So far I have at least one case, where I've been resorting to native
threads because I didn't see another solution. Maybe you/anybody
could enlighten me?
The case at hand: I'm running sqlite queries, which can take a long time.
I don't want to block the chicken system while those are busy. Worse: I'm
using the sqlite VFS to get the actual data blocks ... from chicken, hence
the chicken thread MUST not wait for sqlite... (for reference an old
version of the code is here http://lists.nongnu.org/**
archive/html/chicken-users/**2010-01/msg00046.htmlhttp://lists.nongnu.org/archive/html/chicken-users/2010-01/msg00046.html)
As I said: I'm old school - could I really expect a reasonable performance
from running a separate sqlite process and cop the data back and forth
between the processes? Suggestion how to do this are welcome!
On Sat, May 4, 2013 at 8:39 AM, Dan Leslie d...@ironoxide.ca wrote:
...
Relatedly, is anyone poking at implementing native threads?
I've been digging around a bit but haven't had much time to progress very
far.
In fact I'm contemplating about this for a while and never got the
courage to try. (Especially because I don't see how I could create
a half-way meaningful benchmark beforehand to judge the gain/loss
early in the development process.)
The bottleneck for me is usually blocking on i/o. Wrt. this I do not
expect (as in wild guess - no benchmarks done) much from multiple
threads.
However some things I'm doing are just computations. Am I thinking
early nineties style when I assume that chicken will utilize only
one cpu core? If so, I'd prefer not spare the others all the time.
My problem: I can't see a good way to share chicken-managed memory
between threads. I recall having seen a paper on using a dedicated
thread for memory allocation...but can't find it anymore.
Nevertheless: maybe it would be feasible to do all second gen
gc in a dedicated thread and force a minor gc before passing data
between threads? Would it?
Best
/Jörg
#include sys/types.h
#include sys/mman.h
#include sys/stat.h/* For mode constants */
#include unistd.h
#include fcntl.h /* For O_* constants */
#include stdio.h
#include stdlib.h
#include string.h
#include errno.h
#define BUFFER_SIZE 32768
#define BUFFER_NAME /shmtest1
void buffer_create(int size, int *fileno, char **addr)
{
int perms = 0600;
char *laddr;
*addr = 0;
*fileno = shm_open(BUFFER_NAME, O_CREAT | O_RDWR, perms);
if (errno 0)
{
perror(failed to create shared memory file handle);
exit (1);
}
if ((ftruncate(*fileno, size)) == -1)
{/* Sets the size */
perror(ftruncate failed on shared memory file handle);
exit(1);
}
laddr = (char*)mmap(NULL, size, PROT_WRITE, MAP_SHARED, *fileno, 0);
if (errno 0)
{
perror (failed to mmap shared memory file handle);
exit (1);
}
*addr = laddr;
}
void buffer_clear(char **sbuff, char *start)
{
*sbuff = start;
}
int buffer_size(char *sbuff, char *start)
{
return (sbuff-start);
}
void buffer_close(int fileno, char *start)
{
if ((munmap(start, BUFFER_SIZE)) == -1)
{
perror(munmap failed on shared buffer);
exit(1);
}
if ((close(fileno)) == -1)
{
perror(close failed on shared buffer file handle);
exit(1);
}
}
void insert_item(char *item, char **shared_buffer, char *shared_start)
{
int n = strlen(item);
memcpy (*shared_buffer, item, n);
*shared_buffer = *shared_buffer+n;
}
int main(int argc, const char **argv)
{
char *shared_start, *shared_buffer;
int shared_fileno;
buffer_create(BUFFER_SIZE, shared_fileno,
