I cannot answer your question about performance, but you have prompted me to create a simple example with POSIX shared memory. Attached are two files, producer.c and consumer.scm. The producer (a C program) writes strings to a POSIX shared memory file, and the consumer (a Scheme program) reads them, prints them and then unlinks the buffer. The producer uses mmap, as Dan mentioned in his reply. This may be clunky, but there is no need to worry about synchronization. Using e.g. Linux shmget for shared memory between threads _might_ give you higher performance, but then you have to mess around with semaphores and so on. I will not have time to adapt the code you link to, but I can help if you are having trouble with a particular construct in posix-shm.
-Ivan On Sun, May 5, 2013 at 8:07 AM, Jörg F. Wittenberger < joerg.wittenber...@softeyes.net> wrote: > On May 4 2013, Ivan Raikov wrote: > > I think you can try to have native threads by running a separate instance >> of the Chicken runtime for each thread, but are you sure that you will >> really achieve a significant speedup over Unix processes and/or MPI? >> It's not the early nineties anymore... I suggest prototyping code and >> running real-world benchmarks first. >> > > Hm. I dunno, I'm old school. > > So far I have at least one case, where I've been resorting to native > threads because I didn't see another solution. Maybe you/anybody > could enlighten me? > > The case at hand: I'm running sqlite queries, which can take a long time. > I don't want to block the chicken system while those are busy. Worse: I'm > using the sqlite VFS to get the actual data blocks ... from chicken, hence > the chicken thread MUST not wait for sqlite... (for reference an old > version of the code is here http://lists.nongnu.org/** > archive/html/chicken-users/**2010-01/msg00046.html<http://lists.nongnu.org/archive/html/chicken-users/2010-01/msg00046.html>) > > As I said: I'm old school - could I really expect a reasonable performance > from running a separate sqlite process and cop the data back and forth > between the processes? Suggestion how to do this are welcome! > > > On Sat, May 4, 2013 at 8:39 AM, Dan Leslie <d...@ironoxide.ca> wrote: >> > ... > > Relatedly, is anyone poking at implementing native threads? >>> I've been digging around a bit but haven't had much time to progress very >>> far. >>> >> > In fact I'm contemplating about this for a while and never got the > courage to try. (Especially because I don't see how I could create > a half-way meaningful benchmark beforehand to judge the gain/loss > early in the development process.) > > The bottleneck for me is usually blocking on i/o. Wrt. this I do not > expect (as in wild guess - no benchmarks done) much from multiple > threads. > > However some things I'm doing are just computations. Am I thinking > "early nineties" style when I assume that chicken will utilize only > one cpu core? If so, I'd prefer not spare the others all the time. > > My problem: I can't see a good way to share chicken-managed memory > between threads. I recall having seen a paper on using a dedicated > thread for memory allocation...but can't find it anymore. > > Nevertheless: maybe it would be feasible to do all second gen > gc in a dedicated thread and force a minor gc before passing data > between threads? Would it? > > > Best > > /Jörg > > > > ........ > > > > > >
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/stat.h> /* For mode constants */
#include <unistd.h>
#include <fcntl.h> /* For O_* constants */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#define BUFFER_SIZE 32768
#define BUFFER_NAME "/shmtest1"
void buffer_create(int size, int *fileno, char **addr)
{
int perms = 0600;
char *laddr;
*addr = 0;
*fileno = shm_open(BUFFER_NAME, O_CREAT | O_RDWR, perms);
if (errno > 0)
{
perror("failed to create shared memory file handle");
exit (1);
}
if ((ftruncate(*fileno, size)) == -1)
{ /* Sets the size */
perror("ftruncate failed on shared memory file handle");
exit(1);
}
laddr = (char*)mmap(NULL, size, PROT_WRITE, MAP_SHARED, *fileno, 0);
if (errno > 0)
{
perror ("failed to mmap shared memory file handle");
exit (1);
}
*addr = laddr;
}
void buffer_clear(char **sbuff, char *start)
{
*sbuff = start;
}
int buffer_size(char *sbuff, char *start)
{
return (sbuff-start);
}
void buffer_close(int fileno, char *start)
{
if ((munmap(start, BUFFER_SIZE)) == -1)
{
perror("munmap failed on shared buffer");
exit(1);
}
if ((close(fileno)) == -1)
{
perror("close failed on shared buffer file handle");
exit(1);
}
}
void insert_item(char *item, char **shared_buffer, char *shared_start)
{
int n = strlen(item);
memcpy (*shared_buffer, item, n);
*shared_buffer = *shared_buffer+n;
}
int main(int argc, const char **argv)
{
char *shared_start, *shared_buffer;
int shared_fileno;
buffer_create(BUFFER_SIZE, &shared_fileno, &shared_start);
buffer_clear(&shared_buffer, shared_start); // prepare buffer for jobs
int i = 0;
char s[30];
while(i<1000)
{
memset(s,sizeof(s),0);
sprintf (s, "Hello world #%d", i);
insert_item(s, &shared_buffer, shared_start);
printf ("shared_buffer = %p\n", shared_buffer);
i = i+1;
}
buffer_close (shared_fileno, shared_start);
return 0;
}
consumer.scm
Description: Binary data
_______________________________________________ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
