Re: PThreads and Configure files

[Jukka Jylänki]

Sounds good. Pushed
https://github.com/kripken/emscripten/commit/1ff7294c9c9d629b527b607134b7b5e37d2f23ae
to fix this in a way that does not need to look at the different
ENVIRONMENT_ fields.

2017-10-06 6:47 GMT+03:00 David Claughton :
> Hi jj,
>
> No there are no tests in the configure file specifically for pthreads.  In
> fact the project doesn't use pthreads at all when compiled natively.  It
> does do synchronous read/write to stdin/stdout though and also fork/exec.
> I'm experimenting with using pthreads as a mechanism for making those things
> work - I think for the former I also need --proxy-to-worker, haven't tried
> that yet,  I'm hoping to be able to use posix_spawn in place of the
> fork/exec.
>
> So yes, for my use case it was just that one line that needed changing.  It
> might well be a different story for a project that actually uses pthreads
> natively.
>
> Having said that, out of curiosity I did have a quick go with adding the
> AX_PTHREAD macro to my configure.ac, just for fun, and it appears to work
> but for some reason it doesn't actually seem to run any pthreads code once
> it detects that we're using clang.  So that might not be very conclusive - I
> have no idea whether configure scripts typically use this macro or whether
> they tend to roll their own checks for pthreads.
>
> Cheers,
>
> David.
>
> On Monday, 2 October 2017 21:25:22 UTC+1, jj wrote:
>>
>> Yeah, this is a tricky one. Do you know if the configure tests will
>> actually ever try to pthread_create() or similar? I.e. I wonder how
>> far does node.js need to support -s USE_PTHREADS=1 builds - would it
>> only need to be able to run through them in singlethreaded mode, or
>> does it sometimes need more.
>>
>> Was the above single line change the only thing necessary to get
>> through in node.js, or were there other things that needed changing?
>> If it ends up just being that one line, we could easily add that in
>> trunk.
>>
>> 2017-09-29 0:15 GMT+03:00 David Claughton :
>> > Hi,
>> >
>> > I think I've found a solution, which turned out to be a lot simpler than
>> > I'd
>> > thought ...
>> >
>> > I made a change to line 71 of shell.js :
>> >
>> > From:
>> >var currentScriptUrl = ENVIRONMENT_IS_WORKER ? undefined :
>> > document.currentScript.src;
>> > To:
>> >var currentScriptUrl = (ENVIRONMENT_IS_WORKER || ENVIRONMENT_IS_NODE)
>> > ?
>> > undefined : document.currentScript.src;
>> >
>> > That seems to have done the trick, for me at least.  I haven't done
>> > extensive testing though.
>> >
>> > Cheers,
>> >
>> > David.
>> >
>> > On Thursday, 28 September 2017 15:41:10 UTC+1, David Claughton wrote:
>> >>
>> >> Hi,
>> >>
>> >> I'm having a bit of trouble working out how to compile a project with
>> >> pthreads support where the project is based on autoconf.
>> >>
>> >> The docs say I need to pass -s USE_PTHREADS=1 to the compile stage as
>> >> well
>> >> as when linking.  The simplest way to do that with a configure script
>> >> is to
>> >> pass it in CFLAGS...
>> >>
>> >> e.g. emconfigure ./configure CFLAGS="-s USE_PTHREADS=1"
>> >>
>> >> (That's from memory, I'm not at my home PC at the moment)
>> >>
>> >> The problem with doing that is the flag is used when compiling the
>> >> feature
>> >> tests in the configure script and because those are run by Node, they
>> >> all
>> >> fail.  Node doesn't have Web Workers so there's no way to make that
>> >> work
>> >> AFAIK (although strictly speaking the actual failure is because the
>> >> pthreads
>> >> code references the 'document' object).
>> >>
>> >> As far as solutions go I can think of a few possibilities:
>> >>
>> >> 1. Modify emcc to filter out the USE_PTHREADS flag when compiling a
>> >> conftest snippet.  The downside to that is if the feature test is
>> >> actually
>> >> testing for pthreads support it will probably fail.
>> >>
>> >> 2. Switch to compiling the conftest snippets using native compilation.
>> >> This used to be the default last time I played with emscripten a few
>> >> years
>> >> ago, but has the disadvantage that the features tests may find features
>> >> available on Linux/Windows that are unimplemented on emscripten and/or
>> >> javascript.  I assume this is why the default was switched to running
>> >> them
>> >> in Node.
>> >>
>> >> 3. Somehow make pthreads code work on Node, at least to some extent.
>> >> There are npm modules that claim to implement Web Workers although I
>> >> don't
>> >> know how well these work.  Alternatively maybe emscripten's pthread
>> >> library
>> >> can be modified to run under node with most things doing a no-op, such
>> >> that
>> >> code compiled with USE_PTHREADS but which is actually single-threaded
>> >> will
>> >> run.  That might be enough to make conftests work, but I don't know how
>> >> much
>> >> work that would be.
>> >>
>> >> Thoughts?  I'm prepared to accept I'm missing something obvious here
>> >> ...
>> >> :-)
>> >>
>> >> Cheers,
>> >>

Re: PThreads and Configure files

[David Claughton]

Hi jj,

No there are no tests in the configure file specifically for pthreads.  In 
fact the project doesn't use pthreads at all when compiled natively.  It 
does do synchronous read/write to stdin/stdout though and also fork/exec.  
I'm experimenting with using pthreads as a mechanism for making those 
things work - I think for the former I also need --proxy-to-worker, haven't 
tried that yet,  I'm hoping to be able to use posix_spawn in place of the 
fork/exec.

So yes, for my use case it was just that one line that needed changing.  It 
might well be a different story for a project that actually uses pthreads 
natively.

Having said that, out of curiosity I did have a quick go with adding the 
AX_PTHREAD macro to my configure.ac, just for fun, and it appears to work 
but for some reason it doesn't actually seem to run any pthreads code once 
it detects that we're using clang.  So that might not be very conclusive - 
I have no idea whether configure scripts typically use this macro or 
whether they tend to roll their own checks for pthreads.

Cheers,

David.

On Monday, 2 October 2017 21:25:22 UTC+1, jj wrote:
>
> Yeah, this is a tricky one. Do you know if the configure tests will 
> actually ever try to pthread_create() or similar? I.e. I wonder how 
> far does node.js need to support -s USE_PTHREADS=1 builds - would it 
> only need to be able to run through them in singlethreaded mode, or 
> does it sometimes need more. 
>
> Was the above single line change the only thing necessary to get 
> through in node.js, or were there other things that needed changing? 
> If it ends up just being that one line, we could easily add that in 
> trunk. 
>
> 2017-09-29 0:15 GMT+03:00 David Claughton  >: 
> > Hi, 
> > 
> > I think I've found a solution, which turned out to be a lot simpler than 
> I'd 
> > thought ... 
> > 
> > I made a change to line 71 of shell.js : 
> > 
> > From: 
> >var currentScriptUrl = ENVIRONMENT_IS_WORKER ? undefined : 
> > document.currentScript.src; 
> > To: 
> >var currentScriptUrl = (ENVIRONMENT_IS_WORKER || ENVIRONMENT_IS_NODE) 
> ? 
> > undefined : document.currentScript.src; 
> > 
> > That seems to have done the trick, for me at least.  I haven't done 
> > extensive testing though. 
> > 
> > Cheers, 
> > 
> > David. 
> > 
> > On Thursday, 28 September 2017 15:41:10 UTC+1, David Claughton wrote: 
> >> 
> >> Hi, 
> >> 
> >> I'm having a bit of trouble working out how to compile a project with 
> >> pthreads support where the project is based on autoconf. 
> >> 
> >> The docs say I need to pass -s USE_PTHREADS=1 to the compile stage as 
> well 
> >> as when linking.  The simplest way to do that with a configure script 
> is to 
> >> pass it in CFLAGS... 
> >> 
> >> e.g. emconfigure ./configure CFLAGS="-s USE_PTHREADS=1" 
> >> 
> >> (That's from memory, I'm not at my home PC at the moment) 
> >> 
> >> The problem with doing that is the flag is used when compiling the 
> feature 
> >> tests in the configure script and because those are run by Node, they 
> all 
> >> fail.  Node doesn't have Web Workers so there's no way to make that 
> work 
> >> AFAIK (although strictly speaking the actual failure is because the 
> pthreads 
> >> code references the 'document' object). 
> >> 
> >> As far as solutions go I can think of a few possibilities: 
> >> 
> >> 1. Modify emcc to filter out the USE_PTHREADS flag when compiling a 
> >> conftest snippet.  The downside to that is if the feature test is 
> actually 
> >> testing for pthreads support it will probably fail. 
> >> 
> >> 2. Switch to compiling the conftest snippets using native compilation. 
> >> This used to be the default last time I played with emscripten a few 
> years 
> >> ago, but has the disadvantage that the features tests may find features 
> >> available on Linux/Windows that are unimplemented on emscripten and/or 
> >> javascript.  I assume this is why the default was switched to running 
> them 
> >> in Node. 
> >> 
> >> 3. Somehow make pthreads code work on Node, at least to some extent. 
> >> There are npm modules that claim to implement Web Workers although I 
> don't 
> >> know how well these work.  Alternatively maybe emscripten's pthread 
> library 
> >> can be modified to run under node with most things doing a no-op, such 
> that 
> >> code compiled with USE_PTHREADS but which is actually single-threaded 
> will 
> >> run.  That might be enough to make conftests work, but I don't know how 
> much 
> >> work that would be. 
> >> 
> >> Thoughts?  I'm prepared to accept I'm missing something obvious here 
> ... 
> >> :-) 
> >> 
> >> Cheers, 
> >> 
> >> David. 
> >> 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "emscripten-discuss" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to emscripten-discuss+unsubscr...@googlegroups.com . 
> > For more options, visit 

Re: PThreads and Configure files

[Jukka Jylänki]

Yeah, this is a tricky one. Do you know if the configure tests will
actually ever try to pthread_create() or similar? I.e. I wonder how
far does node.js need to support -s USE_PTHREADS=1 builds - would it
only need to be able to run through them in singlethreaded mode, or
does it sometimes need more.

Was the above single line change the only thing necessary to get
through in node.js, or were there other things that needed changing?
If it ends up just being that one line, we could easily add that in
trunk.

2017-09-29 0:15 GMT+03:00 David Claughton :
> Hi,
>
> I think I've found a solution, which turned out to be a lot simpler than I'd
> thought ...
>
> I made a change to line 71 of shell.js :
>
> From:
>var currentScriptUrl = ENVIRONMENT_IS_WORKER ? undefined :
> document.currentScript.src;
> To:
>var currentScriptUrl = (ENVIRONMENT_IS_WORKER || ENVIRONMENT_IS_NODE) ?
> undefined : document.currentScript.src;
>
> That seems to have done the trick, for me at least.  I haven't done
> extensive testing though.
>
> Cheers,
>
> David.
>
> On Thursday, 28 September 2017 15:41:10 UTC+1, David Claughton wrote:
>>
>> Hi,
>>
>> I'm having a bit of trouble working out how to compile a project with
>> pthreads support where the project is based on autoconf.
>>
>> The docs say I need to pass -s USE_PTHREADS=1 to the compile stage as well
>> as when linking.  The simplest way to do that with a configure script is to
>> pass it in CFLAGS...
>>
>> e.g. emconfigure ./configure CFLAGS="-s USE_PTHREADS=1"
>>
>> (That's from memory, I'm not at my home PC at the moment)
>>
>> The problem with doing that is the flag is used when compiling the feature
>> tests in the configure script and because those are run by Node, they all
>> fail.  Node doesn't have Web Workers so there's no way to make that work
>> AFAIK (although strictly speaking the actual failure is because the pthreads
>> code references the 'document' object).
>>
>> As far as solutions go I can think of a few possibilities:
>>
>> 1. Modify emcc to filter out the USE_PTHREADS flag when compiling a
>> conftest snippet.  The downside to that is if the feature test is actually
>> testing for pthreads support it will probably fail.
>>
>> 2. Switch to compiling the conftest snippets using native compilation.
>> This used to be the default last time I played with emscripten a few years
>> ago, but has the disadvantage that the features tests may find features
>> available on Linux/Windows that are unimplemented on emscripten and/or
>> javascript.  I assume this is why the default was switched to running them
>> in Node.
>>
>> 3. Somehow make pthreads code work on Node, at least to some extent.
>> There are npm modules that claim to implement Web Workers although I don't
>> know how well these work.  Alternatively maybe emscripten's pthread library
>> can be modified to run under node with most things doing a no-op, such that
>> code compiled with USE_PTHREADS but which is actually single-threaded will
>> run.  That might be enough to make conftests work, but I don't know how much
>> work that would be.
>>
>> Thoughts?  I'm prepared to accept I'm missing something obvious here ...
>> :-)
>>
>> Cheers,
>>
>> David.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to emscripten-discuss+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: PThreads and Configure files

[David Claughton]

Hi,

I think I've found a solution, which turned out to be a lot simpler than 
I'd thought ...

I made a change to line 71 of shell.js :

From:
   var currentScriptUrl = ENVIRONMENT_IS_WORKER ? undefined : 
document.currentScript.src;
To:
   var currentScriptUrl = (ENVIRONMENT_IS_WORKER || ENVIRONMENT_IS_NODE) ? 
undefined : document.currentScript.src;

That seems to have done the trick, for me at least.  I haven't done 
extensive testing though.

Cheers,

David.

On Thursday, 28 September 2017 15:41:10 UTC+1, David Claughton wrote:
>
> Hi,
>
> I'm having a bit of trouble working out how to compile a project with 
> pthreads support where the project is based on autoconf.
>
> The docs say I need to pass -s USE_PTHREADS=1 to the compile stage as well 
> as when linking.  The simplest way to do that with a configure script is to 
> pass it in CFLAGS...
>
> e.g. emconfigure ./configure CFLAGS="-s USE_PTHREADS=1"
>
> (That's from memory, I'm not at my home PC at the moment)
>
> The problem with doing that is the flag is used when compiling the feature 
> tests in the configure script and because those are run by Node, they all 
> fail.  Node doesn't have Web Workers so there's no way to make that work 
> AFAIK (although strictly speaking the actual failure is because the 
> pthreads code references the 'document' object).
>
> As far as solutions go I can think of a few possibilities:
>
> 1. Modify emcc to filter out the USE_PTHREADS flag when compiling a 
> conftest snippet.  The downside to that is if the feature test is actually 
> testing for pthreads support it will probably fail.
>
> 2. Switch to compiling the conftest snippets using native compilation.  
> This used to be the default last time I played with emscripten a few years 
> ago, but has the disadvantage that the features tests may find features 
> available on Linux/Windows that are unimplemented on emscripten and/or 
> javascript.  I assume this is why the default was switched to running them 
> in Node.
>
> 3. Somehow make pthreads code work on Node, at least to some extent.  
> There are npm modules that claim to implement Web Workers although I don't 
> know how well these work.  Alternatively maybe emscripten's pthread library 
> can be modified to run under node with most things doing a no-op, such that 
> code compiled with USE_PTHREADS but which is actually single-threaded will 
> run.  That might be enough to make conftests work, but I don't know how 
> much work that would be.
>
> Thoughts?  I'm prepared to accept I'm missing something obvious here ... 
> :-)
>
> Cheers,
>
> David.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

PThreads and Configure files

[David Claughton]

Hi,

I'm having a bit of trouble working out how to compile a project with 
pthreads support where the project is based on autoconf.

The docs say I need to pass -s USE_PTHREADS=1 to the compile stage as well 
as when linking.  The simplest way to do that with a configure script is to 
pass it in CFLAGS...

e.g. emconfigure ./configure CFLAGS="-s USE_PTHREADS=1"

(That's from memory, I'm not at my home PC at the moment)

The problem with doing that is the flag is used when compiling the feature 
tests in the configure script and because those are run by Node, they all 
fail.  Node doesn't have Web Workers so there's no way to make that work 
AFAIK (although strictly speaking the actual failure is because the 
pthreads code references the 'document' object).

As far as solutions go I can think of a few possibilities:

1. Modify emcc to filter out the USE_PTHREADS flag when compiling a 
conftest snippet.  The downside to that is if the feature test is actually 
testing for pthreads support it will probably fail.

2. Switch to compiling the conftest snippets using native compilation.  
This used to be the default last time I played with emscripten a few years 
ago, but has the disadvantage that the features tests may find features 
available on Linux/Windows that are unimplemented on emscripten and/or 
javascript.  I assume this is why the default was switched to running them 
in Node.

3. Somehow make pthreads code work on Node, at least to some extent.  There 
are npm modules that claim to implement Web Workers although I don't know 
how well these work.  Alternatively maybe emscripten's pthread library can 
be modified to run under node with most things doing a no-op, such that 
code compiled with USE_PTHREADS but which is actually single-threaded will 
run.  That might be enough to make conftests work, but I don't know how 
much work that would be.

Thoughts?  I'm prepared to accept I'm missing something obvious here ... :-)

Cheers,

David.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.