Re: managing views after heap realloc
PR made … let me know what you need. > On Apr 10, 2019, at 8:58 AM, Beuc wrote: > > Hi, > > Would you like to propose a pull request? :) > > https://github.com/emscripten-core/emscripten/blob/incoming/site/source/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.rst > https://github.com/emscripten-core/emscripten/edit/incoming/site/source/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.rst > > - Sylvain > > On 10/04/2019 14:53, Eric Mandel wrote: >> I might be really useful to add a sentence to the "Access Memory from >> Javascript" section of >> https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html, >> referencing your illuminating memory growth note >> https://gist.github.com/kripken/949eab99b7bc34f67c12140814d2b595. It's >> a good example of the requirement that "you know what you are doing", >> especially for C programmers who can't wait to get a buffer pointer >> back into Javascript. > > -- > You received this message because you are subscribed to a topic in the Google > Groups "emscripten-discuss" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/emscripten-discuss/hX1KzStIa4E/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > emscripten-discuss+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: managing views after heap realloc
Sure, but after the Event Horizon Telescope press conference, starting in a few minutes! -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: managing views after heap realloc
Hi, Would you like to propose a pull request? :) https://github.com/emscripten-core/emscripten/blob/incoming/site/source/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.rst https://github.com/emscripten-core/emscripten/edit/incoming/site/source/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.rst - Sylvain On 10/04/2019 14:53, Eric Mandel wrote: > I might be really useful to add a sentence to the "Access Memory from > Javascript" section of > https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html, > referencing your illuminating memory growth note > https://gist.github.com/kripken/949eab99b7bc34f67c12140814d2b595. It's > a good example of the requirement that "you know what you are doing", > especially for C programmers who can't wait to get a buffer pointer > back into Javascript. -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: managing views after heap realloc
I might be really useful to add a sentence to the "Access Memory from Javascript" section of https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html, referencing your illuminating memory growth note https://gist.github.com/kripken/949eab99b7bc34f67c12140814d2b595. It's a good example of the requirement that "you know what you are doing", especially for C programmers who can't wait to get a buffer pointer back into Javascript. -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: managing views after heap realloc
In my use case it's not hard to catch the memory growth. There are only a few entry points, and I can check the allocated views afterward. The problem is managing the references to the views that have spread throughout the application state. I can write a Proxy to a view that can swap out the underlying view, but as I mentioned, it's too slow. Turns out it's more performant to subclass the view. The underlying view can be swapped out by setting __proto__. Not a great solution, and still 10x slower to access than the view w/o subclassing, but faster than a Proxy. The main places in the code holding references are memoized functions, so another approach might be to add special handling to the memoizers, to swap out the views in cache when they are invalidated. On Monday, April 8, 2019 at 11:39:11 AM UTC-7, Alon Zakai wrote: > > If you don't capture references to views, then emscripten will update > Module.HEAP8 etc. when memory grows. So only using those should be fine. > > Otherwise, perhaps there should be an option to listen for memory growth > events, so that you can be notified when you need to recreate the views, > but this gets very hard with pthreads (see > https://github.com/WebAssembly/design/issues/1271 ) > > On Sun, Apr 7, 2019 at 1:31 PM Brian Craft > wrote: > >> Copy overhead for large data is too high, so I'm hoping to keep the data >> in the wasm heap and provide access from js via typed array views. >> >> This seems to work well except when the heap grows, invalidating the >> views. Are there any good ways of dealing with this? I'm hoping to avoid >> putting knowledge of the invalidation all over the js code. >> >> I tried writing Proxy which delegates to a view, and updates the view as >> necessary, but using traps kills the performance of the views. E.g. every >> access, like view[i] hits the javascript trap, so iterating the view >> becomes painfully slow. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to emscripten-discuss+unsubscr...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: managing views after heap realloc
If you don't capture references to views, then emscripten will update Module.HEAP8 etc. when memory grows. So only using those should be fine. Otherwise, perhaps there should be an option to listen for memory growth events, so that you can be notified when you need to recreate the views, but this gets very hard with pthreads (see https://github.com/WebAssembly/design/issues/1271 ) On Sun, Apr 7, 2019 at 1:31 PM Brian Craft wrote: > Copy overhead for large data is too high, so I'm hoping to keep the data > in the wasm heap and provide access from js via typed array views. > > This seems to work well except when the heap grows, invalidating the > views. Are there any good ways of dealing with this? I'm hoping to avoid > putting knowledge of the invalidation all over the js code. > > I tried writing Proxy which delegates to a view, and updates the view as > necessary, but using traps kills the performance of the views. E.g. every > access, like view[i] hits the javascript trap, so iterating the view > becomes painfully slow. > > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to emscripten-discuss+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
managing views after heap realloc
Copy overhead for large data is too high, so I'm hoping to keep the data in the wasm heap and provide access from js via typed array views. This seems to work well except when the heap grows, invalidating the views. Are there any good ways of dealing with this? I'm hoping to avoid putting knowledge of the invalidation all over the js code. I tried writing Proxy which delegates to a view, and updates the view as necessary, but using traps kills the performance of the views. E.g. every access, like view[i] hits the javascript trap, so iterating the view becomes painfully slow. -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
