Hi, I'm a pyodide developer amongst other things (webassembly cpython intepreter) and I've got some PRs in progress on arrow relating to webassembly support. I wondered if it might be worth discussing my broader ideas for this on the list or at the biweekly development meeting?
So far I have 35176 in, which makes arrow run on a single thread. This is needed because in a lot of webassembly environments (browsers at least, pyodide), threading isn't available or is heavily constrained. With that I've aimed to make it relatively transparent to users, so that things like datasets and acero mostly just work (but slower obviously). It's kind of fiddly in the arrow code but working, and means users can port things easily. Once that is in, the plan is to submit a following pr that adds cmake presets for emscripten which can build the cpp libraries and pyarrow for pyodide. I've hacked this together in a build already, it's a bit fiddly and needs a load of tidying up, but I'm confident it can be done. Essentially, I'm wanting to get this stuff in because pandas is moving towards arrow as a pretty much required dependency, and webassembly is a pandas platform, as well as being an official python platform, so it would be great to get it working in pyodide without us needing to maintain a load of patches. I guess it could also come in handy with various container platforms that are moving to webassembly. Basically I thought it's probably worth a bit of a heads up relating to this, as I know the bigger picture of things is often hard to see from just pull requests. Thanks Joe This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please contact the sender and delete the email and attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. Email communications with the University of Nottingham may be monitored where permitted by law.
