hi Dominik -- I just wanted to confirm that you're going to wait for the 7.0.0 release? We should still try to get things set up so that JS patch source releases can be made easily.
On Fri, Dec 10, 2021 at 4:39 PM Tim Paine <t.paine...@gmail.com> wrote: > > Just wanted to note that our move from Arrow JS to Wasm arrow resulted in a > pretty substantial speedup, although we solve none of the problems of > exposing wasm arrow over JS since we use it directly from C++. > > Initial PR: https://github.com/finos/perspective/pull/755 > <https://github.com/finos/perspective/pull/755> > Standalone Arrow C++ to wasm via emscripten: > https://github.com/timkpaine/arrow-wasm-cpp > <https://github.com/timkpaine/arrow-wasm-cpp> > (Note that neither utilize all of arrow, just a carve out of IPC stuff) > > > > > Tim Paine > tim.paine.nyc > 908.721.1185 > > > On Dec 9, 2021, at 14:18, Dominik Moritz <domor...@apache.org> wrote: > > > >> > >> Arrow rust implementation is in another repository and has support for > >> > > Javascript/Webassembly : > >> > >> https://github.com/apache/arrow-rs/tree/master/arrow > >> > >> The release cadence for the Rust implementation is much higher than for > >> the C++ implementation. Efficiencies might be gained by releasing Rust > >> and Javascript point implementations together since then the process of > >> creating and verifying signed software would minimize PMC workload.Arrow > >> rust implementation is in another repository and has support for > >> > > Javascript/Webassembly : > >> > >> https://github.com/apache/arrow-rs/tree/master/arrow > >> > >> The release cadence for the Rust implementation is much higher than for > >> the C++ implementation. Efficiencies might be gained by releasing Rust > >> and Javascript point implementations together since then the process of > >> creating and verifying signed software would minimize PMC workload. > > > > > > The biggest challenge with making a web-library for Arrow with WASM is a > > performant JS API. I think realistically, we will have a pure JS Arrow > > library for a few years. Do you think we could sync the release processes > > even if Arrow JS is not in the rust repo? If so, I would love to learn more > > about how that process would work. > > > > On Dec 7, 2021 at 03:43:13, Benson Muite <benson_mu...@emailplus.org> wrote: > > > >> At the moment, the release is not packaged or signed. Thus one can only > >> run the tests on the branch in the git repository. A script to do that > >> on Linux is available at: > >> > >> https://github.com/bkmgit/arrow/blob/ARROW-14801/dev/release/verify-js.sh > >> > >> My understanding is that only PMC members can sign, at the moment not > >> many seem to use Javascript extensively. Can create a script for > >> generating the Javascript only release source package based on the > >> current source packaging and release scripts, but a PMC member would > >> need to have this signed and uploaded. > >> > >> @Dominik - was not aware of arrow-wasm, thanks. > >> > >> Arrow rust implementation is in another repository and has support for > >> Javascript/Webassembly : > >> > >> https://github.com/apache/arrow-rs/tree/master/arrow > >> > >> The release cadence for the Rust implementation is much higher than for > >> the C++ implementation. Efficiencies might be gained by releasing Rust > >> and Javascript point implementations together since then the process of > >> creating and verifying signed software would minimize PMC workload. > >> > >> Benson > >> > >> On 12/6/21 1:01 AM, Wes McKinney wrote: > >> > >> hi Dominik — can you provide instructions for how we should verify the > >> > >> release, aside from checking the GPG signature and checksums? > >> > >> > >> On Sun, Nov 28, 2021 at 12:41 PM Dominik Moritz <domor...@apache.org> > >> wrote: > >> > >>> > >> > >>> Are you talking about https://github.com/domoritz/arrow-wasm? It > >> definitely > >> > >>> isn’t ready for prime time. The overhead of WASM, some issues with the > >> Rust > >> > >>> implementation (some of which I think will be addressed with the Arrow2 > >> > >>> Rust migration), and the much larger bundle size make it not practical > >> > >>> right now. As the WASM ecosystem matures, we can reevaluate and maybe > >> also > >> > >>> consider moving only some of the processing in WASM and leave the rest in > >> > >>> JS. I’m pretty excited about WASM and what it could bring to Arrow > >> > >>> especially when combined with WebGPU. > >> > >>> > >> > >>> Either way, I think we should release the 6.0.2 version soon. @PMC, could > >> > >>> you vote on the patch release? > >> > >>> > >> > >>> On Nov 28, 2021 at 04:33:41, Benson Muite <benson_mu...@emailplus.org> > >> > >>> wrote: > >> > >>> > >> > >>>> Rust implementation can be compiled to WebAssembly and is released > >> > >>>> biweekly. The Javascript version compiled from Rust may not satisfy all > >> > >>>> Javascript users, but maybe there could be some collaboration to reduce > >> > >>>> duplicated efforts? > >> > >>>> > >> > >>>> > >> > >>>> On 11/23/21 9:52 PM, Dominik Moritz wrote: > >> > >>>> > >> > >>>> Ahh, thank you for the clarification. There are no breaking changes > >> in > >> > >>>> > >> > >>>> this point release, just fixes. > >> > >>>> > >> > >>>> > >> > >>>> @PMC, could you please vote on this point release. > >> > >>>> > >> > >>>> > >> > >>>> Would anyone volunteer as the release manager with me to give me a > >> better > >> > >>>> > >> > >>>> understanding of the process? > >> > >>>> > >> > >>>> > >> > >>>> On Nov 23, 2021 at 13:09:47, Benson Muite <benson_mu...@emailplus.org> > >> > >>>> > >> > >>>> wrote: > >> > >>>> > >> > >>>> > >> > >>>>> https://issues.apache.org/jira/browse/ARROW-14801 > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> Rust has its own repository and does frequent point releases: > >> > >>>> > >> > >>>>> https://github.com/apache/arrow-rs/tree/master/dev/release > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> however, even point releases require 3 PMC binding +1 votes and API > >> > >>>> > >> > >>>>> breaking changes can only take place on major releases. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> Many of the tests for releases can be automated, possibly relieving > >> some > >> > >>>> > >> > >>>>> of the PMC burden in the current process. Judgement on code quality > >> and > >> > >>>> > >> > >>>>> software license is still required though[1]. Similarly, releases need > >> > >>>> > >> > >>>>> to be signed. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> [1] https://infra.apache.org/release-publishing.html > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> On 11/23/21 7:33 PM, Dominik Moritz wrote: > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> I tested Node v14.18.1 and tests pass. I think we can go ahead and > >> > >>>> make a > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> release. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> @Benson, could you help me update the script to work off of branches. I > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> don’t know what the expected process for release verification is. I’d > >> be > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> happy to adopt another process. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> On Nov 20, 2021 at 09:57:53, Dominik Moritz <domor...@apache.org> > >> wrote: > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>> Thanks for catching that. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>> Jest is used for running the tests and jest supports node 14.15. Could > >> > >>>> we > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>> switch to node 14.15 instead of 14.0 for this test? > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>> On Nov 20, 2021 at 05:37:00, Benson Muite <benson_mu...@emailplus.org > >>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>> wrote: > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Hi, > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Tested this on AlmaLinux 8. Following steps: > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> export NVM_DIR="`pwd`/.nvm" > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> mkdir -p $NVM_DIR > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> curl -o- > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | \ > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> PROFILE=/dev/null bash > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> nvm install --lts > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> npm install -g yarn > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> git clone https://github.com/apache/arrow > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> cd arrow > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> git checkout release-6.0.2-js > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> cd js > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> yarn --frozen-lockfile > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> yarn run-s clean:all lint build > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> yarn test > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Tests pass. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> yarn 1.22.17 > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> npm 8.1.0 > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> node 16.13.0 > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Tests also pass on > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> node 17.0.0 > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Node 14 is supported until 2023, however if one tries to use Node 14, > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> one gets the error: > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> jest@27.0.6: The engine "node" is incompatible with this module. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Expected version "^10.13.0 || ^12.13.0 || ^14.15.0 || >=15.0.0". Got > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> "14.0.0" > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> error Found incompatible module. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> The current release verification script could be update to support > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> testing directly from a branch if this will be the point release > >> > >>>> process > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> in future. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> On 11/20/21 12:25 AM, Dominik Moritz wrote: > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Hi, > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> I would like to propose a patch release for Arrow JS. The release is > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> forked > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> off of maint-6.0.x and available at > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> https://github.com/apache/arrow/tree/release-6.0.2-js. > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> The release contains two fixes for the js bundle: > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> ARROW-14773: [JS] Fix sourcemap paths > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> <https://github.com/apache/arrow/pull/11741> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> ARROW-14774: [JS] Correct package exports > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> <https://github.com/apache/arrow/pull/11742> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> [ ] +1 Release this as Apache Arrow JS 6.0.2 > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> [ ] +0 > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> [ ] -1 Do not release this as Apache Arrow JS 6.0.2 because... > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Thank you, > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>>>>> Dominik > >> > >>>> > >> > >>>>> > >> > >>>> > >> > >>>> > >> > >> > >> >
