> > Arrow rust implementation is in another repository and has support for > Javascript/Webassembly : > > https://github.com/apache/arrow-rs/tree/master/arrow > > The release cadence for the Rust implementation is much higher than for > the C++ implementation. Efficiencies might be gained by releasing Rust > and Javascript point implementations together since then the process of > creating and verifying signed software would minimize PMC workload.Arrow > rust implementation is in another repository and has support for > Javascript/Webassembly : > > https://github.com/apache/arrow-rs/tree/master/arrow > > The release cadence for the Rust implementation is much higher than for > the C++ implementation. Efficiencies might be gained by releasing Rust > and Javascript point implementations together since then the process of > creating and verifying signed software would minimize PMC workload.
The biggest challenge with making a web-library for Arrow with WASM is a performant JS API. I think realistically, we will have a pure JS Arrow library for a few years. Do you think we could sync the release processes even if Arrow JS is not in the rust repo? If so, I would love to learn more about how that process would work. On Dec 7, 2021 at 03:43:13, Benson Muite <[email protected]> wrote: > At the moment, the release is not packaged or signed. Thus one can only > run the tests on the branch in the git repository. A script to do that > on Linux is available at: > > https://github.com/bkmgit/arrow/blob/ARROW-14801/dev/release/verify-js.sh > > My understanding is that only PMC members can sign, at the moment not > many seem to use Javascript extensively. Can create a script for > generating the Javascript only release source package based on the > current source packaging and release scripts, but a PMC member would > need to have this signed and uploaded. > > @Dominik - was not aware of arrow-wasm, thanks. > > Arrow rust implementation is in another repository and has support for > Javascript/Webassembly : > > https://github.com/apache/arrow-rs/tree/master/arrow > > The release cadence for the Rust implementation is much higher than for > the C++ implementation. Efficiencies might be gained by releasing Rust > and Javascript point implementations together since then the process of > creating and verifying signed software would minimize PMC workload. > > Benson > > On 12/6/21 1:01 AM, Wes McKinney wrote: > > hi Dominik — can you provide instructions for how we should verify the > > release, aside from checking the GPG signature and checksums? > > > On Sun, Nov 28, 2021 at 12:41 PM Dominik Moritz <[email protected]> > wrote: > > > > > > Are you talking about https://github.com/domoritz/arrow-wasm? It > definitely > > > isn’t ready for prime time. The overhead of WASM, some issues with the > Rust > > > implementation (some of which I think will be addressed with the Arrow2 > > > Rust migration), and the much larger bundle size make it not practical > > > right now. As the WASM ecosystem matures, we can reevaluate and maybe > also > > > consider moving only some of the processing in WASM and leave the rest in > > > JS. I’m pretty excited about WASM and what it could bring to Arrow > > > especially when combined with WebGPU. > > > > > > Either way, I think we should release the 6.0.2 version soon. @PMC, could > > > you vote on the patch release? > > > > > > On Nov 28, 2021 at 04:33:41, Benson Muite <[email protected]> > > > wrote: > > > > > >> Rust implementation can be compiled to WebAssembly and is released > > >> biweekly. The Javascript version compiled from Rust may not satisfy all > > >> Javascript users, but maybe there could be some collaboration to reduce > > >> duplicated efforts? > > >> > > >> > > >> On 11/23/21 9:52 PM, Dominik Moritz wrote: > > >> > > >> Ahh, thank you for the clarification. There are no breaking changes > in > > >> > > >> this point release, just fixes. > > >> > > >> > > >> @PMC, could you please vote on this point release. > > >> > > >> > > >> Would anyone volunteer as the release manager with me to give me a > better > > >> > > >> understanding of the process? > > >> > > >> > > >> On Nov 23, 2021 at 13:09:47, Benson Muite <[email protected]> > > >> > > >> wrote: > > >> > > >> > > >>> https://issues.apache.org/jira/browse/ARROW-14801 > > >> > > >>> > > >> > > >>> Rust has its own repository and does frequent point releases: > > >> > > >>> https://github.com/apache/arrow-rs/tree/master/dev/release > > >> > > >>> > > >> > > >>> however, even point releases require 3 PMC binding +1 votes and API > > >> > > >>> breaking changes can only take place on major releases. > > >> > > >>> > > >> > > >>> Many of the tests for releases can be automated, possibly relieving > some > > >> > > >>> of the PMC burden in the current process. Judgement on code quality > and > > >> > > >>> software license is still required though[1]. Similarly, releases need > > >> > > >>> to be signed. > > >> > > >>> > > >> > > >>> > > >> > > >>> [1] https://infra.apache.org/release-publishing.html > > >> > > >>> > > >> > > >>> On 11/23/21 7:33 PM, Dominik Moritz wrote: > > >> > > >>> > > >> > > >>> I tested Node v14.18.1 and tests pass. I think we can go ahead and > > >> make a > > >> > > >>> > > >> > > >>> release. > > >> > > >>> > > >> > > >>> > > >> > > >>> @Benson, could you help me update the script to work off of branches. I > > >> > > >>> > > >> > > >>> don’t know what the expected process for release verification is. I’d > be > > >> > > >>> > > >> > > >>> happy to adopt another process. > > >> > > >>> > > >> > > >>> > > >> > > >>> On Nov 20, 2021 at 09:57:53, Dominik Moritz <[email protected]> > wrote: > > >> > > >>> > > >> > > >>> > > >> > > >>>> Thanks for catching that. > > >> > > >>> > > >> > > >>>> > > >> > > >>> > > >> > > >>>> Jest is used for running the tests and jest supports node 14.15. Could > > >> we > > >> > > >>> > > >> > > >>>> switch to node 14.15 instead of 14.0 for this test? > > >> > > >>> > > >> > > >>>> > > >> > > >>> > > >> > > >>>> On Nov 20, 2021 at 05:37:00, Benson Muite <[email protected] > > > > >> > > >>> > > >> > > >>>> wrote: > > >> > > >>> > > >> > > >>>> > > >> > > >>> > > >> > > >>>>> Hi, > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> Tested this on AlmaLinux 8. Following steps: > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> export NVM_DIR="`pwd`/.nvm" > > >> > > >>> > > >> > > >>>>> mkdir -p $NVM_DIR > > >> > > >>> > > >> > > >>>>> curl -o- > > >> > > >>> > > >> > > >>>>> https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | \ > > >> > > >>> > > >> > > >>>>> PROFILE=/dev/null bash > > >> > > >>> > > >> > > >>>>> [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> nvm install --lts > > >> > > >>> > > >> > > >>>>> npm install -g yarn > > >> > > >>> > > >> > > >>>>> git clone https://github.com/apache/arrow > > >> > > >>> > > >> > > >>>>> cd arrow > > >> > > >>> > > >> > > >>>>> git checkout release-6.0.2-js > > >> > > >>> > > >> > > >>>>> cd js > > >> > > >>> > > >> > > >>>>> yarn --frozen-lockfile > > >> > > >>> > > >> > > >>>>> yarn run-s clean:all lint build > > >> > > >>> > > >> > > >>>>> yarn test > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> Tests pass. > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> yarn 1.22.17 > > >> > > >>> > > >> > > >>>>> npm 8.1.0 > > >> > > >>> > > >> > > >>>>> node 16.13.0 > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> Tests also pass on > > >> > > >>> > > >> > > >>>>> node 17.0.0 > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> Node 14 is supported until 2023, however if one tries to use Node 14, > > >> > > >>> > > >> > > >>>>> one gets the error: > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> [email protected]: The engine "node" is incompatible with this module. > > >> > > >>> > > >> > > >>>>> Expected version "^10.13.0 || ^12.13.0 || ^14.15.0 || >=15.0.0". Got > > >> > > >>> > > >> > > >>>>> "14.0.0" > > >> > > >>> > > >> > > >>>>> error Found incompatible module. > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> The current release verification script could be update to support > > >> > > >>> > > >> > > >>>>> testing directly from a branch if this will be the point release > > >> process > > >> > > >>> > > >> > > >>>>> in future. > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> On 11/20/21 12:25 AM, Dominik Moritz wrote: > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> Hi, > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> I would like to propose a patch release for Arrow JS. The release is > > >> > > >>> > > >> > > >>>>> forked > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> off of maint-6.0.x and available at > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> https://github.com/apache/arrow/tree/release-6.0.2-js. > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> The release contains two fixes for the js bundle: > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> ARROW-14773: [JS] Fix sourcemap paths > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> <https://github.com/apache/arrow/pull/11741> > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> ARROW-14774: [JS] Correct package exports > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> <https://github.com/apache/arrow/pull/11742> > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> [ ] +1 Release this as Apache Arrow JS 6.0.2 > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> [ ] +0 > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> [ ] -1 Do not release this as Apache Arrow JS 6.0.2 because... > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> Thank you, > > >> > > >>> > > >> > > >>>>> > > >> > > >>> > > >> > > >>>>> Dominik > > >> > > >>> > > >> > > >> > > >
