+1, for the new repo. The name sounds fine, but good if we have scope of having “hadoop-” prefix, we have that for almost all of the subprojects/modules
Can hadoop-shims or hadoop-shims-api or something on similar lines work? -Ayush > On 01-Jun-2023, at 1:18 AM, Steve Loughran <ste...@cloudera.com.invalid> > wrote: > > I want to create a new repository to put a shim library to allow previous > releases to access the more recent hadoop filesystem APIs -currently the > open source implementations of parquet, ORC can't use vectored io, in > particular, even though we can in Cloudera. Providing a shim opens them up > to all *and* gets the APIs more broadly stressed/tested. > This needs to be in its own repository, not just for rapid initial release, > but because it is designed to be built as old a version of hadoop we can > reasonably support, which IMO means hadoop 3.1.0+. I know parquet still > wants to build against 2.8.x, but to claim support for hadoop 2 means > "build and test on java7", which is unrealistic in 2023. > > Initial WiP implementation, which works with 3.1.0 and tests against others > https://github.com/steveloughran/fs-api-shim > the complexity is about testing this -I have contract tests which then need > to be executed on every supported hadoop release, which will need a > separate module for each one. > > I can create the repo easily enough, just would like approval. And is the > name OK? > > steve --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
