> Extract go/ in apache/arrow to apache/arrow-go like apache/arrow-rs I am not go maintainer and would defer to Matt and Joel on whether or not this is more work from their end, but it does seem like Go users have been forced to change their import paths quite a lot already and would probably be OK with this update given that it will mean they will have to update it a lot less in the future. There is quite a lot to draw on from arrow-rs, arrow-adbc, arrow-julia, and arrow-nanoarrow such that the initial cost of the CI/release infrastructure is probably not as substantial as it once was (I am happy to help). I have very much enjoyed working in apache/arrow-nanoarrow and apache/arrow-adbc and have heard anecdotal comments from contributors that it has been enjoyable for them as well.
> Prepare integration test CI like apache/arrow-rs does: > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml We just added something similar in arrow-nanoarrow based on the arrow-rs setup as well [1]. > Prepare release script based on apache/arrow-julia, > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql I would defer to Raul and Jacob to corroborate this, but because changes to the CI configuration and release verification scripts don't affect other implementations, I have been able to maintain that infrastructure myself without too much effort and don't have to lean on them for anything except reviews. [1] https://github.com/apache/arrow-nanoarrow/pull/557 On Thu, Jul 18, 2024 at 7:53 PM Matt Topol <zotthewiz...@gmail.com> wrote: > > Part of the goal of splitting out the release processes is that we'd be > able to do minor version releases more frequently instead of major version > releases. > > The general convention in the Go community is to include a major version > "v#" in the import path for all major versions past v1 so that if there's a > breaking change, it's explicit and prevents potential issues from different > major versions being used simultaneously. Being able to do minor version > releases more frequently would lead to not having to change the import > paths every 3-6 months, but only if we actually do a breaking change. > > On Thu, Jul 18, 2024, 3:55 PM George Godik <ggo...@gmail.com> wrote: > > > > If we shift the Go lib to a new/different import > > path we'll end up with the same problem where people will rely on older > > versions and an incorrect path. > > > > Major version upgrades already require changing the import paths by > > increasing the version. The proposed change would require everyone to go > > through a similar process one last time. > > > > > More to the point, there would be the question of whether or not we > > should port over the same major version > > number, i.e. `github.com/apache/arrow-go/v17` > > <http://github.com/apache/arrow-go/v17> > > <http://github.com/apache/arrow-go/v17> or something to that end? Or > > do we restart back at v1 (which I think would be confusing)? > > > > My vote - for whatever it's worth - would be to do away with the > > version-in-path naming convention and relying on the go version/package > > system for major upgrades. > > > > Benefits: I don't have to change import paths every 3-6months > > > > On Thu, Jul 18, 2024 at 3:34 PM Matt Topol <zotthewiz...@gmail.com> wrote: > > > > > My thoughts: > > > > > > > * Go doesn't depend on other components such as C++ > > > > * Go has some active PMC member (Matt) and committer (Joel) > > > > * Could you become a release manager for Go? > > > > > > I'd happily be the release manager for the Go implementation. > > > > > > > Here is my idea how to proceed this: > > > > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like > > > > apache/arrow-rs > > > > * Filter go/ related commits from apache/arrow and create > > > > apache/arrow-go with them like we did for apache/arrow-rs > > > > * Remove go/ related codes from apache/arrow > > > > 2. Prepare integration test CI like apache/arrow-rs does: > > > > > > > > > > > > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml > > > > 3. Prepare release script based on apache/arrow-julia, > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql > > > > > > Personally I would prefer that we do not extract it to its own separate > > > repository purely because I don't want to change the import path for > > users > > > again. We already have this issue from before we introduced the major > > > version into the import path and shifted it up to allow for the Parquet > > lib > > > in the same repository. If you look at [1] you see that there's still > > over > > > 100 projects that never upgraded to v6 or higher because they are still > > > using the old import path. If we shift the Go lib to a new/different > > import > > > path we'll end up with the same problem where people will rely on older > > > versions and an incorrect path. > > > > > > If we as a community decide that splitting out the implementations all > > into > > > separate repositories is the best way forward, I won't hold it up by > > > strictly hammering on this. I'm just concerned about the realities and > > > difficulties of communicating the import path change, ensuring we don't > > > break any consumers, and ensuring that users still end up being able to > > > upgrade easily. > > > > > > > The import path could be "github.com/apache/arrow-go" instead of " > > > github.com/apache/arrow-go/arrow". Since go will allow users to use > > > `arrow.Abc` directly if user imports `github.com/apache/arrow-go` > > <http://github.com/apache/arrow-go> > > > <http://github.com/apache/arrow-go> > > > <http://github.com/apache/arrow-go>. > > > > > > The import path would still have to be ` > > github.com/apache/arrow-go/arrow` > > <http://github.com/apache/arrow-go/arrow> > > > <http://github.com/apache/arrow-go/arrow> > > > since it would also contain the parquet implementation in ` > > > github.com/apache/arrow-go/parquet` > > <http://github.com/apache/arrow-go/parquet> > > > <http://github.com/apache/arrow-go/parquet>. More to the point, there > > > would be the > > > question of whether or not we should port over the same major version > > > number, i.e. `github.com/apache/arrow-go/v17` > > <http://github.com/apache/arrow-go/v17> > > > <http://github.com/apache/arrow-go/v17> or something to that end? Or > > > do we restart back at v1 (which I think would be confusing)? > > > > > > --Matt > > > > > > [1]: https://pkg.go.dev/github.com/apache/arrow/go/arrow > > > > > > On Thu, Jul 18, 2024 at 7:33 AM Antoine Pitrou <anto...@python.org> > > wrote: > > > > > > > > > > > Hi Kou, > > > > > > > > Le 18/07/2024 à 11:33, Sutou Kouhei a écrit : > > > > > > > > > > Here is my idea how to proceed this: > > > > > > > > > > 1. Extract go/ in apache/arrow to apache/arrow-go like > > > > > apache/arrow-rs > > > > > * Filter go/ related commits from apache/arrow and create > > > > > apache/arrow-go with them like we did for apache/arrow-rs > > > > > * Remove go/ related codes from apache/arrow > > > > > 2. Prepare integration test CI like apache/arrow-rs does: > > > > > > > > > > > > > > https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml > > > > > 3. Prepare release script based on apache/arrow-julia, > > > > > apache/arrow-adbc and/or apache/arrow-flight-sql-postgresql > > > > > > > > I think this is a good idea, but I'm not part of the Go maintainers. > > > > > > > > > Cons of this idea: > > > > > > > > > > * This is a backward incompatible change > > > > > * Users need to change their "import" to > > > > > "github.com/apache/arrow-go/arrow" from > > > > > "github.com/apache/arrow/go/arrow" > > > > > > > > Is there no way to leave some kind of alias or redirection in the > > > > apache/arrow repository? > > > > > > > > Regards > > > > > > > > Antoine. > > > > > > > > >
