Thank you kou! I appreciate the help. I'm happy to do whatever is required to facilitate the moving/donating process from JuliaData/Arrow.jl to apache/arrow-julia.
-Jacob On Mon, Apr 12, 2021 at 7:53 PM Sutou Kouhei <k...@clear-code.com> wrote: > Hi Jacob, > > I, a PMC member, talked to Kenta Murata, a commiter and a > Julia user, about this. > > We support that you and Julia folks work on > arrow/arrow-julia until we have enough PMC members from > Julia folks. For example, we'll help IP clearance process to > import the latest JuliaData/Arrow.js changes to apache/ and > we'll start voting on Julia package release. > > > Thanks, > -- > kou > > In <CAKyXBQo2=+NfS=hX6nEDPmMno-bSuXMR+=d-heskgn2mm57...@mail.gmail.com> > "Re: Status of Arrow Julia implementation?" on Sun, 11 Apr 2021 23:06:27 > -0600, > Jacob Quinn <quinn.jac...@gmail.com> wrote: > > > Micah/Wes, > > > > Yes, I've been following the rust proposal thread with great interest. I > do > > think that provides a great path forward: transferring the > > JuliaData/Arrow.jl repo to apache/arrow-julia would help to solve the > > "package history" technical challenges that in part led to the current > > setup and concerns. I think being able to utilize github issues would > also > > be great; as I've mentioned elsewhere, it's much more > traditional/expected > > in the Julia ecosystem. > > > > I think the package could retain an independent versioning scheme. The > >> additional process would be voting on release candidates. If the Julia > >> folks want to try again and move development to a new, Julia-specific > >> apache/* repository and apply the ASF governance to the project, the > >> Arrow PMC could probably fast-track making Jacob a committer. In some > >> code donations / IP clearance, the contributors for the donated code > >> become committers as part of the transaction. > >> > > > > These all sound great and would greatly facilitate a better integration > > under ASF governance. These points definitely resolve my main concerns. > > > > As I commented on the rust thread, I'm mostly interested in the future of > > integration testing for rust/julia if they are split out into separate > > repos. In the current Julia implementation, we have all the code to read > > arrow json, and I just hand-generated the integration test data and > > committed them in the repo itself, but it doesn't interface with other > > languages (just reads arrow json, produces arrow file, reads arrow file, > > compares w/ original arrow json). I'm happy to help work on the details > of > > what that looks like and pilot some solutions. I think with a solid > > inter-repo integration testing framework, we can keep a strong sync > between > > projects. > > > > -Jacob > > > > > > On Sun, Apr 11, 2021 at 5:08 PM Wes McKinney <wesmck...@gmail.com> > wrote: > > > >> On Sat, Apr 10, 2021 at 4:07 PM Micah Kornfield <emkornfi...@gmail.com> > >> wrote: > >> > > >> > > > >> > > Ok, I've had a chance to discuss with a few other Julia developers > and > >> > > review various options. I think it's best to drop the Julia code > from > >> the > >> > > physical apache/arrow repo. The extra overhead on development, > release > >> > > process, and user issue reporting and PR contributing are too much > in > >> > > addition to the technical challenges that we never resolved > involving > >> > > including the past Arrow.jl release version git trees in the > >> apache/arrow > >> > > repo. > >> > > >> > > >> > Hi Jacob, > >> > It seems you are on the new thread discussing a proposal for changing > >> > Rust's development model. Would the proposal [1] address most of > these > >> > concerns if Julia was set up in the same way? > >> > > >> > It seems in the short term the stickiest point would be committer > access > >> > to the new repos, and I suppose the release mechanics still might be > >> > challenging? > >> > >> I think the package could retain an independent versioning scheme. The > >> additional process would be voting on release candidates. If the Julia > >> folks want to try again and move development to a new, Julia-specific > >> apache/* repository and apply the ASF governance to the project, the > >> Arrow PMC could probably fast-track making Jacob a committer. In some > >> code donations / IP clearance, the contributors for the donated code > >> become committers as part of the transaction. > >> > >> > > >> > Thanks, > >> > Micah > >> > > >> > [1] > >> > > >> > https://docs.google.com/document/d/1TyrUP8_UWXqk97a8Hvb1d0UYWigch0HAephIjW7soSI/edit > >> > > >> > On Wed, Apr 7, 2021 at 4:17 AM Wes McKinney <wesmck...@gmail.com> > wrote: > >> > > >> > > I went back and read the mailing list discussions from September > about > >> > > the donation and I would say there was not a clear enough statement > >> > > from us about what the donation and IP clearance meant as far as the > >> > > future of the Julia codebase. This is partly our fault — we have > taken > >> > > in 9 other code donations over the last 5 years, and in all cases > the > >> > > developers understood that they were to move their process to the > >> > > Arrow repositories and communications channels. > >> > > > >> > > It did not occur to me at all that the code that you were putting in > >> > > the Arrow repository would get treated like a read-only fork that > you > >> > > update periodically. If I had realized that, we wouldn't be in this > >> > > situation. > >> > > > >> > > As a reminder about what Arrow and the ASF are all about: Community > >> > > over Code. We think that building a collaborative, open community > that > >> > > works and plans together in public, makes decisions based on > consensus > >> > > with clear meritocratic ("doers decide") governance is the best way > to > >> > > build this project. The concerns that you have around the timing and > >> > > frequency of releases for the Julia codebase are in my mind easy to > >> > > resolve, and if you had indicated that having a customized process > for > >> > > Julia releases was a condition for your joining the community > >> > > wholeheartedly, we would have been happy to help. I think that the > >> > > benefits of common CI/CD infrastructure and opportunities to build > >> > > deeper integrations between the Julia implementation and the other > >> > > implementations (imagine... Julia kernels running in DataFusion?) > >> > > would outweigh the sense of "loss of control" from developing > within a > >> > > larger project. > >> > > > >> > > On Wed, Apr 7, 2021 at 12:16 AM Jacob Quinn <quinn.jac...@gmail.com > > > >> > > wrote: > >> > > > > >> > > > Responses inline below: > >> > > > > >> > > > On Tue, Apr 6, 2021 at 9:46 PM Jorge Cardoso Leitão < > >> > > > jorgecarlei...@gmail.com> wrote: > >> > > > > >> > > > > Hi, > >> > > > > > >> > > > > > you all did not attempt to work in the community for any > >> meaningful > >> > > > > amount of time and > >> > > > > are choosing not to try based on the perception that it will > create > >> > > > > unacceptable overhead for you > >> > > > > > >> > > > > It is not self-evident to me that Julia's community was > >> sufficiently > >> > > > > informed about what they > >> > > > > had to give in in terms of process and release management when > >> merging > >> > > / > >> > > > > donating. > >> > > > > > >> > > > > >> > > > Yes, it was pretty unclear what the process was if we needed to do > >> any > >> > > kind > >> > > > of patch release. I know that has been sorted out better recently, > >> but > >> > > back > >> > > > in November, it didn't really seem like an option (i.e. > independent > >> > > > language patch releases). > >> > > > > >> > > > > >> > > > > IMO this is a plausible explanation as to why the donation was > >> made and > >> > > > > then later abandoned. > >> > > > > > >> > > > > > >> > > > I'll just note that the "abandonment" can only be a perception > from > >> the > >> > > > apache/arrow side of things, but as I mentioned above, I also > tried > >> to > >> > > > clearly state in the julia/Arrow/README that the development > process > >> > > would > >> > > > continue with the JuliaData/Arrow.jl repo as the main "dev" > branch, > >> with > >> > > > changes being upstreamed to the apache/arrow repo, which was > followed > >> > > > through, having an upstream of commits right before the 3.0.0 > >> release, > >> > > and > >> > > > I was planning on doing the same soon for the 4.0.0 release. That > is > >> to > >> > > > say, the Julia implementation has continued progressing forward > quite > >> > > > rapidly, IMO, but I can see that perhaps apache/arrow repo members > >> may > >> > > have > >> > > > viewed it as "abandoned". > >> > > > > >> > > > > >> > > > > I do not fully understand why the pain points Jacob mentioned > were > >> not > >> > > > > brought up to the mailing list sooner, though. > >> > > > > > >> > > > > >> > > > To be honest and frank, I didn't have pain points with the > >> development > >> > > > process I outlined when the code was donated and as stated in the > >> README. > >> > > > That was the process that made the donation possible and I > imagined > >> would > >> > > > work well going forward, and has, until this thread started and it > >> was > >> > > > pointed out that this process isn't viable. The pain points were > >> > > discussed > >> > > > with the initial code donation, but in my mind were resolved with > the > >> > > > development process that was decided upon. > >> > > > > >> > > > > >> > > > > This made us unable to potentially take corrective measures. I > >> think > >> > > that > >> > > > > this is why everyone was taken a bit by surprise with this. > >> > > > > > >> > > > > Best, > >> > > > > Jorge > >> > > > > > >> > > > > > >> > > > > On Fri, Apr 2, 2021 at 10:18 PM Wes McKinney < > wesmck...@gmail.com> > >> > > wrote: > >> > > > > > >> > > > > > hi Jacob — sorry to hear that. It's a bummer that you all did > not > >> > > > > > attempt to work in the community for any meaningful amount of > >> time > >> > > and > >> > > > > > are choosing not to try based on the perception that it will > >> create > >> > > > > > unacceptable overhead for you. I believe the benefits would > >> outweigh > >> > > > > > the costs, but I suppose we will have to agree to disagree. > >> > > > > > > >> > > > > > Can you prepare a pull request to do the requisite repository > >> > > surgery? > >> > > > > > I hope the development goes well in the future and look > forward > >> to > >> > > > > > seeing folks from the Julia ecosystem engaged here on growing > the > >> > > > > > Arrow ecosystem. > >> > > > > > > >> > > > > > Thanks, > >> > > > > > Wes > >> > > > > > > >> > > > > > On Fri, Apr 2, 2021 at 3:03 PM Jacob Quinn < > >> quinn.jac...@gmail.com> > >> > > > > wrote: > >> > > > > > > > >> > > > > > > Ok, I've had a chance to discuss with a few other Julia > >> developers > >> > > and > >> > > > > > > review various options. I think it's best to drop the Julia > >> code > >> > > from > >> > > > > the > >> > > > > > > physical apache/arrow repo. The extra overhead on > development, > >> > > release > >> > > > > > > process, and user issue reporting and PR contributing are > too > >> much > >> > > in > >> > > > > > > addition to the technical challenges that we never resolved > >> > > involving > >> > > > > > > including the past Arrow.jl release version git trees in the > >> > > > > apache/arrow > >> > > > > > > repo. > >> > > > > > > > >> > > > > > > We're still very much committed to working on the Julia > >> > > implementation > >> > > > > > and > >> > > > > > > participating in the broader arrow community. I've enjoyed > >> > > following > >> > > > > the > >> > > > > > > user/dev mailing lists and will continue to do so. We > monitor > >> > > format > >> > > > > > > proposals and try to implement new functionality as quickly > as > >> > > > > possible. > >> > > > > > We > >> > > > > > > got the initial arrow flight proto code generated just last > >> night > >> > > in > >> > > > > > fact. > >> > > > > > > I'd still like to explore official integration with the > archery > >> > > test > >> > > > > > suite > >> > > > > > > to solidify the Julia implementation with integration > tests; I > >> > > think > >> > > > > that > >> > > > > > > would be very valuable for long-term confidence in the > >> > > cross-language > >> > > > > > > support of the Julia implementation. > >> > > > > > > > >> > > > > > > We realize one of the main implications will probably be > >> dropping > >> > > Julia > >> > > > > > > from the list of "official implementations". We're > encouraged > >> by > >> > > the > >> > > > > many > >> > > > > > > users who have already started using the Julia > implementation > >> and > >> > > will > >> > > > > > > strive to maintain a high rate of issue responsiveness and > >> feature > >> > > > > > > development to maintain project confidence. If there's a > >> > > possibility of > >> > > > > > > being included somewhere as an "unofficial" or > "semi-official" > >> > > > > > > implementation, we'd love to still be bundled with the > broader > >> > > arrow > >> > > > > > > project somehow, like, for example, showing how Julia > >> integrates > >> > > with > >> > > > > the > >> > > > > > > archery test suite, once the work there is done. > >> > > > > > > > >> > > > > > > Best, > >> > > > > > > > >> > > > > > > -Jacob > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > On Tue, Mar 30, 2021 at 4:10 PM Wes McKinney < > >> wesmck...@gmail.com> > >> > > > > > wrote: > >> > > > > > > > >> > > > > > > > Also, on the issue that there are no Julia-focused PMC > >> members — > >> > > note > >> > > > > > > > that I helped the JavaScript folks make their own > independent > >> > > > > releases > >> > > > > > > > for quite a while: called the votes (e.g. [1]), helped get > >> > > people to > >> > > > > > > > verify and vote on the releases. After a time, it was > >> decided to > >> > > stop > >> > > > > > > > releasing independently because there wasn't enough > >> development > >> > > > > > > > activity to justify it. > >> > > > > > > > > >> > > > > > > > [1]: > >> > > https://www.mail-archive.com/dev@arrow.apache.org/msg05971.html > >> > > > > > > > > >> > > > > > > > On Tue, Mar 30, 2021 at 4:54 PM Wes McKinney < > >> > > wesmck...@gmail.com> > >> > > > > > wrote: > >> > > > > > > > > > >> > > > > > > > > hi Jacob, > >> > > > > > > > > > >> > > > > > > > > On Tue, Mar 30, 2021 at 4:18 PM Jacob Quinn < > >> > > > > quinn.jac...@gmail.com> > >> > > > > > > > wrote: > >> > > > > > > > > > > >> > > > > > > > > > I can comment as the primary apache arrow liaison for > the > >> > > > > Arrow.jl > >> > > > > > > > > > repository and original code donator. > >> > > > > > > > > > > >> > > > > > > > > > I apologize for the "surprise", but I commented a few > >> times > >> > > in > >> > > > > > various > >> > > > > > > > > > places and put a snippet in the README > >> > > > > > > > > > < > >> > > > > > > > > >> > > > > > > >> > > > > > >> > > > >> > https://github.com/apache/arrow/tree/master/julia/Arrow#difference-between-this-code-and-the-juliadataarrowjl-repository > >> > > > > > > > > > >> > > > > > > > > > about > >> > > > > > > > > > the approach I wanted to take w/ the Julia > >> implementation in > >> > > > > terms > >> > > > > > of > >> > > > > > > > > > keeping the JuliaData/Arrow.jl repository as a "dev > >> branch" > >> > > of > >> > > > > > sorts > >> > > > > > > > of the > >> > > > > > > > > > apache/arrow code, upstreaming changes periodically. > >> There's > >> > > > > even a > >> > > > > > > > script > >> > > > > > > > > > < > >> > > > > > > > > >> > > > > > > >> > > > > > >> > > > >> > https://github.com/JuliaData/Arrow.jl/blob/main/scripts/update_apache_arrow_code.jl > >> > > > > > > > > > >> > > > > > > > > > I wrote to mostly automate this upstreaming. I realize > >> now > >> > > that I > >> > > > > > > > didn't > >> > > > > > > > > > consider the "Arrow PMC" position on this kind of > setup > >> or > >> > > seek > >> > > > > to > >> > > > > > > > affirm > >> > > > > > > > > > that it would be ok to approach things like this. > >> > > > > > > > > > > >> > > > > > > > > > The reality is that Julia users are very engrained to > >> expect > >> > > > > Julia > >> > > > > > > > packages > >> > > > > > > > > > to live in a single stand-alone github repo, where > issues > >> > > can be > >> > > > > > > > opened, > >> > > > > > > > > > and pull requests are welcome. It was hard and still > is > >> hard > >> > > to > >> > > > > > imagine > >> > > > > > > > > > "turning that off", since I believe we would lose a > lot > >> of > >> > > > > > valuable bug > >> > > > > > > > > > reports and first-time contributions. This isn't > >> necessarily > >> > > any > >> > > > > > fault > >> > > > > > > > of > >> > > > > > > > > > how the bug report/contribution process is handled for > >> the > >> > > arrow > >> > > > > > > > project > >> > > > > > > > > > overall, though I'm also aware that there's a desire > to > >> make > >> > > it > >> > > > > > easier > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > < > >> > > > > > > > > >> > > > > > > >> > > > > > >> > > > >> > https://lists.apache.org/x/thread.html/r8817dfba08ef8daa210956db69d513fd27b7a751d28fb8f27e39cc7e@%3Cdev.arrow.apache.org%3E > >> > > > > > > > > > >> > > > > > > > > > and > >> > > > > > > > > > it currently requires more and different effort than > >> Julia > >> > > users > >> > > > > > are > >> > > > > > > > used > >> > > > > > > > > > to. I think it's more from how open, welcoming, and > how > >> > > strong > >> > > > > the > >> > > > > > > > culture > >> > > > > > > > > > is in Julia around encouraging community contributions > >> and > >> > > the > >> > > > > > tight > >> > > > > > > > > > integration with github and its open-source project > >> > > management > >> > > > > > tools. > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > Well, we are on track to having 1000 different people > >> > > contribute to > >> > > > > > > > > the project and have over 12,000 issues, so I don't > think > >> > > there is > >> > > > > > > > > evidence that we are failing to attract new > contributors or > >> > > that > >> > > > > > > > > feature requests / bugs aren't being reported. The way > >> that we > >> > > work > >> > > > > > is > >> > > > > > > > > _different_, so adapting to the Apache process will > require > >> > > change. > >> > > > > > > > > > >> > > > > > > > > > Additionally, I was and still am concerned about the > >> overall > >> > > > > > release > >> > > > > > > > > > process of the apache/arrow project. I know there have > >> been > >> > > > > efforts > >> > > > > > > > there > >> > > > > > > > > > as well to make it easier for individual languages to > >> > > release on > >> > > > > > their > >> > > > > > > > own > >> > > > > > > > > > cadence, but just anecdotally, the JuliaData/Arrow.jl > has > >> > > > > > > > had/needed/wanted > >> > > > > > > > > > 10 patch and minor releases since the original code > >> donation, > >> > > > > > whereas > >> > > > > > > > the > >> > > > > > > > > > apache/arrow project has had one (3.0.0). This leads > to > >> some > >> > > of > >> > > > > the > >> > > > > > > > > > concerns I have with restricting development to just > the > >> > > > > > apache/arrow > >> > > > > > > > > > repository: how exactly does the release process work > for > >> > > > > > individual > >> > > > > > > > > > languages who may desire independent releases apart > from > >> the > >> > > > > > quarterly > >> > > > > > > > > > overall project releases? I think from the Rust > thread I > >> > > remember > >> > > > > > that > >> > > > > > > > you > >> > > > > > > > > > just need a group of language contributors to all > agree, > >> but > >> > > what > >> > > > > > if > >> > > > > > > > I'm > >> > > > > > > > > > the only "active" Julia contributor? It's also unclear > >> what > >> > > the > >> > > > > > > > > > expectations are for actual development: with the > >> original > >> > > code > >> > > > > > > > donation > >> > > > > > > > > > PRs, I know Neal "reviewed" the PRs, but perhaps > missed > >> the > >> > > > > details > >> > > > > > > > around > >> > > > > > > > > > how I proposed development continue going forward. Is > it > >> > > required > >> > > > > > to > >> > > > > > > > have a > >> > > > > > > > > > certain number of reviews before merging? On the Julia > >> side, > >> > > I > >> > > > > can > >> > > > > > try > >> > > > > > > > to > >> > > > > > > > > > encourage/push for those who have contributed to the > >> > > > > > JuliaData/Arrow.jl > >> > > > > > > > > > repository to help review PRs to apache/arrow, but I > also > >> > > can't > >> > > > > > > > guarantee > >> > > > > > > > > > we would always have someone to review. It just feels > >> pretty > >> > > > > > awkward > >> > > > > > > > if I > >> > > > > > > > > > keep needing to ping non-Julia people to "review" a > PR to > >> > > merge > >> > > > > it. > >> > > > > > > > Perhaps > >> > > > > > > > > > this is just a problem of the overall Julia > >> implementation > >> > > > > > "smallness" > >> > > > > > > > in > >> > > > > > > > > > terms of contributors, but I'm not sure on the best > >> answer > >> > > here. > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > Several things here: > >> > > > > > > > > > >> > > > > > > > > * If you want to do separate Julia releases, you are > free > >> to do > >> > > > > that, > >> > > > > > > > > but you have to follow the process (voting on the > mailing > >> list, > >> > > > > > > > > publishing GPG-signed source artifacts) > >> > > > > > > > > * If you had been working "in the community" since > >> November, > >> > > you > >> > > > > > would > >> > > > > > > > > probably already be a committer, so there is a > >> bootstrapping > >> > > here > >> > > > > > that > >> > > > > > > > > has failed to take place. In the meantime, we are more > than > >> > > happy > >> > > > > to > >> > > > > > > > > help you "earn your wings" (as a committer) as quickly > as > >> > > possible. > >> > > > > > > > > But from my perspective, I see a code donation and two > >> other > >> > > > > commits, > >> > > > > > > > > which isn't enough to make a case for committership. > >> > > > > > > > > > >> > > > > > > > > > So in short, I'm not sure on the best path forward. I > >> think > >> > > > > > strictly > >> > > > > > > > > > restricting development to the apache/arrow physical > >> > > repository > >> > > > > > would > >> > > > > > > > > > actively hurt the progress of the Julia > implementation, > >> > > whereas > >> > > > > it > >> > > > > > > > *has* > >> > > > > > > > > > been progressing with increasing momentum since first > >> > > released. > >> > > > > > There > >> > > > > > > > are > >> > > > > > > > > > posts on the Julia discourse forum, in the Julia slack > >> and > >> > > zulip > >> > > > > > > > > > communities, and quite a few issues/PRs being opened > at > >> the > >> > > > > > > > > > JuliaData/Arrow.jl repository. There have been several > >> calls > >> > > for > >> > > > > > arrow > >> > > > > > > > > > flight support, with a member from Julia Computing > >> actually > >> > > close > >> > > > > > to > >> > > > > > > > > > releasing a gRPC client > >> > > > > > > > > > <https://github.com/JuliaComputing/gRPCClient.jl> > >> > > specifically > >> > > > > > > > > > to help with flight support. But in terms of actual > >> > > committers, > >> > > > > > it's > >> > > > > > > > been > >> > > > > > > > > > primarily just myself, with a few minor contributions > by > >> > > others. > >> > > > > > > > > > > >> > > > > > > > > > I guess the big question that comes to mind is what > are > >> the > >> > > hard > >> > > > > > > > > > requirements to be considered an "official > >> implementation"? > >> > > Does > >> > > > > > the > >> > > > > > > > code > >> > > > > > > > > > *have* to live in the same physical repo? Or if it > >> passed the > >> > > > > > series of > >> > > > > > > > > > archery integration tests, would that be enough? I > >> apologize > >> > > for > >> > > > > my > >> > > > > > > > > > naivete/inexperience on all things "apache", but I > >> imagine > >> > > that's > >> > > > > > a big > >> > > > > > > > > > part of it: having official development/releases > through > >> the > >> > > > > > > > apache/arrow > >> > > > > > > > > > community, though again I'm not exactly sure on the > >> formal > >> > > > > > processes > >> > > > > > > > here? > >> > > > > > > > > > I would like to keep Julia as an official > implementation, > >> > > but I'm > >> > > > > > also > >> > > > > > > > > > mostly carrying the maintainership alone at the moment > >> and > >> > > want > >> > > > > to > >> > > > > > be > >> > > > > > > > > > realistic with the future of the project. > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > The critical matter is whether the > development/maintenance > >> > > work is > >> > > > > > > > > conducted by the "Arrow community" in accordance with > the > >> > > Apache > >> > > > > Way, > >> > > > > > > > > which is to say individuals collaborating with each > other > >> on > >> > > Apache > >> > > > > > > > > channels (for communication and development) and > avoiding > >> the > >> > > bad > >> > > > > > > > > patterns you see sometimes in other communities (e.g. > >> > > inconsistent > >> > > > > > > > > openness). > >> > > > > > > > > > >> > > > > > > > > It's fine — really, no pressure — if you want to be > >> > > independent and > >> > > > > > do > >> > > > > > > > > things your own way, you just have to be clear that you > are > >> > > > > > > > > independent and not operating as part of the Apache > Arrow > >> > > > > community. > >> > > > > > > > > You can't have it both ways, though. No hard feelings > >> whatever > >> > > you > >> > > > > > > > > decide, but the current "dump code over the wall > >> occasionally" > >> > > > > > > > > approach but work on independent channels is not > >> compatible. > >> > > > > Building > >> > > > > > > > > healthy open source communities is hard, but this way > has > >> been > >> > > > > shown > >> > > > > > > > > to work well, which is why I've spent the last 6 years > >> working > >> > > hard > >> > > > > > to > >> > > > > > > > > bring people together to build this project and > ecosystem! > >> > > > > > > > > > >> > > > > > > > > If you want to maintain a test harness here to verify an > >> > > > > independent > >> > > > > > > > > Julia implementation, that's fine, too. I'm disappointed > >> that > >> > > > > things > >> > > > > > > > > failed to bootstrap after the code donation, so I want > to > >> see > >> > > if we > >> > > > > > > > > can course correct quickly or if not decide to go our > >> separate > >> > > > > ways. > >> > > > > > > > > > >> > > > > > > > > Thanks, > >> > > > > > > > > Wes > >> > > > > > > > > > >> > > > > > > > > > I'm open to discussion and ideas on the best way > forward. > >> > > > > > > > > > > >> > > > > > > > > > -Jacob > >> > > > > > > > > > > >> > > > > > > > > > On Tue, Mar 30, 2021 at 2:03 PM Wes McKinney < > >> > > > > wesmck...@gmail.com> > >> > > > > > > > wrote: > >> > > > > > > > > > > >> > > > > > > > > > > hi folks, > >> > > > > > > > > > > > >> > > > > > > > > > > I was very surprised today to learn that the Julia > >> Arrow > >> > > > > > > > > > > implementation has continued operating more or less > >> like an > >> > > > > > > > > > > independent open source project since the code > donation > >> > > last > >> > > > > > > > November: > >> > > > > > > > > > > > >> > > > > > > > > > > https://github.com/JuliaData/Arrow.jl/commits/main > >> > > > > > > > > > > > >> > > > > > > > > > > There may have been a misunderstanding about what > was > >> > > expected > >> > > > > to > >> > > > > > > > > > > occur after the code donation, but it's problematic > >> for a > >> > > bunch > >> > > > > > of > >> > > > > > > > > > > reasons (IP lineage / governance / community > >> development) > >> > > to > >> > > > > have > >> > > > > > > > work > >> > > > > > > > > > > happening on the implementation "outside the > >> community". > >> > > > > > > > > > > > >> > > > > > > > > > > In any case, what is done is done, so the Arrow > PMC's > >> > > position > >> > > > > on > >> > > > > > > > this > >> > > > > > > > > > > would be roughly to regard the work as a hard fork > of > >> > > what's in > >> > > > > > > > Apache > >> > > > > > > > > > > Arrow, which given its development activity is more > or > >> less > >> > > > > > inactive > >> > > > > > > > > > > [1]. (I had actually thought the project was simply > >> > > inactive > >> > > > > > after > >> > > > > > > > the > >> > > > > > > > > > > code donation) > >> > > > > > > > > > > > >> > > > > > > > > > > The critical question now is, is there interest from > >> Julia > >> > > > > > developers > >> > > > > > > > > > > in working "in the community", which is to say: > >> > > > > > > > > > > > >> > > > > > > > > > > * Having development discussions on ASF channels > >> (mailing > >> > > list, > >> > > > > > > > > > > GitHub, JIRA), planning and communicating in the > open > >> > > > > > > > > > > * Doing all development in ASF GitHub repositories > >> > > > > > > > > > > > >> > > > > > > > > > > The answer to the question may be "no" (which is > okay), > >> > > but if > >> > > > > > that's > >> > > > > > > > > > > the case, I don't think we should be giving the > >> impression > >> > > that > >> > > > > > we > >> > > > > > > > > > > have an official Julia implementation that is > >> developed and > >> > > > > > > > maintained > >> > > > > > > > > > > by the community (and so my argument would be > >> > > unfortunately to > >> > > > > > drop > >> > > > > > > > > > > the donated code from the project). > >> > > > > > > > > > > > >> > > > > > > > > > > If the answer is "yes", there needs to be a hard > >> > > commitment to > >> > > > > > move > >> > > > > > > > > > > development to Apache channels and not look back. We > >> would > >> > > also > >> > > > > > need > >> > > > > > > > > > > to figure out what to do to document and synchronize > >> the > >> > > new IP > >> > > > > > > > that's > >> > > > > > > > > > > been created since the code donation. > >> > > > > > > > > > > > >> > > > > > > > > > > Thanks, > >> > > > > > > > > > > Wes > >> > > > > > > > > > > > >> > > > > > > > > > > [1]: > >> > > > > https://github.com/apache/arrow/commits/master/julia/Arrow > >> > > > > > > > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > >> > > > >> >
