I apologize for leaving this hanging, but it looks like 4.0 is leaving the station :( Yes, it makes sense to bump it to 1.12, but you can't do that in isolation, because the flatc binary which is fetched as a Maven dependency is only available for 1.9. I will get back onto this and finish it this week.
FWIW, I was looking around and catalogued the various ways of generating flatbuffers for all the languages--you can look at it in my branch: https://github.com/bobtins/arrow/tree/check-in-gen-code/java/dev Let me know if any info is wrong or missing. The methods of generation are all over the map, and some have no script or build file, just doc. Would there be any value in making this more uniform? On 2021/04/14 16:36:47, Nate Bauernfeind <natebauernfe...@deephaven.io> wrote: > It would also be nice to upgrade that java flatbuffer version from 1.9 to > 1.12. Is anyone planning on doing this work (as listed in ARROW-12111)? > > If I did this work today, might it be possible to get it included in the > 4.0.0 release? > > On Fri, Mar 26, 2021 at 3:25 PM bobtins <bobti...@gmail.com> wrote: > > > OK, originally this was part of > > https://issues.apache.org/jira/browse/ARROW-12006 and I was going to just > > add some doc on flatc, but I will make this a new bug because it's a little > > bigger: https://issues.apache.org/jira/browse/ARROW-12111 > > > > On 2021/03/23 23:40:50, Micah Kornfield <emkornfi...@gmail.com> wrote: > > > > > > > > I have a concern, though. Four other languages (Java would be five) > > check > > > > in the generated flatbuffers code, and it appears (based on a quick > > scan of > > > > Git logs) that this is done manually. Is there a danger that the binary > > > > format could change, but some language might get forgotten, and thus be > > > > working with the old format? > > > > > > The format changes relatively slowly and any changes at this point should > > > be backwards compatible. > > > > > > > > > > > > > Or is there enough interop testing that the problem would get caught > > right > > > > away? > > > > > > In most cases I would expect integration tests to catch these types of > > > error. > > > > > > On Tue, Mar 23, 2021 at 4:26 PM bobtins <bobti...@gmail.com> wrote: > > > > > > > I'm happy to check in the generated Java source. I would also update > > the > > > > Java build info to reflect this change and document how to regenerate > > the > > > > source as needed. > > > > > > > > I have a concern, though. Four other languages (Java would be five) > > check > > > > in the generated flatbuffers code, and it appears (based on a quick > > scan of > > > > Git logs) that this is done manually. Is there a danger that the binary > > > > format could change, but some language might get forgotten, and thus be > > > > working with the old format? Or is there enough interop testing that > > the > > > > problem would get caught right away? > > > > > > > > I'm new to the project and don't know how big of an issue this is in > > > > practice. Thanks for any enlightenment. > > > > > > > > On 2021/03/23 07:39:16, Micah Kornfield <emkornfi...@gmail.com> wrote: > > > > > I think checking in the java files is fine and probably better then > > > > relying > > > > > on a third party package. We should make sure there are > > instructions on > > > > > how to regenerate them along with the PR > > > > > > > > > > On Monday, March 22, 2021, Antoine Pitrou <anto...@python.org> > > wrote: > > > > > > > > > > > > > > > > > Le 22/03/2021 à 20:17, bobtins a écrit : > > > > > > > > > > > >> TL;DR: The Java implementation doesn't have generated flatbuffers > > code > > > > > >> under source control, and the code generation depends on an > > > > > >> unofficially-maintained Maven artifact. Other language > > > > implementations do > > > > > >> check in the generated code; would it make sense for this to be > > done > > > > for > > > > > >> Java as well? > > > > > >> > > > > > >> I'm currently focusing on Java development; I started building on > > > > Windows > > > > > >> and got a failure under java/format, because I couldn't download > > the > > > > > >> flatbuffers compiler (flatc) to generate Java source. > > > > > >> The artifact for the flatc binary is provided "unofficially" (not > > by > > > > the > > > > > >> flatbuffers project), and there was no Windows version, so I had > > to > > > > jump > > > > > >> through hoops to build it and proceed. > > > > > >> > > > > > > > > > > > > While this does not answer the more general question of checking > > in the > > > > > > generated Flatbuffers code (which sounds like a good idea, but I'm > > not > > > > a > > > > > > Java developer), note that you could workaround this by installing > > the > > > > > > Conda-provided flatbuffers package: > > > > > > > > > > > > $ conda install flatbuffers > > > > > > > > > > > > which should get you the `flatc` compiler, even on Windows. > > > > > > (see https://docs.conda.io/projects/conda/en/latest/ for > > installing > > > > conda) > > > > > > > > > > > > You may also try other package managers such as Chocolatey: > > > > > > > > > > > > https://chocolatey.org/packages/flatc > > > > > > > > > > > > Regards > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > > > -- >
