[
https://issues.apache.org/jira/browse/ARROW-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-3175:
--------------------------------
Summary: [Java] Upgrade to official FlatBuffers release (Flatbuffers
incompatibility) (was: Arrow Java: Upgrade to official FlatBuffers release
(Flatbuffers incompatibility))
> [Java] Upgrade to official FlatBuffers release (Flatbuffers incompatibility)
> ----------------------------------------------------------------------------
>
> Key: ARROW-3175
> URL: https://issues.apache.org/jira/browse/ARROW-3175
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
> Affects Versions: 0.10.0
> Reporter: Alex Black
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.11.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Arrow Java currently uses an unofficial flatbuffers dependency -
> com.vlkan:flatbuffers:
> [https://github.com/apache/arrow/blob/master/java/pom.xml#L481-L485]
> The likely motivation here is that previously, no Java flatbuffers
> implementation was available on maven central.
> [https://github.com/vy/flatbuffers]
> > Unfortunately, FlatBuffers project does not publish any artifacts to the
> Maven Central Repository
> However, this is no longer the case:
>
> [https://search.maven.org/search?q=g:com.google.flatbuffers%20AND%20a:flatbuffers-java&core=gav]
> The flatbuffers version used in Arrow java is a nearly 3-year-old snapshot,
> not even a version of an official release:
> [https://github.com/vy/flatbuffers#usage]
> The main problem is that this version of flatbuffers is not compatible with
> the official releases of flatbuffers.
> For example, we use the official flatbuffers releases in ND4J and
> Deeplearning4j: [https://github.com/deeplearning4j/deeplearning4j]
> Running Arrow with an official flatbuffers library on the classpath results
> in issues such as:
> {noformat}
> java.lang.NoSuchMethodError:
> com.google.flatbuffers.FlatBufferBuilder.createString(Ljava/lang/String;)I
> at org.apache.arrow.vector.types.pojo.Field.getField(Field.java:154)
> at org.apache.arrow.vector.types.pojo.Schema.getSchema(Schema.java:145)
> at
> org.apache.arrow.vector.ipc.message.MessageSerializer.serialize(MessageSerializer.java:124)
> at
> org.apache.arrow.vector.ipc.ArrowWriter.ensureStarted(ArrowWriter.java:136)
> at org.apache.arrow.vector.ipc.ArrowWriter.start(ArrowWriter.java:97)
> at FlatBuffersDependencyIssue.test(FlatBuffersDependencyIssue.java:56)
> {noformat}
>
> Simply excluding the com.vlkan:flatbuffers dependency in lieu of an official
> flatbuffers release is not a solution (same exception as above) and we aren't
> prepared to downgrade all of our projects to use the flatbuffers version that
> Arrow currently requires.
> Consequently, this is a major issue that prevents us using Arrow in our
> libraries.
> I have prepared a simple repository to reproduce this issue, if required:
> [https://github.com/AlexDBlack/arrowflatbufferstest]
> Is there a reason for using this particular version of flatbuffers, and if
> not, can Arrow java use an official release of flatbuffers instead?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
