[
https://issues.apache.org/jira/browse/BEAM-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040457#comment-16040457
]
Luke Cwik commented on BEAM-2418:
---------------------------------
The way in which you are building your jar file is broken since it includes
multiple copies of the same CoderProviderRegistrar file. Java does not
understand jar files which have multiple copies of the same file within it. In
this specific case the three
*META-INF/services/org.apache.beam.sdk.coders.CoderProviderRegistrar* should
have been concatenated together.
{code}
lcwik@lcwik0:~/beam2418$ jar tvf dataflow-teleport-1.0-Alpha.jar | grep
META-INF/services/org.apache.beam.sdk.coders.CoderProviderRegistrar
71 Fri May 12 17:03:24 PDT 2017
META-INF/services/org.apache.beam.sdk.coders.CoderProviderRegistrar
150 Fri May 12 16:56:14 PDT 2017
META-INF/services/org.apache.beam.sdk.coders.CoderProviderRegistrar
130 Fri May 12 17:03:38 PDT 2017
META-INF/services/org.apache.beam.sdk.coders.CoderProviderRegistrar
{code}
The culprit seems to be that the build file in your project is incorrectly
assembling the *uber* jar:
{code}
task uberjar(type: Jar) {
from files(sourceSets.main.output.classesDir)
from {configurations.compile.collect {zipTree(it)}} {
exclude "META-INF/*.SF"
exclude "META-INF/*.DSA"
exclude "META-INF/*.RSA"
}
manifest {
attributes 'Main-Class': mainClassName
}
}
{code}
Please take a look at the shadow gradle plugin and this section of their
documentation about merging resources (specifically 2.7.1. Merging Service
Descriptor Files):
http://imperceptiblethoughts.com/shadow/#controlling_jar_content_merging
> Datastore IO does not work out of the box
> -----------------------------------------
>
> Key: BEAM-2418
> URL: https://issues.apache.org/jira/browse/BEAM-2418
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-extensions, sdk-java-gcp
> Affects Versions: 2.0.0
> Reporter: Stephen Sisk
> Assignee: Vikas Kedigehalli
> Priority: Blocker
> Fix For: Not applicable
>
>
> We have user reports that DatastoreIO does not work when they try to use it.
> We believe this is a result of our effort to minimize our dependencies in the
> core SDK (protobuf in this case). ProtoCoder is not registered by default, so
> a user would need explicitly include 'beam-sdks-java-extensions-protobuf' in
> their maven dependencies to get it.
> We need to confirm it, but if so, we will probably need to fix this in the
> next release to have ProtoCoder when using DatastoreIO.
> cc [~vikasrk]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
