[
https://issues.apache.org/jira/browse/BEAM-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352587#comment-16352587
]
Jean-Baptiste Onofré edited comment on BEAM-3587 at 2/5/18 4:10 PM:
--------------------------------------------------------------------
As we have:
{code}
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${maven-shade-plugin.version}</version>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<shadedArtifactAttached>true</shadedArtifactAttached>
<shadedClassifierName>shaded</shadedClassifierName>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
{code}
for a Maven build, the same has to be applied to Gradle:
{code}
shadowJar {
transform(AppendingTransformer) {
resource = 'reference.conf'
}
relocate 'com.google.protobuf', 'shaded.protobuf'
mergeServiceFiles()
}
{code}
I'm adding a note in the documentation.
was (Author: jbonofre):
As we have:
{code}
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${maven-shade-plugin.version}</version>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<shadedArtifactAttached>true</shadedArtifactAttached>
<shadedClassifierName>shaded</shadedClassifierName>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
{code}
for a Maven build, the same has to be applied to Gradle.
I'm adding a note in the documentation.
> User reports TextIO failure in FlinkRunner on master
> ----------------------------------------------------
>
> Key: BEAM-3587
> URL: https://issues.apache.org/jira/browse/BEAM-3587
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Kenneth Knowles
> Assignee: Jean-Baptiste Onofré
> Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: screen1.png, screen2.png
>
>
> Reported here:
> [https://lists.apache.org/thread.html/47b16c94032392782505415e010970fd2a9480891c55c2f7b5de92bd@%3Cuser.beam.apache.org%3E]
> "I'm trying to run a pipeline containing just a TextIO.read() step on a Flink
> cluster, using the latest Beam git revision (ff37337). The job fails to start
> with the Exception:
> {{java.lang.UnsupportedOperationException: The transform is currently not
> supported.}}
> It does work with Beam 2.2.0 though. All code, logs, and reproduction steps
> [https://github.com/pelletier/beam-flink-example]";
> My initial thoughts: I have a guess that this has to do with switching to
> running from a portable pipeline representation, and it looks like there's a
> non-composite transform with an empty URN and it threw a bad error message.
> We can try to root cause but may also mitigate short-term by removing the
> round-trip through pipeline proto for now.
> What is curious is that the ValidatesRunner and WordCountIT are working -
> they only run on a local Flink, yet this seems to be a translation issue that
> would occur for local or distributed runs.
> We need to certainly run this repro on the RC if we don't totally get to the
> bottom of it quickly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
