[
https://issues.apache.org/jira/browse/FLINK-11085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16712850#comment-16712850
]
ASF GitHub Bot commented on FLINK-11085:
----------------------------------------
zentol opened a new pull request #7261: [FLINK-11085][s3] Fix inclusion of
presto hadoop classes
URL: https://github.com/apache/flink/pull/7261
## What is the purpose of the change
This PR fixes the bundling of hadoop classes from
`com.facebook.presto.hadoop:hadoop-apache2` in the presto-s3 filesystem module.
As of right now I do not really know why this fixes the issue, but intend to
file an issue with MSHADE.
## Brief change log
* reorder filter definitions
## Verifying this change
Manually verified. Ensure that
`org\apache\flink\fs\s3presto\shaded\com\facebook\presto\hadoop\` is contained
in the jar, and no other contents from this jar are included (for example,
`nativelib`).
Optionally you can also run the presto e2e test if you also include the
commits from https://github.com/zentol/flink/tree/11100.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> NoClassDefFoundError in presto-s3 filesystem
> --------------------------------------------
>
> Key: FLINK-11085
> URL: https://issues.apache.org/jira/browse/FLINK-11085
> Project: Flink
> Issue Type: Bug
> Components: FileSystem
> Affects Versions: 1.7.0
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Priority: Blocker
> Labels: pull-request-available
> Attachments: image-2018-12-07-13-04-06-563.png
>
>
> A user has reporter an issue on the ML where using the presto-s3 filesystem
> fails with an exception due to a missing class. The missing class is indeed
> filtered out in the shade-plugin configuration.
> {code:java}
> java.lang.NoClassDefFoundError:
> org/apache/flink/fs/s3presto/shaded/com/facebook/presto/hadoop/HadoopFileStatus
> at
> org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.directory(PrestoS3FileSystem.java:446)
> at
> org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.delete(PrestoS3FileSystem.java:423)
> at
> org.apache.flink.fs.s3.common.hadoop.HadoopFileSystem.delete(HadoopFileSystem.java:147)
> at
> org.apache.flink.runtime.state.filesystem.FileStateHandle.discardState(FileStateHandle.java:80)
> at
> org.apache.flink.runtime.checkpoint.CompletedCheckpoint.doDiscard(CompletedCheckpoint.java:250)
> at
> org.apache.flink.runtime.checkpoint.CompletedCheckpoint.discardOnSubsume(CompletedCheckpoint.java:219)
> at
> org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore.addCheckpoint(StandaloneCompletedCheckpointStore.java:72)
> at
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:844)
> at
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveAcknowledgeMessage(CheckpointCoordinator.java:756)
> at
> org.apache.flink.runtime.jobmaster.JobMaster.lambda$acknowledgeCheckpoint$8(JobMaster.java:680)
> at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
> at
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
