[
https://issues.apache.org/jira/browse/FLINK-11085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718766#comment-16718766
]
ASF GitHub Bot commented on FLINK-11085:
----------------------------------------
zentol commented on issue #7261: [FLINK-11085][s3] Fix inclusion of presto
hadoop classes
URL: https://github.com/apache/flink/pull/7261#issuecomment-446544953
I have confirmed by theory and opened MSHADE-305. In the meantime we can
simply disable the merging of filters, and instead default to appending them.
Only `flink-fs-s3-presto` should be affected by this in a significant way,
as it is the only module that used inclusion filters, leaving room for an
exclusion filter to be inherited that impacts the packaging.
For the remaining modules,
* if no filter was defined the module was not affected at all as the parent
filters are inherited as is
* if a filter was defined all parent filters were disabled. They were
neither appended, nor could be merged as both the parent and child filters
defined exclusions, and these (apparently) are overridden by children by
default.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> NoClassDefFoundError in presto-s3 filesystem
> --------------------------------------------
>
> Key: FLINK-11085
> URL: https://issues.apache.org/jira/browse/FLINK-11085
> Project: Flink
> Issue Type: Bug
> Components: FileSystem
> Affects Versions: 1.7.0
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.8.0, 1.7.1
>
> Attachments: image-2018-12-07-13-04-06-563.png
>
>
> A user has reporter an issue on the ML where using the presto-s3 filesystem
> fails with an exception due to a missing class. The missing class is indeed
> filtered out in the shade-plugin configuration.
> {code:java}
> java.lang.NoClassDefFoundError:
> org/apache/flink/fs/s3presto/shaded/com/facebook/presto/hadoop/HadoopFileStatus
> at
> org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.directory(PrestoS3FileSystem.java:446)
> at
> org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.delete(PrestoS3FileSystem.java:423)
> at
> org.apache.flink.fs.s3.common.hadoop.HadoopFileSystem.delete(HadoopFileSystem.java:147)
> at
> org.apache.flink.runtime.state.filesystem.FileStateHandle.discardState(FileStateHandle.java:80)
> at
> org.apache.flink.runtime.checkpoint.CompletedCheckpoint.doDiscard(CompletedCheckpoint.java:250)
> at
> org.apache.flink.runtime.checkpoint.CompletedCheckpoint.discardOnSubsume(CompletedCheckpoint.java:219)
> at
> org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore.addCheckpoint(StandaloneCompletedCheckpointStore.java:72)
> at
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:844)
> at
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveAcknowledgeMessage(CheckpointCoordinator.java:756)
> at
> org.apache.flink.runtime.jobmaster.JobMaster.lambda$acknowledgeCheckpoint$8(JobMaster.java:680)
> at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
> at
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
