[
https://issues.apache.org/jira/browse/FLINK-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15114332#comment-15114332
]
ASF GitHub Bot commented on FLINK-3271:
---------------------------------------
GitHub user abhishekagarwal87 opened a pull request:
https://github.com/apache/flink/pull/1543
FLINK-3271: Include jetty-util in the dist jar
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/abhishekagarwal87/flink work
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1543.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1543
----
commit 679318fef7ea8f63f7744c522e3ded7c68983615
Author: Abhishek Agarwal <[email protected]>
Date: 2016-01-24T14:52:45Z
FLINK-3271: Include jetty-util in the dist jar
----
> Using webhdfs in a flink topology throws classnotfound exception
> ----------------------------------------------------------------
>
> Key: FLINK-3271
> URL: https://issues.apache.org/jira/browse/FLINK-3271
> Project: Flink
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.10.1
> Reporter: Abhishek Agarwal
> Assignee: Abhishek Agarwal
>
> I was just trying to run a storm topology on flink using flink-storm. I got
> this exception -
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/mortbay/util/ajax/JSON
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.jsonParse(WebHdfsFileSystem.java:325)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$FsPathResponseRunner.getResponse(WebHdfsFileSystem.java:727)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:610)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:458)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:487)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:483)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.listStatus(WebHdfsFileSystem.java:1277)
> {noformat}
> My topology list some files on hdfs using webhdfs API.
> org.mortbay.util.ajax.JSON was included in the application uber jar. I
> noticed that flink loads the application jar in a child classloader. This is
> what most likely happened -
> 1. WebHdfsFileSystem class was loaded through parent class loader since it is
> included in flink-dist.jar.
> 2. WebHdfsFileSystem has reference to the org.mortbay.util.ajax.JSON but
> since it is loaded through parent class loader, WebHdfsFileSystem can't read
> a class in child class loader.
> Ideally all the referenced classes should be available in the distribution
> jar so that these sort of issues may not occur.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
