[jira] [Commented] (FLINK-1438) ClassCastException for Custom InputSplit in local mode and invalid type code in distributed mode
[
https://issues.apache.org/jira/browse/FLINK-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310891#comment-14310891
]
Robert Metzger commented on FLINK-1438:
---
We should add this also to release-0.8 in my optinion
> ClassCastException for Custom InputSplit in local mode and invalid type code
> in distributed mode
>
>
> Key: FLINK-1438
> URL: https://issues.apache.org/jira/browse/FLINK-1438
> Project: Flink
> Issue Type: Bug
> Components: JobManager
>Affects Versions: 0.8, 0.9
>Reporter: Fabian Hueske
>Assignee: Stephan Ewen
>Priority: Minor
> Fix For: 0.9
>
>
> Jobs with custom InputSplits fail with a ClassCastException such as
> {{org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit
> cannot be cast to
> org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit}}
> if executed on a local setup.
> This issue is probably related to different ClassLoaders used by the
> JobManager when InputSplits are generated and when they are handed to the
> InputFormat by the TaskManager. Moving the class of the custom InputSplit
> into the {{./lib}} folder and removing it from the job's makes the job work.
> To reproduce the bug, run the following job on a local setup.
> {code}
> public class CustomSplitTestJob {
> public static void main(String[] args) throws Exception {
> ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
> DataSet x = env.createInput(new TestFileInputFormat());
> x.print();
> env.execute();
> }
> public static class TestFileInputFormat implements
> InputFormat {
> @Override
> public void configure(Configuration parameters) {
> }
> @Override
> public BaseStatistics getStatistics(BaseStatistics
> cachedStatistics) throws IOException {
> return null;
> }
> @Override
> public TestFileInputSplit[] createInputSplits(int minNumSplits)
> throws IOException {
> return new TestFileInputSplit[]{new
> TestFileInputSplit()};
> }
> @Override
> public InputSplitAssigner
> getInputSplitAssigner(TestFileInputSplit[] inputSplits) {
> return new LocatableInputSplitAssigner(inputSplits);
> }
> @Override
> public void open(TestFileInputSplit split) throws IOException {
> }
> @Override
> public boolean reachedEnd() throws IOException {
> return false;
> }
> @Override
> public String nextRecord(String reuse) throws IOException {
> return null;
> }
> @Override
> public void close() throws IOException {
> }
> }
> public static class TestFileInputSplit extends FileInputSplit {
> }
> }
> {code}
> The same happens in distributed mode just that Akka terminates the
> transmission of the input split with a meaningless {{invalid type code: 00}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1438) ClassCastException for Custom InputSplit in local mode and invalid type code in distributed mode
[
https://issues.apache.org/jira/browse/FLINK-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14299842#comment-14299842
]
Till Rohrmann commented on FLINK-1438:
--
Hard to say without further information about the issue 1390.
> ClassCastException for Custom InputSplit in local mode and invalid type code
> in distributed mode
>
>
> Key: FLINK-1438
> URL: https://issues.apache.org/jira/browse/FLINK-1438
> Project: Flink
> Issue Type: Bug
> Components: JobManager
>Affects Versions: 0.8, 0.9
>Reporter: Fabian Hueske
>Priority: Minor
>
> Jobs with custom InputSplits fail with a ClassCastException such as
> {{org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit
> cannot be cast to
> org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit}}
> if executed on a local setup.
> This issue is probably related to different ClassLoaders used by the
> JobManager when InputSplits are generated and when they are handed to the
> InputFormat by the TaskManager. Moving the class of the custom InputSplit
> into the {{./lib}} folder and removing it from the job's makes the job work.
> To reproduce the bug, run the following job on a local setup.
> {code}
> public class CustomSplitTestJob {
> public static void main(String[] args) throws Exception {
> ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
> DataSet x = env.createInput(new TestFileInputFormat());
> x.print();
> env.execute();
> }
> public static class TestFileInputFormat implements
> InputFormat {
> @Override
> public void configure(Configuration parameters) {
> }
> @Override
> public BaseStatistics getStatistics(BaseStatistics
> cachedStatistics) throws IOException {
> return null;
> }
> @Override
> public TestFileInputSplit[] createInputSplits(int minNumSplits)
> throws IOException {
> return new TestFileInputSplit[]{new
> TestFileInputSplit()};
> }
> @Override
> public InputSplitAssigner
> getInputSplitAssigner(TestFileInputSplit[] inputSplits) {
> return new LocatableInputSplitAssigner(inputSplits);
> }
> @Override
> public void open(TestFileInputSplit split) throws IOException {
> }
> @Override
> public boolean reachedEnd() throws IOException {
> return false;
> }
> @Override
> public String nextRecord(String reuse) throws IOException {
> return null;
> }
> @Override
> public void close() throws IOException {
> }
> }
> public static class TestFileInputSplit extends FileInputSplit {
> }
> }
> {code}
> The same happens in distributed mode just that Akka terminates the
> transmission of the input split with a meaningless {{invalid type code: 00}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1438) ClassCastException for Custom InputSplit in local mode and invalid type code in distributed mode
[
https://issues.apache.org/jira/browse/FLINK-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14299802#comment-14299802
]
Robert Metzger commented on FLINK-1438:
---
Do you think this is related? https://issues.apache.org/jira/browse/FLINK-1390
> ClassCastException for Custom InputSplit in local mode and invalid type code
> in distributed mode
>
>
> Key: FLINK-1438
> URL: https://issues.apache.org/jira/browse/FLINK-1438
> Project: Flink
> Issue Type: Bug
> Components: JobManager
>Affects Versions: 0.8, 0.9
>Reporter: Fabian Hueske
>Priority: Minor
>
> Jobs with custom InputSplits fail with a ClassCastException such as
> {{org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit
> cannot be cast to
> org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit}}
> if executed on a local setup.
> This issue is probably related to different ClassLoaders used by the
> JobManager when InputSplits are generated and when they are handed to the
> InputFormat by the TaskManager. Moving the class of the custom InputSplit
> into the {{./lib}} folder and removing it from the job's makes the job work.
> To reproduce the bug, run the following job on a local setup.
> {code}
> public class CustomSplitTestJob {
> public static void main(String[] args) throws Exception {
> ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
> DataSet x = env.createInput(new TestFileInputFormat());
> x.print();
> env.execute();
> }
> public static class TestFileInputFormat implements
> InputFormat {
> @Override
> public void configure(Configuration parameters) {
> }
> @Override
> public BaseStatistics getStatistics(BaseStatistics
> cachedStatistics) throws IOException {
> return null;
> }
> @Override
> public TestFileInputSplit[] createInputSplits(int minNumSplits)
> throws IOException {
> return new TestFileInputSplit[]{new
> TestFileInputSplit()};
> }
> @Override
> public InputSplitAssigner
> getInputSplitAssigner(TestFileInputSplit[] inputSplits) {
> return new LocatableInputSplitAssigner(inputSplits);
> }
> @Override
> public void open(TestFileInputSplit split) throws IOException {
> }
> @Override
> public boolean reachedEnd() throws IOException {
> return false;
> }
> @Override
> public String nextRecord(String reuse) throws IOException {
> return null;
> }
> @Override
> public void close() throws IOException {
> }
> }
> public static class TestFileInputSplit extends FileInputSplit {
> }
> }
> {code}
> The same happens in distributed mode just that Akka terminates the
> transmission of the input split with a meaningless {{invalid type code: 00}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
