[jira] [Commented] (FLINK-1463) RuntimeStatefulSerializerFactory declares ClassLoader as transient but later tries to use it
[ https://issues.apache.org/jira/browse/FLINK-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298798#comment-14298798 ] ASF GitHub Bot commented on FLINK-1463: --- GitHub user aljoscha opened a pull request: https://github.com/apache/flink/pull/353 [FLINK-1463] Fix stateful/stateless Serializers and Comparators Before, Serializers would announce whether they are stateful or not and rely on RuntimeStatefulSerializerFactory to do the duplication. Comparators, on the other hand, had a duplicate method that the user was required to call. This commit removes the statful/stateless property from Serializers but instead introduces a duplicate() method, similar to Comparators, that can return the same instance. The two serializer factories are merged into one that always calls duplicate() before returning a serializer. You can merge this pull request into a Git repository by running: $ git pull https://github.com/aljoscha/flink serializer-factories-fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/353.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #353 commit 91834fee239b372b9a39f3c8f89ecbe42e2ae23a Author: Aljoscha Krettek aljoscha.kret...@gmail.com Date: 2015-01-30T15:43:31Z [FLINK-1463] Fix stateful/stateless Serializers and Comparators Before, Serializers would announce whether they are stateful or not and rely on RuntimeStatefulSerializerFactory to do the duplication. Comparators, on the other hand, had a duplicate method that the user was required to call. This commit removes the statful/stateless property from Serializers but instead introduces a duplicate() method, similar to Comparators, that can return the same instance. The two serializer factories are merged into one that always calls duplicate() before returning a serializer. RuntimeStatefulSerializerFactory declares ClassLoader as transient but later tries to use it Key: FLINK-1463 URL: https://issues.apache.org/jira/browse/FLINK-1463 Project: Flink Issue Type: Bug Reporter: Aljoscha Krettek At least one user has seen an exception because of this. In theory, the ClassLoader is set again in readParametersFromConfig. But the way it is used in TupleComparatorBase, this method is never called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1463] Fix stateful/stateless Serializer...
GitHub user aljoscha opened a pull request: https://github.com/apache/flink/pull/353 [FLINK-1463] Fix stateful/stateless Serializers and Comparators Before, Serializers would announce whether they are stateful or not and rely on RuntimeStatefulSerializerFactory to do the duplication. Comparators, on the other hand, had a duplicate method that the user was required to call. This commit removes the statful/stateless property from Serializers but instead introduces a duplicate() method, similar to Comparators, that can return the same instance. The two serializer factories are merged into one that always calls duplicate() before returning a serializer. You can merge this pull request into a Git repository by running: $ git pull https://github.com/aljoscha/flink serializer-factories-fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/353.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #353 commit 91834fee239b372b9a39f3c8f89ecbe42e2ae23a Author: Aljoscha Krettek aljoscha.kret...@gmail.com Date: 2015-01-30T15:43:31Z [FLINK-1463] Fix stateful/stateless Serializers and Comparators Before, Serializers would announce whether they are stateful or not and rely on RuntimeStatefulSerializerFactory to do the duplication. Comparators, on the other hand, had a duplicate method that the user was required to call. This commit removes the statful/stateless property from Serializers but instead introduces a duplicate() method, similar to Comparators, that can return the same instance. The two serializer factories are merged into one that always calls duplicate() before returning a serializer. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Comment Edited] (FLINK-1438) ClassCastException for Custom InputSplit in local mode
[
https://issues.apache.org/jira/browse/FLINK-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298847#comment-14298847
]
Till Rohrmann edited comment on FLINK-1438 at 1/30/15 4:48 PM:
---
This is not only happening in local mode but also on cloud-11 with jobs which
contain the InputSplit class information in the user code jars.
The result is that Akka cannot deserialize the object and terminates the
transmission with the following error:
WARN akka.remote.ReliableDeliverySupervisor-
Association with remote system
[akka.tcp://fl...@cloud-22.dima.tu-berlin.de:53627] has failed, address is now
gated for [5000] ms. Reason is: [invalid type code: 00].
was (Author: till.rohrmann):
This is not only happening in local mode but also on cloud-11 with jobs which
contain the InputSplit class information in the user code jars.
ClassCastException for Custom InputSplit in local mode
--
Key: FLINK-1438
URL: https://issues.apache.org/jira/browse/FLINK-1438
Project: Flink
Issue Type: Bug
Components: JobManager
Affects Versions: 0.8
Reporter: Fabian Hueske
Priority: Minor
Jobs with custom InputSplits fail with a ClassCastException such as
{{org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit
cannot be cast to
org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit}}
if executed on a local setup.
This issue is probably related to different ClassLoaders used by the
JobManager when InputSplits are generated and when they are handed to the
InputFormat by the TaskManager. Moving the class of the custom InputSplit
into the {{./lib}} folder and removing it from the job's makes the job work.
To reproduce the bug, run the following job on a local setup.
{code}
public class CustomSplitTestJob {
public static void main(String[] args) throws Exception {
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
DataSetString x = env.createInput(new TestFileInputFormat());
x.print();
env.execute();
}
public static class TestFileInputFormat implements
InputFormatString,TestFileInputSplit {
@Override
public void configure(Configuration parameters) {
}
@Override
public BaseStatistics getStatistics(BaseStatistics
cachedStatistics) throws IOException {
return null;
}
@Override
public TestFileInputSplit[] createInputSplits(int minNumSplits)
throws IOException {
return new TestFileInputSplit[]{new
TestFileInputSplit()};
}
@Override
public InputSplitAssigner
getInputSplitAssigner(TestFileInputSplit[] inputSplits) {
return new LocatableInputSplitAssigner(inputSplits);
}
@Override
public void open(TestFileInputSplit split) throws IOException {
}
@Override
public boolean reachedEnd() throws IOException {
return false;
}
@Override
public String nextRecord(String reuse) throws IOException {
return null;
}
@Override
public void close() throws IOException {
}
}
public static class TestFileInputSplit extends FileInputSplit {
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1438) ClassCastException for Custom InputSplit in local mode
[
https://issues.apache.org/jira/browse/FLINK-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298847#comment-14298847
]
Till Rohrmann commented on FLINK-1438:
--
This is not only happening in local mode but also on cloud-11 with jobs which
contain the InputSplit class information in the user code jars.
ClassCastException for Custom InputSplit in local mode
--
Key: FLINK-1438
URL: https://issues.apache.org/jira/browse/FLINK-1438
Project: Flink
Issue Type: Bug
Components: JobManager
Affects Versions: 0.8
Reporter: Fabian Hueske
Priority: Minor
Jobs with custom InputSplits fail with a ClassCastException such as
{{org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit
cannot be cast to
org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit}}
if executed on a local setup.
This issue is probably related to different ClassLoaders used by the
JobManager when InputSplits are generated and when they are handed to the
InputFormat by the TaskManager. Moving the class of the custom InputSplit
into the {{./lib}} folder and removing it from the job's makes the job work.
To reproduce the bug, run the following job on a local setup.
{code}
public class CustomSplitTestJob {
public static void main(String[] args) throws Exception {
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
DataSetString x = env.createInput(new TestFileInputFormat());
x.print();
env.execute();
}
public static class TestFileInputFormat implements
InputFormatString,TestFileInputSplit {
@Override
public void configure(Configuration parameters) {
}
@Override
public BaseStatistics getStatistics(BaseStatistics
cachedStatistics) throws IOException {
return null;
}
@Override
public TestFileInputSplit[] createInputSplits(int minNumSplits)
throws IOException {
return new TestFileInputSplit[]{new
TestFileInputSplit()};
}
@Override
public InputSplitAssigner
getInputSplitAssigner(TestFileInputSplit[] inputSplits) {
return new LocatableInputSplitAssigner(inputSplits);
}
@Override
public void open(TestFileInputSplit split) throws IOException {
}
@Override
public boolean reachedEnd() throws IOException {
return false;
}
@Override
public String nextRecord(String reuse) throws IOException {
return null;
}
@Override
public void close() throws IOException {
}
}
public static class TestFileInputSplit extends FileInputSplit {
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLINK-1438) ClassCastException for Custom InputSplit in local mode and invalid type code in distributed mode
[
https://issues.apache.org/jira/browse/FLINK-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-1438:
-
Summary: ClassCastException for Custom InputSplit in local mode and invalid
type code in distributed mode (was: ClassCastException for Custom InputSplit
in local mode)
ClassCastException for Custom InputSplit in local mode and invalid type code
in distributed mode
Key: FLINK-1438
URL: https://issues.apache.org/jira/browse/FLINK-1438
Project: Flink
Issue Type: Bug
Components: JobManager
Affects Versions: 0.8
Reporter: Fabian Hueske
Priority: Minor
Jobs with custom InputSplits fail with a ClassCastException such as
{{org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit
cannot be cast to
org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit}}
if executed on a local setup.
This issue is probably related to different ClassLoaders used by the
JobManager when InputSplits are generated and when they are handed to the
InputFormat by the TaskManager. Moving the class of the custom InputSplit
into the {{./lib}} folder and removing it from the job's makes the job work.
To reproduce the bug, run the following job on a local setup.
{code}
public class CustomSplitTestJob {
public static void main(String[] args) throws Exception {
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
DataSetString x = env.createInput(new TestFileInputFormat());
x.print();
env.execute();
}
public static class TestFileInputFormat implements
InputFormatString,TestFileInputSplit {
@Override
public void configure(Configuration parameters) {
}
@Override
public BaseStatistics getStatistics(BaseStatistics
cachedStatistics) throws IOException {
return null;
}
@Override
public TestFileInputSplit[] createInputSplits(int minNumSplits)
throws IOException {
return new TestFileInputSplit[]{new
TestFileInputSplit()};
}
@Override
public InputSplitAssigner
getInputSplitAssigner(TestFileInputSplit[] inputSplits) {
return new LocatableInputSplitAssigner(inputSplits);
}
@Override
public void open(TestFileInputSplit split) throws IOException {
}
@Override
public boolean reachedEnd() throws IOException {
return false;
}
@Override
public String nextRecord(String reuse) throws IOException {
return null;
}
@Override
public void close() throws IOException {
}
}
public static class TestFileInputSplit extends FileInputSplit {
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLINK-1438) ClassCastException for Custom InputSplit in local mode and invalid type code in distributed mode
[
https://issues.apache.org/jira/browse/FLINK-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-1438:
-
Description:
Jobs with custom InputSplits fail with a ClassCastException such as
{{org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit
cannot be cast to
org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit}} if
executed on a local setup.
This issue is probably related to different ClassLoaders used by the JobManager
when InputSplits are generated and when they are handed to the InputFormat by
the TaskManager. Moving the class of the custom InputSplit into the {{./lib}}
folder and removing it from the job's makes the job work.
To reproduce the bug, run the following job on a local setup.
{code}
public class CustomSplitTestJob {
public static void main(String[] args) throws Exception {
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
DataSetString x = env.createInput(new TestFileInputFormat());
x.print();
env.execute();
}
public static class TestFileInputFormat implements
InputFormatString,TestFileInputSplit {
@Override
public void configure(Configuration parameters) {
}
@Override
public BaseStatistics getStatistics(BaseStatistics
cachedStatistics) throws IOException {
return null;
}
@Override
public TestFileInputSplit[] createInputSplits(int minNumSplits)
throws IOException {
return new TestFileInputSplit[]{new
TestFileInputSplit()};
}
@Override
public InputSplitAssigner
getInputSplitAssigner(TestFileInputSplit[] inputSplits) {
return new LocatableInputSplitAssigner(inputSplits);
}
@Override
public void open(TestFileInputSplit split) throws IOException {
}
@Override
public boolean reachedEnd() throws IOException {
return false;
}
@Override
public String nextRecord(String reuse) throws IOException {
return null;
}
@Override
public void close() throws IOException {
}
}
public static class TestFileInputSplit extends FileInputSplit {
}
}
{code}
The same happens in distributed mode just that Akka terminates the transmission
of the input split with a meaningless {{invalid type code: 00}}.
was:
Jobs with custom InputSplits fail with a ClassCastException such as
{{org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit
cannot be cast to
org.apache.flink.examples.java.misc.CustomSplitTestJob$TestFileInputSplit}} if
executed on a local setup.
This issue is probably related to different ClassLoaders used by the JobManager
when InputSplits are generated and when they are handed to the InputFormat by
the TaskManager. Moving the class of the custom InputSplit into the {{./lib}}
folder and removing it from the job's makes the job work.
To reproduce the bug, run the following job on a local setup.
{code}
public class CustomSplitTestJob {
public static void main(String[] args) throws Exception {
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
DataSetString x = env.createInput(new TestFileInputFormat());
x.print();
env.execute();
}
public static class TestFileInputFormat implements
InputFormatString,TestFileInputSplit {
@Override
public void configure(Configuration parameters) {
}
@Override
public BaseStatistics getStatistics(BaseStatistics
cachedStatistics) throws IOException {
return null;
}
@Override
public TestFileInputSplit[] createInputSplits(int minNumSplits)
throws IOException {
return new TestFileInputSplit[]{new
TestFileInputSplit()};
}
@Override
public InputSplitAssigner
getInputSplitAssigner(TestFileInputSplit[] inputSplits) {
return new LocatableInputSplitAssigner(inputSplits);
}
@Override
public void open(TestFileInputSplit split) throws IOException {
}
@Override
public boolean reachedEnd() throws IOException {
return false;
}
@Override
[jira] [Created] (FLINK-1464) Added ResultTypeQueryable interface to TypeSerializerInputFormat.
Alexander Alexandrov created FLINK-1464:
---
Summary: Added ResultTypeQueryable interface to
TypeSerializerInputFormat.
Key: FLINK-1464
URL: https://issues.apache.org/jira/browse/FLINK-1464
Project: Flink
Issue Type: Improvement
Components: Distributed Runtime, Optimizer
Affects Versions: 0.8, 0.9, 0.8.1
Reporter: Alexander Alexandrov
Assignee: Alexander Alexandrov
Priority: Minor
Fix For: 0.9, 0.8.1
It is currently impossible to use the {{TypeSerializerInputFormat}} with
generic Tuple types.
For example, [this example
gist|https://gist.github.com/aalexandrov/90bf21f66bf604676f37] fails with a
{quote}
Exception in thread main org.apache.flink.api.common.InvalidProgramException:
The type returned by the input format could not be automatically determined.
Please specify the TypeInformation of the produced type explicitly.
at
org.apache.flink.api.java.ExecutionEnvironment.readFile(ExecutionEnvironment.java:341)
at SerializedFormatExample$.main(SerializedFormatExample.scala:48)
at SerializedFormatExample.main(SerializedFormatExample.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
{quote}
exaception.
To fix the issue, I changed the constructor to take a {{TypeInformationT}}
instad of a {{TypeSerializerT}} argument. If this is indeed a bug, I think
that this is a good solution.
Unfortunately the fix breaks the API. Feel free to change it if you find a more
elegant solution compatible with the 0.8 branch.
The suggested fix can be found in the GitHub
[PR#349|https://github.com/apache/flink/pull/349].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1464) Added ResultTypeQueryable interface to TypeSerializerInputFormat.
[
https://issues.apache.org/jira/browse/FLINK-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299321#comment-14299321
]
ASF GitHub Bot commented on FLINK-1464:
---
Github user aalexandrov commented on the pull request:
https://github.com/apache/flink/pull/349#issuecomment-72284550
@hsaputra: I just did:
[FLINK-1464](https://issues.apache.org/jira/browse/FLINK-1464).
Added ResultTypeQueryable interface to TypeSerializerInputFormat.
-
Key: FLINK-1464
URL: https://issues.apache.org/jira/browse/FLINK-1464
Project: Flink
Issue Type: Improvement
Components: Distributed Runtime, Optimizer
Affects Versions: 0.8, 0.9, 0.8.1
Reporter: Alexander Alexandrov
Assignee: Alexander Alexandrov
Priority: Minor
Labels: easyfix
Fix For: 0.9, 0.8.1
Original Estimate: 6h
Remaining Estimate: 6h
It is currently impossible to use the {{TypeSerializerInputFormat}} with
generic Tuple types.
For example, [this example
gist|https://gist.github.com/aalexandrov/90bf21f66bf604676f37] fails with a
{quote}
Exception in thread main
org.apache.flink.api.common.InvalidProgramException: The type returned by the
input format could not be automatically determined. Please specify the
TypeInformation of the produced type explicitly.
at
org.apache.flink.api.java.ExecutionEnvironment.readFile(ExecutionEnvironment.java:341)
at SerializedFormatExample$.main(SerializedFormatExample.scala:48)
at SerializedFormatExample.main(SerializedFormatExample.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
{quote}
exaception.
To fix the issue, I changed the constructor to take a {{TypeInformationT}}
instad of a {{TypeSerializerT}} argument. If this is indeed a bug, I think
that this is a good solution.
Unfortunately the fix breaks the API. Feel free to change it if you find a
more elegant solution compatible with the 0.8 branch.
The suggested fix can be found in the GitHub
[PR#349|https://github.com/apache/flink/pull/349].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: Added ResultTypeQueryable interface to TypeSer...
Github user aalexandrov commented on the pull request: https://github.com/apache/flink/pull/349#issuecomment-72284550 @hsaputra: I just did: [FLINK-1464](https://issues.apache.org/jira/browse/FLINK-1464). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (FLINK-1460) Typo fixes
[ https://issues.apache.org/jira/browse/FLINK-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Saputra resolved FLINK-1460. -- Resolution: Fixed Fix Version/s: 0.9 Typo fixes -- Key: FLINK-1460 URL: https://issues.apache.org/jira/browse/FLINK-1460 Project: Flink Issue Type: Improvement Reporter: Shuo Xiang Priority: Minor Fix For: 0.9 Fix some typos. Also fix some inconsistent uses of *partition operator* and *partitioning operator* in the codebase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1460] fix typos
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/346 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1460) Typo fixes
[ https://issues.apache.org/jira/browse/FLINK-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298930#comment-14298930 ] ASF GitHub Bot commented on FLINK-1460: --- Github user hsaputra commented on the pull request: https://github.com/apache/flink/pull/346#issuecomment-72241355 Merged, thanks for the fix @coderxiang Typo fixes -- Key: FLINK-1460 URL: https://issues.apache.org/jira/browse/FLINK-1460 Project: Flink Issue Type: Improvement Reporter: Shuo Xiang Priority: Minor Fix some typos. Also fix some inconsistent uses of *partition operator* and *partitioning operator* in the codebase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1460) Typo fixes
[ https://issues.apache.org/jira/browse/FLINK-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298929#comment-14298929 ] ASF GitHub Bot commented on FLINK-1460: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/346 Typo fixes -- Key: FLINK-1460 URL: https://issues.apache.org/jira/browse/FLINK-1460 Project: Flink Issue Type: Improvement Reporter: Shuo Xiang Priority: Minor Fix some typos. Also fix some inconsistent uses of *partition operator* and *partitioning operator* in the codebase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: Added ResultTypeQueryable interface to TypeSer...
Github user hsaputra commented on the pull request: https://github.com/apache/flink/pull/349#issuecomment-72262547 @aalexandrov, could you file JIRA to associate this PR with? I need it for tracking purposes and releases. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: FLINK-1452: Rename 'flink-addons' to 'flink-st...
GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/355 FLINK-1452: Rename 'flink-addons' to 'flink-staging'; add 'flink-contrib' You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmetzger/flink flink1452 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/355.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #355 commit 7032cffca0ca581cf954ae085ce18a5e1e908c9a Author: Robert Metzger rmetz...@apache.org Date: 2015-01-30T14:16:46Z [FLINK-1452] Rename 'flink-addons' to 'flink-staging' commit 732a9e37846c8e17d12bba1c28d6f62b317206fa Author: Robert Metzger rmetz...@apache.org Date: 2015-01-30T14:29:05Z [FLINK-1452] Add 'flink-contrib' module --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1452) Add flink-contrib maven module and README.md with the rules
[ https://issues.apache.org/jira/browse/FLINK-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298996#comment-14298996 ] ASF GitHub Bot commented on FLINK-1452: --- GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/355 FLINK-1452: Rename 'flink-addons' to 'flink-staging'; add 'flink-contrib' You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmetzger/flink flink1452 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/355.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #355 commit 7032cffca0ca581cf954ae085ce18a5e1e908c9a Author: Robert Metzger rmetz...@apache.org Date: 2015-01-30T14:16:46Z [FLINK-1452] Rename 'flink-addons' to 'flink-staging' commit 732a9e37846c8e17d12bba1c28d6f62b317206fa Author: Robert Metzger rmetz...@apache.org Date: 2015-01-30T14:29:05Z [FLINK-1452] Add 'flink-contrib' module Add flink-contrib maven module and README.md with the rules - Key: FLINK-1452 URL: https://issues.apache.org/jira/browse/FLINK-1452 Project: Flink Issue Type: New Feature Components: flink-contrib Reporter: Robert Metzger Assignee: Robert Metzger I'll also create a JIRA component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1464] Add ResultTypeQueryable interface...
Github user hsaputra commented on the pull request: https://github.com/apache/flink/pull/349#issuecomment-72291369 @aalexandrov, cool! Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1464) Added ResultTypeQueryable interface to TypeSerializerInputFormat.
[
https://issues.apache.org/jira/browse/FLINK-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299395#comment-14299395
]
ASF GitHub Bot commented on FLINK-1464:
---
Github user hsaputra commented on the pull request:
https://github.com/apache/flink/pull/349#issuecomment-72291369
@aalexandrov, cool! Thanks.
Added ResultTypeQueryable interface to TypeSerializerInputFormat.
-
Key: FLINK-1464
URL: https://issues.apache.org/jira/browse/FLINK-1464
Project: Flink
Issue Type: Improvement
Components: Distributed Runtime, Optimizer
Affects Versions: 0.8, 0.9, 0.8.1
Reporter: Alexander Alexandrov
Assignee: Alexander Alexandrov
Priority: Minor
Labels: easyfix
Fix For: 0.9, 0.8.1
Original Estimate: 6h
Remaining Estimate: 6h
It is currently impossible to use the {{TypeSerializerInputFormat}} with
generic Tuple types.
For example, [this example
gist|https://gist.github.com/aalexandrov/90bf21f66bf604676f37] fails with a
{quote}
Exception in thread main
org.apache.flink.api.common.InvalidProgramException: The type returned by the
input format could not be automatically determined. Please specify the
TypeInformation of the produced type explicitly.
at
org.apache.flink.api.java.ExecutionEnvironment.readFile(ExecutionEnvironment.java:341)
at SerializedFormatExample$.main(SerializedFormatExample.scala:48)
at SerializedFormatExample.main(SerializedFormatExample.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
{quote}
exaception.
To fix the issue, I changed the constructor to take a {{TypeInformationT}}
instad of a {{TypeSerializerT}} argument. If this is indeed a bug, I think
that this is a good solution.
Unfortunately the fix breaks the API. Feel free to change it if you find a
more elegant solution compatible with the 0.8 branch.
The suggested fix can be found in the GitHub
[PR#349|https://github.com/apache/flink/pull/349].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1452) Add flink-contrib maven module and README.md with the rules
[ https://issues.apache.org/jira/browse/FLINK-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298630#comment-14298630 ] Fabian Hueske commented on FLINK-1452: -- +1 Add flink-contrib maven module and README.md with the rules - Key: FLINK-1452 URL: https://issues.apache.org/jira/browse/FLINK-1452 Project: Flink Issue Type: New Feature Components: flink-contrib Reporter: Robert Metzger Assignee: Robert Metzger I'll also create a JIRA component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1463) RuntimeStatefulSerializerFactory declares ClassLoader as transient but later tries to use it
Aljoscha Krettek created FLINK-1463: --- Summary: RuntimeStatefulSerializerFactory declares ClassLoader as transient but later tries to use it Key: FLINK-1463 URL: https://issues.apache.org/jira/browse/FLINK-1463 Project: Flink Issue Type: Bug Reporter: Aljoscha Krettek At least one user has seen an exception because of this. In theory, the ClassLoader is set again in readParametersFromConfig. But the way it is used in TupleSerializerBase, this method is never called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLINK-1463) RuntimeStatefulSerializerFactory declares ClassLoader as transient but later tries to use it
[ https://issues.apache.org/jira/browse/FLINK-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek updated FLINK-1463: Description: At least one user has seen an exception because of this. In theory, the ClassLoader is set again in readParametersFromConfig. But the way it is used in TupleComparatorBase, this method is never called. (was: At least one user has seen an exception because of this. In theory, the ClassLoader is set again in readParametersFromConfig. But the way it is used in TupleSerializerBase, this method is never called.) RuntimeStatefulSerializerFactory declares ClassLoader as transient but later tries to use it Key: FLINK-1463 URL: https://issues.apache.org/jira/browse/FLINK-1463 Project: Flink Issue Type: Bug Reporter: Aljoscha Krettek At least one user has seen an exception because of this. In theory, the ClassLoader is set again in readParametersFromConfig. But the way it is used in TupleComparatorBase, this method is never called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1452) Add flink-contrib maven module and README.md with the rules
[
https://issues.apache.org/jira/browse/FLINK-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298498#comment-14298498
]
Robert Metzger commented on FLINK-1452:
---
Okay, so we rename {{flink-addons}} to {{flink-staging}} and add a new module
{{flink-contrib}} ?
Add flink-contrib maven module and README.md with the rules
-
Key: FLINK-1452
URL: https://issues.apache.org/jira/browse/FLINK-1452
Project: Flink
Issue Type: New Feature
Components: flink-contrib
Reporter: Robert Metzger
Assignee: Robert Metzger
I'll also create a JIRA component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1452) Add flink-contrib maven module and README.md with the rules
[ https://issues.apache.org/jira/browse/FLINK-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298626#comment-14298626 ] Ufuk Celebi commented on FLINK-1452: Yes, indeed. Add flink-contrib maven module and README.md with the rules - Key: FLINK-1452 URL: https://issues.apache.org/jira/browse/FLINK-1452 Project: Flink Issue Type: New Feature Components: flink-contrib Reporter: Robert Metzger Assignee: Robert Metzger I'll also create a JIRA component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1452) Add flink-contrib maven module and README.md with the rules
[ https://issues.apache.org/jira/browse/FLINK-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298502#comment-14298502 ] Aljoscha Krettek commented on FLINK-1452: - Yes Add flink-contrib maven module and README.md with the rules - Key: FLINK-1452 URL: https://issues.apache.org/jira/browse/FLINK-1452 Project: Flink Issue Type: New Feature Components: flink-contrib Reporter: Robert Metzger Assignee: Robert Metzger I'll also create a JIRA component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: Corrected some typos in comments (removed doub...
GitHub user mborgmann opened a pull request: https://github.com/apache/flink/pull/352 Corrected some typos in comments (removed double the) You can merge this pull request into a Git repository by running: $ git pull https://github.com/mborgmann/incubator-flink master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/352.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #352 commit aaef0ab4385db529f6bfa5c1ad310a89dbd3 Author: mborgmann moritzborgm...@gmail.com Date: 2015-01-30T10:52:43Z Just a documentation typo in ExecutionEnvironment.java commit b982d129370685a7084b37c05586a2cb2932e5e9 Author: mborgmann moritzborgm...@gmail.com Date: 2015-01-30T10:58:41Z Typos in Codedocumentation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: Corrected some typos in comments (removed doub...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/352#issuecomment-72185322 Thank you. The change is good to merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
