[GitHub] flink pull request: [FLINK-2480][test]add a test for Print Sink wi...

2015-08-28 Thread HuangWHWHW
GitHub user HuangWHWHW opened a pull request: https://github.com/apache/flink/pull/1073 [FLINK-2480][test]add a test for Print Sink with prefix I check the coverage of sink and add a test for sink to print with prefix. You can merge this pull request into a Git repository by running

[jira] [Commented] (FLINK-2480) Improving tests coverage for org.apache.flink.streaming.api

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720994#comment-14720994 ] ASF GitHub Bot commented on FLINK-2480: --- GitHub user HuangWHWHW opened a pull reques

[GitHub] flink pull request: [FLINK1919] add HCatOutputFormat

2015-08-28 Thread jamescao
Github user jamescao commented on the pull request: https://github.com/apache/flink/pull/1064#issuecomment-135925495 I need to work offline to debug the travis builds So close the pr for now. Thanks for all your time and comments! I will reopen once all the tests are fixed. --- If

[GitHub] flink pull request: [FLINK1919] add HCatOutputFormat

2015-08-28 Thread jamescao
Github user jamescao closed the pull request at: https://github.com/apache/flink/pull/1064 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] flink pull request: [FLINK-1725]- New Partitioner for better load ...

2015-08-28 Thread anisnasir
Github user anisnasir commented on the pull request: https://github.com/apache/flink/pull/1069#issuecomment-135905274 @tillrohrmann you are absolutely right with your observation that high skews require more than two workers to process the most frequent keys. However, most of the real

[jira] [Commented] (FLINK-1725) New Partitioner for better load balancing for skewed data

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720719#comment-14720719 ] ASF GitHub Bot commented on FLINK-1725: --- Github user anisnasir commented on the pull

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720690#comment-14720690 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on the pull req

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on the pull request: https://github.com/apache/flink/pull/750#issuecomment-135901379 Travis fails due to instable test. Own Travis is green: https://travis-ci.org/mjsax/flink/builds/77750320 --- If your project is set up for it, you can reply to this email

[jira] [Updated] (FLINK-2596) Failing Test: RandomSamplerTest

2015-08-28 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated FLINK-2596: --- Component/s: Tests > Failing Test: RandomSamplerTest > --- > >

[jira] [Created] (FLINK-2596) Failing Test: RandomSamplerTest

2015-08-28 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2596: -- Summary: Failing Test: RandomSamplerTest Key: FLINK-2596 URL: https://issues.apache.org/jira/browse/FLINK-2596 Project: Flink Issue Type: Bug

[GitHub] flink pull request: [FLINK-2545] add bucket member count verificat...

2015-08-28 Thread greghogan
Github user greghogan commented on the pull request: https://github.com/apache/flink/pull/1067#issuecomment-135885866 I am currently running release-0.10.0-milestone-1. Debugging with Eclipse and looking at MutableHashTable.initTable, numBuckets is computed as 16086. There are

[jira] [Commented] (FLINK-2545) NegativeArraySizeException while creating hash table bloom filters

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720556#comment-14720556 ] ASF GitHub Bot commented on FLINK-2545: --- Github user greghogan commented on the pull

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720362#comment-14720362 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on the pull req

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on the pull request: https://github.com/apache/flink/pull/750#issuecomment-135852748 I just pushed an update addressing your comments. I additionally rework all TaskManager and JobManager test. Open issues: 1. how to deal with (potentially blocking)

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38227043 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -411,6 +411,23 @@ class TaskManager( log.d

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720339#comment-14720339 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on a diff in th

[jira] [Closed] (FLINK-2594) Provide a getAccumulators method on the client

2015-08-28 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels closed FLINK-2594. - Resolution: Implemented Assignee: Maximilian Michels > Provide a getAccumulators met

[jira] [Commented] (FLINK-2594) Provide a getAccumulators method on the client

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720294#comment-14720294 ] ASF GitHub Bot commented on FLINK-2594: --- Github user asfgit closed the pull request

[GitHub] flink pull request: [FLINK-2594][client] implement a method to ret...

2015-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/1072 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (FLINK-1725) New Partitioner for better load balancing for skewed data

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720272#comment-14720272 ] ASF GitHub Bot commented on FLINK-1725: --- Github user tillrohrmann commented on the p

[GitHub] flink pull request: [FLINK-1725]- New Partitioner for better load ...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1069#issuecomment-135838438 @anisnasir thanks for your contribution. Out of curiosity I was wondering why the `PartialPartitioner` distributes the data exactly between two channels. Wouldn't i

[jira] [Commented] (FLINK-2373) Add configuration parameter to createRemoteEnvironment method

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720217#comment-14720217 ] ASF GitHub Bot commented on FLINK-2373: --- Github user akunft commented on the pull re

[GitHub] flink pull request: [FLINK-2373] Add configuration parameter to cr...

2015-08-28 Thread akunft
Github user akunft commented on the pull request: https://github.com/apache/flink/pull/1066#issuecomment-135830165 After thinking about it, I would change the behavior so that the config is only for defining akka specific settings. The default parallelism of the operators can be set v

[jira] [Commented] (FLINK-2594) Provide a getAccumulators method on the client

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720211#comment-14720211 ] ASF GitHub Bot commented on FLINK-2594: --- Github user fhueske commented on the pull r

[GitHub] flink pull request: [FLINK-2594][client] implement a method to ret...

2015-08-28 Thread fhueske
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/1072#issuecomment-135829641 LGTM, ready to merge, IMO. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[jira] [Created] (FLINK-2595) Unclosed JarFile may leak resource in ClassLoaderUtilsTest

2015-08-28 Thread Ted Yu (JIRA)
Ted Yu created FLINK-2595: - Summary: Unclosed JarFile may leak resource in ClassLoaderUtilsTest Key: FLINK-2595 URL: https://issues.apache.org/jira/browse/FLINK-2595 Project: Flink Issue Type: Test

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720045#comment-14720045 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135813751 Right, I am going for an explicit serialized form of the exception at this point. --- If your project is set up for it, you can reply to this email and have your re

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135812417 Sending a stringified exception is not an option between jm and client because the user might rely on the exception in the RemoteExecEnv. --- If your project is set up

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720036#comment-14720036 ] ASF GitHub Bot commented on FLINK-2543: --- Github user rmetzger commented on the pull

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720033#comment-14720033 ] ASF GitHub Bot commented on FLINK-2543: --- Github user tillrohrmann commented on the p

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135811858 I'm not sure whether this would solve the initial problem. I thought the problem was that a user code exception can be sent from the `JobManager` to the `JobClientA

[jira] [Commented] (FLINK-2594) Provide a getAccumulators method on the client

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720024#comment-14720024 ] ASF GitHub Bot commented on FLINK-2594: --- GitHub user mxm opened a pull request:

[jira] [Created] (FLINK-2594) Provide a getAccumulators method on the client

2015-08-28 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-2594: - Summary: Provide a getAccumulators method on the client Key: FLINK-2594 URL: https://issues.apache.org/jira/browse/FLINK-2594 Project: Flink Issue

[GitHub] flink pull request: [FLINK-2594][client] implement a method to ret...

2015-08-28 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/1072 [FLINK-2594][client] implement a method to retrieve the accumulators … …of a job - move SerializedValue from runtime to core - unified code to deserialize accumulators You can merge thi

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720014#comment-14720014 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135808065 I would like to revert the thing back to not use serialized throwable at all. The SerializedThrowable would be used only in archived execution graphs, to ma

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14719304#comment-14719304 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on a diff in th

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38208261 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -411,6 +411,23 @@ class TaskManager( log.d

[jira] [Commented] (FLINK-2125) String delimiter for SocketTextStream

2015-08-28 Thread Hubert Czerpak (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14719291#comment-14719291 ] Hubert Czerpak commented on FLINK-2125: --- I'm not able to complete this task on my ow

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14719290#comment-14719290 ] ASF GitHub Bot commented on FLINK-2111: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38207280 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -411,6 +411,23 @@ class TaskManager(

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135793882 Only disadvantage is that archived execution graphs prevent user class unloading... --- If your project is set up for it, you can reply to this email and have your

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14719285#comment-14719285 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14719286#comment-14719286 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135793797 Hmm... the more I work on it, the trickier it appears. It is efficient when done rigth, but one needs to watch carefully at what places to wrap an exception

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718642#comment-14718642 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on a diff in th

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38204935 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -411,6 +411,23 @@ class TaskManager( log.d

[GitHub] flink pull request: [FLINK-2291] [runtime] Adds high availability ...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1016#issuecomment-135781842 Ok, thanks for your feedback @StephanEwen and @uce. I'll rebase on the latest master and then merge it. I've created two issues [1, 2] to track the problems Stephan

[jira] [Commented] (FLINK-2291) Use ZooKeeper to elect JobManager leader and send information to TaskManagers

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718601#comment-14718601 ] ASF GitHub Bot commented on FLINK-2291: --- Github user tillrohrmann commented on the p

[jira] [Created] (FLINK-2593) Revisit Flink's shading policy

2015-08-28 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-2593: Summary: Revisit Flink's shading policy Key: FLINK-2593 URL: https://issues.apache.org/jira/browse/FLINK-2593 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-2592) Rework of FlinkMiniCluster

2015-08-28 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-2592: Summary: Rework of FlinkMiniCluster Key: FLINK-2592 URL: https://issues.apache.org/jira/browse/FLINK-2592 Project: Flink Issue Type: Improvement

[jira] [Updated] (FLINK-2591) Add configuration parameter for default number of yarn containers

2015-08-28 Thread Robert Metzger (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger updated FLINK-2591: -- Component/s: YARN Client > Add configuration parameter for default number of yarn containers > -

[jira] [Created] (FLINK-2591) Add configuration parameter for default number of yarn containers

2015-08-28 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-2591: - Summary: Add configuration parameter for default number of yarn containers Key: FLINK-2591 URL: https://issues.apache.org/jira/browse/FLINK-2591 Project: Flink

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718549#comment-14718549 ] ASF GitHub Bot commented on FLINK-2543: --- Github user tillrohrmann commented on the p

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135774461 I like this solution :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718538#comment-14718538 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135771604 @tillrohrmann Yes, it is still a subclass of `Exception` and thrown in the regular way by the Akka `Failure` response. --- If your project is set up for it, you can

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718533#comment-14718533 ] ASF GitHub Bot commented on FLINK-2543: --- Github user tillrohrmann commented on the p

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718532#comment-14718532 ] ASF GitHub Bot commented on FLINK-2543: --- Github user rmetzger commented on the pull

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135770745 The `SerializedThrowable` is then a subclass of `Exception`? I think it's a very good solution to not deserialize the exception on the `JobManager`. --- I

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135770247 That sounds very good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718531#comment-14718531 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135769797 The added advantage of that approach s that we keep no user exceptions alive in the execution graph. Not keeping those alive is quite desirable, actually, to

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135769165 I have now actually a pretty interesting solution. I changed the SerializedThrowable such that it looks like the original exception, with respect to message, stack t

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/750#issuecomment-135768996 Wow that is some really big piece of work @mjsax. Thanks for your contribution. To be honest, I actually thought you would only adapt your newly written test. As far

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718529#comment-14718529 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718526#comment-14718526 ] ASF GitHub Bot commented on FLINK-2111: --- Github user tillrohrmann commented on the p

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718512#comment-14718512 ] ASF GitHub Bot commented on FLINK-2543: --- Github user tillrohrmann commented on the p

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135767450 I agree. In both cases the compiler cannot help much to avoid forgetting about it and, thus, it's the responsibility of the programmer to make sure that the failure

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718493#comment-14718493 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135765919 Explicit would mean here to send something where the receiver knows that he has to deserialize. Whether through a dedicated message, or a SerializedThrowable that is

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718489#comment-14718489 ] ASF GitHub Bot commented on FLINK-2111: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38195614 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -411,6 +411,23 @@ class TaskManager(

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718487#comment-14718487 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on a diff in th

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38195523 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -411,6 +411,23 @@ class TaskManager( log.d

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718479#comment-14718479 ] ASF GitHub Bot commented on FLINK-2111: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38195178 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/ExecutionGraphConstructionTest.java --- @@ -450,8 +414,8 @@ private void ve

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718472#comment-14718472 ] ASF GitHub Bot commented on FLINK-2111: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38194969 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -411,6 +411,23 @@ class TaskManager(

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718468#comment-14718468 ] ASF GitHub Bot commented on FLINK-2111: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38194785 --- Diff: flink-clients/src/test/java/org/apache/flink/client/CliFrontendAddressConfigurationTest.java --- @@ -45,163 +43,122 @@ @Rule

[jira] [Commented] (FLINK-2577) Watermarks Stall When a Source Finishes Prematurely

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718460#comment-14718460 ] ASF GitHub Bot commented on FLINK-2577: --- Github user rmetzger commented on the pull

[GitHub] flink pull request: [FLINK-2577] Fix Stalling Watermarks when Sour...

2015-08-28 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1060#issuecomment-135761217 +1 to merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718458#comment-14718458 ] ASF GitHub Bot commented on FLINK-2111: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38194441 --- Diff: flink-clients/src/main/java/org/apache/flink/client/CliFrontend.java --- @@ -588,15 +653,16 @@ protected int cancel(String[] args) {

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718453#comment-14718453 ] ASF GitHub Bot commented on FLINK-2543: --- Github user rmetzger commented on the pull

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135759647 In the first version of this pull request I was sending a `JobFailure(st: SerializedThrowable)` in case of an error. The `SerializedThrowable` did not extend Excepti

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718451#comment-14718451 ] ASF GitHub Bot commented on FLINK-2543: --- Github user tillrohrmann commented on the p

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135758781 What do you mean with making things explicit? Sending a dedicated message or sending a `SerializedThrowable` which extends `Exception`? --- If your project is set

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718445#comment-14718445 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on a diff in th

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38193807 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/ExecutionGraphConstructionTest.java --- @@ -450,8 +414,8 @@ private void verifyTes

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38193714 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -411,6 +411,23 @@ class TaskManager( log.d

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718441#comment-14718441 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on a diff in th

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718393#comment-14718393 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on a diff in th

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38192065 --- Diff: flink-clients/src/test/java/org/apache/flink/client/RemoteExecutorHostnameResolutionTest.java --- @@ -88,15 +65,14 @@ private static void checkPrecond

[jira] [Commented] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718391#comment-14718391 ] ASF GitHub Bot commented on FLINK-2111: --- Github user mjsax commented on a diff in th

[GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...

2015-08-28 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/750#discussion_r38191972 --- Diff: flink-clients/src/test/java/org/apache/flink/client/CliFrontendRunTest.java --- @@ -1,22 +1,16 @@ /* - * Licensed to the Apache Software Found

[jira] [Commented] (FLINK-2543) State handling does not support deserializing classes through the UserCodeClassloader

2015-08-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718389#comment-14718389 ] ASF GitHub Bot commented on FLINK-2543: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2543] Fix user object deserialization f...

2015-08-28 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1048#issuecomment-135746761 Okay, one change I would like to do during merging is to not deserialize exceptions on the JobManager. I would extend the SerializedThrowable to include the original

  1   2   >