[GitHub] flink pull request: [FLINK-1442] Reduce memory consumption of arch...

2015-01-26 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/344 [FLINK-1442] Reduce memory consumption of archived execution graph You can merge this pull request into a Git repository by running: $ git pull https://github.com/mxm/flink flink-1442

[GitHub] flink pull request: [FLINK-1442] Reduce memory consumption of arch...

2015-02-04 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/344#discussion_r24093420 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/MemoryArchivist.scala --- @@ -25,48 +25,82 @@ import

[GitHub] flink pull request: [FLINK-1442] Reduce memory consumption of arch...

2015-02-04 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/344#discussion_r24093445 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/MemoryArchivist.scala --- @@ -25,48 +25,82 @@ import

[GitHub] flink pull request: [FLINK-1442] Reduce memory consumption of arch...

2015-02-04 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/344#discussion_r24093429 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/MemoryArchivist.scala --- @@ -25,48 +25,82 @@ import

[GitHub] flink pull request: [FLINK-1442] Reduce memory consumption of arch...

2015-02-04 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/344#discussion_r24093436 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/MemoryArchivist.scala --- @@ -25,48 +25,82 @@ import

[GitHub] flink pull request: [FLINK-592] Add support for Kerberos secured Y...

2015-02-04 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/358#issuecomment-72870536 @warneke Thanks for reporting. If the above issues have been resolved, I suggest to merge the changes. --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request: [FLINK-592] Add support for Kerberos secured Y...

2015-02-05 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/358#issuecomment-73013486 @warneke Thank you for your help! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-1166] Add qa-check.sh tool

2015-02-05 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/366#issuecomment-73083755 The script builds Flink twice and therefore takes a while. I don't know if we can force users to execute it before making pull requests. However, it would be good to get

[GitHub] flink pull request: [FLINK-1166] Add qa-check.sh tool

2015-02-05 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/366#discussion_r24178788 --- Diff: tools/qa-check.sh --- @@ -0,0 +1,175 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] flink pull request: [FLINK-1486] add print method for prefixing a ...

2015-02-06 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/372 [FLINK-1486] add print method for prefixing a user defined string - extend API to include a `print(String message)` method - change `PrintingOutputformat` to include a message You can merge

[GitHub] flink pull request: [FLINK-1486] add print method for prefixing a ...

2015-02-08 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/372#issuecomment-73410628 Good idea. So we would print `$taskId $outputValue` if the user did not supply a string and `$string:$taskId $outputValue` otherwise. If the parallelization degree is 1, we

[GitHub] flink pull request: [FLINK-1406] update Flink compatibility notice

2015-01-16 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/314 [FLINK-1406] update Flink compatibility notice You can merge this pull request into a Git repository by running: $ git pull https://github.com/mxm/flink flink_1406 Alternatively you can review

[GitHub] flink pull request: Implement the convenience methods count and co...

2015-01-20 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/210#issuecomment-70718732 I've implemented count and collect in the Scala API. There is still a problem with the `ListAccumulator` for non-primitive Objects (e.g. not Integer or Long) probably due

[GitHub] flink pull request: support for secure HDFS access using kerberos

2015-02-11 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/383 support for secure HDFS access using kerberos You can merge this pull request into a Git repository by running: $ git pull https://github.com/mxm/flink kerberos_hdfs Alternatively you can review

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-11 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73931484 Why should the resulting address be different from the one Java uses? What does import socket socket.getaddrinfo(socket.gethostname(), None

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-11 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73926790 No, same result. I tried including my hostname in /etc/hosts which got me past the above exception. However, now I get 02/11/2015 18:41:00 Job execution

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-10 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73726322 With the current version of the pull request, I get the following when executing /pyflink2.sh ../resources/python/flink/example/WordCount.py Job

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-02-10 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-73736669 Now I'm getting: 02/10/2015 17:47:06 Job execution switched to status RUNNING. 02/10/2015 17:47:06 DataSource (ValueSource)(1/1) switched

[GitHub] flink pull request: [FLINK-947] Add a declarative expression API

2015-02-18 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/405#issuecomment-74845397 To me, `flink-expressions` sounds much better than `linq` and it mitigates the risk of law suites :) --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request: Remove extra space after open parenthesis in I...

2015-02-19 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/416#issuecomment-75019775 :+1: Good to merge :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request: Add auto-parallelism to Jobs (0.8 branch)

2015-02-17 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/410#discussion_r24804242 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/jobmanager/JobManager.java --- @@ -374,6 +375,8 @@ public JobSubmissionResult submitJob(JobGraph

[GitHub] flink pull request: Add auto-parallelism to Jobs (0.8 branch)

2015-02-17 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/410#issuecomment-74642763 Right now, the user has to set the parallelism to `ExecutionConfig.PARALLELISM_AUTO_MAX`. Why not use all available task slots by default? I understand, that we shouldn't

[GitHub] flink pull request: [Flink-1436] refactor CLiFrontend to provide m...

2015-01-27 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/331#issuecomment-71649409 Thanks for your feedback, @rmetzger . The error message is at the bottom because that way it is most easily identifiable by the user (no scrolling necessary). Before, we

[GitHub] flink pull request: [FLINK-1442] Reduce memory consumption of arch...

2015-01-27 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/344#issuecomment-71616952 @StephanEwen Yes, that was on purpose. The previous two data structures (`HashMap` and `Queue`) are now replaced by the `LinkedHashMap` which serves the same functionality

[GitHub] flink pull request: [FLINK-1330] [build] Build creates a link in t...

2015-01-27 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/333#issuecomment-71633168 So either we can somehow change the execution order of the clean goal or we fix this in the junction plugin. --- If your project is set up for it, you can reply

[GitHub] flink pull request: [FLINK-1330] [build] Build creates a link in t...

2015-01-25 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/333#issuecomment-71366786 This looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request: [Flink-1436] refactor CLiFrontend to provide m...

2015-01-26 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/331#issuecomment-71446044 @rmetzger @StephanEwen I squashed the commits, merged the recent changes in the CliFrontend on the master, and addressed [FLINK-1424](https://issues.apache.org/jira/browse

[GitHub] flink pull request: [FLINK-1320] Add an off-heap variant of the ma...

2015-01-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/290#issuecomment-69361811 @rmetzger I added some documentation for the config parameter. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request: [FLINK-1486] add print method for prefixing a ...

2015-02-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/372#issuecomment-73504724 To make it a bit more explicit what is sink identifier and what is the task identifier (especially when just one of the two are printed), I prefixed the sink identifier

[GitHub] flink pull request: [FLINK-1436] refactor CLiFrontend to provide m...

2015-02-12 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/331#issuecomment-74088065 @rmetzger Thanks but I think it should be This closes #331 :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a GroupC...

2015-03-18 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/466#issuecomment-82879313 Any further comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request: [FLINK-1679] rename degree of parallelism to p...

2015-03-18 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/488#issuecomment-83095571 Any objections against merging this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-1688] Socket client sink added

2015-03-16 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/484#discussion_r26483463 --- Diff: flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/socket/SocketClientSink.java --- @@ -0,0

[GitHub] flink pull request: [FLINK-1688] Socket client sink added

2015-03-16 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/484#issuecomment-81665515 Thanks for the pull request. Could you add some javadoc to the class and add an entry in the Flink website docs [1]? [1] http://ci.apache.org/projects/flink/flink

[GitHub] flink pull request: [FLINK-1688] Socket client sink added

2015-03-16 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/484#discussion_r26483484 --- Diff: flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/socket/SocketClientSink.java --- @@ -0,0

[GitHub] flink pull request: [FLINK-1688] Socket client sink added

2015-03-16 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/484#discussion_r26483479 --- Diff: flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/socket/SocketClientSink.java --- @@ -0,0

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a GroupC...

2015-03-16 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/466#discussion_r26471022 --- Diff: flink-compiler/src/main/java/org/apache/flink/compiler/operators/GroupCombineProperties.java --- @@ -0,0 +1,112 @@ +/* + * Licensed

[GitHub] flink pull request: [FLINK-1679] rename degree of parallelism to p...

2015-03-16 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/488 [FLINK-1679] rename degree of parallelism to parallelism extend documentation about parallelism You can merge this pull request into a Git repository by running: $ git pull https://github.com

[GitHub] flink pull request: [FLINK-1679] rename degree of parallelism to p...

2015-03-17 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/488#issuecomment-82251340 @uce @StephanEwen I agree with you two that it's better to keep the old API methods and mark them as deprecated while introducing the new ones. When the API has been

[GitHub] flink pull request: [FLINK-1679] rename degree of parallelism to p...

2015-03-20 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/488#issuecomment-84058237 Any other opinions on the API breaking nature of these changes? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a partia...

2015-03-09 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/466 [FLINK-1622][java-api][scala-api] add a partial GroupReduce operator The partial GroupReduce operator acts like a regular GroupReduce operator but does not perform a full group reduce. Instead

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a partia...

2015-03-10 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/466#discussion_r26120661 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/operators/SortedGrouping.java --- @@ -156,6 +156,23 @@ public SortedGrouping(DataSetT set, KeysT

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26215888 --- Diff: docs/_includes/navbar.html --- @@ -24,15 +24,15 @@ We might be on an externally hosted documentation site. Please keep

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-12 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/479#issuecomment-78300593 Nice solution. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26219828 --- Diff: docs/_layouts/default.html --- @@ -23,16 +23,25 @@ meta http-equiv=X-UA-Compatible content=IE=edge meta name=viewport content=width

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a partia...

2015-03-10 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/466#issuecomment-78093403 @aljoscha @fhueske For a general combine, the operator can be used without grouping. When we want to combine elements before performing a proper groupReduce with a groupBy

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a partia...

2015-03-10 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/466#discussion_r26139203 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/operators/GroupReducePartialOperator.java --- @@ -0,0 +1,229 @@ +/* + * Licensed

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a partia...

2015-03-10 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/466#issuecomment-78076027 @aljoscha Thanks for the comments. I agree, the tests are a bit shady because they test the operator by first performing a partial, then a full reduce. Using a custom

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a partia...

2015-03-10 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/466#discussion_r26130234 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/operators/base/GroupReducePartialOperatorBase.java --- @@ -0,0 +1,150 @@ +/* + * Licensed

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a partia...

2015-03-10 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/466#discussion_r26130403 --- Diff: flink-scala/src/main/scala/org/apache/flink/api/scala/GroupedDataSet.scala --- @@ -355,6 +355,63 @@ class GroupedDataSet[T: ClassTag

[GitHub] flink pull request: [FLINK-1629][FLINK-1630][FLINK-1547] Rework Fl...

2015-03-11 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/468#issuecomment-78247421 Nice work. Looks good to me apart from the missing documentation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-11 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/479#issuecomment-78257904 Really nice work, Till! Very nicely documented. Just made a few comments concerning the paths for the documentation. --- If your project is set up for it, you can reply

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-11 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26205970 --- Diff: docs/_includes/navbar.html --- @@ -24,15 +24,15 @@ We might be on an externally hosted documentation site. Please keep

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-11 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26205980 --- Diff: docs/_includes/sidenav.html --- @@ -17,51 +17,51 @@ under the License. -- ul id=flink-doc-sidenav - lidiv class=sidenav-categorya

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-11 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26205978 --- Diff: docs/_includes/navbar.html --- @@ -49,15 +49,15 @@ ul class=nav navbar-nav li - a href=index.html class

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-11 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26205983 --- Diff: docs/_layouts/default.html --- @@ -23,16 +23,25 @@ meta http-equiv=X-UA-Compatible content=IE=edge meta name=viewport content=width

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-11 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26205976 --- Diff: docs/_includes/navbar.html --- @@ -49,15 +49,15 @@ ul class=nav navbar-nav li - a href=index.html class

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-11 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26205975 --- Diff: docs/_includes/navbar.html --- @@ -49,15 +49,15 @@ ul class=nav navbar-nav li - a href=index.html class

[GitHub] flink pull request: [FLINK-1622][java-api][scala-api] add a partia...

2015-03-13 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/466#issuecomment-79096322 @aljoscha @fhueske @hsaputra Thanks for the feedback. Some people suggested that the name is confusing and that my pull request involved too much code duplication. I propose

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-03-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/202#discussion_r26132770 --- Diff: flink-addons/flink-language-binding/src/main/python/org/apache/flink/languagebinding/api/python/dill/__diff.py --- @@ -0,0 +1,247

[GitHub] flink pull request: Kick off of Flink's machine learning library

2015-03-11 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/479#discussion_r26215901 --- Diff: docs/_includes/navbar.html --- @@ -49,15 +49,15 @@ ul class=nav navbar-nav li - a href=index.html class

[GitHub] flink pull request: Flink-1780: Rename FlatCombineFunction to Grou...

2015-03-25 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/530#discussion_r27104700 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java --- @@ -1425,11 +1425,9 @@ public static boolean isClassType(Type t

[GitHub] flink pull request: Flink-1780: Rename FlatCombineFunction to Grou...

2015-03-25 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/530#discussion_r27104945 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/RegularPactTask.java --- @@ -1465,7 +1467,9 @@ public static void cancelChainedTasks

[GitHub] flink pull request: Flink-1780: Rename FlatCombineFunction to Grou...

2015-03-25 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/530#discussion_r27104851 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/RegularPactTask.java --- @@ -525,7 +525,9 @@ protected void run() throws Exception

[GitHub] flink pull request: Flink-1780: Rename FlatCombineFunction to Grou...

2015-03-25 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/530#discussion_r27104741 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/AllGroupCombineDriver.java --- @@ -35,20 +35,20 @@ * Like

[GitHub] flink pull request: Flink-1780: Rename FlatCombineFunction to Grou...

2015-03-25 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/530#issuecomment-85954079 +1 This rename makes a lot of sense. Looks good to me apart from the minor comments. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request: Flink-1780: Rename FlatCombineFunction to Grou...

2015-03-25 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/530#issuecomment-86090016 The documentation also needs to be adapted to use the new interface name. @smarthi If you don't mind I will merge your commit with my remarks addressed. --- If your

[GitHub] flink pull request: [FLINK-1769] Fix deploy bug caused by ScalaDoc...

2015-03-26 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/535#issuecomment-86449530 +1 for merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request: [FLINK-1589] Add option to pass configuration ...

2015-04-01 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/427#discussion_r27564491 --- Diff: flink-clients/src/main/java/org/apache/flink/client/LocalExecutor.java --- @@ -103,6 +110,9 @@ public void start() throws Exception

[GitHub] flink pull request: [FLINK-1589] Add option to pass configuration ...

2015-04-01 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/427#discussion_r27564493 --- Diff: flink-clients/src/main/java/org/apache/flink/client/LocalExecutor.java --- @@ -70,6 +72,11 @@ public LocalExecutor

[GitHub] flink pull request: [FLINK-1589] Add option to pass configuration ...

2015-04-01 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/427#discussion_r27569381 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/ExecutionEnvironment.java --- @@ -1058,6 +1059,20 @@ public static LocalEnvironment

[GitHub] flink pull request: [FLINK-1790]Remove the redundant import code

2015-03-27 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/538#issuecomment-86930773 :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request: Add auto-parallelism to Jobs (0.8 branch)

2015-03-03 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/410#issuecomment-76978687 @rmetzger I don't see a reason why this should not go to master as well. After all, it's optional and quite useful if you want to run a job on the full cluster with as many

[GitHub] flink pull request: [FLINK-1624] fix failing build due to maven gi...

2015-03-02 Thread mxm
Github user mxm closed the pull request at: https://github.com/apache/flink/pull/445 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled

[GitHub] flink pull request: Implement the convenience methods count and co...

2015-02-27 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/210#issuecomment-76385275 I moved the `writeInto(ByteBuf buf)` method to the subclasses of `AbstractID` used by `NettyMessage`. This is the only context where the method is being used

[GitHub] flink pull request: [FLINK-1501] Add metrics library for monitorin...

2015-02-23 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/421#issuecomment-75566621 @fhueske The user could do so by selecting Show all task managers and then identify the struggling task manager. For large cluster setups, it makes sense to sample just from

[GitHub] flink pull request: [FLINK-1501] Add metrics library for monitorin...

2015-02-23 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/421#issuecomment-75554187 Looks really nice and informative :+1: Some suggestions: - It would be great if one could specify the number of task managers to see. If the number of task

[GitHub] flink pull request: [FLINK-1501] Add metrics library for monitorin...

2015-02-23 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/421#issuecomment-75600898 I didn't fully understand that you wanted to have one chart containing all task managers' load. That's a good thing. If it is only one chart, the overhead to update it should

[GitHub] flink pull request: Implement the convenience methods count and co...

2015-02-26 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/210#discussion_r25426681 --- Diff: flink-core/pom.xml --- @@ -58,6 +58,12 @@ under the License. version0.5.1/version /dependency

[GitHub] flink pull request: Implement the convenience methods count and co...

2015-02-26 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/210#issuecomment-76151146 I squashed the commits and rebased to the current master. Any objections against merging this? --- If your project is set up for it, you can reply to this email and have

[GitHub] flink pull request: [Flink-1436] refactor CLiFrontend to provide m...

2015-01-23 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/331 [Flink-1436] refactor CLiFrontend to provide more identifiable and meaningful error messages You can merge this pull request into a Git repository by running: $ git pull https://github.com/mxm

[GitHub] flink pull request: [FLINK-1679] rename degree of parallelism to p...

2015-03-23 Thread mxm
Github user mxm closed the pull request at: https://github.com/apache/flink/pull/488 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled

[GitHub] flink pull request: [FLINK-1679] rename degree of parallelism to p...

2015-03-23 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/488#issuecomment-84875935 Merged in master with 126f9f799071688fe80955a7e7cfa991f53c95af --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request: [FLINK-1679] rename degree of parallelism to p...

2015-03-23 Thread mxm
GitHub user mxm reopened a pull request: https://github.com/apache/flink/pull/488 [FLINK-1679] rename degree of parallelism to parallelism extend documentation about parallelism https://issues.apache.org/jira/browse/FLINK-1679 You can merge this pull request into a Git repository

[GitHub] flink pull request: [FLINK-1768]Fix the bug of BlobServerConnectio...

2015-03-23 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/514#issuecomment-84933938 Thanks for spotting this. We will merge your change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request: Stream graph + internal refactor

2015-04-14 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/594#issuecomment-92780564 This is a huge PR but from what I've seen you made very sensible changes. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink pull request: [scheduling] implement backtracking with inter...

2015-04-14 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/595#discussion_r28319765 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -339,6 +339,12 @@ extends Actor with ActorLogMessages

[GitHub] flink pull request: [FLINK-1887]Fix the message in runtime excepti...

2015-04-14 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/599#issuecomment-92775321 Good to merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request: [scheduling] implement backtracking with inter...

2015-04-13 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/595 [scheduling] implement backtracking with intermediate result checking - backtracks from the sinks of an ExecutionGraph - checks the availability of IntermediatePartitionResults - marks

[GitHub] flink pull request: [FLINK-1908] JobManager startup delay isn't co...

2015-04-20 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/609#issuecomment-94401080 Thanks for the pull request. Seems to work fine. I was wondering, shouldn't the task managers repeatably try to build up a connection to the job manager? For me, that seems

[GitHub] flink pull request: [FLINK-1486] add print method for prefixing a ...

2015-04-21 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/372#issuecomment-94703910 I've added documentation for the new print method. Will merge later on. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] flink pull request: [FLINK-377] [FLINK-671] Generic Interface / PA...

2015-04-21 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/202#issuecomment-94798580 Congraluations! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request: [FLINK-1486] add print method for prefixing a ...

2015-04-24 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/372#issuecomment-95880258 Yes, it should be simple for the user. It makes sense to have one print method which just prints the output on the client. In addition, we could have another *advanced* print

[GitHub] flink pull request: [FLINK-1486] add print method for prefixing a ...

2015-04-24 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/372#issuecomment-95855310 Just saying that a prefix helps to identify output, even if everything is printed on the client. Additionally, including the task id in the output can be useful for debugging

[GitHub] flink pull request: [FLINK-938] Auomatically configure the jobmana...

2015-04-24 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/248#issuecomment-95990793 This will work for most use cases where the JobManager is started within the same cluster as the TaskManagers. It will certainly fail if the JobManager resides in a different

[GitHub] flink pull request: [FLINK-1486] add print method for prefixing a ...

2015-04-23 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/372#issuecomment-95488183 Do we want to break backwards compatibility or include a new method for printing on the client? After all, printing on the workers is a useful tool to debug the dataflow

[GitHub] flink pull request: [FLINK-1486] add print method for prefixing a ...

2015-04-20 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/372#issuecomment-94502825 I've updated the pull request. I decided to implement the concise method: ``` sinkId:taskId output - sink id provided, parallelism 1 sinkId output - sink

[GitHub] flink pull request: [FLINK-1927] [py] Operator distribution rework

2015-04-29 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/638#issuecomment-97353102 Wow! Great to see that we get rid of the only python-side dependency. It was a bit unclear under which terms we could ship the library anyways. Have you done any measurements

[GitHub] flink pull request: [FLINK-1843] remove SoftReferences on archived...

2015-04-29 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/639 [FLINK-1843] remove SoftReferences on archived ExecutionGraphs The previously introduced SoftReferences to store archived execution graphs cleared old graphs in a non-transparent order. This pull

[GitHub] flink pull request: [scheduling] implement backtracking of interme...

2015-04-30 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/640#discussion_r29432849 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Backtracking.java --- @@ -0,0 +1,252 @@ +/* + * Licensed to the Apache

  1   2   3   4   5   6   7   8   9   10   >