[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40442440 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40442383 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40442370 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40442340 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40442163 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40442050 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40441936 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40441863 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40441800 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40441437 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40441149 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-1719] [ml] Add Multinomial Naive Bayes ...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1156#discussion_r40440950 --- Diff: flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/classification/MultinomialNaiveBayes.scala --- @@ -0,0 +1,900

[GitHub] flink pull request: [FLINK-2761][scala-shell]Prevent creation of n...

2015-09-25 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1180#issuecomment-143206548 @chiwanpark , done. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] flink pull request: [FLINK-2761][scala-shell]Prevent creation of n...

2015-09-25 Thread sachingoel0101
GitHub user sachingoel0101 opened a pull request: https://github.com/apache/flink/pull/1180 [FLINK-2761][scala-shell]Prevent creation of new environment in Scala shell You can merge this pull request into a Git repository by running: $ git pull https://github.com

[GitHub] flink pull request: Parameter Server: Distributed Key-Value store,...

2015-09-21 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1003#issuecomment-142190932 Yes. I agree. I'm currently working on finalizing the interface. Just waiting for another PR to get in. And involving the Job Manager is not str

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-21 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-141926745 Okay. Great. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] flink pull request: [FLINK-2488][FLINK-2496] Expose Task Manager c...

2015-09-19 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1026#issuecomment-141653638 Unrelated failure on travis. Filed a jira [2711] --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-18 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-141471679 Rebased to reflect the changes in the scala utility functions. Travis failure on an unrelated error. Reported at jira id 2700. This has already

[GitHub] flink pull request: [FLINK-2488][FLINK-2496] Expose Task Manager c...

2015-09-18 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1026#issuecomment-141461094 Is it possible to get this in soon? I need access to the task manager configuration for something I'm working on. @StephanEwen --- If your project is s

[GitHub] flink pull request: [FLINK-2312][utils] Randomly Splitting a Data ...

2015-09-18 Thread sachingoel0101
Github user sachingoel0101 closed the pull request at: https://github.com/apache/flink/pull/921 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-2312][utils] Randomly Splitting a Data ...

2015-09-18 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/921#issuecomment-141406447 Right now, there is no way to achieve this. After support for persisted results is added, this can be re-visited again. Closing for now. --- If your project is

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-18 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-141404362 I've already removed the line break. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-18 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-141391475 Unrelated failures. Already filed jiras for those. 2700 and 2612. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-17 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-141148749 Hey @StephanEwen, apologies for being too eager but is it possible to get this in soon? --- If your project is set up for it, you can reply to this email and

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-09-11 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/945#discussion_r39268191 --- Diff: flink-core/src/main/java/org/apache/flink/util/VersionUtils.java --- @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-10 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-139137139 @StephanEwen , can you look this over again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] flink pull request: [FLINK-1730]Persist operator on Data Sets

2015-09-09 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1083#issuecomment-138926576 Aha. I had only searched Jira for any existing work on this. Anyways, I'm assuming you'll be rebasing the same PR after session management is in.

[GitHub] flink pull request: [FLINK-1730]Persist operator on Data Sets

2015-09-09 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1083#issuecomment-138921118 Great! One step closer. Would love to see this feature soon. :) --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] flink pull request: [FLINK-1730]Persist operator on Data Sets

2015-09-09 Thread sachingoel0101
Github user sachingoel0101 closed the pull request at: https://github.com/apache/flink/pull/1083 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] flink pull request: [FLINK-1730]Persist operator on Data Sets

2015-09-09 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1083#issuecomment-138868165 I apologize. Like I said before, the way I've implemented this, it seemed pretty trivial. Foremost, I needed this for something else and decided to take a

[GitHub] flink pull request: [FLINK-1730]Persist operator on Data Sets

2015-09-09 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1083#issuecomment-138844139 Hey @fhueske , thanks for your comments. I was not aware this was intended to allow for recovery on failed jobs. For reusing among different jobs in the

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-08 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-138530591 Travis passes successfully. I've squashed the commits. This should be mergeable now. --- If your project is set up for it, you can reply to this emai

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-08 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-138504819 Ah. Thank you @aljoscha. Travis should pass. I've already pushed a fix. --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink pull request: [FLINK-2488][FLINK-2496] Expose Task Manager c...

2015-09-08 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1026#issuecomment-138503010 Sure. No worries. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-08 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-138502471 I think I agree with that. I wasn't too happy about using implicit arguments here; we're constructing the type information explicitly anyway. W

[GitHub] flink pull request: Parameter Server: Distributed Key-Value store,...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1003#issuecomment-138419004 @StephanEwen , I had a look at #967 and I quite agree with the interface designed by @nltran. I believe I can separate out my parameter server implementation

[GitHub] flink pull request: Stale Synchronous Parallel Iterations

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/967#issuecomment-138416633 @nltran , I went through you code and if I understand correctly, the slack-related synchronization is done by the Runtime itself using events. I like that idea

[GitHub] flink pull request: [FLINK-2488][FLINK-2496] Expose Task Manager c...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1026#issuecomment-138404446 Will there be any further review of this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-138401226 @StephanEwen , I have created a separate function to create type information for 2-tuple. One question though. Why is there a need to generate type

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1099#discussion_r38869305 --- Diff: flink-scala/src/main/scala/org/apache/flink/api/scala/utils/package.scala --- @@ -0,0 +1,124 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1099#discussion_r38868511 --- Diff: flink-scala/src/main/scala/org/apache/flink/api/scala/utils/package.scala --- @@ -0,0 +1,124 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-138318843 @tillrohrmann , can you do a final review of this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink pull request: [FLINK-2626][CORE]add averageaccumulator

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1096#issuecomment-138279804 @mjsax, I'm not sure what's the need for those. The `add(Double)` is only needed because the interface requires this. `add(Long)` is equivalent to

[GitHub] flink pull request: [FLINK-2626][CORE]add averageaccumulator

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1096#issuecomment-138277626 @mjsax , primitive add methods for accumulators were introduced in #942 to get rid of potential unnecessary boxing and unboxing. --- If your project is set up

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-138277563 I get the following error on using that: `macro implementation not found: createTypeInformation (the most common reason is that you cannot use macro

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1099#issuecomment-138258862 I am unable to get rid of the implicit type information for the `zip` functions, presumably because the type information for `(Long,T)` isn't found. --- If

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1099#discussion_r38842188 --- Diff: flink-scala/src/main/scala/org/apache/flink/api/scala/utils/package.scala --- @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-2626][CORE]add averageaccumulator

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1096#discussion_r38839638 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/accumulators/AverageAccumulator.java --- @@ -0,0 +1,98 @@ +/* + * Licensed to

[GitHub] flink pull request: [FLINK-2626][CORE]add averageaccumulator

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1096#discussion_r38838895 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/accumulators/AverageAccumulator.java --- @@ -0,0 +1,98 @@ +/* + * Licensed to

[GitHub] flink pull request: [FLINK-2626][CORE]add averageaccumulator

2015-09-07 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/1096#discussion_r38838868 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/accumulators/AverageAccumulator.java --- @@ -0,0 +1,98 @@ +/* + * Licensed to

[GitHub] flink pull request: [FLINK-2627][utils]Make Scala Data Set utils e...

2015-09-07 Thread sachingoel0101
GitHub user sachingoel0101 opened a pull request: https://github.com/apache/flink/pull/1099 [FLINK-2627][utils]Make Scala Data Set utils easier to access Introduces a package object for Scala data set utils to simplify usage. New usage: `import org.apache.flink.api.scala.utils

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-05 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137952549 @thvasilo , @chiwanpark I have added a wrapper for the scala utils class. Further, there are a few more methods: 1. For Discrete histograms: *entropy* and

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-05 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137937841 Okay. Let me revert the commit and make this independently mergeable --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137802768 I have split this into two commits. @thvasilo, if there is need to file a jira, I'll update the second commit message to indicate that. --- If your pr

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137778976 Okay. Instead of adding the Scala api wrappers and add to the complexity [we might have another import such as

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137776491 @thvasilo , I'm not sure I got that right. I hadn't tested the changed `DataSetUtils`. I'm not sure how to handle the implicit types and class

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137768632 IMO this isn't a good idea. We're adding an additional mapping step, and creating as many objects unnecessarily. Sure, chaining will make the mapping

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137763865 1. @thvasilo , I have changed the scala `DataSetUtils` class to make it a bit more easy to use. Now, it doesn't require an impo

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137748205 Hello @thvasilo , I was unable to access the `create...` functions from the scala API. I'm not sure how to pass a `DataSet[Double]` to a `DataSet[java.lang.D

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137695084 Haha. This patch is 1000+, 0-. :laughing: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/861#discussion_r38736955 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/accumulators/ContinuousHistogram.java --- @@ -0,0 +1,534 @@ +/* + * Licensed to

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-04 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/861#discussion_r38736695 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/accumulators/ContinuousHistogram.java --- @@ -0,0 +1,534 @@ +/* + * Licensed to

[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...

2015-09-03 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1088#issuecomment-137421268 Sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...

2015-09-03 Thread sachingoel0101
Github user sachingoel0101 closed the pull request at: https://github.com/apache/flink/pull/1088 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...

2015-09-03 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1088#issuecomment-137420928 Aha. I just had a brief look at the PR. I had initially thought of implementing a start and stop method in the `ExecutionEnvironment` but decided against it

[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...

2015-09-03 Thread sachingoel0101
GitHub user sachingoel0101 opened a pull request: https://github.com/apache/flink/pull/1088 [FLINK-2615]Preserve executors for the entire run of program. 1. Prevents re-starts of local cluster several times on multiple executions. 2. Preserves the executor states for the entire

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-03 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137399688 Hey @chiwanpark , travis passes. Let me know if I should squash the commits. I'm only keeping them in case we need to go back to the Scala implementation. [

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-03 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137382016 Okay. Please take a look now. [There is an unintentional formatting introduced by IDE in Histogram.java. Will remove it while squashing.] --- If your

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-03 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137380847 Yes. For Discrete fields, quantiles do not make sense. In the paper, they only cover the continuous fields, since the Discrete fields are more or less trivial to

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-03 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137378160 Yes. I was concerned about the API breaking part too. What about keeping the original histogram as such, and add something like `DoubleHistogram`, which

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-03 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-137358805 Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request: [FLINK-2488][FLINK-2496] Expose Task Manager c...

2015-09-02 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1026#issuecomment-137116109 Travis Passes successfully. https://travis-ci.org/apache/flink/builds/78368635 --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink pull request: [FLINK-2472]Make JobClientActor poll JobManage...

2015-09-02 Thread sachingoel0101
Github user sachingoel0101 closed the pull request at: https://github.com/apache/flink/pull/979 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-2472]Make JobClientActor poll JobManage...

2015-09-02 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/979#issuecomment-137010239 This most likely requires a re-work because of the latest changes in Job Manager and Client. Closing for now while I re-examine all the logic. Will reopen it in a

[GitHub] flink pull request: [FLINK-2488][FLINK-2496] Expose Task Manager c...

2015-09-02 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1026#issuecomment-137009547 Rebased to the current master. This should be mergeable now. Travis fails on unrelated kafka and flink-fs-tests. Just re-triggered another build. @StephanEwen

[GitHub] flink pull request: [FLINK-2379][ml]Add column wise statistics for...

2015-09-02 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1032#issuecomment-136978688 @tillrohrmann , can you review this? This will be a starting point for a package to provide more statistical methods, such as hypothesis testing, correlation

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-01 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-136791499 Okay. Sure. :) No problem. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][core][utils]Histogram...

2015-09-01 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-136789371 @chiwanpark , can you review this again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request: [FLINK-2448]Create new Test environments on ge...

2015-09-01 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1031#issuecomment-136780181 Okay. Great. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request: [FLINK-1730]Persist operator on Data Sets

2015-09-01 Thread sachingoel0101
GitHub user sachingoel0101 opened a pull request: https://github.com/apache/flink/pull/1083 [FLINK-1730]Persist operator on Data Sets This PR introduces a `persist` operation on `DataSet` which allows persisting the data set in memory, allowing for direct access if this data set

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-31 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/945#issuecomment-136356151 Rebased to the current master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request: [FLINK-2448]Create new Test environments on ge...

2015-08-31 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1031#issuecomment-136349300 @StephanEwen, this should be mergeable now. I've separated the Gelly change as a second commit. --- If your project is set up for it, you can reply to

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-27 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/945#issuecomment-135372659 @tillrohrmann , I have modified the way version information is passed to the Job Manager. It is now through the Configuration object passed. Further, rolled

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-26 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/945#issuecomment-135045372 Hi @tillrohrmann , thanks for the review. You're right. `getClass...` is not the right way to go. I had decided to just use this for the moment. We can cert

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-26 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/945#discussion_r37989581 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/jobmanager/JobSubmitTest.java --- @@ -170,4 +170,43 @@ public void initializeOnMaster

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-26 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/945#discussion_r37989473 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -547,7 +552,7 @@ class TaskManager

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-26 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/945#discussion_r37989277 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/Configuration.java --- @@ -54,8 +54,12 @@ /** Stores the

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-26 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/945#discussion_r37989214 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/Configuration.java --- @@ -54,8 +54,12 @@ /** Stores the

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-26 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/945#discussion_r37988692 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -524,13 +534,24 @@ class JobManager

[GitHub] flink pull request: [FLINK-2399] Version checks for Job Manager an...

2015-08-26 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/945#discussion_r37988579 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -166,7 +169,8 @@ class JobManager

[GitHub] flink pull request: [FLINK-2472]Make JobClientActor poll JobManage...

2015-08-24 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/979#discussion_r37768306 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java --- @@ -49,27 +57,127 @@ // Actor which submits a

[GitHub] flink pull request: [FLINK-2448]Create new Test environments on ge...

2015-08-24 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1031#issuecomment-134265716 Yes, that's perfectly doable. I actually did that initially. The only reason to add it to the `ExecutionEnvironment` was to remove that little bit of

[GitHub] flink pull request: [FLINK-2472]Make JobClientActor poll JobManage...

2015-08-24 Thread sachingoel0101
Github user sachingoel0101 commented on a diff in the pull request: https://github.com/apache/flink/pull/979#discussion_r37767947 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java --- @@ -144,11 +268,25 @@ else if (message instanceof

[GitHub] flink pull request: [FLINK-1901] [core] Create sample operator for...

2015-08-24 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/949#issuecomment-134081197 Yes. I was only wondering if we should at least ensure this when it is done right at the source though. --- If your project is set up for it, you can reply to

[GitHub] flink pull request: [FLINK-1901] [core] Create sample operator for...

2015-08-24 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/949#issuecomment-134070592 Hey @ChengXiangLi, I have another concern, regarding the seed for sampling. It doesn't seem to serve its purpose. I tried sampling with fraction three times

[GitHub] flink pull request: [FLINK-2448]Clear cache file list in Execution...

2015-08-22 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1031#issuecomment-133788218 Travis passes. :) I asked on the dev list about the two small changes in flink-gelly. @vasia --- If your project is set up for it, you can reply to this

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][ml]Online Histograms ...

2015-08-22 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-133707118 @tillrohrmann, thanks for the brilliant suggestions. Using a `TreeMap` and `PriorityQueue` with invalidation, I've managed to bring down the complexity of

[GitHub] flink pull request: [FLINK-1901] [core] Create sample operator for...

2015-08-21 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/949#issuecomment-133461987 Hey @ChengXiangLi , I just observed a failure on a test case: https://travis-ci.org/sachingoel0101/flink/jobs/76649177 Here is the relevant statement

[GitHub] flink pull request: [FLINK-2448]Clear cache file list in Execution...

2015-08-21 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/1031#issuecomment-133449024 Okay, I figured out a way to share the environments. :) Let me know if this is a valid approach. Waiting for travis to pass. --- If your project is set

[GitHub] flink pull request: [FLINK-2030][FLINK-2274][ml]Online Histograms ...

2015-08-21 Thread sachingoel0101
Github user sachingoel0101 commented on the pull request: https://github.com/apache/flink/pull/861#issuecomment-133410381 Hi @tillrohrmann , I have addressed most of the comments. 1. Improved the documentation 2. Ported the continuous histogram implementation to `TreeMap` 3

<    1   2   3   4   5   6   7   >