Re: MultipleLinearRegression - Strange results

2015-06-02 Thread Till Rohrmann
Great to hear. This should no longer be a pain point once we support proper cross validation. On Tue, Jun 2, 2015 at 11:11 AM, Felix Neutatz neut...@googlemail.com wrote: Yes, grid search solved the problem :) 2015-06-02 11:07 GMT+02:00 Till Rohrmann till.rohrm...@gmail.com: The SGD

Re: MultipleLinearRegression - Strange results

2015-06-02 Thread Till Rohrmann
The SGD algorithm adapts the learning rate accordingly. However, this does not help if you choose the initial learning rate too large because then you calculate a weight vector in the first iterations from which it takes really long to recover. Cheer, Till On Mon, Jun 1, 2015 at 7:15 PM, Sachin

[jira] [Created] (FLINK-2132) Java version parsing is not working for OpenJDK

2015-06-02 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-2132: -- Summary: Java version parsing is not working for OpenJDK Key: FLINK-2132 URL: https://issues.apache.org/jira/browse/FLINK-2132 Project: Flink Issue Type: Bug

Re: MultipleLinearRegression - Strange results

2015-06-02 Thread Felix Neutatz
Yes, grid search solved the problem :) 2015-06-02 11:07 GMT+02:00 Till Rohrmann till.rohrm...@gmail.com: The SGD algorithm adapts the learning rate accordingly. However, this does not help if you choose the initial learning rate too large because then you calculate a weight vector in the

Re: Validate (commons) versus checkArgument (guava)

2015-06-02 Thread Robert Metzger
We have now replaced all commons validate calls with guava preconditions but its not written down anywhere or enforced by anything. Who would like to take care of that? On Thu, Mar 26, 2015 at 11:03 AM, Robert Metzger rmetz...@apache.org wrote: I didn't know that there was already an issue for

Re: ClassNotFoundException : org.apache.flink.api.common.operators.util.UserCodeObjectWrapper, while trying to run locally

2015-06-02 Thread santosh_rajaguru
Thanks Robert and Stephan. I executed the program in maven environment. It works fine. However, in Plugin environment, though i have included all the dependencies and dependent jars, it says classnotfoundexception while executing the env. -- View this message in context:

Memory management overhaul

2015-06-02 Thread Alexander Alexandrov
During an offline chat some time ago Stephan Ewen mentioned that there is an ongoing effort for a dynamic memory allocation in some feature branch lying around. Can you point me to that, as I would like to look at the code? Thanks.

Send events to parallel operator instances

2015-06-02 Thread Gyula Fóra
Hi, I am wondering, what is the suggested way to send some events directly to another parallel instance in a flink job? For example from one mapper to another mapper (of the same operator). Do we have any internal support for this? The first thing that we thought of is iterations but that is

[jira] [Created] (FLINK-2138) PartitionCustom for streaming

2015-06-02 Thread JIRA
Márton Balassi created FLINK-2138: - Summary: PartitionCustom for streaming Key: FLINK-2138 URL: https://issues.apache.org/jira/browse/FLINK-2138 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-2139) Test Streaming Outputformats

2015-06-02 Thread JIRA
Márton Balassi created FLINK-2139: - Summary: Test Streaming Outputformats Key: FLINK-2139 URL: https://issues.apache.org/jira/browse/FLINK-2139 Project: Flink Issue Type: Test

Re: Validate (commons) versus checkArgument (guava)

2015-06-02 Thread Lokesh Rajaram
Hello Ufuk, Robert, @Ufuk if you can create a ticket and assign it to me that would be very helpful @Robert I can definitely update those two documents. Just now I was checking and looks like we can control this using checkstyle http://checkstyle.sourceforge.net/config_imports.html#IllegalImport

Re: Changed the behavior of DataSet.print()

2015-06-02 Thread Kostas Tzoumas
+1 for printOnTaskManager(prefix) On Tue, Jun 2, 2015 at 1:35 PM, Till Rohrmann trohrm...@apache.org wrote: +1 for printOnTaskManager(prefix) On Tue, Jun 2, 2015 at 12:08 PM, Fabian Hueske fhue...@gmail.com wrote: +1 for writeToWorkerStdOut(prefix) On Jun 2, 2015 11:42, Aljoscha Krettek

Re: [FLINK-1731] [ML] Issues with vector to breeze converter while implementing KMeans

2015-06-02 Thread Till Rohrmann
Hi Florian, I just wrote a patch for this problem. I wait until all tests pass and then I’ll merge the fix. Thus, it will be included in the current master in the late afternoon. If you don’t want to wait that long, then you can also solve the issue with

Re: Changed the behavior of DataSet.print()

2015-06-02 Thread Till Rohrmann
+1 for printOnTaskManager(prefix) On Tue, Jun 2, 2015 at 12:08 PM, Fabian Hueske fhue...@gmail.com wrote: +1 for writeToWorkerStdOut(prefix) On Jun 2, 2015 11:42, Aljoscha Krettek aljos...@apache.org wrote: +1 for printOnTaskManager(prefix) On Tue, Jun 2, 2015 at 11:35 AM, Robert

Re: Travis build failing

2015-06-02 Thread Aljoscha Krettek
The last one is actually SuccessAfterNetworkBuffersFailureITCase which uses ConnectedComponents. On Tue, Jun 2, 2015 at 5:42 PM, Robert Metzger rmetz...@apache.org wrote: Hi, I think there are currently at least three tests randomly failing on Travis: - the yarn failure you've just reported -

Travis build failing

2015-06-02 Thread Sachin Goel
I recently merged the master branch to something I've been working on. However, the travis build is failing with this error: I have made no changes to the existing files, only added some new files and the tests for those actually pass properly. The build is here:

Writing test suites

2015-06-02 Thread Sachin Goel
Hi What's the convention with using an ITSuite in the name instead of just Suite? Regards Sachin Goel

[jira] [Created] (FLINK-2135) Java plan translation fails with ClassCastException (probably in first())

2015-06-02 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-2135: - Summary: Java plan translation fails with ClassCastException (probably in first()) Key: FLINK-2135 URL: https://issues.apache.org/jira/browse/FLINK-2135 Project:

Re: Writing test suites

2015-06-02 Thread Aljoscha Krettek
Hi, the difference is that the IT (Integration Test) suites are only run when mvn verify or mvn install is executed. The other suites (or tests) are ran when mvn test is executed. Typically the non-IT tests are more lightweight while the IT tests can do heavy stuff that takes a bit longer. When

Re: [2/2] flink git commit: [ml] Replaces RichMapFunctions with mapWithBcVariable in FlinkML

2015-06-02 Thread Till Rohrmann
Yes you’re right Sachin. The mapWithBcVariable is only syntactic sugar if you have a broadcast DataSet which contains only one element. If you have multiple elements in your DataSet then you can’t use this method. But we can define another method mapWithBcSet which takes a function f: (element: T,

Re: [DISCUSS] Consolidate method naming between the batch and streaming API

2015-06-02 Thread Szabó Péter
Great proposal! We should use consistent naming for the two API. Peter 2015-06-01 21:11 GMT+02:00 Márton Balassi balassi.mar...@gmail.com: @Fabian: I hope that this is the complete list, correct me f I am wrong. :) I am opening a small PR with the changes on top of Aljoscha's one that