[jira] [Created] (FLINK-2189) NullPointerException in MutableHashTable

2015-06-09 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-2189: Summary: NullPointerException in MutableHashTable Key: FLINK-2189 URL: https://issues.apache.org/jira/browse/FLINK-2189 Project: Flink Issue Type: Bug

Re: Checkstyle in IntelliJ

2015-06-09 Thread Maximilian Michels
Hi Pieter-Jan, It would be great to have a plugin for IntelliJ/Eclipse to make new code stylecheck-compliant. However, as Till mentioned, the problem is that most such plugins touch more lines than necessary. We try to only commit changes to the Git repository which are related to the

Re: Checkstyle in IntelliJ

2015-06-09 Thread Pieter-Jan Van Aeken
Hi Till, If I recall correctly, there is a possibility to import checkstyle XML's into Eclipse so that the auto format feature would result in style compliant code. This imported Eclipse config could then be exported and reimported into IntelliJ but you can imagine that is not a reason for me to

Re: Checkstyle in IntelliJ

2015-06-09 Thread Aljoscha Krettek
By the way, do you have the Flink checkstyle and scalastyle profiles set in IntelliJ? This way you at least get red errors directly in the IDE. For checkstyle there is Checkstyle-IDEA and for scalastyle you can put the scalastyle config of Flink into the .idea directory to have it recognised: cp

Re: ALS implementation

2015-06-09 Thread Till Rohrmann
I think I found the possible error. I suspect that the empirical risk calculation causes the problem with the *Hash join exceeded maximum number of recursions*. What you do for this calculation is to provide the training data set DataSet[(Int, Int, Double)] and you calculate for each item the

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Till Rohrmann
I also encountered a failing TaskManagerFailsWithSlotSharingITCase using Java8. I could, however, not reproduce the error a second time. The stack trace is: The JobManager should handle hard failing task manager with slot

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Maximilian Michels
The name of the Git branch was not correct. Thank you, Aljoscha, for noticing. I've changed it from release-0.9-rc1 to release-0.9.0-rc1. This has no affect on the validity of the release candidate.

[jira] [Created] (FLINK-2188) Reading from big HBase Tables

2015-06-09 Thread Hilmi Yildirim (JIRA)
Hilmi Yildirim created FLINK-2188: - Summary: Reading from big HBase Tables Key: FLINK-2188 URL: https://issues.apache.org/jira/browse/FLINK-2188 Project: Flink Issue Type: Bug

Checkstyle in IntelliJ

2015-06-09 Thread Pieter-Jan Van Aeken
Hi Flink Community, I would like to contribute to the code base of Flink but I find myself unable to built my changed project due to errors in my code style. My IDE of choice is IntelliJ and unfortunately, I can not import tools/maven/checkstyle.xml nor tools/maven/scalastyle-config.xml. From

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Aljoscha Krettek
I would suggest we use this format to notify others that we did a task: Assignees: - Aljoscha: done - Ufuk: found bug in such an such... - Chiwan Park: done, ... The simple status doesn't work with multiple people on one task. On Tue, Jun 9, 2015 at 9:40 AM, Ufuk Celebi u...@apache.org

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Maximilian Michels
+1 makes sense. On Tue, Jun 9, 2015 at 10:48 AM, Aljoscha Krettek aljos...@apache.org wrote: I would suggest we use this format to notify others that we did a task: Assignees: - Aljoscha: done - Ufuk: found bug in such an such... - Chiwan Park: done, ... The simple status doesn't work

Re: Checkstyle in IntelliJ

2015-06-09 Thread Till Rohrmann
Hi Pieter-Jan, I'm not aware of an Eclipse or IntellJ auto format profile. I think that all Flink contributors apply their style changes manually. The maven output should tell you quite precisely what's wrong and in which file the checkstyle errors occur. Moreover, applying an IDE auto format is

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Aljoscha Krettek
I also saw the same error on my third mvn clean verify run. Before it always failed in the YARN tests. On Tue, Jun 9, 2015 at 12:23 PM, Till Rohrmann trohrm...@apache.org wrote: I also encountered a failing TaskManagerFailsWithSlotSharingITCase using Java8. I could, however, not reproduce the

Re: Checkstyle in IntelliJ

2015-06-09 Thread Till Rohrmann
But then we should also provide a code style profile for Eclipse and have to keep them in sync. On Tue, Jun 9, 2015 at 12:33 PM Aljoscha Krettek aljos...@apache.org wrote: code_scheme name=Flink option name=CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND value=100 / option name=RIGHT_MARGIN value=100

Re: Checkstyle in IntelliJ

2015-06-09 Thread Aljoscha Krettek
code_scheme name=Flink option name=CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND value=100 / option name=RIGHT_MARGIN value=100 / XML option name=XML_LEGACY_SETTINGS_IMPORTED value=true / /XML codeStyleSettings language=JAVA option name=ALIGN_MULTILINE_PARAMETERS value=false / option

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Sachin Goel
On my local machine, several flink runtime tests are failing on mvn clean verify. Here is the log output: http://pastebin.com/raw.php?i=VWbx2ppf -- ​ Sachin​ On Tue, Jun 9, 2015 at 4:04 PM, Aljoscha Krettek aljos...@apache.org wrote: I also saw the same error on my third mvn clean verify run.

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Aljoscha Krettek
I did five mvn clean verify runs by now. All of them failed. One with the TaskmanagerFailsWithSlotSharingITCase and the other ones with YARNSessionFIFOITCase On Tue, Jun 9, 2015 at 12:34 PM, Aljoscha Krettek aljos...@apache.org wrote: I also saw the same error on my third mvn clean verify run.

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Ufuk Celebi
On 09 Jun 2015, at 13:58, Sachin Goel sachingoel0...@gmail.com wrote: On my local machine, several flink runtime tests are failing on mvn clean verify. Here is the log output: http://pastebin.com/raw.php?i=VWbx2ppf Thanks for reporting this. Have you tried it multiple times? Is it failing

Code Style

2015-06-09 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, I was told that Flink follows a strict code style (eg, putting a blank between 'if' and the opening parentheses). However, I could not find any documentation about it. I would like to follow those rules if there are any. Can you please give

Re: Code Style

2015-06-09 Thread Aljoscha Krettek
These are the only format coding guidelines I'm aware of: http://flink.apache.org/coding-guidelines.html Some other stuff (such as space between 'if' and open parentheses) is implicit. It seems to be common Java style and the code that we already have follows it. I would say that the existing

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Aljoscha Krettek
I discovered something that might be a feature, rather than a bug. When you submit an example using the web client without giving parameters the program fails with this: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error. at

RE: Buffer re-ordering problem

2015-06-09 Thread Kruse, Sebastian
Hi everyone, I just wanted to let you know, that after quite a few more runs on different machines the buffer reordering problem did not happen to appear again. I don't know what caused the problem, maybe it really was due to the potentially illegal UTF-16 code within strings. If the error

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Chiwan Park
I attached jps and jstack log about hanging TaskManagerFailsWithSlotSharingITCase to JIRA FLINK-2183. Regards, Chiwan Park On Jun 10, 2015, at 12:28 AM, Aljoscha Krettek aljos...@apache.org wrote: I discovered something that might be a feature, rather than a bug. When you submit an example

Re: Checkstyle in IntelliJ

2015-06-09 Thread Chiwan Park
Hi. IntelliJ IDEA can reformat only changed code. In Reformat File dialog (you can open this by pressing Alt + Cmd + L in OS X), you can choose scope to reformat style. After the scope is changed, you can reformat only changed code without opening the dialog. Following links will be helpful.

[jira] [Created] (FLINK-2192) ComplexIntegrationTest.complexIntegrationTest1 fails

2015-06-09 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-2192: Summary: ComplexIntegrationTest.complexIntegrationTest1 fails Key: FLINK-2192 URL: https://issues.apache.org/jira/browse/FLINK-2192 Project: Flink Issue

Re: Integration : Flink with OSGI

2015-06-09 Thread Robert Metzger
Hi Santosh, I'm not aware of anybody in the community who has done anything like that before. So I guess there is no existing infrastructure to run Flink in an OSGi container. I don't know the limitations of an OSGi container, but from what I understood, you can run arbitrary java code in such a

Re: Force enabling checkpoints for iterative streaming jobs

2015-06-09 Thread Ufuk Celebi
Hey Gyula, I understand your reasoning, but I don't think its worth to rush this into the release. As you've said, we cannot give precise guarantees. But this is arguably one of the key requirements for any fault tolerance mechanism. Therefore I disagree that this is better than not having

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Ufuk Celebi
While looking into FLINK-2188 (HBase input) I've discovered that Hadoop input formats implementing Configurable (like mapreduce.TableInputFormat) don't have the Hadoop configuration set via setConf(Configuration). I have a small fix for this, which I have to clean up. First, I wanted to check

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Sachin Goel
A re-ran lead to reproducibility of 11 failures again. TaskManagerTest.testSubmitAndExecuteTask was failing with a time-out but managed to succeed in a re-run. Here is the log output again: http://pastebin.com/raw.php?i=N4cm1J18 Setup: JDK 1.8.0_40 on windows 8.1 System memory: 8GB, quad-core

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Aljoscha Krettek
I found the bug in the failing YARNSessionFIFOITCase: It was comparing the hostname to a hostname in some yarn config. In one case it was capitalised, in the other case it wasn't. Pushing fix to master and release-0.9 branch. On Tue, Jun 9, 2015 at 2:18 PM, Sachin Goel sachingoel0...@gmail.com

Integration : Flink with OSGI

2015-06-09 Thread santosh_rajaguru
Hi guys, how can i run flink in OSGI container? Is there any way to deploy the osgi bundles in flink and execute the job. Regards, Santosh -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Integration-Flink-with-OSGI-tp6278.html Sent from the

[jira] [Created] (FLINK-2191) Inconsistent use of Closure Cleaner in Streaming API

2015-06-09 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-2191: --- Summary: Inconsistent use of Closure Cleaner in Streaming API Key: FLINK-2191 URL: https://issues.apache.org/jira/browse/FLINK-2191 Project: Flink

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Aljoscha Krettek
I discovered another problem: https://issues.apache.org/jira/browse/FLINK-2191 The closure cleaner cannot be disabled in part of the Streaming Java API and all of the Streaming Scala API. I think this is a release blocker (in addition the the other bugs found so far.) On Tue, Jun 9, 2015 at 2:35