Re: Testing Apache Flink 0.9.0-rc2

2015-06-16 Thread Till Rohrmann
I merged the legal and scheduler PR. On Tue, Jun 16, 2015 at 11:17 PM Márton Balassi wrote: > @Max: The PR is good to go on my side. Does the job, could be a bit nicer > though. Added to the document. > > On Tue, Jun 16, 2015 at 10:54 PM, Aljoscha Krettek > wrote: > > > I added the two relevant

Re: Testing Apache Flink 0.9.0-rc2

2015-06-16 Thread Márton Balassi
@Max: The PR is good to go on my side. Does the job, could be a bit nicer though. Added to the document. On Tue, Jun 16, 2015 at 10:54 PM, Aljoscha Krettek wrote: > I added the two relevant Table API commits to the release doc. > > On Tue, 16 Jun 2015 at 21:49 Maximilian Michels wrote: > > > +1

Re: Testing Apache Flink 0.9.0-rc2

2015-06-16 Thread Aljoscha Krettek
I added the two relevant Table API commits to the release doc. On Tue, 16 Jun 2015 at 21:49 Maximilian Michels wrote: > +1 for adding the Table API fixes. > > @Till: It seems like you fixed the bug in the scheduler. Is > https://github.com/apache/flink/pull/843 fixing it? > > @Marton: What's the

Re: Testing Apache Flink 0.9.0-rc2

2015-06-16 Thread Maximilian Michels
+1 for adding the Table API fixes. @Till: It seems like you fixed the bug in the scheduler. Is https://github.com/apache/flink/pull/843 fixing it? @Marton: What's the state of your pull request to solve the license issue of the shaded dependencies? https://github.com/apache/flink/pull/837 I'm as

Re: Testing Apache Flink 0.9.0-rc2

2015-06-16 Thread Fabian Hueske
Important fixes, IMO. +1 for adding. 2015-06-16 18:50 GMT+02:00 Aljoscha Krettek : > I would like to include the fixes for the Table API: RowSerializer supports > null values, Aggregations properly deal with null values. > > What do you think? > > On Mon, 15 Jun 2015 at 17:31 Aljoscha Krettek wr

Re: Testing Apache Flink 0.9.0-rc2

2015-06-16 Thread Aljoscha Krettek
I would like to include the fixes for the Table API: RowSerializer supports null values, Aggregations properly deal with null values. What do you think? On Mon, 15 Jun 2015 at 17:31 Aljoscha Krettek wrote: > I created this to help with release testing: > https://github.com/aljoscha/FliRTT > > Y

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Aljoscha Krettek
I created this to help with release testing: https://github.com/aljoscha/FliRTT You just start your cluster and then point the tool to the Flink directory. It will then run all the examples with both builtin data and external data. On Mon, 15 Jun 2015 at 17:15 Maximilian Michels wrote: > Hmm. M

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Maximilian Michels
Hmm. Might be interesting to check out whether this is a regression from rc1 to rc2. In any case, it is a serious release blocker and we need to fix it. On Mon, Jun 15, 2015 at 5:04 PM, Till Rohrmann wrote: > I might have found another release blocker. While running some cluster > tests I also t

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Till Rohrmann
I might have found another release blocker. While running some cluster tests I also tried to run the `ConnectedComponents` example. However, sometimes the example couldn't be executed because the scheduler could not schedule co-located tasks, `NoResourceAvailableException`, even though it should ha

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Ufuk Celebi
Please continue the discussion in the issue Aljoscha opened: https://issues.apache.org/jira/browse/FLINK-2221 I think it is better to only point to issues in this mail thread. Otherwise the discussions are very hard to follow.

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Gyula Fóra
The checkpoint cleanup works for HDFS right? I assume the job manager should see that as well. This is not a trivial problem in general, so the assumptions we were making now that the JM can actually execute the cleanup logic. Aljoscha Krettek ezt írta (időpont: 2015. jún. 15., H, 15:40): > @Uf

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Aljoscha Krettek
Oh yes, on that I agree. I'm just saying that the checkpoint setting should maybe be a central setting. On Mon, 15 Jun 2015 at 15:38 Matthias J. Sax wrote: > Hi, > > IMHO, it is very common that Workers do have their own config files (eg, > Storm works the same way). And I think it make a lot of

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Aljoscha Krettek
@Ufuk The cleanup bug for file:// checkpoints is not easy to fix IMHO. On Mon, 15 Jun 2015 at 15:39 Aljoscha Krettek wrote: > Oh yes, on that I agree. I'm just saying that the checkpoint setting > should maybe be a central setting. > > On Mon, 15 Jun 2015 at 15:38 Matthias J. Sax < > mj...@infor

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Matthias J. Sax
Hi, IMHO, it is very common that Workers do have their own config files (eg, Storm works the same way). And I think it make a lot of senses. You might run Flink in an heterogeneous cluster and you want to assign different memory and slots for different hardware. This would not be possible using a

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Ufuk Celebi
On Mon, Jun 15, 2015 at 3:30 PM, Aljoscha Krettek wrote: > Regarding 1), thats why I said "bugs and features". :D But I think of it as > a bug, since people will normally set in in the flink-conf.yaml on the > master and assume that it works. That's what I assumed and it took me a > while to figu

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Aljoscha Krettek
Regarding 1), thats why I said "bugs and features". :D But I think of it as a bug, since people will normally set in in the flink-conf.yaml on the master and assume that it works. That's what I assumed and it took me a while to figure out that the task managers don't respect this setting. Regardin

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Márton Balassi
@Aljoscha: 1) I think this just means that you can set the state backend on a taskmanager basis. 3) This is a serious issue then. Is it work when you set it in the flink-conf.yaml? On Mon, Jun 15, 2015 at 3:17 PM, Aljoscha Krettek wrote: > So, during my testing of the state checkpointing on a cl

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Aljoscha Krettek
So, during my testing of the state checkpointing on a cluster I discovered several things (bugs and features): - If you have a setup where the configuration is not synced to the workers they do not pick up the state back-end configuration. The workers do not respect the setting in the flink-cont.

Re: Testing Apache Flink 0.9.0-rc2

2015-06-14 Thread Maximilian Michels
Hi Henry, This is just a dry run. The goal is to get everything in shape for a proper vote. Kind regards, Max On Sun, Jun 14, 2015 at 7:58 PM, Henry Saputra wrote: > Hi Max, > > Are you doing official VOTE on the RC on 0.9 release or this is just a dry > run? > > > - Henry > > On Sun, Jun 14,

Re: Testing Apache Flink 0.9.0-rc2

2015-06-14 Thread Henry Saputra
Hi Max, Are you doing official VOTE on the RC on 0.9 release or this is just a dry run? - Henry On Sun, Jun 14, 2015 at 9:11 AM, Maximilian Michels wrote: > Dear Flink community, > > Here's the second release candidate for the 0.9.0 release. We haven't had a > formal vote on the previous relea

Testing Apache Flink 0.9.0-rc2

2015-06-14 Thread Maximilian Michels
Dear Flink community, Here's the second release candidate for the 0.9.0 release. We haven't had a formal vote on the previous release candidate but it received an implicit -1 because of a couple of issues. Thanks to the hard-working Flink devs these issues should be solved now. The following comm