[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2016-03-19 Thread Astralidea
Github user Astralidea commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r56613503 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/FlinkScheduler.scala --- @@ -0,0 +1,223 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2016-03-18 Thread ankurcha
Github user ankurcha closed the pull request at: https://github.com/apache/flink/pull/948 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2016-03-11 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-195398349 With the recently introduced changes to the resource management in order to better be in line with Mesos' model, I think we can close this PR. Sorry! --- If your project

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-10-15 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-148403603 I tried running the code from this pull request again, this time using the `mesos-playa` vagrant image, and it does not work for me. I was following your

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-10-13 Thread hsaputra
Github user hsaputra commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-147937031 Thx again for updating the patch, @ankurcha . Apologize for the delay of the review. --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-10-11 Thread ankurcha
Github user ankurcha commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r41721162 --- Diff: flink-mesos/pom.xml --- @@ -0,0 +1,203 @@ + + +http://maven.apache.org/POM/4.0.0; +

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-10-11 Thread hsaputra
Github user hsaputra commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r41721026 --- Diff: flink-mesos/pom.xml --- @@ -0,0 +1,203 @@ + + +http://maven.apache.org/POM/4.0.0; +

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-10-11 Thread hsaputra
Github user hsaputra commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r41721041 --- Diff: flink-mesos/pom.xml --- @@ -0,0 +1,203 @@ + + +http://maven.apache.org/POM/4.0.0; +

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-10-01 Thread hsaputra
Github user hsaputra commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-144858877 Quick comment on the PR, could you add Javadoc header information to give short description on why each new class/ trait is created and what role do they play to

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-10-01 Thread hsaputra
Github user hsaputra commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r40970091 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/executor/FlinkExecutor.scala --- @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-10-01 Thread hsaputra
Github user hsaputra commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r40970304 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/executor/TaskManagerExecutor.scala --- @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-09-29 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-144074252 Thank you for the update @ankurcha. I'm trying to review your PR in the next few days. Sorry for the delay. --- If your project is set up for it, you can reply to

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-09-24 Thread ankurcha
Github user ankurcha commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-142825340 Hi @rmetzger, This completely got buried in scheduler but i think i have finally worked out the kinks. Thanks for spending the time to try it out. I

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-09-08 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r38957167 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/SchedulerUtils.scala --- @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-09-08 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-138640533 I'm trying out the code again in GCE ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-09-08 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r38958292 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/SchedulerUtils.scala --- @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-09-08 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-138649089 it seems that the taskmanagers are failing and the scheduler keeps scheduling new ones: http://i.imgur.com/iZiYa4u.png --- If your project is set up for it, you

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-09-07 Thread ankurcha
Github user ankurcha commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-138417721 @rmetzger This PR should be good to test. I have embedded a simple http server that can serve the `log4j.configuration=...` file or the a default file to the task

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-31 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-136377793 Thank you for the response. I was asking because I think its a requirement that the JobManager is running as a mesos task in the cluster as well. But as far as I

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-30 Thread ankurcha
Github user ankurcha commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r38281174 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/SchedulerUtils.scala --- @@ -0,0 +1,348 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-30 Thread ankurcha
Github user ankurcha commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-136236719 @rmetzger I have finally got some time to work on this again. Let me address your question one by one: Why did you decide to start the JobManager alongside the

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-19 Thread ankurcha
Github user ankurcha commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-132529165 Thanks all for the comments. I am out for MesosCon and work this week. I'll try to address the feedback and push some changes later this weekend. --- If your project

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-19 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-132503290 Sorry for not getting back on time @ankurcha. Robert did a good review in the meantime. Thanks! --- If your project is set up for it, you can reply to this email and have

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-17 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-131796852 Hi @ankurcha, I've started a Mesos cluster on Google Compute Engine to try out your pull request. I've used this configuration: ```

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-17 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-131812796 It seems that you are ignoring methods such as `error(driver: SchedulerDriver, message: String)` or `frameworkMessage()`. Are they application specific (e.g. send by

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-17 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r37184472 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/executor/FlinkExecutor.scala --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-17 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r37185622 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/SchedulerUtils.scala --- @@ -0,0 +1,348 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-17 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/948#discussion_r37183905 --- Diff: flink-mesos/pom.xml --- @@ -0,0 +1,188 @@ +?xml version=1.0 encoding=UTF-8? +!-- +Licensed to the Apache Software Foundation (ASF) under

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-17 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-131803783 For the tests: For HDFS, Kafka and YARN, we are using MiniClusters which start all needed services in one JVM. I don't think we can do something similar with Mesos

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-11 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-129785986 Thanks for the reply at the mailing list. I will try out your PR this week and have a look at the code. Sorry for the delay. I needed to clear some more time, because it is

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-11 Thread ankurcha
Github user ankurcha commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-129765378 @StephanEwen Thanks for the pointer, I replied on the mailing list thread. Any code-review comments for this pull request? --- If your project is set up for

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-08-02 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-127048248 Here is the thread where the discussion started:

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-07-30 Thread ankurcha
Github user ankurcha commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-126522087 Clearly i don't know how to add stuff correctly to the .travis file. If I could get some help with that, it'll be great. @uce @StephanEwen - I haven't seen a

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-07-28 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/948#issuecomment-125782171 Hey Ankur! Welcome to the Flink community. I didn't have a proper look at your PR yet, but regarding the question you've asked in the issue: @StephanEwen kicked off a

[GitHub] flink pull request: [FLINK-1984] Integrate Flink with Apache Mesos

2015-07-28 Thread ankurcha
GitHub user ankurcha opened a pull request: https://github.com/apache/flink/pull/948 [FLINK-1984] Integrate Flink with Apache Mesos This pull requests adds a mesos scheduler and an executor (inspired from the work done in, now abandoned, PR #251). The highlights are as follows: