I'll take a look.
Cheers,
Till
On Tue, Jul 23, 2019 at 3:07 PM Oleksandr Nitavskyi
wrote:
> Hey guys.
>
>
>
> We have also made implementation in Flink on Mesos component in order to
> support network bandwidth configuration.
>
>
>
> Will somebody be able
Hey guys.
We have also made implementation in Flink on Mesos component in order to
support network bandwidth configuration.
Will somebody be able to have a look on our PR:
https://github.com/apache/flink/pull/8652
There are for sure some details to clarify.
Cheers
Oleksandr
From: Till
Hello!
We are trying to run in Mesos a job which will launch its own cluster (as
opposed to launching the cluster and then submitting jobs to it)
We have a couple of questions/issues:
1. Is there any easier way to achieve this without having to generate a
graph file before submitting the jo
llo!
>
>
>
> We are having a small problem while trying to deploy Flink on Mesos using
> marathon. In our set up of Mesos we are required to specify the amount of
> disk space we want to have for the applications we deploy there.
>
> The current default value in Flink is
Hello!
We are having a small problem while trying to deploy Flink on Mesos using
marathon. In our set up of Mesos we are required to specify the amount of disk
space we want to have for the applications we deploy there.
The current default value in Flink is 0 and it’s currently is not
gt;
> The only Docker container is the one that runs Job Manager.
>
>
>
> What am I missing?
>
>
>
> Thanks,
>
> Alex
>
>
>
> *From:* Renjie Liu [mailto:liurenjie2...@gmail.com]
> *Sent:* Friday, July 20, 2018 8:56 PM
> *To:* Till Rohrmann
> *Cc:* N
PM
To: Till Rohrmann
Cc: NEKRASSOV, ALEXEI ; Fabian Hueske ; user
Subject: Re: Flink on Mesos: containers question
Hi， Alexei:
What you paste is expected behavior. Jobmanager, two task managers each should
run in a docker instance.
13276 is should be the process of job manager, and it'
From:* Fabian Hueske [mailto:fhue...@gmail.com]
>> *Sent:* Monday, July 16, 2018 7:57 AM
>> *To:* NEKRASSOV, ALEXEI
>> *Cc:* user@flink.apache.org; Till Rohrmann
>> *Subject:* Re: Flink on Mesos: containers question
>>
>>
>>
>> Hi Alexei,
>>
>
erized in Mesos?
>
>
>
> Thanks!
>
> Alex
>
>
>
> *From:* Fabian Hueske [mailto:fhue...@gmail.com]
> *Sent:* Monday, July 16, 2018 7:57 AM
> *To:* NEKRASSOV, ALEXEI
> *Cc:* user@flink.apache.org; Till Rohrmann
> *Subject:* Re: Flink on Mesos: conta
Till,
Any insight into how Flink components are containerized in Mesos?
Thanks!
Alex
From: Fabian Hueske [mailto:fhue...@gmail.com]
Sent: Monday, July 16, 2018 7:57 AM
To: NEKRASSOV, ALEXEI
Cc: user@flink.apache.org; Till Rohrmann
Subject: Re: Flink on Mesos: containers question
Hi Alexei
Hi Alexei,
Till (in CC) is familiar with Flink's Mesos support in 1.4.x.
Best, Fabian
2018-07-13 15:07 GMT+02:00 NEKRASSOV, ALEXEI :
> Can someone please clarify how Flink on Mesos in containerized?
>
>
>
> On 5-node Mesos cluster I started Flink (1.4.2) with two Task Man
Can someone please clarify how Flink on Mesos in containerized?
On 5-node Mesos cluster I started Flink (1.4.2) with two Task Managers. Mesos
shows "flink" task and two "taskmanager" tasks, all on the same VM.
On that VM I see one Docker container running a process that se
lto:lassenederga...@gmail.com>> wrote:
>
> Hi.
> Go to Catalog, Search for Flink and click deploy
>
> Med venlig hilsen / Best regards
> Lasse Nedergaard
>
>
> Den 18. mar. 2018 kl. 16.18 skrev miki haiat <mailto:miko5...@gmail.com>
dergaard
>
>
> Den 18. mar. 2018 kl. 16.18 skrev miki haiat :
>
>
> Hi ,
>
> Im trying to run flink on mesos iv installed mesos and marathon
> successfully but im unable to create flink job/task manager
>
> im running this command but mesos wont start any task
>
Hi.
Go to Catalog, Search for Flink and click deploy
Med venlig hilsen / Best regards
Lasse Nedergaard
> Den 18. mar. 2018 kl. 16.18 skrev miki haiat :
>
>
> Hi ,
>
> Im trying to run flink on mesos iv installed mesos and marathon
> successfully but im unable to
Hi ,
Im trying to run flink on mesos iv installed mesos and marathon
successfully but im unable to create flink job/task manager
im running this command but mesos wont start any task
./mesos-appmaster-flip6-session.sh -n 1
i cant figure out the proper way to run flink on mesos
Hi Bo,
I think the by saying mesos.constraings.hard.hostattribute:
rack:ak03-07,rack:ak16-10, you define two hard constraints which are
attribute rack must equal ak03-07 AND rack must equal ak16-10. Since a task
offer must come from both racks, it will never complete a task request. So
at the mome
Thanks, Till
I tried to set hard host attribute constraints in "flink-conf.yaml" as
mesos.constraints.hard.hostattribute: rack:ak03-07,rack:ak16-10,rack:ak03-04
where "rack:akXX-XX" is the MESOS_attributes of each slave.
Then I get to the situation that the mesos app master doesn't accept the
off
Hi Bo,
you can still use Flink with Marathon, because Marathon will only schedule
the cluster entrypoint which is the MesosApplicationMasterRunner.
Everything else will be scheduled via Fenzo. Moreover, by using Marathon
you gain high availability because Marathon makes sure that the
ApplicationMa
Thanks for your reply, Till
We will use without Marathon, and hope the PR is merged to latest version soon.
Best regards,
Bo
> On Oct 9, 29 Heisei, at 6:36 PM, Till Rohrmann wrote:
>
> Hi Bo,
>
> Flink uses internally Fenzo to match tasks and offers. Fenzo does not support
> the Marathon con
Hi Bo,
Flink uses internally Fenzo to match tasks and offers. Fenzo does not
support the Marathon constraints syntax you are referring to. At the
moment, Flink only allows to define hard host attribute constraints which
means that you define a host attribute which has to match exactly. Fenzo
also
Hi Bo,
I'm not familiar with Mesos deployments, but I'll forward this to Till or Eron
(in CC) who perhaps could provide some help here.
Cheers,
Gordon
On 2 October 2017 at 8:49:32 PM, Bo Yu (yubo1...@gmail.com) wrote:
Hello all,
This is Bo, I met some problems when I tried to use flink in my
Hello all,
This is Bo, I met some problems when I tried to use flink in my mesos
cluster (1 master, 2 slaves (cpu has 32 cores)).
I tried to start the mesos-appmaster.sh in marathon, the job manager is
started without problem.
mesos-appmaster.sh -Djobmanager.heap.mb=1024 -Dtaskmanager.heap.mb=1024
Hi,
You do not need to install Flink onto the nodes. The approach used by
Flink is that the task managers download a copy of Flink from the
appmaster. The entire installation tree of Flink is downloaded (i.e. the
bin/lib/conf directories). The only assumed dependency is Java, which may
be prov
Hi,
I am newbie in Apache Flink and our team is trying to set up an Apache
Flink Cluster on Apaches Mesos. We have already installed Apache Mesos &
Marathon with 3 Master nodes and 3 Slaves and now we are trying to install
Apache Flink without DC/OS as mentioned here
https://ci.apache.org/project
Cool, good to hear!
It is one of those "it a feature, not a bug" situations ;-)
Flink's HA mode supports multiple masters, so the CLI needs to have a way
to find which master is "leader" (active, versus the passive masters on
standby). That discovery goes through ZooKeeper as well (which is the
g
Hey! It´s working now!!
I will do a summary for those who might have the same problem in the future:
- Flink 1.3.0 dockerized on Mesos:
- Add the HA configuration values in your flink app:
https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/jobmanager_high_availability.html#config-
Hi Stephan,
So, do you mean to remove the “-m” param from the flink CLI call? And on the
other hand, that I should add the Zookeeper configuration in both sides, the
remote flink and locally in the flink CLI config, right?
Regards
On 31 Jul 2017, at 22:21, Stephan Ewen
mailto:se...@apache.or
Hi Francisco!
Can you drop the explicit address of the jobmanager? The client should pick
up that address automatically from ZooKeeper as well (together with the HA
leader session ID).
Please check if you have the ZooKeeper HA config entries in the config used
by the CLI.
Stephan
On Mon, Jul 3
Hi again,
On the other hand, we are running the following flink CLI command:
./flink run -d -m ${jobmanager.rpc.address}:${jobmanager.rpc.port}
${our-program-jar} ${our-program-params}
Maybe is the command what we are using wrongly?
Thank you
On 28 Jul 2017, at 11:07, Till Rohrmann
mailto:t
Hi Till,
Thanks for your answer.
We have reviewed the configuration and everything seems fine in our side… But
we´re still getting the message:
“Discard message
LeaderSessionMessage(----,SubmitJob(JobGraph(jobId:
041b67c7ef765c2f61bd69c2b9dacbce),DETACHED)) be
Hi Francisco,
have you set the right high-availability configuration options in your
client configuration as described here [1]? If not, then Flink is not able
to find the correct JobManager because it retrieves the address as well as
a fencing token (called leader session id) from the HA store (Z
Hello,
We´re having lot of issues while trying to submit a job remotely using the
Flink CLI command line tool. We have tried different configurations but in all
of them we get errors from AKKA while trying to connect. I will try to
summarise the configurations we´ve tried.
- Flink 1.3.0 deploy
33 matches
Mail list logo
