Congratulations! Very well deserved! Always so helpful :)
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150
On 20 December 2016 at 01:24, Jie Yu <yujie@gmail.com> wrote:
> Congrats! Well deserved!!
>
> Always wondering why yo
Kubernetes uses its own
containeriser to do this?).
We'd be very keen to have this ability in Mesos too.
Thanks,
Aaron
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150
From: Vinod Kone [vinodk...@apache.org]
Just had a brief look over, this is great, a huge leap forward.. we were
considering moving to Kubernetes because of the pod support, having this in
mesos would allow us to do a lot more!
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751
As you're writing the framework, have you looked at reserving persistent
volumes? I think it might help in your use case:
http://mesos.apache.org/documentation/latest/persistent-volume/
Aaron
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751
#3 would be very helpful for us. Also related:
https://issues.apache.org/jira/browse/MESOS-3059
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150
From: Du, Fan [fan...@intel.com]
Sent: 14 June 2016 0
Would this perhaps make sense as a mesos module which can automatically assigns
labels to the agents, rather than something in the core itself?
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150
From
What exactly do you mean by deploying a mesos cluster to run on ceph etc?
Do you mean having a clustered file system mounted via nfs to the hosts which
contains the mesos binaries?
Or something to do with how jobs are executed?
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial
We'd be very excited to see a pluggable mesos fetcher!
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150
From: Ken Sipe [kens...@gmail.com]
Sent: 11 May 2016 08:40
To: user@mesos.apache.org
Subject: Re: Enable
Then you need to tell marathon to run the mysql container first, and then
submit the wordpress container.
Sorry I think I misunderstood!
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150
From: Stefano Bia
If you run the wordpress container on a different host to the mysql container
and use --link on the command line, does that work?
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150
From: Stefano Bianchi [jaz
or try dns based
solutions like Mesos-dns or Consul (along with mesos-consul). I think marathon
also has some concept of service discovery built in too if you use something
like haproxy.
I hope this helps!
Aaron
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & M
Out of curiosity... is this for fun or production workloads? I'd be curious to
hear about raspis being used in production!
--
Aaron Carey
Production Engineer - Cloud Pipeline
Industrial Light & Magic
London
020 3751 9150
From: Sharma Podila [
Thanks Klaus,
After reading Haosdent's response I have a feeling a ticket may already exist:
MESOS-3059 which would work for our usecase, however I can't actually access
that ticket.
I'm happy to create a new one if needed though?
Thanks,
Aaron
--
Aaron Carey
Production Engineer - Cloud
Ah thank you! I tried searching Jira but didn't find that ticket.
Yes I think you might be right about the attributes, although I don't seem to
be able to get to the MESOS-3059 ticket in Jira, do you know if it's on the
roadmap?
Thanks,
Aaron
--
Aaron Carey
Production Engineer - Cloud
Hi All,
I was wondering if it was possible somehow to alter an agent's resources after
it has started?
Example: we are dynamically attaching and detaching EBS volumes to EC2 hosts
running as agents. (This is part of our docker volume setup using RexRay). When
a host has an EBS volume attached
-1.8.2_ubuntu-14.04.3
Do you know how one can trigger an update to keep it on par with
mesosphere/mesos-slave?
On Tue, Mar 15, 2016 at 1:53 AM, Aaron Carey
<aca...@ilm.com<mailto:aca...@ilm.com>> wrote:
Would the officially provided docker-in-docker image help?
mesosphere/meso
Would the officially provided docker-in-docker image help?
mesosphere/mesos-slave-dind
From: Yuri Finkelstein [yurif2...@gmail.com]
Sent: 15 March 2016 04:25
To: user@mesos.apache.org
Subject: Re: running mesos slave in a docker container
Sure, but my point
Ah I think I've solved my own problem:
We are using the dind mesos slave container and hadn't mounted the host's
/var/lib/mesos folder within the container so it wasn't showing up in the other
containers.
Oops!
Thanks for your help,
Aaron
From: Aaron Carey
Sorry apparently the inline image didn't work:
http://i.imgur.com/x1cPXvW.png
From: Aaron Carey [aca...@ilm.com]
Sent: 29 February 2016 08:50
To: user@mesos.apache.org
Subject: RE: Downloading s3 uris
To illustrate:
[X]
/var/lib/mesos/slaves/20160212-131720
3 root root 4.0K Feb 26 18:10 ..
From: Aaron Carey [aca...@ilm.com]
Sent: 29 February 2016 08:45
To: user@mesos.apache.org
Subject: RE: Downloading s3 uris
Yeah, I've managed to find the sandbox itself on disk, but it's empty, even
though the file shows up
you don't want to configure hadoop on your mesos slaves, the only workaround
I see is to write a "hadoop" script and put it in your PATH. It need to support
the following usage patterns:
- hadoop version
- hadoop fs -copyToLocal s3n://path /target/directory/
On Sat, Feb 27, 2016 at
want to use "s3n://xxx". :-(
On Sat, Feb 27, 2016 at 12:39 AM, Abhishek Amralkar
<abhishek.amral...@talentica.com<mailto:abhishek.amral...@talentica.com>> wrote:
Agreed with @aaron, it will be too much manual work to generate S3 url
everytime.
Thanks
Abhishek
From: Aaron
t the files) is to use
http<http://stackoverflow.com/questions/18239567/how-can-i-download-a-file-from-an-s3-bucket-with-wget>
url instead.
On Feb 26, 2016, at 8:17 AM, Aaron Carey
<aca...@ilm.com<mailto:aca...@ilm.com>> wrote:
I'm attempting to fetch files from s3 uris i
I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs
in our cluster... however I believe I need the client installed.
Is it possible to just have the client running without a full hdfs setup?
I haven't been able to find much information in the docs, could someone point
:32, Aaron Carey wrote:
Has anyone had a good experience recording mesos metrics into influxdb?
I've found a couple of options, a collectd plugin which doesn't appear to work
with version 0.24.x and a more up to date containerised option which randomly
crashes regularly and doesn't appear to actua
Has anyone had a good experience recording mesos metrics into influxdb?
I've found a couple of options, a collectd plugin which doesn't appear to work
with version 0.24.x and a more up to date containerised option which randomly
crashes regularly and doesn't appear to actually post any stats.
If this is of any use to anyone: There is also an outstanding branch of Docker
which has checkpoint/restore functionality in it (based on CRIU I believe)
which is hopefully being merged into experimental soon.
From: Sharma Podila [spod...@netflix.com]
Sent: 19
Would you be able to elaborate a bit more on how you did this?
From: Mauricio Garavaglia [mauri...@medallia.com]
Sent: 19 February 2016 19:20
To: user@mesos.apache.org
Subject: Re: AW: Feature request: move in-flight containers w/o stopping them
Mesos is not only
https://github.com/udacity/ansible-marathon
https://github.com/AnsibleShipyard/ansible-marathon
From: Antonio Fernandez [antonio.fernan...@bq.com]
Sent: 08 February 2016 15:08
To: user@mesos.apache.org
Subject: Re: ansible modules?
René,
take a look on this
:
https://github.com/kubernetes/kubernetes/blob/master/docs/getting-started-guides/mesos.md
Kind Regards,
Bartek Plotka
From: Aaron Carey [mailto:aca...@ilm.com]
Sent: Tuesday, November 10, 2015 4:33 PM
To: user@mesos.apache.org
Subject: RE: Custom python executor with Docker
We would also
We would also be interested in some sort of standardised DockerExecutor which
would allow us to add pre and post launch steps.
Also having the ability to run two containers together as one task would be
very useful (ie on the same host and linked together)
Ah interesting.. I reported exactly the same problem with NFS the other day!
@Luke: we also had some issues with 1.8.0, 1.8.3 seems to have fixed things for
us though!
From: craig w [codecr...@gmail.com]
Sent: 23 October 2015 14:54
To: user@mesos.apache.org
We run both our Master and Agent processes as docker containers.. it works well
although we don't have strict security requirements..
From: Krish [krishnan.k.i...@gmail.com]
Sent: 30 September 2015 13:58
To: user@mesos.apache.org
Subject: Mesos slave in docker
"
<spod...@netflix.com<mailto:spod...@netflix.com>> wrote:
Ah, OK, thanks. Yes, Fenzo is a Java library.
It might be a nice addition to Mesos master to get a global view of contention
for resources. In addition to autoscaling, it would be useful in the allocator.
On Wed, S
?
Does /metrics/snapshot not satisfy your requirement?
On Wed, Sep 23, 2015 at 4:50 PM, Aaron Carey
<aca...@ilm.com<mailto:aca...@ilm.com>> wrote:
Hi all,
Is there any way to get a metric of all tasks currently waiting/queued in Mesos
(across all schedulers)? The snapshot metrics s
I have limited knowledge of Azkaban/Spark, but this sounds like a good fit for
the Chronos (or Aurora) schedulers.
Mesos itself doesn't provide the scheduling logic (ie the dependency
information, the 'run this task at x time' info etc). Mesos is the framework,
which Chronos or Aurora run on
Hi All,
I'm Aaron and I work as a production/rd engineer at Industrial Light and
Magic. We've been experimenting with Mesos and Docker over the last 4 or 5
months for a variety of purposes.
I'm really looking forward to hearing more on scheduling algorithms and per
container IP applications.
-- just add it
as a URL and it will be downloaded to the sandbox like any other resource!
On Jul 29, 2015, at 02:47, Aaron Carey aca...@ilm.commailto:aca...@ilm.com
wrote:
Ah I see.. so is it simply a case of making the executor file executable,
putting it on the slave, and supplying the path
are and what your new custom executor
will do?
Tim
On Tue, Jul 28, 2015 at 5:08 AM, Aaron Carey
aca...@ilm.commailto:aca...@ilm.com wrote:
Hi,
Is it possible to build a custom executor which is not associated with a
particular scheduler framework? I want to be able to write a custom executor
and match languages for schedulers and executors? (ie
one is python one is C++)
Yes, could use different languages for different components. Just need
implement the interfaces and make sure the executor could run in slaves.
On Tue, Jul 28, 2015 at 8:08 PM, Aaron Carey
aca...@ilm.commailto:aca
Hi,
Is it possible to build a custom executor which is not associated with a
particular scheduler framework? I want to be able to write a custom executor
which is available to multiple schedulers (eg Marathon, Chronos and our own
custom scheduler). Is this possible? I couldn't quite figure out
There's nothing stopping you running the mesos master and slave process on the
same machine, so you could run the master process on your non-desktop machine
if you're worried.
We have the master and slave processes run as docker containers and they can
both end up on the same machine without
+1 for mesos-consul
We've been using it to great effect!
From: Dave Lester [d...@davelester.org]
Sent: 30 June 2015 06:38
To: user@mesos.apache.org
Subject: Re: service discovery in Mesos on CoreOS
It would be great to have a documentation page devoted to
Thanks James,
Interesting background!
From: CCAAT [cc...@tampabay.rr.com]
Sent: 04 June 2015 14:05
To: user@mesos.apache.org
Cc: cc...@tampabay.rr.com
Subject: Re: [DISCUSS] Renaming Mesos Slave
On 06/04/2015 02:32 AM, Aaron Carey wrote:
+1 to Itamar
RAM and Mesos offers 1.9G - so there should is no problem related to
your mesos setup (mesos in both cases offers around 63% of RAM).
About manual setup: you can use some automation tool (Ansible, Puppet) if you
plan to setup large number of nodes.
2015-05-21 13:10 GMT+02:00 Aaron Carey aca
(info.freeram);
# endif
On Thu, May 21, 2015 at 6:51 AM, Aaron Carey
aca...@ilm.commailto:aca...@ilm.com wrote:
I've managed to increase the disksize by playing with some docker options,
Anyone have any idea about the memory?
Thanks,
Aaron
From: Aaron Carey [aca
Hi,
I was just trying to figure out how Mesos slaves report the amount of resources
available to them on the host?
We have some slaves running on AWS t2.medium machines (2cpu, 4Gb RAM) with 32GB
disks.
The slaves are running inside docker containers.
They report 2 cpus (correct), 2.5GB RAM
I've managed to increase the disksize by playing with some docker options,
Anyone have any idea about the memory?
Thanks,
Aaron
From: Aaron Carey [aca...@ilm.com]
Sent: 21 May 2015 11:19
To: user@mesos.apache.org
Subject: How slaves calculate resources
Hi,
I
at stolos too:
https://github.com/sailthru/stolos
Andras
From: Aaron Carey [mailto:aca...@ilm.commailto:aca...@ilm.com]
Sent: Wednesday, May 13, 2015 11:54 AM
To: user@mesos.apache.orgmailto:user@mesos.apache.org
Subject: RE: Batch Scheduler with dependency support
Thanks! I hadn't come across
You might want to have a look at stolos too:
https://github.com/sailthru/stolos
Andras
From: Aaron Carey [mailto:aca...@ilm.com]
Sent: Wednesday, May 13, 2015 11:54 AM
To: user@mesos.apache.org
Subject: RE: Batch Scheduler with dependency support
Thanks! I hadn't come across that one before
13, 2015 at 9:46 AM, Andras Kerekes
andras.kere...@ishisystems.commailto:andras.kere...@ishisystems.com wrote:
You might want to have a look at stolos too:
https://github.com/sailthru/stolos
Andras
From: Aaron Carey [mailto:aca...@ilm.commailto:aca...@ilm.com]
Sent: Wednesday, May 13, 2015 11
Hi All,
I was just wondering if anyone out there knew of a good mesos batch scheduler
which supports dependencies between tasks? (ie Task B cannot run until Task A
is complete)
Thanks,
Aaron
2015 13:12
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support
Apache Aurora does this and you can be explicit about the ordering
On Wednesday, May 13, 2015, Aaron Carey aca...@ilm.commailto:aca...@ilm.com
wrote:
Hi All,
I was just wondering if anyone out there knew
Lookup Hubspot's Singularity
On Wednesday, May 13, 2015, Aaron Carey aca...@ilm.commailto:aca...@ilm.com
wrote:
Thanks Jeff,
Any other options around as well?
From: jeffschr...@gmail.comUrlBlockedError.aspx
[jeffschr...@gmail.comUrlBlockedError.aspx] on behalf
Thanks for this... very useful!
From: Christos Kozyrakis [kozyr...@gmail.com]
Sent: 07 April 2015 23:25
To: user@mesos.apache.org
Cc: John Omernik
Subject: Re: Using mesos-dns in an enterprise
This is a great thread, thanks for starting it John.
I will transcode
Not sure if this helps, but we've been using docker to run Mesos on Centos 7
hosts.
From: craig w [codecr...@gmail.com]
Sent: 23 March 2015 12:06
To: user@mesos.apache.org
Subject: Unable to install subversion-devel 1.8+ on CentOS 7
Mesos 0.21.0+ requires
, Aaron Carey
aca...@ilm.commailto:aca...@ilm.com wrote:
Not sure if this helps, but we've been using docker to run Mesos on Centos 7
hosts.
From: craig w [codecr...@gmail.commailto:codecr...@gmail.com]
Sent: 23 March 2015 12:06
To: user@mesos.apache.orgmailto:user
at 8:30 AM, Aaron Carey
aca...@ilm.commailto:aca...@ilm.com wrote:
ah interesting.. what causes this difference?
I think this probably makes sense for our setup currently..
From: craig w [codecr...@gmail.commailto:codecr...@gmail.com]
Sent: 23 March 2015 12:20
the service.
Sent from my iPhone
On Mar 23, 2015, at 4:25 AM, Aaron Carey
aca...@ilm.commailto:aca...@ilm.com wrote:
Hey,
I don't suppose there is anything like Mesos-DNS but for services/users outside
the mesos cluster? So having a service which updates a DNS provider with task
port/ips running
address. Do get port
information you have to query mesos-dns for its SRV records.
On Mon, Mar 23, 2015 at 11:29 AM, Ken Sipe
kens...@gmail.commailto:kens...@gmail.com wrote:
roger that
On Mar 23, 2015, at 9:22 AM, Aaron Carey
aca...@ilm.commailto:aca...@ilm.com wrote:
Thanks Ken,
So basically
Hey,
I don't suppose there is anything like Mesos-DNS but for services/users outside
the mesos cluster? So having a service which updates a DNS provider with task
port/ips running inside the cluster so that external users are able to find
those services? Am I correct in thinking Mesos-DNS only
Hi All,
In setting up our cluster, we require things like consul to be running on all
of our nodes. I was just wondering if there was any sort of best practice (or a
scheduler perhaps) that people could share for this sort of thing?
Currently the approach is to use salt to provision each node
62 matches
Mail list logo
