[VOTE] Release Apache Mesos 0.24.2 (rc1)

[Michael Park]

Hi all,

Please vote on releasing the following candidate as Apache Mesos 0.24.2.


0.24.2 includes the following:


   - Improvements
  - Allocator filter performance
  - Port Ranges performance
  - UUID performance
  - `/state` endpoint performance
  - GLOG performance
  - Configurable task/framework history
  - Offer filter timeout fix for backlogged allocator


   - Bugs
   - SSL
  - Libevent
  - Fixed point resources math
   - HDFS
  - Agent upgrade compatibility
  - Health checks

The CHANGELOG for the release is available at:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.24.2-rc1


The candidate for Mesos 0.24.2 release is available at:
https://dist.apache.org/repos/dist/dev/mesos/0.24.2-rc1/mesos-0.24.2.tar.gz

The tag to be voted on is 0.24.2-rc1:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.24.2-rc1

The MD5 checksum of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.24.2-rc1/mesos-0.24.2.tar.gz.md5

The signature of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.24.2-rc1/mesos-0.24.2.tar.gz.asc

The PGP key used to sign the release is here:
https://dist.apache.org/repos/dist/release/mesos/KEYS

The JAR is up in Maven in a staging repository here:
https://repository.apache.org/content/repositories/orgapachemesos-1110

Please vote on releasing this package as Apache Mesos 0.24.2!

The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a majority
of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Mesos 0.24.2
[ ] -1 Do not release this package because ...

Thanks,

Joris, Kapil, MPark

[VOTE] Release Apache Mesos 0.25.1 (rc1)

[Michael Park]

Hi all,

Please vote on releasing the following candidate as Apache Mesos 0.25.1.


0.25.1 includes the following:


   - Improvements
  - `/state` endpoint performance
  - systemd integration
  - GLOG performance
  - Configurable task/framework history
  - Offer filter timeout fix for backlogged allocator


   - Bugs
   - SSL
  - Libevent
  - Fixed point resources math
   - HDFS
  - Agent upgrade compatibility
  - Health checks

The CHANGELOG for the release is available at:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.25.1-rc1


The candidate for Mesos 0.25.1 release is available at:
https://dist.apache.org/repos/dist/dev/mesos/0.25.1-rc1/mesos-0.25.1.tar.gz

The tag to be voted on is 0.25.1-rc1:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.25.1-rc1

The MD5 checksum of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.25.1-rc1/mesos-0.25.1.tar.gz.md5

The signature of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.25.1-rc1/mesos-0.25.1.tar.gz.asc

The PGP key used to sign the release is here:
https://dist.apache.org/repos/dist/release/mesos/KEYS

The JAR is up in Maven in a staging repository here:
https://repository.apache.org/content/repositories/orgapachemesos-1108

Please vote on releasing this package as Apache Mesos 0.25.1!

The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a majority
of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Mesos 0.25.1
[ ] -1 Do not release this package because ...

Thanks,

Joris, Kapil, MPark

[VOTE] Release Apache Mesos 0.26.1 (rc1)

[Michael Park]

Hi all,

Please vote on releasing the following candidate as Apache Mesos 0.26.1.


0.26.1 includes the following:


   - Improvements
  - `/state` endpoint performance
  - systemd integration
  - GLOG performance
  - Configurable task/framework history
  - Offer filter timeout fix for backlogged allocator


   - Bugs
   - SSL
  - Libevent
  - Fixed point resources math
   - HDFS
  - Agent upgrade compatibility

The CHANGELOG for the release is available at:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.1-rc1


The candidate for Mesos 0.26.1 release is available at:
https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz

The tag to be voted on is 0.26.1-rc1:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.1-rc1

The MD5 checksum of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz.md5

The signature of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.26.1-rc1/mesos-0.26.1.tar.gz.asc

The PGP key used to sign the release is here:
https://dist.apache.org/repos/dist/release/mesos/KEYS

The JAR is up in Maven in a staging repository here:
https://repository.apache.org/content/repositories/orgapachemesos-1106

Please vote on releasing this package as Apache Mesos 0.26.1!

The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a majority
of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Mesos 0.26.1
[ ] -1 Do not release this package because ...

Thanks,

Joris, Kapil, MPark

[VOTE] Release Apache Mesos 0.27.2 (rc1)

[Michael Park]

Hi all,

Please vote on releasing the following candidate as Apache Mesos 0.27.2.


0.27.2 includes the following:


   - MESOS-4693  -
   Variable shadowing in HookManager::slavePreLaunchDockerHook.
   - MESOS-4711  - Race
   condition in libevent poll implementation causes crash.
   - MESOS-4754  - The
   "executors" field is exposed under a backwards incompatible schema.
   - MESOS-4687  -
   Implement reliable floating point for scalar resources.


The CHANGELOG for the release is available at:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.2-rc1


The candidate for Mesos 0.27.2 release is available at:
https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz

The tag to be voted on is 0.27.2-rc1:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.27.2-rc1

The MD5 checksum of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.md5

The signature of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.27.2-rc1/mesos-0.27.2.tar.gz.asc

The PGP key used to sign the release is here:
https://dist.apache.org/repos/dist/release/mesos/KEYS

The JAR is up in Maven in a staging repository here:
https://repository.apache.org/content/repositories/orgapachemesos-1104

Please vote on releasing this package as Apache Mesos 0.27.2!

The vote is open until Wed Mar 2 23:59:59 PST 2016 and passes if a majority
of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Mesos 0.27.2
[ ] -1 Do not release this package because ...

Thanks,

MPark, Joris, Kapil

Re: mesos agent not recovering after ZK init failure

[Sharma Podila]

MESOS-4795 created.

I don't have the exit status. We haven't seen a repeat yet, will catch the
exit status next time it happens.

Yes, removing the metadata directory was the only way it was resolved. This
happened on multiple hosts requiring the same resolution.


On Thu, Feb 25, 2016 at 6:37 PM, Benjamin Mahler  wrote:

> Feel free to create one. I don't have enough information to know what the
> issue is without doing some further investigation, but if the situation you
> described is accurate it seems like a there are two strange bugs:
>
> -the silent exit (do you not have the exit status?), and
> -the flapping from ZK errors that needed the meta data directory to be
> removed to resolve (are you convinced the removal of the meta directory is
> what solved it?)
>
> It would be good to track these issues in case they crop up again.
>
> On Tue, Feb 23, 2016 at 2:51 PM, Sharma Podila 
> wrote:
>
>> Hi Ben,
>>
>> Let me know if there is a new issue created for this, I would like to add
>> myself to watch it.
>> Thanks.
>>
>>
>>
>> On Wed, Feb 10, 2016 at 9:54 AM, Sharma Podila 
>> wrote:
>>
>>> Hi Ben,
>>>
>>> That is accurate, with one additional line:
>>>
>>> -Agent running fine with 0.24.1
>>> -Transient ZK issues, slave flapping with zookeeper_init failure
>>> -ZK issue resolved
>>> -Most agents stop flapping and function correctly
>>> -Some agents continue flapping, but silent exit after printing the
>>> detector.cpp:481 log line.
>>> -The agents that continue to flap repaired with manual removal of
>>> contents in mesos-slave's working dir
>>>
>>>
>>>
>>> On Wed, Feb 10, 2016 at 9:43 AM, Benjamin Mahler 
>>> wrote:
>>>
 Hey Sharma,

 I didn't quite follow the timeline of events here or how the agent logs
 you posted fit into the timeline of events. Here's how I interpreted:

 -Agent running fine with 0.24.1
 -Transient ZK issues, slave flapping with zookeeper_init failure
 -ZK issue resolved
 -Most agents stop flapping and function correctly
 -Some agents continue flapping, but silent exit after printing the
 detector.cpp:481 log line.

 Is this accurate? What is the exit code from the silent exit?

 On Tue, Feb 9, 2016 at 9:09 PM, Sharma Podila 
 wrote:

> Maybe related, but, maybe different since a new process seems to find
> the master leader and still aborts, never recovering with restarts until
> work dir data is removed.
> It is happening in 0.24.1.
>
>
>
>
> On Tue, Feb 9, 2016 at 11:53 AM, Vinod Kone 
> wrote:
>
>> MESOS-1326 was fixed in 0.19.0 (set the fix version now). But I guess
>> you are saying it is somehow related but not exactly the same issue?
>>
>> On Tue, Feb 9, 2016 at 11:46 AM, Raúl Gutiérrez Segalés <
>> r...@itevenworks.net> wrote:
>>
>>> On 9 February 2016 at 11:04, Sharma Podila 
>>> wrote:
>>>
 We had a few mesos agents stuck in an unrecoverable state after a
 transient ZK init error. Is this a known problem? I wasn't able to 
 find an
 existing jira item for this. We are on 0.24.1 at this time.

 Most agents were fine, except a handful. These handful of agents
 had their mesos-slave process constantly restarting. The .INFO logfile 
 had
 the following contents below, before the process exited, with no error
 messages. The restarts were happening constantly due to an existing 
 service
 keep alive strategy.

 To fix it, we manually stopped the service, removed the data in the
 working dir, and then restarted it. The mesos-slave process was able to
 restart then. The manual intervention needed to resolve it is 
 problematic.

 Here's the contents of the various log files on the agent:

 The .INFO logfile for one of the restarts before mesos-slave
 process exited with no other error messages:

 Log file created at: 2016/02/09 02:12:48
 Running on machine: titusagent-main-i-7697a9c5
 Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
 I0209 02:12:48.502403 97255 logging.cpp:172] INFO level logging
 started!
 I0209 02:12:48.502938 97255 main.cpp:185] Build: 2015-09-30
 16:12:07 by builds
 I0209 02:12:48.502974 97255 main.cpp:187] Version: 0.24.1
 I0209 02:12:48.503288 97255 containerizer.cpp:143] Using isolation:
 posix/cpu,posix/mem,filesystem/posix
 I0209 02:12:48.507961 97255 main.cpp:272] Starting Mesos slave
 I0209 02:12:48.509827 97296 slave.cpp:190] Slave started on 1)@
 10.138.146.230:7101
 I0209 02:12:48.510074 97296 slave.cpp:191] Flags at startup:
 

Re: Recent UI enhancements & Managed Service Providers

[Vinod Kone]

On Thu, Feb 25, 2016 at 10:31 AM, Paul Bell  wrote:

> I'm not sure that I can be more specific. MSPs are interested in a "view
> by tenant", e.g., "show me all applications that are allotted to Tenant
> X".  I suppose that the standard Mesos UI could, with properly named task
> IDs and the UI's "Find" filter, accomplish part of "view by tenant". But in
> order to see the resources consumed by Tenant X's tasks, you have to visit
> each task individually and look at their "Resources" table (add them all
> up).
>
> It'd be cool if when a filter is in effect, the Resources table was
> updated to reflect only the resources consumed by the filter-selected tasks.
>
>
There has been no work on this (i.e., some way to filter the UI view w.r.t
a group of tasks), but this sounds like a good use case. Can you file a
ticket?



> There's also the question of the units/meaning of Resources. Through
> Marathon I give each of my Dockerized tasks .1 CPU. As I understand it,
> Docker multiplies this value times 1024 which is Docker's representation of
> all the cores on a host. So when I do "docker inspect " I will see
> CpuShares of 102. But in the Mesos UI each of my 6 tasks shows .2 CPUs
> allocated. I'm simply not sure what this means or how it's arrived at. I
> suspect that an MSP will ask the same questions.
>

You see 0.2 because Mesos adds 0.1 overhead for the default executor that
runs the docker task.

Re: How did the mesos master detect the disconnect of a framework (scheduler)

[Joseph Wu]

Here's a brief(?) run-down:

   1.
   
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/src/master/master.cpp#L5739-L5748
   

   When a new framework is added, the master opens a socket connection with
   the framework.
   - If this is a scheduler-driver-based framework, this is a plain socket
  connection.
  - If this is a new HTTP API framework, the master uses the streaming
  HTTP connection instead.
   2. The HTTP API framework's exit logic is simpler to explain.  When the
   streaming connection closes, the master considers the framework to have
   exited.  In the above code, see this chunk of code:
   http.closed()
 .onAny(defer(self(), ::exited, framework->id(), http));
   3. The scheduler-driver-based framework exit is a bit more involved:
  1.
  
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/3rdparty/libprocess/src/process.cpp#L1326
  Libprocess has a SocketManager which, as the name suggests, managed
  sockets.  Linking the master <-> framework spawns a socket here.
  2.
  
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/3rdparty/libprocess/src/process.cpp#L1394-L1400
  Linking will install a dispatch loop, which continually reads the
  data from the socket until the socket closes.
  3.
  
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/3rdparty/libprocess/src/process.cpp#L1300-L1312
  The dispatch loop calls "ignore_recv_data".  This detects when the
  socket closes and calls "SocketManager->close(s)".
  4.
  
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/3rdparty/libprocess/src/process.cpp#L1928
  "SocketManager->close" will generate a libprocess "ExitedEvent".
  5.
  
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/src/master/master.cpp#L1352
  Master has a listener for "ExitedEvent" which rate-limits these
  events.
  6.
  
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/src/master/master.cpp#L1161
  The "ExitedEvent" eventually gets propagated to that ^ method
  (through a libprocess event visitor).
  7.
  
https://github.com/apache/mesos/blob/4376803007446b949840d53945547d8a61b91339/src/master/master.cpp#L1165
  Finally, the framework gets removed.

Hope that helps,

~Joseph

On Fri, Feb 26, 2016 at 10:45 AM, Chong Chen  wrote:

> Hi,
>
> When a running framework was disconnected (manually terminated), the Mesos
> master will detect it immediately.  The master::exited() function will be
> invoked with log info “framework disconnected”.
>
> I just wondering, how this disconnect detection was implemented in Mesos?
> I can’t find any place in mesos src directory where the Master::exit()
> function was called.
>
>
>
> Thanks!
>
>
>
> Best Regards,
>
> Chong
>

Re: How did the mesos master detect the disconnect of a framework (scheduler)

[haosdent]

I think this is control through SocketManager in process.cpp. When the
connection broken, exit would be trigged in socket_manager.

On Sat, Feb 27, 2016 at 2:45 AM, Chong Chen  wrote:

> Hi,
>
> When a running framework was disconnected (manually terminated), the Mesos
> master will detect it immediately.  The master::exited() function will be
> invoked with log info “framework disconnected”.
>
> I just wondering, how this disconnect detection was implemented in Mesos?
> I can’t find any place in mesos src directory where the Master::exit()
> function was called.
>
>
>
> Thanks!
>
>
>
> Best Regards,
>
> Chong
>



-- 
Best Regards,
Haosdent Huang

How did the mesos master detect the disconnect of a framework (scheduler)

[Chong Chen]

Hi,
When a running framework was disconnected (manually terminated), the Mesos 
master will detect it immediately.  The master::exited() function will be 
invoked with log info "framework disconnected".
I just wondering, how this disconnect detection was implemented in Mesos? I 
can't find any place in mesos src directory where the Master::exit() function 
was called.

Thanks!

Best Regards,
Chong

Re: Bangalore Mesos User Group [http://www.meetup.com/Bangalore-Mesos-User-Group/]

[Vinod Kone]

On Wed, Feb 24, 2016 at 10:53 PM, Dhilip Kumar S 
wrote:

> http://www.meetup.com/Bangalore-Mesos-User-Group/


Added to user-groups

list. Thanks for starting it!

Re: Downloading s3 uris

[Joseph Wu]

The sandbox directory structure is a bit deep...  See the "Where is the
sandbox?" section here:
http://mesos.apache.org/documentation/latest/sandbox/


On Fri, Feb 26, 2016 at 10:15 AM, Aaron Carey  wrote:

> A second question for you all..
>
> I'm testing http uri downloads, and all the logs say that the file has
> downloaded (it even shows up in the mesos UI in the sandbox) but I can't
> find the file on disk anywhere. It doesn't appear in the docker container
> I'm running either (shouldn't it be in /mnt/mesos/sandbox?)
>
> Am I missing something here?
>
> Thanks for your help,
>
> Aaron
>
>
> --
> *From:* Radoslaw Gruchalski [ra...@gruchalski.com]
> *Sent:* 26 February 2016 17:41
>
> *To:* user@mesos.apache.org; user@mesos.apache.org
> *Subject:* Re: Downloading s3 uris
>
> Just keep in mind that every execution of such command starts a jvm and
> is, generally, heavyweight. Use WebHDFS if you can.
>
> Sent from Outlook Mobile 
>
>
>
>
> On Fri, Feb 26, 2016 at 9:13 AM -0800, "Shuai Lin"  > wrote:
>
> If you don't want to configure hadoop on your mesos slaves, the only
>> workaround I see is to write a "hadoop" script and put it in your PATH. It
>> need to support the following usage patterns:
>>
>> - hadoop version
>> - hadoop fs -copyToLocal s3n://path /target/directory/
>>
>> On Sat, Feb 27, 2016 at 12:31 AM, Aaron Carey  wrote:
>>
>>> I was trying to avoid generating urls for everything as this will
>>> complicate things a lot.
>>>
>>> Is there a straight forward way to get the fetcher to do it directly?
>>>
>>> --
>>> *From:* haosdent [haosd...@gmail.com]
>>> *Sent:* 26 February 2016 16:27
>>> *To:* user
>>> *Subject:* Re: Downloading s3 uris
>>>
>>> I think still could pass AWSAccessKeyId if it is private?
>>> http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html
>>>
>>> On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar <
>>> abhishek.amral...@talentica.com> wrote:
>>>
 In that case do we need to keep bucket/files public?

 -Abhishek

 From: Zhitao Li 
 Reply-To: "user@mesos.apache.org" 
 Date: Friday, 26 February 2016 at 8:23 AM
 To: "user@mesos.apache.org" 
 Subject: Re: Downloading s3 uris

 Haven't directly used s3 download, but I think a workaround (if you
 don't care ACL about the files) is to use http
 
  url
 instead.

 On Feb 26, 2016, at 8:17 AM, Aaron Carey  wrote:

 I'm attempting to fetch files from s3 uris in mesos, but we're not
 using hdfs in our cluster... however I believe I need the client installed.

 Is it possible to just have the client running without a full hdfs
 setup?

 I haven't been able to find much information in the docs, could someone
 point me in the right direction?

 Thanks!

 Aaron



>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>

RE: Downloading s3 uris

[Aaron Carey]

A second question for you all..

I'm testing http uri downloads, and all the logs say that the file has 
downloaded (it even shows up in the mesos UI in the sandbox) but I can't find 
the file on disk anywhere. It doesn't appear in the docker container I'm 
running either (shouldn't it be in /mnt/mesos/sandbox?)

Am I missing something here?

Thanks for your help,

Aaron



From: Radoslaw Gruchalski [ra...@gruchalski.com]
Sent: 26 February 2016 17:41
To: user@mesos.apache.org; user@mesos.apache.org
Subject: Re: Downloading s3 uris

Just keep in mind that every execution of such command starts a jvm and is, 
generally, heavyweight. Use WebHDFS if you can.

Sent from Outlook Mobile




On Fri, Feb 26, 2016 at 9:13 AM -0800, "Shuai Lin" 
> wrote:

If you don't want to configure hadoop on your mesos slaves, the only workaround 
I see is to write a "hadoop" script and put it in your PATH. It need to support 
the following usage patterns:

- hadoop version
- hadoop fs -copyToLocal s3n://path /target/directory/

On Sat, Feb 27, 2016 at 12:31 AM, Aaron Carey 
> wrote:
I was trying to avoid generating urls for everything as this will complicate 
things a lot.

Is there a straight forward way to get the fetcher to do it directly?


From: haosdent [haosd...@gmail.com]
Sent: 26 February 2016 16:27
To: user
Subject: Re: Downloading s3 uris

I think still could pass AWSAccessKeyId if it is private? 
http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html

On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar 
> wrote:
In that case do we need to keep bucket/files public?

-Abhishek

From: Zhitao Li >
Reply-To: "user@mesos.apache.org" 
>
Date: Friday, 26 February 2016 at 8:23 AM
To: "user@mesos.apache.org" 
>
Subject: Re: Downloading s3 uris

Haven't directly used s3 download, but I think a workaround (if you don't care 
ACL about the files) is to use 
http
 url instead.
On Feb 26, 2016, at 8:17 AM, Aaron Carey 
> wrote:

I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point 
me in the right direction?

Thanks!

Aaron




--
Best Regards,
Haosdent Huang

Re: Downloading s3 uris

[Radoslaw Gruchalski]

Just keep in mind that every execution of such command starts a jvm and is, 
generally, heavyweight. Use WebHDFS if you can.

Sent from Outlook Mobile




On Fri, Feb 26, 2016 at 9:13 AM -0800, "Shuai Lin"  
wrote:










If you don't want to configure hadoop on your mesos slaves, the only workaround 
I see is to write a "hadoop" script and put it in your PATH. It need to support 
the following usage patterns:
- hadoop version- hadoop fs -copyToLocal s3n://path /target/directory/
On Sat, Feb 27, 2016 at 12:31 AM, Aaron Carey  wrote:






I was trying to avoid generating urls for everything as this will complicate 
things a lot.



Is there a straight forward way to get the fetcher to do it directly?





From: haosdent [haosd...@gmail.com]

Sent: 26 February 2016 16:27

To: user

Subject: Re: Downloading s3 uris






I think still could pass AWSAccessKeyId if it is private? 
http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html


On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar 
 wrote:



In that case do we need to keep bucket/files public?



-Abhishek





From: Zhitao Li 

Reply-To: "user@mesos.apache.org" 

Date: Friday, 26 February 2016 at 8:23 AM

To: "user@mesos.apache.org" 

Subject: Re: Downloading s3 uris








Haven't directly used s3 download, but I think a workaround (if you don't care 
ACL about the files) is to use
 http url instead.



On Feb 26, 2016, at 8:17 AM, Aaron Carey  wrote:




I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
in our cluster... however I believe I need the client installed.



Is it possible to just have the client running without a full hdfs setup?



I haven't been able to find much information in the docs, could someone point 
me in the right direction?



Thanks!



Aaron



















-- 

Best Regards,

Haosdent Huang

RE: Downloading s3 uris

[Aaron Carey]

I know this is a stupid question...

but how do I just install the client without all the rest of the stuff?


From: haosdent [haosd...@gmail.com]
Sent: 26 February 2016 16:50
To: user
Subject: Re: Downloading s3 uris

So far have to install a hdfs client if you want to use "s3n://xxx". :-(

On Sat, Feb 27, 2016 at 12:39 AM, Abhishek Amralkar 
> wrote:
Agreed with @aaron, it will be too much manual work to generate S3 url 
everytime.

Thanks
Abhishek

From: Aaron Carey >
Reply-To: "user@mesos.apache.org" 
>
Date: Friday, 26 February 2016 at 8:31 AM
To: "user@mesos.apache.org" 
>
Subject: RE: Downloading s3 uris

I was trying to avoid generating urls for everything as this will complicate 
things a lot.

Is there a straight forward way to get the fetcher to do it directly?


From: haosdent [haosd...@gmail.com]
Sent: 26 February 2016 16:27
To: user
Subject: Re: Downloading s3 uris

I think still could pass AWSAccessKeyId if it is private? 
http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html

On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar 
> wrote:
In that case do we need to keep bucket/files public?

-Abhishek

From: Zhitao Li >
Reply-To: "user@mesos.apache.org" 
>
Date: Friday, 26 February 2016 at 8:23 AM
To: "user@mesos.apache.org" 
>
Subject: Re: Downloading s3 uris

Haven't directly used s3 download, but I think a workaround (if you don't care 
ACL about the files) is to use 
http
 url instead.
On Feb 26, 2016, at 8:17 AM, Aaron Carey 
> wrote:

I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point 
me in the right direction?

Thanks!

Aaron




--
Best Regards,
Haosdent Huang



--
Best Regards,
Haosdent Huang

Re: Downloading s3 uris

[Shuai Lin]

If you don't want to configure hadoop on your mesos slaves, the only
workaround I see is to write a "hadoop" script and put it in your PATH. It
need to support the following usage patterns:

- hadoop version
- hadoop fs -copyToLocal s3n://path /target/directory/

On Sat, Feb 27, 2016 at 12:31 AM, Aaron Carey  wrote:

> I was trying to avoid generating urls for everything as this will
> complicate things a lot.
>
> Is there a straight forward way to get the fetcher to do it directly?
>
> --
> *From:* haosdent [haosd...@gmail.com]
> *Sent:* 26 February 2016 16:27
> *To:* user
> *Subject:* Re: Downloading s3 uris
>
> I think still could pass AWSAccessKeyId if it is private?
> http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html
>
> On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar <
> abhishek.amral...@talentica.com> wrote:
>
>> In that case do we need to keep bucket/files public?
>>
>> -Abhishek
>>
>> From: Zhitao Li 
>> Reply-To: "user@mesos.apache.org" 
>> Date: Friday, 26 February 2016 at 8:23 AM
>> To: "user@mesos.apache.org" 
>> Subject: Re: Downloading s3 uris
>>
>> Haven't directly used s3 download, but I think a workaround (if you don't
>> care ACL about the files) is to use http
>> 
>>  url
>> instead.
>>
>> On Feb 26, 2016, at 8:17 AM, Aaron Carey  wrote:
>>
>> I'm attempting to fetch files from s3 uris in mesos, but we're not using
>> hdfs in our cluster... however I believe I need the client installed.
>>
>> Is it possible to just have the client running without a full hdfs setup?
>>
>> I haven't been able to find much information in the docs, could someone
>> point me in the right direction?
>>
>> Thanks!
>>
>> Aaron
>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: Downloading s3 uris

[haosdent]

So far have to install a hdfs client if you want to use "s3n://xxx". :-(

On Sat, Feb 27, 2016 at 12:39 AM, Abhishek Amralkar <
abhishek.amral...@talentica.com> wrote:

> Agreed with @aaron, it will be too much manual work to generate S3 url
> everytime.
>
> Thanks
> Abhishek
>
> From: Aaron Carey 
> Reply-To: "user@mesos.apache.org" 
> Date: Friday, 26 February 2016 at 8:31 AM
> To: "user@mesos.apache.org" 
> Subject: RE: Downloading s3 uris
>
> I was trying to avoid generating urls for everything as this will
> complicate things a lot.
>
> Is there a straight forward way to get the fetcher to do it directly?
>
> --
> *From:* haosdent [haosd...@gmail.com]
> *Sent:* 26 February 2016 16:27
> *To:* user
> *Subject:* Re: Downloading s3 uris
>
> I think still could pass AWSAccessKeyId if it is private?
> http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html
>
> On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar <
> abhishek.amral...@talentica.com> wrote:
>
>> In that case do we need to keep bucket/files public?
>>
>> -Abhishek
>>
>> From: Zhitao Li 
>> Reply-To: "user@mesos.apache.org" 
>> Date: Friday, 26 February 2016 at 8:23 AM
>> To: "user@mesos.apache.org" 
>> Subject: Re: Downloading s3 uris
>>
>> Haven't directly used s3 download, but I think a workaround (if you don't
>> care ACL about the files) is to use http
>> 
>>  url
>> instead.
>>
>> On Feb 26, 2016, at 8:17 AM, Aaron Carey  wrote:
>>
>> I'm attempting to fetch files from s3 uris in mesos, but we're not using
>> hdfs in our cluster... however I believe I need the client installed.
>>
>> Is it possible to just have the client running without a full hdfs setup?
>>
>> I haven't been able to find much information in the docs, could someone
>> point me in the right direction?
>>
>> Thanks!
>>
>> Aaron
>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>



-- 
Best Regards,
Haosdent Huang

Re: Downloading s3 uris

[Abhishek Amralkar]

Agreed with @aaron, it will be too much manual work to generate S3 url everytime.


Thanks
Abhishek




From: Aaron Carey 
Reply-To: "user@mesos.apache.org" 
Date: Friday, 26 February 2016 at 8:31 AM
To: "user@mesos.apache.org" 
Subject: RE: Downloading s3 uris





I was trying to avoid generating urls for everything as this will complicate things a lot.

Is there a straight forward way to get the fetcher to do it directly?



From: haosdent [haosd...@gmail.com]
Sent: 26 February 2016 16:27
To: user
Subject: Re: Downloading s3 uris




I think still could pass AWSAccessKeyId if it is private? http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html

On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar 
 wrote:


In that case do we need to keep bucket/files public?


-Abhishek




From: Zhitao Li 
Reply-To: "user@mesos.apache.org" 
Date: Friday, 26 February 2016 at 8:23 AM
To: "user@mesos.apache.org" 
Subject: Re: Downloading s3 uris






Haven't directly used s3 download, but I think a workaround (if you don't care ACL about the files) is to use
 http url instead.


On Feb 26, 2016, at 8:17 AM, Aaron Carey  wrote:



I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point me in the right direction?

Thanks!

Aaron















-- 
Best Regards,
Haosdent Huang

Re: Mesos metrics -> influxdb

[craig w]

Niklas -- is the mesos collector ready? it's in the "wish list" on that
page.

On Fri, Feb 26, 2016 at 11:12 AM, Niklas Nielsen  wrote:

> Or take SNAP for a spin:
> https://github.com/intelsdi-x/snap/blob/master/docs/PLUGIN_CATALOG.md
> Can collect metrics from Mesos and output in several TSDBs and you can do
> transformations on the metrics in flight.
>
> Niklas
>
> On Fri, Feb 26, 2016 at 1:05 AM, Aaron Carey  wrote:
>
>> Thanks Alberto, those are the two projects I was having issues with. I
>> was hoping not to have to fork the first one (to add 0.24 functionality)
>> and the second just keeps timing out and crashing (!)
>>
>> I'll have a look at telegraph!
>>
>> Thanks all for your help,
>>
>> Aaron
>>
>> --
>> *From:* Alberto del Barrio [alberto.delbarrio.albe...@gmail.com]
>> *Sent:* 26 February 2016 08:04
>> *To:* user@mesos.apache.org
>> *Subject:* Re: Mesos metrics -> influxdb
>>
>> I am using collectd to gather those metrics and I am pretty happy with it.
>> I use two plugins:
>>  - To gather mesos metrics (System metrics, number of slaves connected...
>> )
>> https://github.com/rayrod2030/collectd-mesos
>>  - To gather CPU and memory used per mesos tasks:
>> https://github.com/bobrik/collectd-mesos-tasks
>>
>> The setup is stable, it never crashed. Regarding the version 0.24 it can
>> work with few modifications.
>>
>>
>> On 02/25/16 18:32, Aaron Carey wrote:
>>
>> Has anyone had a good experience recording mesos metrics into influxdb?
>>
>> I've found a couple of options, a collectd plugin which doesn't appear to
>> work with version 0.24.x and a more up to date containerised option which
>> randomly crashes regularly and doesn't appear to actually post any stats.
>>
>> Anyone have any good solutions?
>>
>>
>>
>>
>
>
> --
> Niklas
>



-- 

https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links

RE: Downloading s3 uris

[Aaron Carey]

I was trying to avoid generating urls for everything as this will complicate 
things a lot.

Is there a straight forward way to get the fetcher to do it directly?


From: haosdent [haosd...@gmail.com]
Sent: 26 February 2016 16:27
To: user
Subject: Re: Downloading s3 uris

I think still could pass AWSAccessKeyId if it is private? 
http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html

On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar 
> wrote:
In that case do we need to keep bucket/files public?

-Abhishek

From: Zhitao Li >
Reply-To: "user@mesos.apache.org" 
>
Date: Friday, 26 February 2016 at 8:23 AM
To: "user@mesos.apache.org" 
>
Subject: Re: Downloading s3 uris

Haven't directly used s3 download, but I think a workaround (if you don't care 
ACL about the files) is to use 
http
 url instead.
On Feb 26, 2016, at 8:17 AM, Aaron Carey 
> wrote:

I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point 
me in the right direction?

Thanks!

Aaron




--
Best Regards,
Haosdent Huang

Re: Downloading s3 uris

[haosdent]

I think still could pass AWSAccessKeyId if it is private?
http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html

On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar <
abhishek.amral...@talentica.com> wrote:

> In that case do we need to keep bucket/files public?
>
> -Abhishek
>
> From: Zhitao Li 
> Reply-To: "user@mesos.apache.org" 
> Date: Friday, 26 February 2016 at 8:23 AM
> To: "user@mesos.apache.org" 
> Subject: Re: Downloading s3 uris
>
> Haven't directly used s3 download, but I think a workaround (if you don't
> care ACL about the files) is to use http
> 
>  url
> instead.
>
> On Feb 26, 2016, at 8:17 AM, Aaron Carey  wrote:
>
> I'm attempting to fetch files from s3 uris in mesos, but we're not using
> hdfs in our cluster... however I believe I need the client installed.
>
> Is it possible to just have the client running without a full hdfs setup?
>
> I haven't been able to find much information in the docs, could someone
> point me in the right direction?
>
> Thanks!
>
> Aaron
>
>
>


-- 
Best Regards,
Haosdent Huang

Re: Downloading s3 uris

[Abhishek Amralkar]

In that case do we need to keep bucket/files public?


-Abhishek




From: Zhitao Li 
Reply-To: "user@mesos.apache.org" 
Date: Friday, 26 February 2016 at 8:23 AM
To: "user@mesos.apache.org" 
Subject: Re: Downloading s3 uris





Haven't directly used s3 download, but I think a workaround (if you don't care ACL about the files) is to use http url instead.


On Feb 26, 2016, at 8:17 AM, Aaron Carey  wrote:



I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point me in the right direction?

Thanks!

Aaron

Re: Downloading s3 uris

[Zhitao Li]

Haven't directly used s3 download, but I think a workaround (if you don't care 
ACL about the files) is to use http 

 url instead.
> On Feb 26, 2016, at 8:17 AM, Aaron Carey  wrote:
> 
> I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
> in our cluster... however I believe I need the client installed.
> 
> Is it possible to just have the client running without a full hdfs setup?
> 
> I haven't been able to find much information in the docs, could someone point 
> me in the right direction?
> 
> Thanks!
> 
> Aaron

Downloading s3 uris

[Aaron Carey]

I'm attempting to fetch files from s3 uris in mesos, but we're not using hdfs 
in our cluster... however I believe I need the client installed.

Is it possible to just have the client running without a full hdfs setup?

I haven't been able to find much information in the docs, could someone point 
me in the right direction?

Thanks!

Aaron

Re: Mesos metrics -> influxdb

[Niklas Nielsen]

Or take SNAP for a spin:
https://github.com/intelsdi-x/snap/blob/master/docs/PLUGIN_CATALOG.md
Can collect metrics from Mesos and output in several TSDBs and you can do
transformations on the metrics in flight.

Niklas

On Fri, Feb 26, 2016 at 1:05 AM, Aaron Carey  wrote:

> Thanks Alberto, those are the two projects I was having issues with. I was
> hoping not to have to fork the first one (to add 0.24 functionality) and
> the second just keeps timing out and crashing (!)
>
> I'll have a look at telegraph!
>
> Thanks all for your help,
>
> Aaron
>
> --
> *From:* Alberto del Barrio [alberto.delbarrio.albe...@gmail.com]
> *Sent:* 26 February 2016 08:04
> *To:* user@mesos.apache.org
> *Subject:* Re: Mesos metrics -> influxdb
>
> I am using collectd to gather those metrics and I am pretty happy with it.
> I use two plugins:
>  - To gather mesos metrics (System metrics, number of slaves connected... )
> https://github.com/rayrod2030/collectd-mesos
>  - To gather CPU and memory used per mesos tasks:
> https://github.com/bobrik/collectd-mesos-tasks
>
> The setup is stable, it never crashed. Regarding the version 0.24 it can
> work with few modifications.
>
>
> On 02/25/16 18:32, Aaron Carey wrote:
>
> Has anyone had a good experience recording mesos metrics into influxdb?
>
> I've found a couple of options, a collectd plugin which doesn't appear to
> work with version 0.24.x and a more up to date containerised option which
> randomly crashes regularly and doesn't appear to actually post any stats.
>
> Anyone have any good solutions?
>
>
>
>


-- 
Niklas

Re: Question regarding docker containerizer

[Pradeep Chhetri]

Actually the reason i asked this question is because i am unable to
generate these warning messages by setting MEMORY < MIN_MEM and CPU <
MIN_CPUS in case of docker containers started using marathon.

https://github.com/apache/mesos/blob/4d4d7166414f0ebd8d5e40df34070446098a3c91/src/master/validation.cpp#L397-L425

So i was trying to figure out whether

https://github.com/apache/mesos/blob/4d4d7166414f0ebd8d5e40df34070446098a3c91/src/master/validation.cpp#L399
is resulting in true or false in case of docker containers started using
marathon.


On Fri, Feb 26, 2016 at 8:20 AM, Pradeep Chhetri <
pradeep.chhetr...@gmail.com> wrote:

> Thank you Jie and Jojy for the help.
>
> On Fri, Feb 26, 2016 at 4:50 AM, Jojy Varghese  wrote:
>
>> Hi Pradeep,
>>
>> The relevant code if you are interested is at :
>>
>> https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L3561
>>
>> https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L3672
>>
>> -jojy
>>
>>
>> On Feb 25, 2016, at 4:10 PM, Pradeep Chhetri 
>> wrote:
>>
>> Hi Jie,
>>
>> I see executor_id as empty. Does this means that it is running as task
>> and there is no executor for it ? Although, in the ps output, I see
>> mesos-slave is spawning a mesos-docker-executor which in turn spawning the
>> docker command. Here is the detail of one of the marathon task:
>>
>> {"id": "logs_role_kibana.3b024b6e-dbf3-11e5-bc67-56847afe9799","name": "
>> kibana.role.logs","framework_id": "
>> 20150904-093718-2198675372-5050-22379-","executor_id": "","slave_id
>> ": "a4f30c35-eee7-4110-a82f-cab5522c9b1b-S4","state": "TASK_RUNNING","
>> resources": {"cpus": 0.5,"disk": 0,"mem": 1024,"ports": "[31050-31050]"},
>> "statuses": [{"state": "TASK_RUNNING","timestamp": 1456427344.63661,"
>> labels": [{"key": "Docker.NetworkSettings.IPAddress","value": "
>> 172.17.0.11"}],"container_status": {"network_infos": [{"ip_address": "
>> 172.17.0.11","ip_addresses": [{"ip_address": "172.17.0.11"}]}]}}],"
>> container": {"type": "DOCKER","docker": {"image": "kibana:4.3.1","network
>> ": "BRIDGE","port_mappings": [{"host_port": 31050,"container_port": 5601,
>> "protocol": "tcp"}],"privileged": false,"parameters": [{"key": "
>> publish-all","value": "true"}],"force_pull_image": true}}},
>>
>>
>>
>> On Thu, Feb 25, 2016 at 11:59 PM, Jie Yu  wrote:
>>
>>> You can checkout the state.json endpoint on the master
>>>
>>> On Thu, Feb 25, 2016 at 3:53 PM, Pradeep Chhetri <
>>> pradeep.chhetr...@gmail.com> wrote:
>>>
 Hello Jie,

 Thank you for the quick reply. Sorry for asking silly question. How can
 i look for the taskinfo of a running container ? Can i see the TaskInfo
 details in the mesos master UI for a task?

 On Thu, Feb 25, 2016 at 10:52 PM, Jie Yu  wrote:

> You can take a look at the TaskInfo. If the TaskInfo does not have
> ExecutorInfo set, then it's a task. Otherwise, Mesos will launch the
> executor and send the task to the executor.
>
> - Jie
>
> On Thu, Feb 25, 2016 at 2:50 PM, Pradeep Chhetri <
> pradeep.chhetr...@gmail.com> wrote:
>
>> Hello,
>>
>> From docker containerizer documentation (
>> http://mesos.apache.org/documentation/latest/docker-containerizer/),
>>
>> *"Users can either launch a Docker image as a Task, or as an
>> Executor."*
>>
>> How can i identify whether a docker container started by lets say
>> marathon is running as a task or as an executor ?
>>
>>
>> Thank you.
>>
>> -Pradeep
>>
>>
>>
>


 --
 Pradeep Chhetri

 In the world of Linux, who needs Windows and Gates...

>>>
>>>
>>
>>
>> --
>> Pradeep Chhetri
>>
>> In the world of Linux, who needs Windows and Gates...
>>
>>
>>
>
>
> --
> Pradeep Chhetri
>
> In the world of Linux, who needs Windows and Gates...
>



-- 
Pradeep Chhetri

In the world of Linux, who needs Windows and Gates...

RE: Mesos metrics -> influxdb

[Aaron Carey]

Thanks Alberto, those are the two projects I was having issues with. I was 
hoping not to have to fork the first one (to add 0.24 functionality) and the 
second just keeps timing out and crashing (!)

I'll have a look at telegraph!

Thanks all for your help,

Aaron


From: Alberto del Barrio [alberto.delbarrio.albe...@gmail.com]
Sent: 26 February 2016 08:04
To: user@mesos.apache.org
Subject: Re: Mesos metrics -> influxdb

I am using collectd to gather those metrics and I am pretty happy with it.
I use two plugins:
 - To gather mesos metrics (System metrics, number of slaves connected... )
https://github.com/rayrod2030/collectd-mesos
 - To gather CPU and memory used per mesos tasks: 
https://github.com/bobrik/collectd-mesos-tasks

The setup is stable, it never crashed. Regarding the version 0.24 it can work 
with few modifications.


On 02/25/16 18:32, Aaron Carey wrote:
Has anyone had a good experience recording mesos metrics into influxdb?

I've found a couple of options, a collectd plugin which doesn't appear to work 
with version 0.24.x and a more up to date containerised option which randomly 
crashes regularly and doesn't appear to actually post any stats.

Anyone have any good solutions?

Re: Question regarding docker containerizer

[Pradeep Chhetri]

Thank you Jie and Jojy for the help.

On Fri, Feb 26, 2016 at 4:50 AM, Jojy Varghese  wrote:

> Hi Pradeep,
>
> The relevant code if you are interested is at :
>
> https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L3561
>
> https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L3672
>
> -jojy
>
>
> On Feb 25, 2016, at 4:10 PM, Pradeep Chhetri 
> wrote:
>
> Hi Jie,
>
> I see executor_id as empty. Does this means that it is running as task and
> there is no executor for it ? Although, in the ps output, I see mesos-slave
> is spawning a mesos-docker-executor which in turn spawning the docker
> command. Here is the detail of one of the marathon task:
>
> {"id": "logs_role_kibana.3b024b6e-dbf3-11e5-bc67-56847afe9799","name": "
> kibana.role.logs","framework_id": "
> 20150904-093718-2198675372-5050-22379-","executor_id": "","slave_id":
> "a4f30c35-eee7-4110-a82f-cab5522c9b1b-S4","state": "TASK_RUNNING","
> resources": {"cpus": 0.5,"disk": 0,"mem": 1024,"ports": "[31050-31050]"},"
> statuses": [{"state": "TASK_RUNNING","timestamp": 1456427344.63661,"labels
> ": [{"key": "Docker.NetworkSettings.IPAddress","value": "172.17.0.11"}],"
> container_status": {"network_infos": [{"ip_address": "172.17.0.11","
> ip_addresses": [{"ip_address": "172.17.0.11"}]}]}}],"container": {"type":
> "DOCKER","docker": {"image": "kibana:4.3.1","network": "BRIDGE","
> port_mappings": [{"host_port": 31050,"container_port": 5601,"protocol": "
> tcp"}],"privileged": false,"parameters": [{"key": "publish-all","value": "
> true"}],"force_pull_image": true}}},
>
>
>
> On Thu, Feb 25, 2016 at 11:59 PM, Jie Yu  wrote:
>
>> You can checkout the state.json endpoint on the master
>>
>> On Thu, Feb 25, 2016 at 3:53 PM, Pradeep Chhetri <
>> pradeep.chhetr...@gmail.com> wrote:
>>
>>> Hello Jie,
>>>
>>> Thank you for the quick reply. Sorry for asking silly question. How can
>>> i look for the taskinfo of a running container ? Can i see the TaskInfo
>>> details in the mesos master UI for a task?
>>>
>>> On Thu, Feb 25, 2016 at 10:52 PM, Jie Yu  wrote:
>>>
 You can take a look at the TaskInfo. If the TaskInfo does not have
 ExecutorInfo set, then it's a task. Otherwise, Mesos will launch the
 executor and send the task to the executor.

 - Jie

 On Thu, Feb 25, 2016 at 2:50 PM, Pradeep Chhetri <
 pradeep.chhetr...@gmail.com> wrote:

> Hello,
>
> From docker containerizer documentation (
> http://mesos.apache.org/documentation/latest/docker-containerizer/),
>
> *"Users can either launch a Docker image as a Task, or as an
> Executor."*
>
> How can i identify whether a docker container started by lets say
> marathon is running as a task or as an executor ?
>
>
> Thank you.
>
> -Pradeep
>
>
>

>>>
>>>
>>> --
>>> Pradeep Chhetri
>>>
>>> In the world of Linux, who needs Windows and Gates...
>>>
>>
>>
>
>
> --
> Pradeep Chhetri
>
> In the world of Linux, who needs Windows and Gates...
>
>
>


-- 
Pradeep Chhetri

In the world of Linux, who needs Windows and Gates...

Re: Mesos metrics -> influxdb

[Alberto del Barrio]

I am using collectd to gather those metrics and I am pretty happy with it.
I use two plugins:
 - To gather mesos metrics (System metrics, number of slaves connected... )
https://github.com/rayrod2030/collectd-mesos
 - To gather CPU and memory used per mesos tasks: 
https://github.com/bobrik/collectd-mesos-tasks


The setup is stable, it never crashed. Regarding the version 0.24 it can 
work with few modifications.



On 02/25/16 18:32, Aaron Carey wrote:

Has anyone had a good experience recording mesos metrics into influxdb?

I've found a couple of options, a collectd plugin which doesn't appear 
to work with version 0.24.x and a more up to date containerised option 
which randomly crashes regularly and doesn't appear to actually post 
any stats.


Anyone have any good solutions?