Re: cron-like scheduling in mesos framework?

[l vic]

suppose i have more than one db container on the same node... I don't think
I can constrain to the certain container-id, or can i?

On Tue, Jan 10, 2017 at 11:58 AM, Diogo Gomes  wrote:

> For this purpose, you should use constraints.
>
> https://mesos.github.io/chronos/docs/api.html#constraints
>
> Diogo Gomes
>
>
>
>
> On 10 de jan de 2017, at 14:31, l vic  wrote:
>
> OK, got it, thanks.
> That makes it useless for data backup though, if chronos starts new
> container on the node where it does not have an access to db data
>
> On Tue, Jan 10, 2017 at 11:14 AM, Stephen Gran 
> wrote:
>
>> Hi,
>>
>> This is what I'm trying to say, perhaps not very clearly:
>> the task 'ls /' will be run in a 4th container.  It is a new process.
>>
>> Cheers,
>>
>> On 10/01/17 15:54, l vic wrote:
>> > Great, but I have several servers: 3 different containers running on 3
>> > different nodes. If i try to schedule the job :
>> > curl -L -H 'Content-Type: application/json' -X POST
>> > http://|chronos-node:|2002/scheduler/iso8601
>> >  -d '
>> > {
>> >   "schedule": "R10/2015-06-05T13:36:00Z/PT1M",
>> >   "name": "test-container-job",
>> >  "container": {
>> >   "type": "DOCKER",
>> >   "image": "",
>> >   "network": "BRIDGE"
>> >   },
>> >   "cpus": "0.1",
>> >   "mem": "512",
>> >  "uris" : ["file:///.dockercfg"],
>> >   "command": "ls /"
>> > }'
>> > Will it schedule the job on all 3 nodes, or only the first one? Can I
>> > actually choose which one in my json?
>> > Thks
>> >
>> > On Tue, Jan 10, 2017 at 10:47 AM, Stephen Gran > > > wrote:
>> >
>> > Hi,
>> >
>> > Chronos launches a new task to perform the job.  Think of it more
>> like
>> > cron launching a new process when this is run on a single server.
>> >
>> > Cheers,
>> >
>> > On 10/01/17 15:31, l vic wrote:
>> > > If I have cluster of 3 running containers - how Chronos will
>> select one
>> > > to schedule the job?
>> > >
>> > > On Fri, Jan 6, 2017 at 1:33 PM, Radek Gruchalski <
>> ra...@gruchalski.com 
>> > > >>
>> wrote:
>> > >
>> > > Chronos launches a task on Mesos whenever a scheduled event
>> is bound
>> > > to happen.
>> > > You can run a container task with Chronos, n problem:
>> > >
>> > > https://mesos.github.io/chronos/docs/api.html#job-configura
>> tion
>> > 
>> > > > ation
>> > >
>> > >
>> > > Check the container field.
>> > >
>> > > –
>> > > Best regards,
>> > > Radek Gruchalski
>> > > ra...@gruchalski.com 
>> > >
>> > >
>> > >
>> > > On January 6, 2017 at 7:17:36 PM, l vic (lvic4...@gmail.com
>> 
>> > > >)
>> wrote:
>> > >
>> > >> Chronos is a separate service... I have database cluster
>> running
>> > >> as Mesos framework. I have to  periodically schedule data
>> backups
>> > >> using shell scripts within running docker container. Is that
>> > >> possible with Chronos?
>> > >>
>> > >> On Fri, Jan 6, 2017 at 11:47 AM, Radek Gruchalski
>> > >> 
>> > >> wrote:
>> > >>
>> > >> There are two options:
>> > >>
>> > >> - https://github.com/dcos/metronome <
>> https://github.com/dcos/metronome>
>> > >> > https://github.com/dcos/metronome>>
>> > >> - https://github.com/mesos/chronos <
>> https://github.com/mesos/chronos>
>> > >> > https://github.com/mesos/chronos>>
>> > >>
>> > >> while Metronome, apparently, being a successor of
>> Chronos.
>> > >>
>> > >> –
>> > >> Best regards,
>> > >> Radek Gruchalski
>> > >> ra...@gruchalski.com 
>> > >
>> > >>
>> > >>
>> > >> On January 6, 2017 at 5:40:45 PM, l vic (
>> lvic4...@gmail.com 
>> > >> >)
>> > wrote:
>> > >>
>> > >>> Hi,
>> > >>> Is there a way to schedule mesos framework task for
>> > execution
>> > >>> at certain 

Re: cron-like scheduling in mesos framework?

[Diogo Gomes]

For this purpose, you should use constraints. 

https://mesos.github.io/chronos/docs/api.html#constraints

Diogo Gomes




> On 10 de jan de 2017, at 14:31, l vic  wrote:
> 
> OK, got it, thanks. 
> That makes it useless for data backup though, if chronos starts new container 
> on the node where it does not have an access to db data
> 
> On Tue, Jan 10, 2017 at 11:14 AM, Stephen Gran  > wrote:
> Hi,
> 
> This is what I'm trying to say, perhaps not very clearly:
> the task 'ls /' will be run in a 4th container.  It is a new process.
> 
> Cheers,
> 
> On 10/01/17 15:54, l vic wrote:
> > Great, but I have several servers: 3 different containers running on 3
> > different nodes. If i try to schedule the job :
> > curl -L -H 'Content-Type: application/json' -X POST
> > http://|chronos-node:|2002/scheduler/iso8601
> >  > > -d '
> > {
> >   "schedule": "R10/2015-06-05T13:36:00Z/PT1M",
> >   "name": "test-container-job",
> >  "container": {
> >   "type": "DOCKER",
> >   "image": "",
> >   "network": "BRIDGE"
> >   },
> >   "cpus": "0.1",
> >   "mem": "512",
> >  "uris" : ["file:///.dockercfg"],
> >   "command": "ls /"
> > }'
> > Will it schedule the job on all 3 nodes, or only the first one? Can I
> > actually choose which one in my json?
> > Thks
> >
> > On Tue, Jan 10, 2017 at 10:47 AM, Stephen Gran  > 
> > >> wrote:
> >
> > Hi,
> >
> > Chronos launches a new task to perform the job.  Think of it more like
> > cron launching a new process when this is run on a single server.
> >
> > Cheers,
> >
> > On 10/01/17 15:31, l vic wrote:
> > > If I have cluster of 3 running containers - how Chronos will select 
> > one
> > > to schedule the job?
> > >
> > > On Fri, Jan 6, 2017 at 1:33 PM, Radek Gruchalski 
> >  
> > >
> > >  
> >  > >
> > > Chronos launches a task on Mesos whenever a scheduled event is 
> > bound
> > > to happen.
> > > You can run a container task with Chronos, n problem:
> > >
> > > https://mesos.github.io/chronos/docs/api.html#job-configuration 
> > 
> >  > >
> > >  > 
> >  > >>
> > >
> > > Check the container field.
> > >
> > > –
> > > Best regards,
> 
> > > Radek Gruchalski
> > > ra...@gruchalski.com  
> > >
> >  
> > >>
> > >
> > >
> > > On January 6, 2017 at 7:17:36 PM, l vic (lvic4...@gmail.com 
> >   > >
> > >  
> > >>) wrote:
> > >
> > >> Chronos is a separate service... I have database cluster running
> > >> as Mesos framework. I have to  periodically schedule data backups
> > >> using shell scripts within running docker container. Is that
> > >> possible with Chronos?
> > >>
> > >> On Fri, Jan 6, 2017 at 11:47 AM, Radek Gruchalski
> > >>  
> > >
> >  
> >  > >>
> > >> There are two options:
> > >>
> > >> - https://github.com/dcos/metronome 
> >   > >
> > >>  >   > >>
> > >> - https://github.com/mesos/chronos 
> >  

Re: Providing end-user feedback on Docker image download progress

[Zhitao Li]

Hi Franck,

Can you enable comment on the doc? Thanks!

On Tue, Jan 10, 2017 at 6:52 AM, Frank Scholten 
wrote:

> Here is a very rudimentary design doc on image download progress:
> https://docs.google.com/document/d/1x9dtcNgwecAp1xAHeDH-
> FJOJRIWC3OfC2akX8LLWNcs/edit?usp=sharing
>
> Feel free to comment.
>
> Cheers,
>
> Frank
>
>
>
> On Tue, Jan 10, 2017 at 8:49 AM, Frank Scholten 
> wrote:
> > Hi Jie,
> >
> > Great!
> >
> > What are the next steps here? Create a design document?
> >
> > Cheers,
> >
> > Frank
> >
> >
> > On Mon, Jan 9, 2017 at 8:38 PM, Jie Yu  wrote:
> >> Frank,
> >>
> >> Thanks for reaching out! I think this is definitely something we've
> thought
> >> about, just don't have the cycle to get it prioritized.
> >> https://issues.apache.org/jira/browse/MESOS-2256
> >>
> >> The idea is around re-using STAGING state with more information about
> the
> >> progress of the provisioning (and fetching). cc Vinod, BenM, BenH
> >>
> >>> We want to contribute this work back to the project and like to know
> >>>
> >>> which of the above and other options are the most viable.
> >>
> >>
> >> That's great!
> >>
> >> - Jie
> >>
> >> On Mon, Jan 9, 2017 at 3:07 AM, Frank Scholten 
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Together with a client we are looking into ways to provide end-user
> >>> feedback on Docker image downloads for Mesos tasks.
> >>>
> >>> The idea is that an end user who uses a cli tool that connects to a
> >>> Mesos scheduler gets image dowload and provisioning progress on it's
> >>> standard out stream every few seconds.
> >>>
> >>> Our client has a Mesos framework that runs large Docker images via the
> >>> Mesos Containerizer and we like to know what our options are of adding
> >>> this feature either to the framework, to a Mesos module, or to Mesos
> >>> itself using some sort of API or a new task state next to STAGING like
> >>> DOWNLOADING including detailed progress information.
> >>>
> >>> We want to contribute this work back to the project and like to know
> >>> which of the above and other options are the most viable.
> >>>
> >>> Cheers,
> >>>
> >>> Frank
> >>
> >>
>



-- 
Cheers,

Zhitao Li

Re: cron-like scheduling in mesos framework?

[l vic]

OK, got it, thanks.
That makes it useless for data backup though, if chronos starts new
container on the node where it does not have an access to db data

On Tue, Jan 10, 2017 at 11:14 AM, Stephen Gran 
wrote:

> Hi,
>
> This is what I'm trying to say, perhaps not very clearly:
> the task 'ls /' will be run in a 4th container.  It is a new process.
>
> Cheers,
>
> On 10/01/17 15:54, l vic wrote:
> > Great, but I have several servers: 3 different containers running on 3
> > different nodes. If i try to schedule the job :
> > curl -L -H 'Content-Type: application/json' -X POST
> > http://|chronos-node:|2002/scheduler/iso8601
> >  -d '
> > {
> >   "schedule": "R10/2015-06-05T13:36:00Z/PT1M",
> >   "name": "test-container-job",
> >  "container": {
> >   "type": "DOCKER",
> >   "image": "",
> >   "network": "BRIDGE"
> >   },
> >   "cpus": "0.1",
> >   "mem": "512",
> >  "uris" : ["file:///.dockercfg"],
> >   "command": "ls /"
> > }'
> > Will it schedule the job on all 3 nodes, or only the first one? Can I
> > actually choose which one in my json?
> > Thks
> >
> > On Tue, Jan 10, 2017 at 10:47 AM, Stephen Gran  > > wrote:
> >
> > Hi,
> >
> > Chronos launches a new task to perform the job.  Think of it more
> like
> > cron launching a new process when this is run on a single server.
> >
> > Cheers,
> >
> > On 10/01/17 15:31, l vic wrote:
> > > If I have cluster of 3 running containers - how Chronos will
> select one
> > > to schedule the job?
> > >
> > > On Fri, Jan 6, 2017 at 1:33 PM, Radek Gruchalski <
> ra...@gruchalski.com 
> > > >>
> wrote:
> > >
> > > Chronos launches a task on Mesos whenever a scheduled event is
> bound
> > > to happen.
> > > You can run a container task with Chronos, n problem:
> > >
> > > https://mesos.github.io/chronos/docs/api.html#job-
> configuration
> > 
> > >  configuration
> > >
> > >
> > > Check the container field.
> > >
> > > –
> > > Best regards,
> > > Radek Gruchalski
> > > ra...@gruchalski.com 
> > >
> > >
> > >
> > > On January 6, 2017 at 7:17:36 PM, l vic (lvic4...@gmail.com
> 
> > > >)
> wrote:
> > >
> > >> Chronos is a separate service... I have database cluster
> running
> > >> as Mesos framework. I have to  periodically schedule data
> backups
> > >> using shell scripts within running docker container. Is that
> > >> possible with Chronos?
> > >>
> > >> On Fri, Jan 6, 2017 at 11:47 AM, Radek Gruchalski
> > >> 
> > >> wrote:
> > >>
> > >> There are two options:
> > >>
> > >> - https://github.com/dcos/metronome <
> https://github.com/dcos/metronome>
> > >>  https://github.com/dcos/metronome>>
> > >> - https://github.com/mesos/chronos <
> https://github.com/mesos/chronos>
> > >>  https://github.com/mesos/chronos>>
> > >>
> > >> while Metronome, apparently, being a successor of Chronos.
> > >>
> > >> –
> > >> Best regards,
> > >> Radek Gruchalski
> > >> ra...@gruchalski.com 
> > >
> > >>
> > >>
> > >> On January 6, 2017 at 5:40:45 PM, l vic (
> lvic4...@gmail.com 
> > >> >)
> > wrote:
> > >>
> > >>> Hi,
> > >>> Is there a way to schedule mesos framework task for
> > execution
> > >>> at certain day/time?
> > >>> Thank youm
> > >>> -V
> > >>
> > >>
> > >
> >
> > --
> > Stephen Gran
> > Senior Technical Architect
> >
> > picture the possibilities | piksel.com 
> >
> >
>
> --
> Stephen Gran
> Senior Technical Architect
>
> picture the possibilities | piksel.com
>

Re: cron-like scheduling in mesos framework?

[Stephen Gran]

Hi,

This is what I'm trying to say, perhaps not very clearly:
the task 'ls /' will be run in a 4th container.  It is a new process.

Cheers,

On 10/01/17 15:54, l vic wrote:
> Great, but I have several servers: 3 different containers running on 3
> different nodes. If i try to schedule the job :
> curl -L -H 'Content-Type: application/json' -X POST
> http://|chronos-node:|2002/scheduler/iso8601
>  -d '
> {
>   "schedule": "R10/2015-06-05T13:36:00Z/PT1M",
>   "name": "test-container-job",
>  "container": {
>   "type": "DOCKER",
>   "image": "",
>   "network": "BRIDGE"
>   },
>   "cpus": "0.1",
>   "mem": "512",
>  "uris" : ["file:///.dockercfg"],
>   "command": "ls /"
> }'
> Will it schedule the job on all 3 nodes, or only the first one? Can I
> actually choose which one in my json?
> Thks
>
> On Tue, Jan 10, 2017 at 10:47 AM, Stephen Gran  > wrote:
>
> Hi,
>
> Chronos launches a new task to perform the job.  Think of it more like
> cron launching a new process when this is run on a single server.
>
> Cheers,
>
> On 10/01/17 15:31, l vic wrote:
> > If I have cluster of 3 running containers - how Chronos will select one
> > to schedule the job?
> >
> > On Fri, Jan 6, 2017 at 1:33 PM, Radek Gruchalski  
> > >> wrote:
> >
> > Chronos launches a task on Mesos whenever a scheduled event is bound
> > to happen.
> > You can run a container task with Chronos, n problem:
> >
> > https://mesos.github.io/chronos/docs/api.html#job-configuration
> 
> >  >
> >
> > Check the container field.
> >
> > –
> > Best regards,

> > Radek Gruchalski
> > ra...@gruchalski.com 
> >
> >
> >
> > On January 6, 2017 at 7:17:36 PM, l vic (lvic4...@gmail.com 
> 
> > >) wrote:
> >
> >> Chronos is a separate service... I have database cluster running
> >> as Mesos framework. I have to  periodically schedule data backups
> >> using shell scripts within running docker container. Is that
> >> possible with Chronos?
> >>
> >> On Fri, Jan 6, 2017 at 11:47 AM, Radek Gruchalski
> >> 
> >> wrote:
> >>
> >> There are two options:
> >>
> >> - https://github.com/dcos/metronome 
> 
> >>  >
> >> - https://github.com/mesos/chronos 
> 
> >>  >
> >>
> >> while Metronome, apparently, being a successor of Chronos.
> >>
> >> –
> >> Best regards,

> >> Radek Gruchalski
> >> ra...@gruchalski.com 
> >
> >>
> >>
> >> On January 6, 2017 at 5:40:45 PM, l vic (lvic4...@gmail.com 
> 
> >> >)
> wrote:
> >>
> >>> Hi,
> >>> Is there a way to schedule mesos framework task for
> execution
> >>> at certain day/time?
> >>> Thank youm
> >>> -V
> >>
> >>
> >
>
> --
> Stephen Gran
> Senior Technical Architect
>
> picture the possibilities | piksel.com 
>
>

-- 
Stephen Gran
Senior Technical Architect

picture the possibilities | piksel.com

Re: cron-like scheduling in mesos framework?

[l vic]

Great, but I have several servers: 3 different containers running on 3
different nodes. If i try to schedule the job :
curl -L -H 'Content-Type: application/json' -X POST http://chronos-node:
2002/scheduler/iso8601  -d '
{
  "schedule": "R10/2015-06-05T13:36:00Z/PT1M",
  "name": "test-container-job",
 "container": {
  "type": "DOCKER",
  "image": "",
  "network": "BRIDGE"
  },
  "cpus": "0.1",
  "mem": "512",
 "uris" : ["file:///.dockercfg"],
  "command": "ls /"
}'
Will it schedule the job on all 3 nodes, or only the first one? Can I
actually choose which one in my json?
Thks

On Tue, Jan 10, 2017 at 10:47 AM, Stephen Gran 
wrote:

> Hi,
>
> Chronos launches a new task to perform the job.  Think of it more like
> cron launching a new process when this is run on a single server.
>
> Cheers,
>
> On 10/01/17 15:31, l vic wrote:
> > If I have cluster of 3 running containers - how Chronos will select one
> > to schedule the job?
> >
> > On Fri, Jan 6, 2017 at 1:33 PM, Radek Gruchalski  > > wrote:
> >
> > Chronos launches a task on Mesos whenever a scheduled event is bound
> > to happen.
> > You can run a container task with Chronos, n problem:
> >
> > https://mesos.github.io/chronos/docs/api.html#job-configuration
> > 
> >
> > Check the container field.
> >
> > –
> > Best regards,
> > Radek Gruchalski
> > ra...@gruchalski.com 
> >
> >
> > On January 6, 2017 at 7:17:36 PM, l vic (lvic4...@gmail.com
> > ) wrote:
> >
> >> Chronos is a separate service... I have database cluster running
> >> as Mesos framework. I have to  periodically schedule data backups
> >> using shell scripts within running docker container. Is that
> >> possible with Chronos?
> >>
> >> On Fri, Jan 6, 2017 at 11:47 AM, Radek Gruchalski
> >> > wrote:
> >>
> >> There are two options:
> >>
> >> - https://github.com/dcos/metronome
> >> 
> >> - https://github.com/mesos/chronos
> >> 
> >>
> >> while Metronome, apparently, being a successor of Chronos.
> >>
> >> –
> >> Best regards,
> >> Radek Gruchalski
> >> ra...@gruchalski.com 
> >>
> >>
> >> On January 6, 2017 at 5:40:45 PM, l vic (lvic4...@gmail.com
> >> ) wrote:
> >>
> >>> Hi,
> >>> Is there a way to schedule mesos framework task for execution
> >>> at certain day/time?
> >>> Thank youm
> >>> -V
> >>
> >>
> >
>
> --
> Stephen Gran
> Senior Technical Architect
>
> picture the possibilities | piksel.com
>

Re: cron-like scheduling in mesos framework?

[Stephen Gran]

Hi,

Chronos launches a new task to perform the job.  Think of it more like 
cron launching a new process when this is run on a single server.

Cheers,

On 10/01/17 15:31, l vic wrote:
> If I have cluster of 3 running containers - how Chronos will select one
> to schedule the job?
>
> On Fri, Jan 6, 2017 at 1:33 PM, Radek Gruchalski  > wrote:
>
> Chronos launches a task on Mesos whenever a scheduled event is bound
> to happen.
> You can run a container task with Chronos, n problem:
>
> https://mesos.github.io/chronos/docs/api.html#job-configuration
> 
>
> Check the container field.
>
> –
> Best regards,

> Radek Gruchalski
> 
ra...@gruchalski.com 
>
>
> On January 6, 2017 at 7:17:36 PM, l vic (lvic4...@gmail.com
> ) wrote:
>
>> Chronos is a separate service... I have database cluster running
>> as Mesos framework. I have to  periodically schedule data backups
>> using shell scripts within running docker container. Is that
>> possible with Chronos?
>>
>> On Fri, Jan 6, 2017 at 11:47 AM, Radek Gruchalski
>> > wrote:
>>
>> There are two options:
>>
>> - https://github.com/dcos/metronome
>> 
>> - https://github.com/mesos/chronos
>> 
>>
>> while Metronome, apparently, being a successor of Chronos.
>>
>> –
>> Best regards,

>> Radek Gruchalski
>> 
ra...@gruchalski.com 
>>
>>
>> On January 6, 2017 at 5:40:45 PM, l vic (lvic4...@gmail.com
>> ) wrote:
>>
>>> Hi,
>>> Is there a way to schedule mesos framework task for execution
>>> at certain day/time?
>>> Thank youm
>>> -V
>>
>>
>

-- 
Stephen Gran
Senior Technical Architect

picture the possibilities | piksel.com

Re: cron-like scheduling in mesos framework?

[l vic]

If I have cluster of 3 running containers - how Chronos will select one to
schedule the job?

On Fri, Jan 6, 2017 at 1:33 PM, Radek Gruchalski 
wrote:

> Chronos launches a task on Mesos whenever a scheduled event is bound to
> happen.
> You can run a container task with Chronos, n problem:
>
> https://mesos.github.io/chronos/docs/api.html#job-configuration
>
> Check the container field.
>
> –
> Best regards,
> Radek Gruchalski
> ra...@gruchalski.com
>
>
> On January 6, 2017 at 7:17:36 PM, l vic (lvic4...@gmail.com) wrote:
>
> Chronos is a separate service... I have database cluster running as Mesos
> framework. I have to  periodically schedule data backups using shell
> scripts within running docker container. Is that possible with Chronos?
>
> On Fri, Jan 6, 2017 at 11:47 AM, Radek Gruchalski 
> wrote:
>
>> There are two options:
>>
>> - https://github.com/dcos/metronome
>> - https://github.com/mesos/chronos
>>
>> while Metronome, apparently, being a successor of Chronos.
>>
>> –
>> Best regards,
>> Radek Gruchalski
>> ra...@gruchalski.com
>>
>>
>> On January 6, 2017 at 5:40:45 PM, l vic (lvic4...@gmail.com) wrote:
>>
>> Hi,
>> Is there a way to schedule mesos framework task for execution at certain
>> day/time?
>> Thank youm
>> -V
>>
>>
>

Re: 答复: Optimize libprocess performance

[tommy xiao]

Looking forward to seeing the benchmarks.

2017-01-06 5:52 GMT+08:00 Benjamin Mahler :

> Ok great, thanks for sharing. Looking forward to seeing the benchmarks.
>
> On Wed, Jan 4, 2017 at 10:31 PM, pangbingqiang 
> wrote:
>
> > We write a light k-v database ,use for metadata store and nameservice
> like
> > etcd, but we test its TPS just 1200+(one client), the network is not the
> > bottleneck, so the RPC layer is too heavy.
> >
> > -邮件原件-
> > 发件人: Benjamin Mahler [mailto:bmah...@apache.org]
> > 发送时间: 2017年1月5日 9:26
> > 收件人: dev
> > 抄送: user@mesos.apache.org
> > 主题: Re: Optimize libprocess performance
> >
> > Which areas does the performance not meet your needs? There are a lot of
> > aspects to libprocess that can be optimized, so it would be good to focus
> > on each of your particular use cases via benchmarks, this allows us to
> have
> > a shared way to profile and measure improvements.
> >
> > Copy elimination is one area where a lot of improvement can be made
> across
> > libprocess, note that libprocess was implemented before we had C++11 move
> > support available. We've recently made some improvements to update the
> HTTP
> > serving path towards zero-copies but it's not completely done. Can you
> > submit patches for the ProcessBase::send() path copy elimination? We can
> > have a move overload for ProcessBase::send and have
> ProtobufProcess::send()
> > and encode() perform moves instead of a copy.
> >
> > With respect to the MessageEncoder, since it's less trivial, you can
> > submit a benchmark that captures the use case you care about and we can
> > drive improvements using it. I have some suggestions here as well but we
> > can discuss once we have the benchmarks committed.
> >
> > How does that sound to start?
> >
> > On Tue, Jan 3, 2017 at 7:31 PM, pangbingqiang 
> > wrote:
> >
> > > Hi All:
> > >
> > >   We use libprocess as our underlying communication library, but we
> > > find it’s performance don’t meet, we want to optimize it, for example:
> > >
> > > *  ‘send’ function *implementation one metadata has four times memory
> > > copy,
> > >
> > > *1. ProtobufMessage SerializeToString then processbase ‘encode’
> > > construct string once;*
> > >
> > > *2. In ‘encode’ function Message body copy again;*
> > >
> > > *3. In MessageEncoder in order to construct HTTP Request, copy again;*
> > >
> > > *4.   **MessageEncoder return copy again;*
> > >
> > >   How to optimize this scenario may be useful.
> > >
> > >   Also , in libprocess it has so many lock:
> > >
> > > *1.   **SocketManager:   std::recursive_mutex mutex;*
> > >
> > > *2.   **ProcessManager:  std::recursive_mutex processes_mutex;*
> > *std::recursive_mutex
> > > runq_mutex; std::recursive_mutex firewall_mutex;*
> > >
> > > In particular, everytime event enqueue/dequeue both need to get lock,
> > > maybe use lookfree struct is better.
> > >
> > >
> > >
> > > If have any optimize suggestion or discussion, please let me know,
> > thanks.
> > >
> > >
> > >
> > > [image: cid:image001.png@01D0E8C5.8D08F440]
> > >
> > >
> > >
> > > Bingqiang Pang(庞兵强)
> > >
> > >
> > >
> > > Distributed and Parallel Software Lab
> > >
> > > Huawei Technologies Co., Ltd.
> > >
> > > Email:pangbingqi...@huawei.com 
> > >
> > >
> > >
> > >
> > >
> >
>



-- 
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com

Re: Providing end-user feedback on Docker image download progress

[Frank Scholten]

Here is a very rudimentary design doc on image download progress:
https://docs.google.com/document/d/1x9dtcNgwecAp1xAHeDH-FJOJRIWC3OfC2akX8LLWNcs/edit?usp=sharing

Feel free to comment.

Cheers,

Frank



On Tue, Jan 10, 2017 at 8:49 AM, Frank Scholten  wrote:
> Hi Jie,
>
> Great!
>
> What are the next steps here? Create a design document?
>
> Cheers,
>
> Frank
>
>
> On Mon, Jan 9, 2017 at 8:38 PM, Jie Yu  wrote:
>> Frank,
>>
>> Thanks for reaching out! I think this is definitely something we've thought
>> about, just don't have the cycle to get it prioritized.
>> https://issues.apache.org/jira/browse/MESOS-2256
>>
>> The idea is around re-using STAGING state with more information about the
>> progress of the provisioning (and fetching). cc Vinod, BenM, BenH
>>
>>> We want to contribute this work back to the project and like to know
>>>
>>> which of the above and other options are the most viable.
>>
>>
>> That's great!
>>
>> - Jie
>>
>> On Mon, Jan 9, 2017 at 3:07 AM, Frank Scholten 
>> wrote:
>>>
>>> Hi,
>>>
>>> Together with a client we are looking into ways to provide end-user
>>> feedback on Docker image downloads for Mesos tasks.
>>>
>>> The idea is that an end user who uses a cli tool that connects to a
>>> Mesos scheduler gets image dowload and provisioning progress on it's
>>> standard out stream every few seconds.
>>>
>>> Our client has a Mesos framework that runs large Docker images via the
>>> Mesos Containerizer and we like to know what our options are of adding
>>> this feature either to the framework, to a Mesos module, or to Mesos
>>> itself using some sort of API or a new task state next to STAGING like
>>> DOWNLOADING including detailed progress information.
>>>
>>> We want to contribute this work back to the project and like to know
>>> which of the above and other options are the most viable.
>>>
>>> Cheers,
>>>
>>> Frank
>>
>>