Re: scp with mesos fetcher

That's so cool, otherwise I am going to contribute it. Cheers guys On Wed, Jul 6, 2016 at 10:47 AM, Jie Yu wrote: > Bryan, not yet. But we are trying to make fetcher pluggable so that we can > easily extend it to support scp. You can monitor the progress on this epic: >

Re: scp with mesos fetcher

Bryan, not yet. But we are trying to make fetcher pluggable so that we can easily extend it to support scp. You can monitor the progress on this epic: https://issues.apache.org/jira/browse/MESOS-3918 - Jie On Tue, Jul 5, 2016 at 7:42 PM, haosdent wrote: > Hi, @Bryan. So far

Re: scp with mesos fetcher

Hi, @Bryan. So far mesos fetcher only support * ftps,ftp,https,http (via libcurl) * hdfs, hftp, s3, s3n (via hadoop client) On Wed, Jul 6, 2016 at 9:41 AM, Bryan Fok wrote: > Hi all > > Is it possible to do scp over the mesos fetcher? > > BR > Bryan > -- Best

Re: Return a vector value from custom executor

Understood, moving to that direction now. Thank you for your advance. BR Bryan On Wed, Jul 6, 2016 at 1:29 AM, haosdent wrote: > I afraid protobuf don't allow so large message size? I think you may store > your result in HDFS or other distributed filesystem. And executor

scp with mesos fetcher

Hi all Is it possible to do scp over the mesos fetcher? BR Bryan

Re: A Redis Framework for Apache Mesos

Thanks Avinash, It is done now. On Mon, Jul 4, 2016 at 12:01 PM, Avinash Sridharan wrote: > > > On Mon, Jul 4, 2016 at 12:11 AM, DhilipKumar Sankaranarayanan < > s.dhilipku...@gmail.com> wrote: > >> *"* >> *Would be nice to get on the frameworks page:* >> >>

Allocator slack channel

Hi, I created an #allocator slack channel in mesos.slack.com, please join this if you want to have some discussion for allocator related issues, such as allocator performance, optimistic offer, revocable resource etc. @bmahler and @vinodkone, I also posted a RR here

Re: Documentation for ACCEPT HTTP API

- the list Hi Giulio, You probably remember me, I used to work with Predrag and Jakob and we met when BenH was at CERN. Please let me know how it goes with using HTTP API. The documentation is indeed rough on edges so I am not surprised you're having questions. I'll be happy to setup a google

Re: On a whim I sync'd my local master branch

You're welcome! Kudos to Gilbert and Jie for landing this :). On Wednesday, June 29, 2016, Charles Allen wrote: > And ran the unit tests (through jenkins) > > [--] Global test environment tear-down > [==] 1209 tests from 140 test cases ran. (360352

Re: ScyllaDB

+1 Le 5 juil. 2016 17:38, "José Guilherme Vanz" a écrit : > Hi guys! > > Have you ever consider make ScyllaDB run on > Mesos? > I took a look on it, it seems very powerful and cool! It would be nice run > it on Mesos =) > > Thanks >

Re: Return a vector value from custom executor

I afraid protobuf don't allow so large message size? I think you may store your result in HDFS or other distributed filesystem. And executor just notify framework the progress of store. On Tue, Jul 5, 2016 at 2:17 PM, Bryan Fok wrote: > Hi all > > What is the best

RE: Fetcher cache: caching even more while an executor is alive

As you're writing the framework, have you looked at reserving persistent volumes? I think it might help in your use case: http://mesos.apache.org/documentation/latest/persistent-volume/ Aaron -- Aaron Carey Production Engineer - Cloud Pipeline Industrial Light & Magic London 020 3751 9150

Fetcher cache: caching even more while an executor is alive

Hi, I'm developing my own framework - that distributes >100 independent tasks across the cluster and just run them arbitrarily. My problem is, each task execution environment is a bit large tarball (2~6GB, mostly application jar files) and task itself finishes within 1~200 seconds, while tarball

Re: Mesos API - How to send argument to task?

Thanks bro, thanks for your advance. I will try it out and return feedback here On Tue, Jul 5, 2016 at 2:23 PM, Gilbert Song wrote: > Bryan, > > You can define your own arguments in your > 'TaskInfo::CommandInfo::arguments', > while having your executable as 'value'.

Return a vector value from custom executor

Hi all What is the best practice to have a custom executor to return large size (~200mb) value (e.g. matrix array, or numpy array) ? B.R Bryan

Mesos API - How to send argument to task?

Hi all I am writing a python framework which has a custom executor. Each task I submit will need to pass 2 string as arguments, each task also need 2 unique xml configuration files in the slave for the custom