spark and mesos issue

2014-07-04 Thread Gurvinder Singh
We are getting this issue when we are running jobs with close to 1000 workers. Spark is from the github version and mesos is 0.19.0 ERROR storage.BlockManagerMasterActor: Got two different block manager registrations on 201407031041-1227224054-5050-24004-0 Googling about it seems that mesos is

Re: spark and mesos issue

2014-07-04 Thread Ray Rodriguez
I've been running into the same issue with task counts greater than 600 or so using spark with mesos in fine grain mode. On Fri, Jul 4, 2014 at 5:06 AM, Gurvinder Singh gurvinder.si...@uninett.no wrote: We are getting this issue when we are running jobs with close to 1000 workers. Spark is

Re: 0.19.1

2014-07-04 Thread Dominic Hamon
Hi Can you give some background as to why this is a critical fix? We try to minimise what we include in bug fix releases to avoid feature creep. Thanks On Jul 4, 2014 12:31 PM, Tom Arnfeld t...@duedil.com wrote: Any chance we can get https://issues.apache.org/jira/browse/MESOS-1448 too? On

Re: 0.19.1

2014-07-04 Thread Tom Arnfeld
Happy to. It surprised me that this wasn't supported, especially considering the fetcher is supposed to be able to download URIs from any URL using http(s). This is most useful (and in my opinion quite an important issue) for downloading executors from S3 in situations a redirect is incurred,