spark and mesos issue
We are getting this issue when we are running jobs with close to 1000 workers. Spark is from the github version and mesos is 0.19.0 ERROR storage.BlockManagerMasterActor: Got two different block manager registrations on 201407031041-1227224054-5050-24004-0 Googling about it seems that mesos is starting slaves at the same time and giving them the same id. So may bug in mesos ? Thanks, Gurvinder On 07/04/2014 01:03 AM, Vinod Kone wrote: correct url: https://issues.apache.org/jira/issues/?jql=project%20%3D%20MESOS%20AND%20%22Target%20Version%2Fs%22%20%3D%200.19.1 On Thu, Jul 3, 2014 at 1:40 PM, Vinod Kone vinodk...@gmail.com mailto:vinodk...@gmail.com wrote: Hi, We are planning to release 0.19.1 (likely next week) which will be a bug fix release. Specifically, these are the fixes that we are planning to cherry pick. https://issues.apache.org/jira/issues/?filter=12326191jql=project%20%3D%20MESOS%20AND%20%22Target%20Version%2Fs%22%20%3D%200.19.1 If there are other critical fixes that need to be backported to 0.19.1 please reply here as soon as possible. Thanks,
Re: spark and mesos issue
I've been running into the same issue with task counts greater than 600 or so using spark with mesos in fine grain mode. On Fri, Jul 4, 2014 at 5:06 AM, Gurvinder Singh gurvinder.si...@uninett.no wrote: We are getting this issue when we are running jobs with close to 1000 workers. Spark is from the github version and mesos is 0.19.0 ERROR storage.BlockManagerMasterActor: Got two different block manager registrations on 201407031041-1227224054-5050-24004-0 Googling about it seems that mesos is starting slaves at the same time and giving them the same id. So may bug in mesos ? Thanks, Gurvinder On 07/04/2014 01:03 AM, Vinod Kone wrote: correct url: https://issues.apache.org/jira/issues/?jql=project%20%3D%20MESOS%20AND%20%22Target%20Version%2Fs%22%20%3D%200.19.1 On Thu, Jul 3, 2014 at 1:40 PM, Vinod Kone vinodk...@gmail.com mailto:vinodk...@gmail.com wrote: Hi, We are planning to release 0.19.1 (likely next week) which will be a bug fix release. Specifically, these are the fixes that we are planning to cherry pick. https://issues.apache.org/jira/issues/?filter=12326191jql=project%20%3D%20MESOS%20AND%20%22Target%20Version%2Fs%22%20%3D%200.19.1 If there are other critical fixes that need to be backported to 0.19.1 please reply here as soon as possible. Thanks,
Re: 0.19.1
Hi Can you give some background as to why this is a critical fix? We try to minimise what we include in bug fix releases to avoid feature creep. Thanks On Jul 4, 2014 12:31 PM, Tom Arnfeld t...@duedil.com wrote: Any chance we can get https://issues.apache.org/jira/browse/MESOS-1448 too? On 3 Jul 2014, at 21:40, Vinod Kone vinodk...@gmail.com wrote: Hi, We are planning to release 0.19.1 (likely next week) which will be a bug fix release. Specifically, these are the fixes that we are planning to cherry pick. https://issues.apache.org/jira/issues/?filter=12326191jql=project%20%3D%20MESOS%20AND%20%22Target%20Version%2Fs%22%20%3D%200.19.1 If there are other critical fixes that need to be backported to 0.19.1 please reply here as soon as possible. Thanks,
Re: 0.19.1
Happy to. It surprised me that this wasn't supported, especially considering the fetcher is supposed to be able to download URIs from any URL using http(s). This is most useful (and in my opinion quite an important issue) for downloading executors from S3 in situations a redirect is incurred, and more specifically, github tar archives which almost always go through a 301. Don't mind if going into the next non-bugfix release if you don't agree it's that important. On 4 Jul 2014, at 20:48, Dominic Hamon dha...@twopensource.com wrote: Hi Can you give some background as to why this is a critical fix? We try to minimise what we include in bug fix releases to avoid feature creep. Thanks On Jul 4, 2014 12:31 PM, Tom Arnfeld t...@duedil.com wrote: Any chance we can get https://issues.apache.org/jira/browse/MESOS-1448 too? On 3 Jul 2014, at 21:40, Vinod Kone vinodk...@gmail.com wrote: Hi, We are planning to release 0.19.1 (likely next week) which will be a bug fix release. Specifically, these are the fixes that we are planning to cherry pick. https://issues.apache.org/jira/issues/?filter=12326191jql=project%20%3D%20MESOS%20AND%20%22Target%20Version%2Fs%22%20%3D%200.19.1 If there are other critical fixes that need to be backported to 0.19.1 please reply here as soon as possible. Thanks,
