Hey Evan, Sorry about the issue with the Mesos urls. I submitted the patch that broke it originally -- at the time, I did not know about the zk:// urls mapping to Mesos. I went ahead and started PR #217<https://github.com/apache/incubator-spark/pull/217> to correct this, which also includes a unit test for creating the schedulers to hopefully avoid this type of issue in the future (at least at this most basic level).
I cannot speak for #2, though, which seems to be the more serious issue. On Wed, Nov 27, 2013 at 10:01 PM, Evan Chan <e...@ooyala.com> wrote: > 1) zk:// URIs can no longer be passed to SparkContext as the first > argument, as the MESOS_REGEX was changed to take only mesos:// URIs. > This essentially renders Spark incompatible with real production Mesos > setups. > > 2) Spinning up tasks on a Mesos cluster, all of them fail right away, > with no log message visible. This might possibly be me, but this was > working with 0.8.0. > > I can submit a pull request for #1, but a better fix would be to have > some kind of MesosSuite so that any kind of breakages can be caught > systematically. Maybe some kind of VM, but ideas are welcome. What > do you guys think? > > (I've cc'ed two folks from Mesosphere, esp on setting up a test suite) > > -Evan > > -- > -- > Evan Chan > Staff Engineer > e...@ooyala.com | >
