Hi Gino, It does not solve the problem of running a Spark job (on Yarn) remotely from a Jupyter notebook which is running on say in a laptop/some machine.
The issue is in yarn-client mode the laptop needs to get access to all the slave nodes where the executors would be running. In a typical security scenario of an organization the slave nodes are behind firewall and cannot be accessed from any random machine outside. Regards, Sourav On Wed, May 4, 2016 at 11:47 AM, Gino Bustelo <[email protected]> wrote: > Wiki has been updated for May 2016 > > On Fri, Apr 29, 2016 at 11:32 AM, Gino Bustelo <[email protected]> wrote: > > > My bad... copy paste error. @Luciano, I agree... my intention was to > leave > > that section the same > > > > 1. Resolve LGPL dependency: This is in progress, and hopefully we will > have the dependency license changed to a license that is compatible > with AL2. 2. Make a release: nearing completion 3. Grow a diverse > community: We should put some emphasis on growing the community and > making it diverse (the rule is at least three independent contributors) > > > > > > On Fri, Apr 29, 2016 at 9:45 AM, Luciano Resende <[email protected]> > > wrote: > > > >> I still think the important issues toward graduation are similar to what > >> was reported in the previous report: > >> > >> Three most important issues to address in the move towards graduation: > >> 1. Resolve LGPL dependency: This is in progress, and hopefully we > >> will have the dependency license changed to a license that is > >> compatible with AL2. 2. Make a release: nearing completion 3. > >> Grow a diverse community: We should put some emphasis on growing the > >> community and making it diverse (the rule is at least three > >> independent contributors) > >> > >> > >> > >> On Fri, Apr 29, 2016 at 6:44 AM, Gino Bustelo <[email protected]> > wrote: > >> > >> > Last month can be found http://wiki.apache.org/incubator/April2016. > >> > Here is the draft for this month. Please give feedback... > >> > > >> > -------------------- > >> > Toree > >> > > >> > Toree provides applications with a mechanism to interactively and > >> remotely > >> > access Apache Spark. It enables interactive workloads between > >> applications > >> > and a Spark cluster. As a Jupyter Notebook extension, it provides the > >> user > >> > with a preconfigured environment for interacting with Spark using > Scala, > >> > Python, R or SQL. > >> > > >> > Toree has been incubating since 2015-12-02. > >> > > >> > Three most important issues to address in the move towards graduation: > >> > > >> > 1. Removal of LGPL dependency > >> > 2. Ease of installation > >> > 3. Documentation of APIs > >> > > >> > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be > >> > aware of? > >> > > >> > TOREE-262 - Progress on removal of LGPL dependency > >> > > >> > How has the community developed since the last report? > >> > > >> > 1. Active communication in mailing list and gitter with early > adopters > >> > 2. Still working on transitioning users from Spark Kernel project > into > >> > Toree. > >> > 3. More external contributions being made > >> > > >> > How has the project developed since the last report? > >> > > >> > 1. Contrinue working with JeroMQ community to further their > transition > >> > into > >> > MPL v2 and away from LGPL. 3 committers to go. > >> > 2. Substantial effort in our build system to make Apache compliant > >> > releases. > >> > 3. Addressing issues opened by community > >> > > >> > Date of last release: > >> > > >> > None since incubation. > >> > > >> > When were the last committers or PMC members elected? > >> > > >> > No new additions since incubation > >> > > >> > >> > >> > >> -- > >> Luciano Resende > >> http://twitter.com/lresende1975 > >> http://lresende.blogspot.com/ > >> > > > > >
