Hi all,
Workers will exchange data in between, right?
What classes are in charge of these actions?
-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org
Hi Guys,
I am trying something very basic. I am using GraphX to load a graph from an
edge list file which is like this:
*220 224*
*400 401*
*220 221*
So it has following nodes (just for the sake of understanding - bear with
me for drawing):
*220 = 224 400 = 401 *
* ||*
* v 221*
Well - resolved.
The problem was in my understanding. It returns the graph with vertex
data set to the connected components.
Thanks.
On Wed, Oct 22, 2014 at 6:56 PM, Manoj Awasthi awasthi.ma...@gmail.com
wrote:
Hi Guys,
I am trying something very basic. I am using GraphX to load a graph
Hi Sean,
I've pushed a PR for this https://github.com/apache/spark/pull/2893 :)
Cheers,
Holden :)
On Tue, Oct 21, 2014 at 4:41 AM, Sean Owen so...@cloudera.com wrote:
This one can be resolved, I think, with a bit of help from someone who
understands SBT + plugin config:
Since this task involves making changes to some of core functionality, I
figured it's best if I share my intents for completing this task.
This change is a little more involved as it requires modifying the Catalog
trait.
My current plan is to add an abstract method to the Catalog trait and have
The best documentation about communication interfaces is the
SecurityManager doc written by Tom Graves. With this as a starting
point I'd recommend digging through the code for each component.
Hi Spark devs/users,
One of the things we are investigating here at Netflix is if Spark would
suit us for our ETL needs, and one of requirements is multi tenancy.
I did read the official doc
http://spark.apache.org/docs/latest/job-scheduling.html and the book, but
I'm still not clear on certain
Hi Ashwin,
Let me try to answer to the best of my knowledge.
On Wed, Oct 22, 2014 at 11:47 AM, Ashwin Shankar
ashwinshanka...@gmail.com wrote:
Here are my questions :
1. Sharing spark context : How exactly multiple users can share the cluster
using same spark
context ?
That's not
Thanks Marcelo, that was helpful ! I had some follow up questions :
That's not something you might want to do usually. In general, a
SparkContext maps to a user application
My question was basically this. In this
http://spark.apache.org/docs/latest/job-scheduling.html page in the
official doc,
We want to run multiple instances of spark sql cli on our yarn cluster.
Each instance of the cli is to be used by a different user. This would be
non-optimal if each user brings up a different cli given how spark works on
yarn by running executor processes (and hence consuming resources) on
worker
We want to run multiple instances of spark sql cli on our yarn cluster.
Each instance of the cli is to be used by a different user. This looks
non-optimal if each user brings up a different cli given how spark works on
yarn by running executor processes (and hence consuming resources) on
worker
On Wed, Oct 22, 2014 at 2:17 PM, Ashwin Shankar
ashwinshanka...@gmail.com wrote:
That's not something you might want to do usually. In general, a
SparkContext maps to a user application
My question was basically this. In this page in the official doc, under
Scheduling within an application
Hi,
If developing in python, what is preffered way to do unit testing?
Do I use pyunit framework or I need to go with scalaTest?
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/Development-testing-code-tp8911.html
Sent from the Apache Spark
Hi,
Many tests in pyspark are implemented as doctests and the python
unittesting framework is also used for additional tests.
Cheers,
Holden :)
On Wed, Oct 22, 2014 at 4:13 PM, catchmonster skacan...@gmail.com wrote:
Hi,
If developing in python, what is preffered way to do unit testing?
Do
14 matches
Mail list logo
