Re: Spark SQL and confused about number of partitions/tasks to do a simple join.

and confused about number of partitions/tasks to do a simple join. I have a SchemaRDD with 100 records in 1 partition.  We'll call this baseline. I have a SchemaRDD with 11 records in 1 partition.  We'll call this daily. After a fairly basic join of these two tables JavaSchemaR

Re: Spark SQL and confused about number of partitions/tasks to do a simple join.

2014 1:55 PM Subject: Spark SQL and confused about number of partitions/tasks to do a simple join. I have a SchemaRDD with 100 records in 1 partition.  We'll call this baseline. I have a SchemaRDD with 11 records in 1 partition.  We'll call this daily. After a fairly basic join

Spark SQL and confused about number of partitions/tasks to do a simple join.

I have a SchemaRDD with 100 records in 1 partition.  We'll call this baseline. I have a SchemaRDD with 11 records in 1 partition.  We'll call this daily. After a fairly basic join of these two tables JavaSchemaRDD results = sqlContext.sql("SELECT id, action, daily.epoch, daily.version FROM baselin