Can you give us a bit more information ?
how you packaged the code into jar
command you used for execution
version of Spark
related log snippet
Thanks
On Mon, Jun 6, 2016 at 10:43 AM, Daniel Haviv <
daniel.ha...@veracity-group.com> wrote:
> Hi,
> I'm wrapped the following code into a jar:
>
> v
Hi,
I'm wrapped the following code into a jar:
val test = sc.parallelize(Seq(("daniel", "a"), ("daniel", "b"), ("test", "1)")))
val agg = test.groupByKey()
agg.collect.foreach(r=>{println(r._1)})
The result of groupByKey is an empty RDD, when I'm trying the same
code using the spark-shell it's
like this:
emptyRDD.mapPartitionsWithIndex(...), but it doesn't work. So it seems it is
not suitable to use emptyRDD to start tasks on the worker side.It is so
appreciated if you can give me some suggestions.Thanks.
It looks like an Scala issue. Seems like the implicit conversion to
ArrayOps does not apply if the type is Array[Nothing].
Try giving a type to the empty RDD:
val emptyRdd: RDD[Any] = sc.EmptyRDD
emptyRdd.collect.foreach(println) // prints a line return
-kr, Gerard.
On Fri, Nov 14, 2014 at
If I remember correctly, EmptyRDD is private [spark]
You can create an empty RDD using the spark context:
val emptyRdd = sc.emptyRDD
-kr, Gerard.
On Fri, Nov 14, 2014 at 11:22 AM, Deep Pradhan
wrote:
> To get an empty RDD, I did this:
>
> I have an rdd with one element. I create
To get an empty RDD, I did this:
I have an rdd with one element. I created another rdd using filter so that
the second rdd does not contain anything. I achieved what I wanted but I
want to know whether there is an efficient way to achieve this. This is a
very crude way of creating an empty RDD. Is
See
http://spark.apache.org/docs/0.8.1/api/core/org/apache/spark/rdd/EmptyRDD.html
On Nov 14, 2014, at 2:09 AM, Deep Pradhan wrote:
> How to create an empty RDD in Spark?
>
> Thank You
How to create an empty RDD in Spark?
Thank You
