Hi all,I have some data on the driver side. Then I will broadcast the data to
all workers side to ensure each worker has same data. Due to there is no RDD in
the memory, I don't know how to make workers to start tasks to do some
transformation based on the data. I have try to write code like thi
Hi kdmxen,You want to delete the broadcast variables on the executors to avoid
executors lost failure, right?Have you try to use the unpersist method? Like
this way:itemSplitBroadcast.destroy(true); =>
itemSplitBroadcast.unpersist(true);
LIN Chen
Date: Thu, 7 Jan 2016 22:01:27 +0800
Subject: H
Hi Marco,In your case, since you don't need to perform an aggregation (such as
a sum or average) over each key, using groupByKey may perform better.
groupByKey inherently utilizes compactBuffer which is much more efficient than
ArrayBuffer.
Thanks.LIN Chen
Date: Tue, 5 Jan 2016 21:13:40 +
S
of combination grows.
Thanks,
From: LinChen
Date: Saturday, 2 January 2016 14:48
To: Roberto Pagliari
Cc: "user@spark.apache.org"
Subject: RE: frequent itemsets
Hi Roberto,
What is the minimum support threshold you set?
Could you check which stage you ran into Sta
Hi Roberto,What is the minimum support threshold you set? Could you check which
stage you ran into StackOverFlow exception?
Thanks.
From: roberto.pagli...@asos.com
To: yblia...@gmail.com
CC: user@spark.apache.org
Subject: Re: frequent itemsets
Date: Sat, 2 Jan 2016 12:01:31 +
Hi Yanbo,
Screenshot1(Normal WebUI)
Screenshot2(Corrupted WebUI)
As screenshot2 shows, the format of my Spark WebUI looks strange and I cannot
click the description of active jobs. It seems there is something missing in my
opearing system. I googled it but find nothing. Could anybody help me?
---
