Hi Sean Owen, Thank you for your attention.
I know spark.hadoop.validateOutputSpecs.
I restart the job, the application id is application_1428664056212_0017 and it
recovery from checkpoint, and it will write eventLog into
application_1428664056212_0016 dir, I think it shoud write to
applica
This is why spark.hadoop.validateOutputSpecs exists, really:
https://spark.apache.org/docs/latest/configuration.html
On Mon, Apr 20, 2015 at 3:40 AM, wyphao.2007 wrote:
> Hi,
>When I recovery from checkpoint in yarn-cluster mode using Spark
> Streaming, I found it will reuse the application
Hi,
When I recovery from checkpoint in yarn-cluster mode using Spark Streaming,
I found it will reuse the application id (In my case is
application_1428664056212_0016) before falied to write spark eventLog, But now
my application id is application_1428664056212_0017,then spark write eventLog
Hi Cesar,
Can you try 1.3.1 (
https://spark.apache.org/releases/spark-release-1-3-1.html) and see if it
still shows the error?
Thanks,
Yin
On Fri, Apr 17, 2015 at 1:58 PM, Reynold Xin wrote:
> This is strange. cc the dev list since it might be a bug.
>
>
>
> On Thu, Apr 16, 2015 at 3:18 PM, C
Definitely a bug. I just checked and it looks like we don't actually have a
function that takes a Scala RDD and Seq[String].
cc Davies who added this code a while back.
On Sun, Apr 19, 2015 at 2:56 PM, Justin Uang wrote:
> Hi,
>
> I have a question regarding SQLContext#createDataFrame(JavaRDD[
Hi,
I have a question regarding SQLContext#createDataFrame(JavaRDD[Row],
java.util.List[String]). It looks like when I try to call it, it results in
an infinite recursion that overflows the stack. I filed it here:
https://issues.apache.org/jira/browse/SPARK-6999.
What is the best way to fix this?