been reported before,
>> e.g. https://issues.apache.org/jira/browse/HDFS-770 which is *very* old.
>>
>>
>>
>> If you’re pretty sure Spark couldn’t be responsible for issues at this
>> level I’ll stick to the Hadoop mailing list.
>>
>>
>>
>>
, but rare, cases have been reported before,
> e.g. https://issues.apache.org/jira/browse/HDFS-770 which is *very* old.
>
>
>
> If you’re pretty sure Spark couldn’t be responsible for issues at this
> level I’ll stick to the Hadoop mailing list.
>
>
>
> Thanks
>
>
.
Thanks
---
Joe Naegele
Grier Forensics
From: Michael Stratton [mailto:michael.strat...@komodohealth.com]
Sent: Monday, December 19, 2016 10:00 AM
To: Joseph Naegele
Cc: user
Subject: Re: [Spark SQL] Task failed while writing rows
It seems like an issue w/ Hadoop. What do you get when
It seems like an issue w/ Hadoop. What do you get when you run hdfs
dfsadmin -report?
Anecdotally(And w/o specifics as it has been a while), I've generally used
Parquet instead of ORC as I've gotten a bunch of random problems reading
and writing ORC w/ Spark... but given ORC performs a lot better
Hi all,
I'm having trouble with a relatively simple Spark SQL job. I'm using Spark
1.6.3. I have a dataset of around 500M rows (average 128 bytes per record).
It's current compressed size is around 13 GB, but my problem started when it
was much smaller, maybe 5 GB. This dataset is generated by
