Pls paste code and sample CSV I m guessing it has to do with formatting time? Kr
On 8 Sep 2016 12:38 am, "Daniel Lopes" <dan...@onematch.com.br> wrote: > Hi, > > I'm* importing a few CSV*s with spark-csv package, > Always when I give a select at each one looks ok > But when i join then with sqlContext.sql give me this error > > all tables has fields timestamp > > joins are not with this dates > > > *Py4JJavaError: An error occurred while calling o643.showString.* > : org.apache.spark.SparkException: Job aborted due to stage failure: Task > 54 in stage 92.0 failed 10 times, most recent failure: Lost task 54.9 in > stage 92.0 (TID 6356, yp-spark-dal09-env5-0036): > org.apache.spark.api.python.PythonException: Traceback (most recent call > last): > File "/usr/local/src/spark160master/spark-1.6.0- > bin-2.6.0/python/lib/pyspark.zip/pyspark/worker.py", line 111, in main > process() > File "/usr/local/src/spark160master/spark-1.6.0- > bin-2.6.0/python/lib/pyspark.zip/pyspark/worker.py", line 106, in process > serializer.dump_stream(func(split_index, iterator), outfile) > File "/usr/local/src/spark160master/spark-1.6.0- > bin-2.6.0/python/lib/pyspark.zip/pyspark/serializers.py", line 263, in > dump_stream > vs = list(itertools.islice(iterator, batch)) > File "/usr/local/src/spark160master/spark/python/pyspark/sql/functions.py", > line 1563, in <lambda> > func = lambda _, it: map(lambda x: returnType.toInternal(f(*x)), it) > File "/usr/local/src/spark160master/spark-1.6.0- > bin-2.6.0/python/lib/pyspark.zip/pyspark/sql/types.py", line 191, in > toInternal > else time.mktime(dt.timetuple())) > *ValueError: year out of range * > > Any one knows this problem? > > Best, > > *Daniel Lopes* > Chief Data and Analytics Officer | OneMatch > c: +55 (18) 99764-2733 | https://www.linkedin.com/in/dslopes > > www.onematch.com.br > <http://www.onematch.com.br/?utm_source=EmailSignature&utm_term=daniel-lopes> >
