Re: Failed to generate predicate Error when using dropna

2015-12-08 Thread Chang Ya-Hsuan
https://issues.apache.org/jira/browse/SPARK-12231 this is my first time to create JIRA ticket. is this ticket proper? thanks On Tue, Dec 8, 2015 at 9:59 PM, Reynold Xin wrote: > Can you create a JIRA ticket for this? Thanks. > > > On Tue, Dec 8, 2015 at 5:25 PM, Chang Ya-Hsuan wrote: > >> spar

Re: Failed to generate predicate Error when using dropna

2015-12-08 Thread Reynold Xin
Can you create a JIRA ticket for this? Thanks. On Tue, Dec 8, 2015 at 5:25 PM, Chang Ya-Hsuan wrote: > spark version: spark-1.5.2-bin-hadoop2.6 > python version: 2.7.9 > os: ubuntu 14.04 > > code to reproduce error > > # write.py > > import pyspark > sc = pyspark.SparkContext() > sqlc = pyspark

Failed to generate predicate Error when using dropna

2015-12-08 Thread Chang Ya-Hsuan
spark version: spark-1.5.2-bin-hadoop2.6 python version: 2.7.9 os: ubuntu 14.04 code to reproduce error # write.py import pyspark sc = pyspark.SparkContext() sqlc = pyspark.SQLContext(sc) df = sqlc.range(10) df1 = df.withColumn('a', df['id'] * 2) df1.write.partitionBy('id').parquet('./data') #