Seems like a bug.
Suggest filing an issue with code snippet if this can be reproduced on 1.6
branch.
Cheers
On Fri, Feb 12, 2016 at 4:25 AM, Zsolt Tóth
wrote:
> Sure. I ran the same job with fewer columns, the exception:
>
> java.lang.IllegalArgumentException: requirement failed: DataFrame mus
Sure. I ran the same job with fewer columns, the exception:
java.lang.IllegalArgumentException: requirement failed: DataFrame must
have the same schema as the relation to which is inserted.
DataFrame schema: StructType(StructField(pixel0,ByteType,true),
StructField(pixel1,ByteType,true), StructFie
Can you pastebin the full error with all column types ?
There should be a difference between some column(s).
Cheers
> On Feb 11, 2016, at 2:12 AM, Zsolt Tóth wrote:
>
> Hi,
>
> I'd like to append a column of a dataframe to another DF (using Spark 1.5.2):
>
> DataFrame outputDF = unlabelledDF
Hi,
thanks for the answers. If joining the DataFrames is the solution, then why
does the simple withColumn() succeed for some datasets and fail for others?
2016-02-11 11:53 GMT+01:00 Michał Zieliński :
> I think a good idea would be to do a join:
>
> outputDF = unlabelledDF.join(predictedDF.sele
I think a good idea would be to do a join:
outputDF = unlabelledDF.join(predictedDF.select(“id”,”predicted”),”id”)
On 11 February 2016 at 10:12, Zsolt Tóth wrote:
> Hi,
>
> I'd like to append a column of a dataframe to another DF (using Spark
> 1.5.2):
>
> DataFrame outputDF = unlabelledDF.with
Hi,
I'd like to append a column of a dataframe to another DF (using Spark
1.5.2):
DataFrame outputDF = unlabelledDF.withColumn("predicted_label",
predictedDF.col("predicted"));
I get the following exception:
java.lang.IllegalArgumentException: requirement failed: DataFrame must have
the same sc