[
https://issues.apache.org/jira/browse/SPARK-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565891#comment-14565891
]
Will Chen commented on SPARK-7967:
----------------------------------
BTW, how could I give the new column a default value?
I checked the source code for withColumn:
def withColumn(colName: String, col: Column): DataFrame = select(Column("*"),
col.as(colName))
It seems like this method just add a new column without any default value.Can
anyone help me? For now, I just use RDD's map and join
> cannot resolve 'count' given input columns when using DataFrame.withColumn
> --------------------------------------------------------------------------
>
> Key: SPARK-7967
> URL: https://issues.apache.org/jira/browse/SPARK-7967
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.3.0
> Environment: spark 1.3.0 standalone
> Reporter: Will Chen
> Labels: dataFrame, sparksql
>
> Code:
> val userDF = app_user_register_log.filter($"add_time" >
> startDay).filter($"add_time" < endDay)
> .select("id").as("userReg")
> .join(activeDF.as("ad"), $"userReg.id" === $"ad.uid")
> .select("ad.uid","ad.clientVerion","ad.loc","ad.auth_status"
> ,"ad.channel","ad.bd_area","ad.mobile_area","ad.idcard_area")
> .withColumn("count", $"count") // Exception came from this line
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
