[
https://issues.apache.org/jira/browse/SPARK-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370028#comment-15370028
]
Dongjoon Hyun commented on SPARK-16467:
---------------------------------------
Hi, you missed `"` in the last command. :)
{code}
> collect(describe(sdfIris, "Species"))
summary Species
1 count 150
2 mean <NA>
3 stddev <NA>
4 min setosa
5 max virginica
{code}
> After importing R data.frame, although DataFrame columns show . replaced by
> _, the describe() function gives warnings on . in the name
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-16467
> URL: https://issues.apache.org/jira/browse/SPARK-16467
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 1.6.1
> Reporter: Neil Dewar
> Priority: Minor
>
> When importing an R data.frame into a Spark DataFrame using
> createDataFrame(), if the R data.frame column names contain period characters
> "." they are converted to underscores "_". Most SparkR functions then
> display the column names with the underscores.
> If the describe() function is used, with the option for a specific function
> name, warning messages are displayed about the use of period characters.
> Example:
> sdfIris <- createDataFrame(sqlContext, iris)
> str(sdfIris) # works fine
> collect(describe(sdfIris, Species))
> the last function above throws a string of warnings such as:
> 1: In FUN(X[[i]], ...) : Use Sepal_Length instead of Sepal.Length as column
> name
> Note, the describe() function appears to work ok if no column name is
> specified but fails if a column name is specified, even if the specified name
> did not contain a period.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
