[
https://issues.apache.org/jira/browse/SPARK-11781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sun Rui updated SPARK-11781:
----------------------------
Description:
{code}
> bytes <- as.raw(c(1, 2 ,3))
> bytes
[1] 01 02 03
> df <- createDataFrame(sqlContext, list(list(bytes)))
> printSchema(df)
root
|-- _1: array (nullable = true)
| |-- element: binary (containsNull = true)
> SparkR:::infer_type(bytes)
[1] "array<binary>"
{code}
The schema is incorrect. It should be "binary" instead of "array<binary>".
Also there is problem when collecting a DataFrame of which any column is of
binary type.
was:
```
> bytes <- as.raw(c(1, 2 ,3))
> bytes
[1] 01 02 03
> df <- createDataFrame(sqlContext, list(list(bytes)))
> printSchema(df)
root
|-- _1: array (nullable = true)
| |-- element: binary (containsNull = true)
> SparkR:::infer_type(bytes)
[1] "array<binary>"
```
The schema is incorrect. It should be "binary" instead of "array<binary>".
Also there is problem when collecting a DataFrame of which any column is of
binary type.
> SparkR has problem in inferring type of raw type
> ------------------------------------------------
>
> Key: SPARK-11781
> URL: https://issues.apache.org/jira/browse/SPARK-11781
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 1.5.1
> Reporter: Sun Rui
>
> {code}
> > bytes <- as.raw(c(1, 2 ,3))
> > bytes
> [1] 01 02 03
> > df <- createDataFrame(sqlContext, list(list(bytes)))
> > printSchema(df)
> root
> |-- _1: array (nullable = true)
> | |-- element: binary (containsNull = true)
> > SparkR:::infer_type(bytes)
> [1] "array<binary>"
> {code}
> The schema is incorrect. It should be "binary" instead of "array<binary>".
> Also there is problem when collecting a DataFrame of which any column is of
> binary type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
