[
https://issues.apache.org/jira/browse/SPARK-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617267#comment-14617267
]
Evgeny SInelnikov commented on SPARK-8840:
------------------------------------------
I tested it on SparkSQL - problem not reproduced on it.
I looked into spark/R sources and found, than deserialization for _float_ type
not implemented. But not for double and it works:
{code}
> sql(hiveContext, "CREATE TABLE float_table (fl float, db double) row format
> delimited fields terminated by ','")
> result <- sql(hiveContext, "SELECT * from float_table")
> head(result)
Error in readTypedObject(con, type) :
Unsupported type for deserialization
> result <- sql(hiveContext, "LOAD DATA INPATH 'data.csv' INTO TABLE
> float_table")
> result <- sql(hiveContext, "SELECT * from float_table")
> head(result)
Error in as.data.frame.default(x[[i]], optional = TRUE) :
cannot coerce class ""jobj"" to a data.frame
> result <- sql(hiveContext, "SELECT db from float_table")
> head(result)
db
1 1.1
2 2.0
{code}
> Float type coercion with hiveContext
> ------------------------------------
>
> Key: SPARK-8840
> URL: https://issues.apache.org/jira/browse/SPARK-8840
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 1.4.0
> Reporter: Evgeny SInelnikov
>
> Problem with +float+ type coercion on SparkR with hiveContext.
> {code}
> > result <- sql(hiveContext, "SELECT offset, percentage from data limit 100")
> > show(result)
> DataFrame[offset:float, percentage:float]
> > head(result)
> Error in as.data.frame.default(x[[i]], optional = TRUE) :
> cannot coerce class ""jobj"" to a data.frame
> {code}
> This trouble looks like already exists (SPARK-2863 - Emulate Hive type
> coercion in native reimplementations of Hive functions) with same
> reason - not completed "native reimplementations of Hive..." not
> "...functions" only.
> I used spark 1.4.0 binaries from official site:
> http://spark.apache.org/downloads.html
> And running it on:
> * Hortonworks HDP 2.2.0.0-2041
> * with Hive 0.14
> * with disabled hooks for Application Timeline Servers (ATSHook) in
> hive-site.xml, commented:
> ** hive.exec.failure.hooks,
> ** hive.exec.post.hooks,
> ** hive.exec.pre.hooks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
