[ https://issues.apache.org/jira/browse/SPARK-11374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephane Maarek updated SPARK-11374: ------------------------------------ Comment: was deleted (was: I may add that more metadata isn't processed, namely TBLPROPERTIES ('serialization.null.format'='') Also, another issue (may still be related to Spark not reading Hive Metadata or not properly using Hive), but if you create a csv with the following (spaces intended) 1, 2,3 4, 5,6 use Hive as this: CREATE EXTERNAL TABLE `my_table`( `c1` DECIMAL, `c2` DECIMAL, `c3` DECIMAL) ... etc select * from my_table will return in Hive 1,2,3 4,5,6 But using a hive context, in Spark 1,null,3 4,null,6) > skip.header.line.count is ignored in HiveContext > ------------------------------------------------ > > Key: SPARK-11374 > URL: https://issues.apache.org/jira/browse/SPARK-11374 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.5.1 > Reporter: Daniel Haviv > > csv table in Hive which is configured to skip the header row using > TBLPROPERTIES("skip.header.line.count"="1"). > When querying from Hive the header row is not included in the data, but when > running the same query via HiveContext I get the header row. > "show create table " via the HiveContext confirms that it is aware of the > setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org