Re: skip.header.line.count is ignored in HiveContext

2017-11-09 Thread Dongjoon Hyun
Hi, For 2.1.X, 2.1.2 was already out and I don't think 2.1.3 will be out. At that time, I made a PR for SPARK-11374 based on Spark 2.0. I believe the patch works in 2.1, too. If you need, you can try. https://github.com/apache/spark/pull/14638 At that time, it was not accepted as you know, 'Won

skip.header.line.count is ignored in HiveContext

2017-11-09 Thread sunerhan1...@sina.com
hello, I've got a table in Hive(path located to csv formatted files) which is configured to skip the header row using TBLPROPERTIES("skip.header.line.count"="1"). When querying from Hive the header row is not included in the data, but when running the same query via HiveContext I get the hea