AngersZhuuuu edited a comment on pull request #30850: URL: https://github.com/apache/spark/pull/30850#issuecomment-748436853
`orc/rcfile/sequencefile` is ok to write data. 1. hive version < 2.3, write data into dir with all this serde will have column names delimiter problem. 2. hive version > 2.3, parquet serde and avro serde have schema issue with column name contains `,` 3. hive version > 2.3, orc, rcfile, textfile, sequencefile can write data correcttly 4. we need to keep data metadata when use parqeut/orc etc.., but hive version< 2.3, serde just split column name use `,`, there maybe some conflicts. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
