Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19871#discussion_r154707557
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -363,6 +363,14 @@ object SQLConf {
.checkValues(Set("none", "uncompressed", "snappy", "zlib", "lzo"))
.createWithDefault("snappy")
+ val ORC_USE_NEW_VERSION = buildConf("spark.sql.orc.useNewVersion")
+ .doc("When true, use new OrcFileFormat in sql/core module instead of
the one in sql/hive. " +
+ "Since new OrcFileFormat uses Apache ORC library instead of ORC
library Hive 1.2.1, it is " +
+ "more stable and faster.")
--- End diff --
Thank you for review, @HyukjinKwon .
Do you mean `Apache ORC library is more stable, but new OrcFileFormat is
not` because it's introduced newly?
Actually, that's true in the Spark's viewpoint, but new OrcFileFormat
contains more bug fixes and new features too. If you allow, I want to keep
this. :)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
