Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2715#discussion_r227241517
--- Diff:
integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/dataload/TestLoadDataWithCompression.scala
---
@@ -42,6 +44,112 @@ case class Rcd(booleanField: Boolean, shortField:
Short, intField: Int, bigintFi
dateField: String, charField: String, floatField: Float,
stringDictField: String,
stringSortField: String, stringLocalDictField: String,
longStringField: String)
+/**
+ * This compressor actually will not compress or decompress anything.
+ * It is used for test case of specifying customized compressor.
+ */
+class CustomizeCompressor extends Compressor {
+ override def getName: String =
"org.apache.carbondata.integration.spark.testsuite.dataload.CustomizeCompressor"
--- End diff --
I'm refering to the implementation of 'CompressionCodec' in spark.
Due to yesterday's github problem, some content of my previous was missing.
The corresponding code in spark is here:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/io/CompressionCodec.scala#L57-L82
For a customize compressionCodec, the line#75 use the name as the class
name.
I think in your example, spark file format can handle the short name of
'carbon' because we are using CarbonSession and in this session we have
registered the short name of carbon.
---
