Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1595#discussion_r155251715
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/strategy/CarbonLateDecodeStrategy.scala
---
@@ -413,19 +414,35 @@ private[sql] class CarbonLateDecodeStrategy extends
SparkStrategy {
relation: BaseRelation,
predicates: Seq[Expression]): (Seq[Expression], Seq[Filter]) = {
+ def isComplexAttribute(attribute: Attribute) = attribute.dataType
match {
+ case ArrayType(dataType, _) => true
+ case StructType(_) => true
+ case _ => false
+ }
+
+ // In case of ComplexType dataTypes no filters should be pushed down.
IsNotNull is being
+ // explicitly added by spark and pushed. That also has to be handled
and pushed back to
+ // Spark for handling.
+ val predicatesWithoutComplex = predicates.filter(predicate =>
predicate.collect {
--- End diff --
format properly.
use
```
predicates.filter{predicate =>
predicate.collect {
```
---
