Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2453#discussion_r202244060
--- Diff:
datamap/mv/core/src/main/scala/org/apache/carbondata/mv/datamap/MVHelper.scala
---
@@ -147,6 +149,34 @@ object MVHelper {
}.filter(_.isDefined).map(_.get)
}
+
+ /**
+ * Check if we can do incremental load on the mv table. Some cases like
aggregation functions
+ * which are present inside other expressions like sum(a)+sum(b) cannot
be incremental loaded.
+ */
+ private def isFullReload(logicalPlan: LogicalPlan): Boolean = {
+ var isFullReload = false
+ logicalPlan.transformAllExpressions {
+ case a: Alias =>
+ a
+ case agg: AggregateExpression => agg
+ case c: Cast =>
+ isFullReload = c.child.find {
+ case agg: AggregateExpression => false
+ case _ => false
+ }.isDefined || isFullReload
+ c
+ case exp: Expression =>
+ // Check any aggregation function present inside other expression.
+ isFullReload = exp.find {
+ case agg: AggregateExpression => true
--- End diff --
no, it should be true. If the aggregation function present inside any other
expression like add(sum(a) + sum(b)) then we should always do full refresh
instead of incremental refresh
---
