[
https://issues.apache.org/jira/browse/TAJO-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900312#comment-14900312
]
ASF GitHub Bot commented on TAJO-1493:
--------------------------------------
Github user jihoonson commented on a diff in the pull request:
https://github.com/apache/tajo/pull/772#discussion_r39945793
--- Diff:
tajo-plan/src/main/java/org/apache/tajo/plan/expr/AlgebraicUtil.java ---
@@ -493,4 +497,190 @@ public Object visitTimeLiteral(Object ctx,
Stack<Expr> stack, TimeLiteral expr)
return super.visitTimeLiteral(ctx, stack, expr);
}
}
+
+ /**
+ * Find the top expr matched to type from the given expr
+ *
+ * @param expr start expr
+ * @param type to find
+ * @return a found expr
+ */
+ public static <T extends Expr> T findTopExpr(Expr expr, OpType type)
throws TajoException {
+ Preconditions.checkNotNull(expr);
+ Preconditions.checkNotNull(type);
+
+ ExprFinder finder = new ExprFinder(type);
+ finder.visit(null, new Stack<Expr>(), expr);
+
+ if (finder.getFoundExprs().size() == 0) {
+ return null;
+ }
+ return (T) finder.getFoundExprs().get(0);
+ }
+
+ private static class ExprFinder extends SimpleAlgebraVisitor<Object,
Expr> {
+ private List<Expr> list = new ArrayList<Expr>();
+ private final OpType[] tofind;
+ private boolean topmost = false;
+ private boolean finished = false;
+
+ public ExprFinder(OpType... type) {
+
+ this.tofind = type;
+ }
+
+ public ExprFinder(OpType[] type, boolean topmost) {
+ this(type);
+ this.topmost = topmost;
+ }
+
+ @Override
+ public Expr visit(Object ctx, Stack<Expr> stack, Expr expr) throws
TajoException {
+ if (!finished) {
+ for (OpType type : tofind) {
+ if (expr.getType() == type) {
+ list.add(expr);
+ }
+ if (topmost && list.size() > 0) {
+ finished = true;
+ }
+ }
+ }
+ return super.visit(ctx, stack, expr);
+ }
+
+ public List<Expr> getFoundExprs() {
+ return list;
+ }
+
+ }
+
+ public static Expr[] toConjunctiveNormalFormArray(Expr expr) {
--- End diff --
Unnecessary change.
> Make partition pruning based on catalog informations
> ----------------------------------------------------
>
> Key: TAJO-1493
> URL: https://issues.apache.org/jira/browse/TAJO-1493
> Project: Tajo
> Issue Type: Sub-task
> Components: Catalog, Planner/Optimizer
> Reporter: Jaehwa Jung
> Assignee: Jaehwa Jung
> Fix For: 0.11.0, 0.12.0
>
> Attachments: TAJO-1493.patch, TAJO-1493_2.patch, TAJO-1493_3.patch,
> TAJO-1493_4.patch
>
>
> Currently, PartitionedTableRewriter take a look into partition directories
> for rewriting filter conditions. It get all sub directories of table path
> because catalog doesn’t provide partition directories. But if there are lots
> of sub directories on HDFS, such as, more than 10,000 directories, it might
> be cause overload to NameNode. Thus, CatalogStore need to provide partition
> directories for specified filter conditions. I designed new method to
> CatalogStore as follows:
> * method name: getPartitionsWithConditionFilters
> * first parameter: database name
> * second parameter: table name
> * third parameter: where clause (included target column name and partition
> value)
> * return values:
> List<org.apache.tajo.catalog.proto.CatalogProtos.TablePartitionProto>
> * description: It scan right partition directories on CatalogStore with where
> caluse.
> For examples, users set parameters as following:
> ** first parameter: default
> ** second parameter: table1
> ** third parameter: COLUMN_NAME = 'col1' AND PARTITION_VALUE = '3
> In the previous cases, this method will create select clause as follows.
> {code:xml}
> SELECT DISTINCT A.PATH
> FROM PARTITIONS A, (
> SELECT B.PARTITION_ID
> FROM PARTITION_KEYS B
> WHERE B.PARTITION_ID > 0
> AND (
> COLUMN_NAME = 'col1' AND PARTITION_VALUE = '3'
> )
> ) B
> WHERE A.PARTITION_ID > 0
> AND A.TID = ${table_id}
> AND A.PARTITION_ID = B.PARTITION_ID
> {code}
> At the first time, I considered to use EvalNode instead of where clause. But
> I can’t use it because of recursive related problems between tajo-catalog
> module and tajo-plan module. So, I’ll implement utility class to convert
> EvalNode to SQL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
