Thanks, will see if I can influence the code path here. Not sure how I can skip HepPlanner findBestExp() in production code path, but I'm in my early days here.
Can I suggest that, for at least a smattering of query forms, that FINER and FINEST tracing be included in integration if not also unit tests? On Fri, Jul 10, 2015 at 1:22 PM, Jinfeng Ni <[email protected]> wrote: > DRILL-3156 was filed to track the calcite trace issue [1]. > > Basically, the HepPlanner used for window function planning caused the > tracing issue. I have a prototype patch to fix this issue. I'll try to see > if I can get it ready for 1.2.0 release. > > As a workaround, if you do not use window function, you may consider > skipping the HepPlanner findBestExp() call in > DefaultSqlHandler.:convertToRel(SqlNode node)[2]. > > > 1. https://issues.apache.org/jira/browse/DRILL-3156 > 2. > > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java#L451 > > > > On Fri, Jul 10, 2015 at 9:19 AM, George Spofford <[email protected]> > wrote: > > > (If there's a better target for an issue request, please let me know!) > > > > While trying to understand the details Calcite rule execution, I turned > on > > the Calcite tracing per > > https://calcite.incubator.apache.org/docs/howto.html#tracing . At that > > point (running a query from the web UI) I get the error > > > > Query Failed: An Error Occurred > > org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > > AssertionError: Internal error: should never get here ... > > > > > > The query is pretty straightforward: > > > > select Person, sum(Qty1) from mongo.mine.test group by Person > > > > > > > > A simple trial with the same settings against the Apache Calcite > > example/csv doesn't show the same behavior. (Calcite query: SELECT > DEPTNO, > > SUM(EMPNO) FROM emps GROUP BY DEPTNO ; ) > > > > In Drill, I'm querying against a Mongo db but the code path in the > > exception trace doesn't immediately appear to be relevant for that. It > > seems to happen the very first time dumpGraph is called. > > > > The innermost cause (in all its glory) is due to the method: > > > > @Override public RelOptCost computeSelfCost(RelOptPlanner planner) { > > // HepRelMetadataProvider is supposed to intercept this > > // and redirect to the real rels. > > throw Util.newInternal("should never get here"); > > } > > > > > > and the trace is: > > > > cause { > > exception_class: "java.lang.AssertionError" > > message: "Internal error: should never get here" > > stack_trace { > > class_name: "org.apache.calcite.util.Util" > > file_name: "Util.java" > > line_number: 775 > > method_name: "newInternal" > > is_native_method: false > > } > > stack_trace { > > class_name: "org.apache.calcite.plan.hep.HepRelVertex" > > file_name: "HepRelVertex.java" > > line_number: 68 > > method_name: "computeSelfCost" > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows" > > file_name: "RelMdPercentageOriginalRows.java" > > line_number: 165 > > method_name: "getNonCumulativeCost" > > is_native_method: false > > } > > stack_trace { > > class_name: "..." > > line_number: 0 > > method_name: "..." > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1" > > file_name: "ReflectiveRelMetadataProvider.java" > > line_number: 194 > > method_name: "invoke" > > is_native_method: false > > } > > stack_trace { > > class_name: "..." > > line_number: 0 > > method_name: "..." > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.RelMetadataQuery" > > file_name: "RelMetadataQuery.java" > > line_number: 115 > > method_name: "getNonCumulativeCost" > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows" > > file_name: "RelMdPercentageOriginalRows.java" > > line_number: 151 > > method_name: "getCumulativeCost" > > is_native_method: false > > } > > stack_trace { > > class_name: "..." > > line_number: 0 > > method_name: "..." > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1" > > file_name: "ReflectiveRelMetadataProvider.java" > > line_number: 194 > > method_name: "invoke" > > is_native_method: false > > } > > stack_trace { > > class_name: "..." > > line_number: 0 > > method_name: "..." > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.RelMetadataQuery" > > file_name: "RelMetadataQuery.java" > > line_number: 101 > > method_name: "getCumulativeCost" > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows" > > file_name: "RelMdPercentageOriginalRows.java" > > line_number: 154 > > method_name: "getCumulativeCost" > > is_native_method: false > > } > > stack_trace { > > class_name: "..." > > line_number: 0 > > method_name: "..." > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1" > > file_name: "ReflectiveRelMetadataProvider.java" > > line_number: 194 > > method_name: "invoke" > > is_native_method: false > > } > > stack_trace { > > class_name: "..." > > line_number: 0 > > method_name: "..." > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.calcite.rel.metadata.RelMetadataQuery" > > file_name: "RelMetadataQuery.java" > > line_number: 101 > > method_name: "getCumulativeCost" > > is_native_method: false > > } > > stack_trace { > > class_name: "org.apache.calcite.plan.AbstractRelOptPlanner" > > file_name: "AbstractRelOptPlanner.java" > > line_number: 251 > > method_name: "getCost" > > is_native_method: false > > } > > stack_trace { > > class_name: "org.apache.calcite.plan.hep.HepPlanner" > > file_name: "HepPlanner.java" > > line_number: 962 > > method_name: "dumpGraph" > > is_native_method: false > > } > > stack_trace { > > class_name: "org.apache.calcite.plan.hep.HepPlanner" > > file_name: "HepPlanner.java" > > line_number: 151 > > method_name: "setRoot" > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler" > > file_name: "DefaultSqlHandler.java" > > line_number: 449 > > method_name: "convertToRel" > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler" > > file_name: "DefaultSqlHandler.java" > > line_number: 191 > > method_name: "validateAndConvert" > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler" > > file_name: "DefaultSqlHandler.java" > > line_number: 157 > > method_name: "getPlan" > > is_native_method: false > > } > > stack_trace { > > class_name: > > "org.apache.drill.exec.planner.sql.DrillSqlWorker" > > file_name: "DrillSqlWorker.java" > > line_number: 178 > > method_name: "getPlan" > > is_native_method: false > > } > > stack_trace { > > class_name: "org.apache.drill.exec.work.foreman.Foreman" > > file_name: "Foreman.java" > > line_number: 903 > > method_name: "runSQL" > > is_native_method: false > > } > > stack_trace { > > class_name: "org.apache.drill.exec.work.foreman.Foreman" > > file_name: "Foreman.java" > > line_number: 242 > > method_name: "run" > > is_native_method: false > > } > > stack_trace { > > class_name: "..." > > line_number: 0 > > method_name: "..." > > is_native_method: false > > } > > } > > > > > > > > The mongo database is simplicity itself, for repro purposes: > > > > { > > "Date" : "4/4/2014", > > "Person" : "Bob", > > "Qty1" : 45, > > "Qty2" : 52 > > } > > { > > "Date" : "4/12/2014", > > "Person" : "Sarah", > > "Qty1" : 10, > > "Qty2" : 14 > > } > > { > > "Date" : "4/12/2014", > > "Person" : "Bob", > > "Qty1" : 72, > > "Qty2" : 33 > > } > > { > > "Date" : "5/1/2014", > > "Person" : "Agnes", > > "Qty1" : 24, > > "Qty2" : 25 > > } > > { > > "Date" : "5/2/2014", > > "Person" : "Sarah", > > "Qty1" : 86, > > "Qty2" : 4 > > } > > { > > "Date" : "5/6/2014", > > "Person" : "Bob", > > "Qty1" : 17, > > "Qty2" : 4 > > } > > > > loaded with `mongoimport --db mine --collection test --drop --file > > c:/tmp/work1a.json` > > > > Thank you for any pointers. > > >
