[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688811#comment-16688811 ] ASF GitHub Bot commented on DRILL-6770: --- asfgit closed pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/pom.xml b/pom.xml index a75e251d2f7..c49f1ede4bb 100644 --- a/pom.xml +++ b/pom.xml @@ -53,8 +53,8 @@ 2.9.5 2.9.5 3.4.12 -6.0.1-mapr -2.0.1-mapr-1804 +6.1.0-mapr +3.0-mapr-1808 1.0.0-RC2 3.0.0 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Parai >Priority: Critical > Labels: ready-to-commit > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643489#comment-16643489 ] Anton Gozhiy commented on DRILL-6770: - [~agirish], these are different failures. > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Labels: ready-to-commit > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643036#comment-16643036 ] Vitalii Diravka commented on DRILL-6770: Most likely the reason of these failures is the different versions of _mapr-release_ libraries version - [6.1.0-mapr|https://github.com/apache/drill/pull/1489/files#diff-600376dffeb79835ede4a0b285078036R56] and mapr-core version on test cluster. > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Labels: ready-to-commit > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642953#comment-16642953 ] Volodymyr Vysotskyi commented on DRILL-6770: [~agirish], sent via mail. > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Labels: ready-to-commit > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642313#comment-16642313 ] Abhishek Girish commented on DRILL-6770: The failures in Functional/hive/hive_storage/hive_maprdb_json suite are seen on apache/drill master branch as well, so we could ignore those until they are fixed. cc: [~agozhiy]. The other Functional test failures aren't seen on our test runs. [~vvysotskyi], can you share details on the failures - will help with narrowing down. > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Labels: ready-to-commit > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641746#comment-16641746 ] ASF GitHub Bot commented on DRILL-6770: --- vvysotskyi commented on issue #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#issuecomment-427811290 @gparai, this PR causes the next functional tests failures: ``` Execution Failures: /root/drillAutomation/drill-test-framework/framework/resources/Functional/hive/hive_storage/hive_maprdb_json/less_than.sql.explain Plan Verification Failures: /root/drillAutomation/drill-test-framework/framework/resources/Functional/hive/hive_storage/hive_maprdb_json/between.sql.explain Random Failures: /root/drillAutomation/drill-test-framework/framework/resources/External/mapr-drill-tests/Testcases/maprdb/limit0/plan/pushdown_q20.q /root/drillAutomation/drill-test-framework/framework/resources/External/mapr-drill-tests/Testcases/maprdb/pushdown/query/pushdown_q8.q /root/drillAutomation/drill-test-framework/framework/resources/External/mapr-drill-tests/Testcases/maprdb/limit0/data/pushdown_q18.q /root/drillAutomation/drill-test-framework/framework/resources/External/mapr-drill-tests/Testcases/maprdb/limit0/plan/rowkey-range.q /root/drillAutomation/drill-test-framework/framework/resources/External/mapr-drill-tests/Testcases/maprdb/limit0/plan/avg.q /root/drillAutomation/drill-test-framework/framework/resources/Functional/schema_change_empty_batch/maprdb/binary_maprdb/emptyMaprDBUnion.sql /root/drillAutomation/drill-test-framework/framework/resources/External/mapr-drill-tests/Testcases/maprdb/sanity/date_add.q /root/drillAutomation/drill-test-framework/framework/resources/Functional/schema_change_empty_batch/maprdb/json/emptyMaprDBJsonLeftJoin.sql /root/drillAutomation/drill-test-framework/framework/resources/Functional/schema_change_empty_batch/maprdb/binary_hbase/emptyMaprDBScan1.sql /root/drillAutomation/drill-test-framework/framework/resources/Functional/hive/hive_storage/hive_maprdb_json/project.sql /root/drillAutomation/drill-test-framework/framework/resources/Functional/hive/hive_storage/hive_maprdb_json/like.sql.explain /root/drillAutomation/drill-test-framework/framework/resources/Functional/hive/hive_storage/hive_maprdb_json/filter_id.sql.explain /root/drillAutomation/drill-test-framework/framework/resources/Functional/hive/hive_storage/hive_maprdb_json/filter_id.sql ``` and advanced tests failures: ``` Execution Failures: /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/sanity/maprdb/json/query23_max.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/smoke/maprdb/json/join03-merge.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/smoke/maprdb/json/group-by01.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/smoke/maprdb/json/group-by02.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/smoke/maprdb/json/failed04.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/original/maprdb/json/query10.sql Data Verification Failures: /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/sanity/maprdb/json/query30_distinct.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/original/maprdb/json/query13.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/original/maprdb/json/query18.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/original/maprdb/json/query4.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/original/maprdb/json/query8.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/smoke/maprdb/json/failed02.sql Timeouts: /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/original/maprdb/json/query1.sql /root/drillAutomation/drill-test-framework/framework/resources/Advanced/tpch/tpch_sf1/sql_functions/strings/maprdb/json/query10.sql ``` Could you please take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640219#comment-16640219 ] Vitalii Diravka commented on DRILL-6770: [~gparai] Could you change the status of Jira to {{Reviwable}} to add it to the next batch-commit? > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Labels: ready-to-commit > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640069#comment-16640069 ] ASF GitHub Bot commented on DRILL-6770: --- amansinha100 commented on issue #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#issuecomment-427428588 +1 to get it merged to get past the current blocking issue. There could be potential plan changes though (which are not tracked by functional tests) since the filtered row count now will be higher than before (even though it will be more accurate). So we should follow up with some more targeted tests. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640048#comment-16640048 ] ASF GitHub Bot commented on DRILL-6770: --- vdiravka commented on a change in pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#discussion_r223065491 ## File path: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java ## @@ -184,11 +184,9 @@ public MapRDBSubScan getSpecificScan(int minorFragmentId) { @Override public ScanStats getScanStats() { -//TODO: look at stats for this. -long rowCount = (long) ((scanSpec.getSerializedFilter() != null ? .5 : 1) * totalRowCount); Review comment: Even better :) Thanks for explanation This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639965#comment-16639965 ] ASF GitHub Bot commented on DRILL-6770: --- gparai commented on a change in pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#discussion_r223052988 ## File path: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java ## @@ -184,11 +184,9 @@ public MapRDBSubScan getSpecificScan(int minorFragmentId) { @Override public ScanStats getScanStats() { -//TODO: look at stats for this. -long rowCount = (long) ((scanSpec.getSerializedFilter() != null ? .5 : 1) * totalRowCount); Review comment: The totalrowcount already factored in the scanspec condition - please see the metatable call above. This was a bug in changes made for DRILL-5429. I missed it while reviewing the PR. The original rowcount did not factor the condition earlier hence it was estimating it here based on whether the filter is present or not. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639629#comment-16639629 ] ASF GitHub Bot commented on DRILL-6770: --- vdiravka commented on a change in pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#discussion_r222957449 ## File path: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java ## @@ -184,11 +184,9 @@ public MapRDBSubScan getSpecificScan(int minorFragmentId) { @Override public ScanStats getScanStats() { -//TODO: look at stats for this. -long rowCount = (long) ((scanSpec.getSerializedFilter() != null ? .5 : 1) * totalRowCount); Review comment: Why usage of `JsonScanSpec` is no more relevant here? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639077#comment-16639077 ] ASF GitHub Bot commented on DRILL-6770: --- gparai commented on issue #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#issuecomment-427207684 @vdiravka yes these changes will be overridden by the Secondary Index PR which is currently in review. For this PR, we just target to fix the failure since those other changes are bigger in scope. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639073#comment-16639073 ] ASF GitHub Bot commented on DRILL-6770: --- gparai commented on a change in pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#discussion_r222862324 ## File path: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java ## @@ -148,11 +150,16 @@ private void init() { table = MapRDB.getTable(scanSpec.getTableName()); tabletInfos = table.getTabletInfos(scanSpec.getCondition()); - // Calculate totalRowCount for the table from tabletInfos estimatedRowCount. - // This will avoid calling expensive MapRDBTableStats API to get total rowCount, avoiding - // duplicate work and RPCs to MapR DB server. - for (TabletInfo tabletInfo : tabletInfos) { -totalRowCount += tabletInfo.getEstimatedNumRows(); + final MetaTable metaTable = table.getMetaTable(); + com.mapr.db.scan.ScanStats stats = (scanSpec.getCondition() == null) + ? metaTable.getScanStats() : metaTable.getScanStats(scanSpec.getCondition()); + totalRowCount = stats.getEstimatedNumRows(); + // MapRDB client can return invalid rowCount i.e. 0, especially right after table + // creation. It takes 15 minutes before table stats are obtained and cached in client. + // If we get 0 rowCount, fallback to getting rowCount using old admin API. + if (totalRowCount == 0) { Review comment: Sure, I will remove it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638714#comment-16638714 ] ASF GitHub Bot commented on DRILL-6770: --- amansinha100 commented on a change in pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#discussion_r222793328 ## File path: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java ## @@ -148,11 +150,16 @@ private void init() { table = MapRDB.getTable(scanSpec.getTableName()); tabletInfos = table.getTabletInfos(scanSpec.getCondition()); - // Calculate totalRowCount for the table from tabletInfos estimatedRowCount. - // This will avoid calling expensive MapRDBTableStats API to get total rowCount, avoiding - // duplicate work and RPCs to MapR DB server. - for (TabletInfo tabletInfo : tabletInfos) { -totalRowCount += tabletInfo.getEstimatedNumRows(); + final MetaTable metaTable = table.getMetaTable(); + com.mapr.db.scan.ScanStats stats = (scanSpec.getCondition() == null) + ? metaTable.getScanStats() : metaTable.getScanStats(scanSpec.getCondition()); + totalRowCount = stats.getEstimatedNumRows(); + // MapRDB client can return invalid rowCount i.e. 0, especially right after table + // creation. It takes 15 minutes before table stats are obtained and cached in client. + // If we get 0 rowCount, fallback to getting rowCount using old admin API. + if (totalRowCount == 0) { Review comment: Previously, we were not checking this condition of 0 rowcount. I am thinking since this part of the code has anyways changed in the indexing PR [1], should we just wait until that change is merged ? [1] https://github.com/apache/drill/pull/1466 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638701#comment-16638701 ] ASF GitHub Bot commented on DRILL-6770: --- vdiravka commented on issue #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#issuecomment-427132855 The change should solve the issue. Looks like to avoid similar in future it would be good to get rid of using `com.mapr.db.impl.TabletInfoImpl` in Drill code, because it is marked as [`@Internal`](https://github.com/ojai/ojai/blob/master/core/src/main/java/org/ojai/annotation/API.java#L44). @gparai Is it possible to do it? Or maybe mark it with `TODO` comment somewhere in the code. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638655#comment-16638655 ] ASF GitHub Bot commented on DRILL-6770: --- Agirish commented on a change in pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489#discussion_r222780156 ## File path: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java ## @@ -148,11 +150,16 @@ private void init() { table = MapRDB.getTable(scanSpec.getTableName()); tabletInfos = table.getTabletInfos(scanSpec.getCondition()); - // Calculate totalRowCount for the table from tabletInfos estimatedRowCount. - // This will avoid calling expensive MapRDBTableStats API to get total rowCount, avoiding - // duplicate work and RPCs to MapR DB server. - for (TabletInfo tabletInfo : tabletInfos) { -totalRowCount += tabletInfo.getEstimatedNumRows(); + final MetaTable metaTable = table.getMetaTable(); + com.mapr.db.scan.ScanStats stats = (scanSpec.getCondition() == null) + ? metaTable.getScanStats() : metaTable.getScanStats(scanSpec.getCondition()); + totalRowCount = stats.getEstimatedNumRows(); + // MapRDB client can return invalid rowCount i.e. 0, especially right after table + // creation. It takes 15 minutes before table stats are obtained and cached in client. Review comment: I think it takes 5 min now instead of 15. Need to confirm. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table:
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638648#comment-16638648 ] ASF GitHub Bot commented on DRILL-6770: --- gparai opened a new pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs URL: https://github.com/apache/drill/pull/1489 MapRDB 6.1.0 has removed support for some APIs which breaks Drill code (Deprecation should have been used instead). This PR uses the new APIs instead. @amansinha100 can you please review the PR? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Gautam Kumar Parai >Priority: Critical > Fix For: 1.15.0 > > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > >
[jira] [Commented] (DRILL-6770) Queries on MapR-DB JSON tables fail with UnsupportedOperationException: Getting number of rows for tablet not supported
[ https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637317#comment-16637317 ] Chunhui Shi commented on DRILL-6770: For now, Apache Drill does not talk to MapRDB 6.X unless the index based commits Aman is filing got checked in. > Queries on MapR-DB JSON tables fail with UnsupportedOperationException: > Getting number of rows for tablet not supported > --- > > Key: DRILL-6770 > URL: https://issues.apache.org/jira/browse/DRILL-6770 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization, Storage - MapRDB >Affects Versions: 1.15.0 > Environment: MapR 6.1.0 > Drill 1.15.0 >Reporter: Abhishek Girish >Assignee: Pritesh Maker >Priority: Critical > > Create a simple MapR-DB JSON table > {code} > $ mapr dbshell > MapR-DB Shell > maprdb root:> create /tmp/t1 > Table /tmp/t1 created. > maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}' > Document with id: "1" inserted. > maprdb root:> find /tmp/t1 > {"_id":"1","a":1} > 1 document(s) found. > {code} > Querying this from Drill fails: > {code} > > select * from mfs.`/tmp/t1`; > Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows > for tablet not supported > {code} > Stack Trace: > {code} > (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception > during fragment initialization: Error while applying rule DrillTableRule, > args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, > /tmp/t1])] > org.apache.drill.exec.work.foreman.Foreman.run():300 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (java.lang.RuntimeException) Error while applying rule > DrillTableRule, args > [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])] > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 > org.apache.drill.exec.work.foreman.Foreman.runSQL():584 > org.apache.drill.exec.work.foreman.Foreman.run():272 > java.util.concurrent.ThreadPoolExecutor.runWorker():1149 > java.util.concurrent.ThreadPoolExecutor$Worker.run():624 > java.lang.Thread.run():748 > Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error > getting region info for table: maprfs:///tmp/t1 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161 > org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.():83 > org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81 > org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117 > org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112 > org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99 > org.apache.drill.exec.planner.logical.DrillScanRel.():90 > org.apache.drill.exec.planner.logical.DrillScanRel.():70 > org.apache.drill.exec.planner.logical.DrillScanRel.():63 > org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38 > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212 > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648 > org.apache.calcite.tools.Programs$RuleSetProgram.run():339 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252 > > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314 > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145 > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83 >