[
https://issues.apache.org/jira/browse/DRILL-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638714#comment-16638714
]
ASF GitHub Bot commented on DRILL-6770:
---------------------------------------
amansinha100 commented on a change in pull request #1489: DRILL-6770:
JsonTableGroupScan should use new MapRDB 6.1.0 APIs
URL: https://github.com/apache/drill/pull/1489#discussion_r222793328
##########
File path:
contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java
##########
@@ -148,11 +150,16 @@ private void init() {
table = MapRDB.getTable(scanSpec.getTableName());
tabletInfos = table.getTabletInfos(scanSpec.getCondition());
- // Calculate totalRowCount for the table from tabletInfos
estimatedRowCount.
- // This will avoid calling expensive MapRDBTableStats API to get total
rowCount, avoiding
- // duplicate work and RPCs to MapR DB server.
- for (TabletInfo tabletInfo : tabletInfos) {
- totalRowCount += tabletInfo.getEstimatedNumRows();
+ final MetaTable metaTable = table.getMetaTable();
+ com.mapr.db.scan.ScanStats stats = (scanSpec.getCondition() == null)
+ ? metaTable.getScanStats() :
metaTable.getScanStats(scanSpec.getCondition());
+ totalRowCount = stats.getEstimatedNumRows();
+ // MapRDB client can return invalid rowCount i.e. 0, especially right
after table
+ // creation. It takes 15 minutes before table stats are obtained and
cached in client.
+ // If we get 0 rowCount, fallback to getting rowCount using old admin
API.
+ if (totalRowCount == 0) {
Review comment:
Previously, we were not checking this condition of 0 rowcount. I am
thinking since this part of the code has anyways changed in the indexing PR
[1], should we just wait until that change is merged ?
[1] https://github.com/apache/drill/pull/1466
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Queries on MapR-DB JSON tables fail with UnsupportedOperationException:
> Getting number of rows for tablet not supported
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: DRILL-6770
> URL: https://issues.apache.org/jira/browse/DRILL-6770
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization, Storage - MapRDB
> Affects Versions: 1.15.0
> Environment: MapR 6.1.0
> Drill 1.15.0
> Reporter: Abhishek Girish
> Assignee: Gautam Kumar Parai
> Priority: Critical
> Fix For: 1.15.0
>
>
> Create a simple MapR-DB JSON table
> {code}
> $ mapr dbshell
> MapR-DB Shell
> maprdb root:> create /tmp/t1
> Table /tmp/t1 created.
> maprdb root:> insert /tmp/t1 --id '1' --v '{"a":1}'
> Document with id: "1" inserted.
> maprdb root:> find /tmp/t1
> {"_id":"1","a":1}
> 1 document(s) found.
> {code}
> Querying this from Drill fails:
> {code}
> > select * from mfs.`/tmp/t1`;
> Error: SYSTEM ERROR: UnsupportedOperationException: Getting number of rows
> for tablet not supported
> {code}
> Stack Trace:
> {code}
> (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception
> during fragment initialization: Error while applying rule DrillTableRule,
> args [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs,
> /tmp/t1])]
> org.apache.drill.exec.work.foreman.Foreman.run():300
> java.util.concurrent.ThreadPoolExecutor.runWorker():1149
> java.util.concurrent.ThreadPoolExecutor$Worker.run():624
> java.lang.Thread.run():748
> Caused By (java.lang.RuntimeException) Error while applying rule
> DrillTableRule, args
> [rel#1400499:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[mfs, /tmp/t1])]
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():236
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648
> org.apache.calcite.tools.Programs$RuleSetProgram.run():339
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83
> org.apache.drill.exec.work.foreman.Foreman.runSQL():584
> org.apache.drill.exec.work.foreman.Foreman.run():272
> java.util.concurrent.ThreadPoolExecutor.runWorker():1149
> java.util.concurrent.ThreadPoolExecutor$Worker.run():624
> java.lang.Thread.run():748
> Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) Error
> getting region info for table: maprfs:///tmp/t1
> org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():161
> org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.<init>():83
> org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81
> org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170
> org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117
> org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112
> org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>():90
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>():70
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>():63
> org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648
> org.apache.calcite.tools.Programs$RuleSetProgram.run():339
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83
> org.apache.drill.exec.work.foreman.Foreman.runSQL():584
> org.apache.drill.exec.work.foreman.Foreman.run():272
> java.util.concurrent.ThreadPoolExecutor.runWorker():1149
> java.util.concurrent.ThreadPoolExecutor$Worker.run():624
> java.lang.Thread.run():748
> Caused By (java.lang.UnsupportedOperationException) Getting number of rows
> for tablet not supported
> com.mapr.db.impl.TabletInfoImpl.getEstimatedNumRows():71
> org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.init():155
> org.apache.drill.exec.store.mapr.db.json.JsonTableGroupScan.<init>():83
> org.apache.drill.exec.store.mapr.db.MapRDBFormatPlugin.getGroupScan():81
> org.apache.drill.exec.store.dfs.FileSystemPlugin.getPhysicalScan():170
> org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():117
> org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan():112
> org.apache.drill.exec.planner.logical.DrillTable.getGroupScan():99
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>():90
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>():70
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>():63
> org.apache.drill.exec.planner.logical.DrillScanRule.onMatch():38
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():212
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():648
> org.apache.calcite.tools.Programs$RuleSetProgram.run():339
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():425
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform():365
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel():252
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():314
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():179
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():145
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():83
> org.apache.drill.exec.work.foreman.Foreman.runSQL():584
> org.apache.drill.exec.work.foreman.Foreman.run():272
> java.util.concurrent.ThreadPoolExecutor.runWorker():1149
> java.util.concurrent.ThreadPoolExecutor$Worker.run():624
> java.lang.Thread.run():748 (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
