Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14188 )
Change subject: IMPALA-8923: remove synchronized in HBaseTable.getEstimatedRowStats ...................................................................... IMPALA-8923: remove synchronized in HBaseTable.getEstimatedRowStats HBaseTable.getEstimatedRowStats() estimates #rows and row size by sampling on hbase table in target key range. It requires HBase RPCs so could be slow. Currently, HBaseTable.getEstimatedRowStats() is marked as synchronized. The purpose is to protect the HTable (old HBase API) object in legacy codes (before commit cf9d2485dd4e6544f6f1f407e2ad0b43eba31874). However, after commit cf9d248, we create org.apache.hadoop.hbase.client.Table object for each task (See comments and usages of getHBaseTable() in FeHBaseTable.Util). So we don't need the "synchronized" keyword anymore in HBaseTable.getEstimatedRowStats(). Keeping this method "synchronized" is further harmful. In high qps workload, queries on the same table will wait for entering this method and cost a lot of time in waiting (if this method is comparably slow). Added some useful tracing logs to detect slow HBase RPCs. Tests: - Manually adding a 100ms latency (e.g. 100ms) in FeHBaseTable.Util.getEstimatedRowStats() and run concurrent queries on the same hbase table. In my experiment, removing "synchronized" gains 40% boost in 95% percentile query time. Change-Id: Ifa23c16ee662c4f22851c700aea2ea5be847b64d Reviewed-on: http://gerrit.cloudera.org:8080/14188 Reviewed-by: Quanlong Huang <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/FeHBaseTable.java M fe/src/main/java/org/apache/impala/catalog/HBaseTable.java 2 files changed, 26 insertions(+), 15 deletions(-) Approvals: Quanlong Huang: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/14188 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifa23c16ee662c4f22851c700aea2ea5be847b64d Gerrit-Change-Number: 14188 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
