vinothchandar commented on a change in pull request #849: [HUDI-62] Added index
handle times
URL: https://github.com/apache/incubator-hudi/pull/849#discussion_r318183172
##########
File path: hudi-client/src/main/java/org/apache/hudi/HoodieWriteClient.java
##########
@@ -152,9 +153,12 @@ public static SparkConf registerClasses(SparkConf conf) {
// Create a Hoodie table which encapsulated the commits and files visible
HoodieTable<T> table = HoodieTable.getHoodieTable(
createMetaClient(true), config, jsc);
-
+ indexLookupContext=metrics.getIndexLookupCtx();
JavaRDD<HoodieRecord<T>> recordsWithLocation =
index.tagLocation(hoodieRecords, jsc, table);
- return recordsWithLocation.filter(v1 -> !v1.isCurrentLocationKnown());
+ recordsWithLocation=recordsWithLocation.filter(v1 ->
!v1.isCurrentLocationKnown());
Review comment:
we already report num inserts and updates already IIUC... For the index, I
think the one metric we can do in this PR is actually index lookup time. i.e
simply time index.tagLocation() call?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
