Hi Prashant, Thanks for bringing this up. I don't know if its a good idea to move "all" of metrics code to hudi-common, since some things are purely write specific..
But, we can selectively move some base classes there and have the actual metrics live across both hudi-client and hudi-common? If you are interested, in the specific problem alone, then having just a clear Metrics class that tracks calls from the Wrapper FileSystem and reports it back to either query engine or write client via a callback should suffice? Thanks Vinoth On Tue, Mar 10, 2020 at 1:02 PM Prashant Wason <[email protected]> wrote: > Hi Team, > > I am interested in adding metrics support to HoodieWrapperFileSystem. This > will help track the counts of operations and their latencies and will > provide valuable data to implement and test newer ideas (e.g. RFC 15 > < > https://cwiki.apache.org/confluence/display/HUDI/RFC+-+15:+HUDI+File+Listing+and+Query+Planning+Improvements > >which > is > proposing a consolidated metadata reducing the number of file system > operations). > > HUDI metrics are currently implemented in module hudi-client. Modules like > hudi-utilities have hudi-client as their dependency (via pom.xml). But this > cannot be done for hudi-common as this module is itself a dependency for > hudi-client. > > Hence, I feel it may be better to move the HUID metrics code to hudi-common > as most modules anyways depend on hudi-common. > > What do you think about this? Any other ideas of how to approach this? > > Thanks > Prashant >
