danielcweeks commented on pull request #4050:
URL: https://github.com/apache/iceberg/pull/4050#issuecomment-1035680716
> Yea it looks good to me too, definitely have heard some asks for it! Two
overall questions:
>
> 1. How about the implementation for S3FileOutputStream to track
write_bytes, write_records?
> 2. Not sure what is the right way, but if I am implementing a
FileIOMetrics for S3FileIO, the FileIOMetrics interface does not seem
descriptive enough what I should do. Example, how will I know what kind of
counter name/type to support? Maybe a wrapper S3FileIOMetrics interface
extend FileIO interface with methods like:
>
> ```
> Counter<Long> readBytesCounter() {
> counter(READ_BYTES, Long.class);
> }
> ```
>
> that S3FileIO calls instead, so it's more clear what to implement?
@szehon-ho I think we actually want to keep this rather generic, so we don't
want specific counters called out since they may vary by engine and
implementation, so that could get quite messy.
After talking with @jackye1995, @nastra and @rdblue about this, I think
we're going to up-level the Metrics interfaces so they can be reused for
capturing metrics for other areas of the codebase as well. I'll update with
the proposed changes and we can follow on with other meter types (e.g. timers)
and incorporate them into areas like scan planning in new PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
