EugeneYushin opened a new pull request, #2666:
URL: https://github.com/apache/iceberg-go/pull/2666

   Fixes #2665
   
   ## Summary
   
   The commit `*.total` metrics are table totals after the commit, read from the
   snapshot summary's `total-*` entries in `table/commit_metrics.go`. The OTel
   reporter recorded them as monotonic counters, so summing them or taking
   `rate()`/`increase()` over commits gave inflated values (for example 100, 
then 250,
   reported as 350). They are now `Int64Gauge`s, so each series holds the last
   reported value. This matches Java's `OtelMetricsReporter`
   (apache/iceberg#16250), which records these same six metrics as `LongGauge`.
   
   ## Changes
   
   - `metrics/otel/reporter.go`: moved the six totals out of 
`commitCounterSpecs` into a
     new `commitGaugeSpecs`. They are created with `meter.Int64Gauge` and 
written with
     `Record`:
     - `iceberg.commit.records.total`
     - `iceberg.commit.data_files.total`
     - `iceberg.commit.delete_files.total`
     - `iceberg.commit.file_size.total_bytes` (unit `By`)
     - `iceberg.commit.positional_deletes.total`
     - `iceberg.commit.equality_deletes.total`
   - Names, descriptions, units and attributes are unchanged. Cardinality is 
unchanged.
   - Unset totals are skipped rather than recorded as 0, so a missing value 
never
     overwrites the last known one.
   - Added/removed/attempts/manifest metrics and all scan metrics remain 
counters.
     Durations remain histograms.
   - Updated the package doc to describe the three instrument types, and noted 
that
     gauges are last-write-wins per attribute set. Dropping 
`iceberg.table.name` via
     `WithAttributes` makes different tables overwrite each other's totals.
   - `website/src/metrics-reporting.md`: documented which instruments are 
gauges,
     counters and histograms.
   
   ## Tests
   
   In `metrics/otel/reporter_test.go`:
   
   - `TestReporterCommitFullCoverage` now asserts counters are `Sum[int64]` and 
totals
     are `Gauge[int64]`, with expected values and units.
   - `TestReporterCommitTotalsAreGauges`: totals 100 then 250 give 250, while
     `records.added` still sums to 300.
   - `TestReporterCommitTotalsPerTable`: one gauge series per table.
   - `TestReporterMissingTotalKeepsLastGauge`: an unset total does not 
overwrite the
     last value.
   - The `attributeKeys` helper now also walks gauge data points. Previously it 
only
     looked at counters, so attribute assertions would have skipped gauges.
   
   Verified locally with:
   * `make lint`
   * `make test`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to