> Returning a success code when there's a failure is clearly wrong and an > implementation issue. However, whether a client retries or ignores the > failure response from the metrics endpoint is a client-side implementation > issue.
+1 on that. I'm OK with leaving the spec unchanged. In that case, I think the reasonable interpretation is that the metrics endpoint behaves like a standard REST endpoint and can return an error status (e.g., 4xx or 5xx) when an operation fails. Returning 204 when the operation actually failed would be misleading. Whether to retry is a client decision. The current Java implementation doesn't retry, which I believe is reasonable. This is consistent with the point I raised in the Polaris dev thread: https://lists.apache.org/thread/0r7fpq153d2ny0tbt162051dh0jv3wjb Yufei Yufei On Tue, Nov 3, 2026 at 5:15 PM Daniel Weeks via dev <[email protected]> wrote: > > I don't think we need to explicitly define the behavior as best effort in the > spec. The endpoint has defined behavior and return codes. It being separate > from a commit inherently makes it a separate operation disconnected from the > commit itself. The idea of "best effort" is probably misleading here. The > critical operation is the commit. Metrics are separate and reporting is not > required for the commit. > > Yufei, I think your example is comparing something different. It's not > reasonable for the server to return a code that doesn't represent what > actually happened within the request. Returning a success code when there's > a failure is clearly wrong and an implementation issue. However, whether a > client retries or ignores the failure response from the metrics endpoint is a > client-side implementation issue. > > -Dan > > On Tue, Nov 3, 2026 at 4:37 PM Yufei Gu <[email protected]> wrote: >> >> I think it's reasonable to treat the metrics endpoint as best effort. >> However, we should explicitly document that behavior in the spec rather than >> leave it up to individual implementations to interpret or guess. >> >> Otherwise, where do we draw the line? Could we also consider `updateTable` >> best effort and allow the server to return 204 even if the operation failed >> internally? >> >> Yufei >> >> >> On Tue, Nov 3, 2026 at 6:30 AM Anand Kumar Sankaran via dev >> <[email protected]> wrote: >>> >>> Thanks Eduard. Appreciate the response. I will take this back to the >>> Polaris dev mailing list. >>> >>> On Tue, Nov 3, 2026 at 1:28 AM Eduard Tudenhöfner via dev >>> <[email protected]> wrote: >>>> >>>> Hey Anand, The metrics endpoint and metrics reporting is intended to be >>>> best-effort and the RESTMetricsReporter implementation also ensures it >>>> does not bubble up any errors. I don't think we want reads/writes to fail >>>> just because the server >>>> Hey Anand, >>>> >>>> The metrics endpoint and metrics reporting is intended to be best-effort >>>> and the RESTMetricsReporter implementation also ensures it does not bubble >>>> up any errors. >>>> I don't think we want reads/writes to fail just because the server >>>> couldn't store information about the scan/commit. >>>> >>>> We document 204, 400, 401, 403, 404, 419, 503 and 5XX on the /metrics >>>> endpoint because those are all potential errors. It is up to the catalog >>>> implementation to decide whether to throw a 500 or just log an error. The >>>> catalog can also decide to discard metrics reports and return 204. >>>> >>>> Clients are not expected to retry, simply because it would increase the >>>> overall duration of a scan/commit and we don't want to negatively affect >>>> the read/write path. >>>> >>>> I'm not sure we need to update the wording in the spec. As I mentioned, it >>>> is up to catalog implementations to decide what to do and we don't want to >>>> be prescriptive. >>>> >>>> I hope that helps and answers your questions. >>>> >>>> Eduard >>>> >>>> On Tue, Nov 3, 2026 at 1:42 AM Anand Kumar Sankaran via dev >>>> <[email protected]> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I'm a contributor to Apache Polaris. We have a question about the REST >>>>> catalog's metrics endpoint (POST >>>>> /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics, >>>>> reportMetrics) that we'd like to settle with the Iceberg community before >>>>> changing server behavior. >>>>> >>>>> Background >>>>> >>>>> Polaris can persist metrics reports sent to this endpoint. Today, if the >>>>> database write fails, the endpoint returns a 500. A pending Polaris PR >>>>> [1] would log the failure, increment a drop counter, and return 204 >>>>> instead. The motivation is that a degraded metrics store shouldn't show >>>>> up as a catalog failure. The discussion on the Polaris dev list [2] has >>>>> split on this. >>>>> >>>>> What the spec says >>>>> >>>>> The spec defines the operation as "Send a metrics report to this endpoint >>>>> to be processed by the backend". It documents 204, 400, 401, 403, 404, >>>>> 419, 503 and 5XX. It doesn't say whether reporting is best-effort or >>>>> whether a catalog must signal a failure to process a report. It also >>>>> doesn't say what a client should do on a 5xx. >>>>> >>>>> What clients do >>>>> >>>>> As far as I know, the Java client's REST metrics reporter catches and >>>>> logs errors from this endpoint and does not retry. I haven't checked the >>>>> other implementations (PyIceberg, iceberg-rust, iceberg-go), so please >>>>> correct me. >>>>> >>>>> The two readings >>>>> >>>>> 1. Best-effort telemetry. The client sends the report and moves on. A >>>>> catalog may drop a report it can't persist, as long as it surfaces that >>>>> to operators. A 5xx tells the client nothing it can act on. >>>>> >>>>> 2. A normal REST write. If the catalog can't process the report, it says >>>>> so with a 5xx, and clients may retry. This matters for anyone using the >>>>> reports for auditing or billing, though it only helps if clients actually >>>>> retry. >>>>> >>>>> Questions >>>>> >>>>> 1. Is reportMetrics intended to be best-effort? Is a catalog allowed to >>>>> accept a report and discard it, or fail to process it, without signaling >>>>> failure? >>>>> 2. If a catalog can't process a report, what should it return? Is 5xx the >>>>> expected answer, or is 204 acceptable? >>>>> 3. Should clients be expected to retry a failed report? If so, the spec >>>>> should say that. If not, it should say reporting is not guaranteed >>>>> delivery. >>>>> 4. Should the spec state any of this in the endpoint description, so >>>>> implementations behave consistently? >>>>> >>>>> We'd like to follow whatever the community decides rather than invent >>>>> Polaris-specific behavior. I'm happy to send a spec PR with clarifying >>>>> wording once there's a direction. >>>>> >>>>> Thanks, >>>>> Anand >>>>> >>>>> [1] https://github.com/apache/polaris/pull/5395 >>>>> [2] https://lists.apache.org/thread/0r7fpq153d2ny0tbt162051dh0jv3wjb >>>>> >>>>> -- >>>>> Anand Kumar Sankaran >>>>> Sr Principal, Software Engineer (Distributed Systems) >>>>> Office of the CTO >>>>> [email protected] >>> >>> >>> >>> -- >>> Anand Kumar Sankaran >>> Sr Principal, Software Engineer (Distributed Systems) >>> Office of the CTO >>> [email protected]
