pratapaditya04 commented on code in PR #4206:
URL: https://github.com/apache/gobblin/pull/4206#discussion_r3676723698


##########
gobblin-metrics-libs/gobblin-metrics-base/src/main/avro/GaaSJobObservabilityEvent.avsc:
##########
@@ -264,6 +264,18 @@
                 "type": ["null","string"],
                 "doc": "Whether the dataset passed the overall data quality 
check",
                 "default": null
+              },
+              {
+                "name": "snapshotsCommitted",
+                "type": "long",
+                "doc": "Number of table snapshots committed for the dataset in 
this run (e.g. Iceberg snapshot-replication commits), -1 if unsupported/unknown 
by the writer",
+                "default": -1
+              },
+              {
+                "name": "partitionsCommitted",
+                "type": "long",
+                "doc": "Number of partitions committed for the dataset in this 
run, -1 if unsupported/unknown by the writer",

Review Comment:
   Correction to my earlier reply — I kept the plural `partitionsCommitted` 
after checking the producer side. A single run can commit **more than one** 
partition (an Iceberg snapshot replication can span many), so singular would be 
misleading.
   
   To honour the `["null", "string"]` type you asked for, both 
`snapshotsCommitted` and `partitionsCommitted` are now nullable strings 
carrying a **comma-separated list** — snapshot *ids* and partition values 
respectively, rather than counts. They are `null`/absent when the writer 
commits none. Happy to switch to an `array` type instead if you would prefer 
that over a delimited string.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to