c-thiel commented on code in PR #12584:
URL: https://github.com/apache/iceberg/pull/12584#discussion_r2206764361
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3964,6 +4144,292 @@ components:
metadata:
$ref: '#/components/schemas/TableMetadata'
+ EventsResponse:
+ type: object
+ required:
+ - highest-processed-timestamp-ms
+ - events
+ properties:
+ next-page-token:
+ $ref: "#/components/schemas/PageToken"
+ highest-processed-timestamp-ms:
+ description: >
+ The highest event timestamp processed when generating this
response.
+ This may not necessarily appear in the returned changes if it was
filtered out.
+ type: integer
+ format: int64
+ events:
+ type: array
+ items:
+ $ref: "#/components/schemas/Event"
+
+ Event:
+ type: object
+ required:
+ - event-id
+ - request-id
+ - event-count
+ - timestamp-ms
+ - operation
+ properties:
+ event-id:
+ type: string
+ description: Unique ID of this event. Clients should perform
deduplication based on this ID.
+ request-id:
+ description: >
+ Opaque ID of the request this change belongs to.
+ This ID can be used to identify events that were part of the same
request.
+ Servers generate this ID randomly.
+ type: string
+ event-count:
+ type: integer
+ description: Number of events in the request / batch of events
+ timestamp-ms:
+ type: integer
+ format: int64
+ description: >
+ Timestamp when this transaction occurred (epoch milliseconds).
+ Timestamps are not guaranteed to be unique. Typically all events in
+ a transaction will have the same timestamp.
+ actor:
+ type: string
+ description: >
+ The actor who performed the operation, such as a user or service
account.
+ The content of this field is implementation specific.
+ operation:
+ type: object
+ description: >
+ The operation that was performed, such as creating or updating a
table.
+ Clients should discard events with unknown operation types.
+ discriminator:
+ propertyName: operation-type
+ mapping:
+ create-table: "#/components/schemas/CreateTableOperation"
+ register-table: "#/components/schemas/RegisterTableOperation"
+ drop-table: "#/components/schemas/DropTableOperation"
+ update-table: "#/components/schemas/UpdateTableOperation"
+ rename-table: "#/components/schemas/RenameTableOperation"
+ create-view: "#/components/schemas/CreateViewOperation"
+ drop-view: "#/components/schemas/DropViewOperation"
+ update-view: "#/components/schemas/UpdateViewOperation"
+ rename-view: "#/components/schemas/RenameTableOperation"
+ create-namespace: "#/components/schemas/CreateNamespaceOperation"
+ update-namespace-properties:
"#/components/schemas/UpdateNamespacePropertiesOperation"
+ drop-namespace: "#/components/schemas/DropNamespaceOperation"
+ custom: "#/components/schemas/CustomOperation"
+ oneOf:
+ - $ref: "#/components/schemas/CreateTableOperation"
+ - $ref: "#/components/schemas/RegisterTableOperation"
+ - $ref: "#/components/schemas/DropTableOperation"
+ - $ref: "#/components/schemas/UpdateTableOperation"
+ - $ref: "#/components/schemas/RenameTableOperation"
+ - $ref: "#/components/schemas/CreateViewOperation"
+ - $ref: "#/components/schemas/DropViewOperation"
+ - $ref: "#/components/schemas/UpdateViewOperation"
+ - $ref: "#/components/schemas/RenameTableOperation"
+ - $ref: "#/components/schemas/CreateNamespaceOperation"
+ - $ref: "#/components/schemas/UpdateNamespacePropertiesOperation"
+ - $ref: "#/components/schemas/DropNamespaceOperation"
+ - $ref: "#/components/schemas/CustomOperation"
+
+ CreateTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "create-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+
+ RegisterTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "register-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+
+ DropTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "drop-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ purge:
+ type: boolean
+ description: Whether purge flag was set
+
+ UpdateTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ - updates
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "update-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableUpdate"
+
+ RenameTableOperation:
+ allOf:
+ - $ref: "#/components/schemas/RenameTableRequest"
+ required:
+ - operation-type
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "rename-table"
+ table-uuid:
+ type: string
+ format: uuid
+
+ CreateViewOperation:
+ required:
+ - operation-type
+ - identifier
+ - view-uuid
+ - metadata
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "create-view"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ view-uuid:
+ type: string
+ format: uuid
+
+ DropViewOperation:
+ required:
+ - operation-type
+ - identifier
+ - view-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "drop-view"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ view-uuid:
+ type: string
+ format: uuid
+
+ UpdateViewOperation:
+ required:
+ - operation-type
+ - identifier
+ - view-uuid
+ - updates
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "update-view"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ view-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/ViewUpdate"
+
+ CreateNamespaceOperation:
+ allOf:
+ - $ref: "#/components/schemas/CreateNamespaceResponse"
+ required:
+ - operation-type
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "create-namespace"
+
+ UpdateNamespacePropertiesOperation:
+ allOf:
+ - $ref: "#/components/schemas/UpdateNamespacePropertiesResponse"
+ required:
+ - operation-type
+ - namespace
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "update-namespace-properties"
+ namespace:
+ $ref: "#/components/schemas/Namespace"
+
+ DropNamespaceOperation:
+ required:
+ - operation-type
+ - namespace
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "drop-namespace"
+ namespace:
+ $ref: "#/components/schemas/Namespace"
+
+ CustomOperation:
+ type: object
+ description: >
+ Extension point for catalog-specific operations not defined in the
standard.
+ required:
+ - operation-type
+ - custom-type
Review Comment:
We had this discussion also in the sync. I believe that having them
separated is better for a couple of reasons:
1. It is rare that a custom `operation-type` makes it to the spec unchanged.
It is even more pain if the standard now supports this new `operation-type` but
is incompatible with what the custom implementation offered before. Separating
custom operations into a separate Namespace solves this. Catalog
implementations can provide both event types for some time.
1. It is easy to identify what belongs to the standard and what not. While
it might be unexpected to see an additional standardized event (we might want
to alert on this and recommend a client update), we might want to treat missing
custom events differently.
1. Deserialization / Code generation is much easier. I already have my
`LakekeeperEvent` class that I can use to deserialize custom events, which are
provided to me cleanly by the outer wrapping class from the standard
implementation. One big Event is less extendable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
