stevenzwu commented on code in PR #12584:
URL: https://github.com/apache/iceberg/pull/12584#discussion_r2519483685
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3405,6 +3513,66 @@ components:
allOf:
- $ref: '#/components/schemas/ScanTasks'
+ QueryEventsRequest:
+ type: object
+ properties:
+ page-token:
+ $ref: "#/components/schemas/PageToken"
+ page-size:
+ type: integer
+ format: int32
+ description: >
+ The maximum number of events to return in a single response.
+ If not provided, the server may choose a default page size.
+ Servers may return less results than requested for various
reasons, such as
+ server side limits, payload size or processing time.
+ after-timestamp-ms:
+ type: integer
+ format: int64
+ description: >
+ The (server) timestamp in milliseconds to start consuming events
from (inclusive).
+ If not provided, the first available timestamp is used.
Review Comment:
> If not provided, the first available timestamp is used.
I found this description unclear. Removing it doesn't seem to decrease the
clarity. It is an additional filter (just like operation-types or
catalog-objects-by-name etc.).
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3405,6 +3513,66 @@ components:
allOf:
- $ref: '#/components/schemas/ScanTasks'
+ QueryEventsRequest:
+ type: object
+ properties:
+ page-token:
Review Comment:
nit: I know `page-token` and `continuation-token` often mean the same thing
in most context. Maybe it is just me. `page-token` implies a pagination through
a large static result (like list of tables). the next request with `page-token`
should get a new page from the same static result.
`continuation-token` is more natural with dynamically changing result (like
events api here). E.g., it could be a catalog server-side sequence number or
monotonically increasing clock.
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3964,6 +4132,332 @@ components:
metadata:
$ref: '#/components/schemas/TableMetadata'
+ EventsResponse:
+ type: object
+ required:
+ - highest-processed-timestamp-ms
+ - events
+ properties:
+ next-page-token:
+ $ref: "#/components/schemas/PageToken"
+ highest-processed-timestamp-ms:
+ description: >
+ The highest event timestamp processed when generating this
response.
+ This may not necessarily appear in the returned changes if it was
filtered out.
+ type: integer
+ format: int64
+ events:
+ type: array
+ items:
+ $ref: "#/components/schemas/Event"
+
+ Event:
+ type: object
+ required:
+ - event-id
+ - request-id
+ - event-count
+ - timestamp-ms
+ - operation
+ properties:
+ event-id:
+ type: string
+ description: Unique ID of this event. Clients should perform
deduplication based on this ID.
+ request-id:
+ description: >
+ Opaque ID of the request this change belongs to.
+ This ID can be used to identify events that were part of the same
request.
+ Servers generate this ID randomly.
+ type: string
+ event-count:
+ type: integer
+ description: >
+ Total number of events in this batch or request.
+ Some endpoints, such as "updateTable" and "commitTransaction", can
perform multiple updates in a single atomic request.
+ Each update is modeled as a separate event. All events generated
by the same request share the same `request-id`.
+ The `event-count` field indicates the total number of events
generated by that request.
+ timestamp-ms:
+ type: integer
+ format: int64
+ description: >
+ Timestamp when this event occurred (epoch milliseconds).
+ Timestamps are not guaranteed to be unique. Typically all events in
+ a transaction will have the same timestamp.
+ actor:
+ type: string
+ description: >
+ The actor who performed the operation, such as a user or service
account.
+ The content of this field is implementation specific.
+ operation:
+ type: object
+ description: >
+ The operation that was performed, such as creating or updating a
table.
+ Clients should discard events with unknown operation types.
+ discriminator:
+ propertyName: operation-type
+ mapping:
+ create-table: "#/components/schemas/CreateTableOperation"
+ register-table: "#/components/schemas/RegisterTableOperation"
+ drop-table: "#/components/schemas/DropTableOperation"
+ update-table: "#/components/schemas/UpdateTableOperation"
+ rename-table: "#/components/schemas/RenameTableOperation"
+ create-view: "#/components/schemas/CreateViewOperation"
+ drop-view: "#/components/schemas/DropViewOperation"
+ update-view: "#/components/schemas/UpdateViewOperation"
+ rename-view: "#/components/schemas/RenameViewOperation"
+ create-namespace: "#/components/schemas/CreateNamespaceOperation"
+ update-namespace-properties:
"#/components/schemas/UpdateNamespacePropertiesOperation"
+ drop-namespace: "#/components/schemas/DropNamespaceOperation"
+ custom: "#/components/schemas/CustomOperation"
+ oneOf:
+ - $ref: "#/components/schemas/CreateTableOperation"
+ - $ref: "#/components/schemas/RegisterTableOperation"
+ - $ref: "#/components/schemas/DropTableOperation"
+ - $ref: "#/components/schemas/UpdateTableOperation"
+ - $ref: "#/components/schemas/RenameTableOperation"
+ - $ref: "#/components/schemas/CreateViewOperation"
+ - $ref: "#/components/schemas/DropViewOperation"
+ - $ref: "#/components/schemas/UpdateViewOperation"
+ - $ref: "#/components/schemas/RenameViewOperation"
+ - $ref: "#/components/schemas/CreateNamespaceOperation"
+ - $ref: "#/components/schemas/UpdateNamespacePropertiesOperation"
+ - $ref: "#/components/schemas/DropNamespaceOperation"
+ - $ref: "#/components/schemas/CustomOperation"
+
+ CreateTableOperation:
+ description: >
+ Operation to create a new table in the catalog.
+ Events for this Operation must be issued when the create is finalized
and committed, not when the create is staged.
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ - updates
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "create-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableUpdate"
+
+ RegisterTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "register-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableUpdate"
+
+ DropTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "drop-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ purge:
+ type: boolean
+ description: Whether purge flag was set
+
+ UpdateTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ - updates
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "update-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableUpdate"
+ requirements:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableRequirement"
+
+ RenameTableOperation:
+ allOf:
+ - $ref: "#/components/schemas/RenameTableRequest"
+ required:
+ - operation-type
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "rename-table"
+ table-uuid:
+ type: string
+ format: uuid
+
+ RenameViewOperation:
+ allOf:
+ - $ref: "#/components/schemas/RenameTableRequest"
+ required:
+ - operation-type
+ - view-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "rename-view"
+ view-uuid:
+ type: string
+ format: uuid
+
+ CreateViewOperation:
+ required:
+ - operation-type
+ - identifier
+ - view-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "create-view"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ view-uuid:
+ type: string
+ format: uuid
+
+ DropViewOperation:
+ required:
+ - operation-type
+ - identifier
+ - view-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "drop-view"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ view-uuid:
+ type: string
+ format: uuid
+
+ UpdateViewOperation:
+ required:
+ - operation-type
+ - identifier
+ - view-uuid
+ - updates
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "update-view"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ view-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/ViewUpdate"
+ requirements:
+ type: array
+ items:
+ $ref: "#/components/schemas/ViewRequirement"
+
+ CreateNamespaceOperation:
+ allOf:
+ - $ref: "#/components/schemas/CreateNamespaceResponse"
Review Comment:
curious why the response (not request) is used here? is it for the
additional props like `created_at`?
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3405,6 +3490,101 @@ components:
allOf:
- $ref: '#/components/schemas/ScanTasks'
+ GetEventsRequest:
+ type: object
+ properties:
+ page-token:
+ $ref: "#/components/schemas/PageToken"
+ page-size:
+ type: integer
+ format: int32
+ description: >
+ The maximum number of events to return in a single response.
+ If not provided, the server may choose a default page size.
+ Servers may return less results than requested for various
reasons, such as
+ server side limits, payload size or processing time.
+ after-timestamp-ms:
+ type: integer
+ format: int64
+ description: >
+ The (server) timestamp in milliseconds to start consuming events
from (inclusive).
+ If not provided, the first available timestamp is used.
+ operation-types:
+ type: array
+ items:
+ $ref: "#/components/schemas/OperationType"
+ description: >
+ Filter events by the type of operation.
+ If not provided, all types are returned.
+ catalog-objects:
+ type: array
+ discriminator:
+ propertyName: reference-type
+ mapping:
+ namespace: "#/components/schemas/NamespaceReference"
+ table: "#/components/schemas/TableReference"
+ view: "#/components/schemas/ViewReference"
+ items:
+ oneOf:
+ - $ref: "#/components/schemas/NamespaceReference"
+ - $ref: "#/components/schemas/TableReference"
+ - $ref: "#/components/schemas/ViewReference"
+ description: >
+ List of catalog objects (namespaces, tables, views) to get events
for.
+ If not provided, events for all objects will be returned subject
to other filters.
+ For specified namespaces, events for the namespaces and all
containing objects
+ (namespaces, tables, views) will be returned.
+ custom-filters:
Review Comment:
Do we already have custom filter use cases in mind?
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3405,6 +3513,66 @@ components:
allOf:
- $ref: '#/components/schemas/ScanTasks'
+ QueryEventsRequest:
+ type: object
+ properties:
+ page-token:
+ $ref: "#/components/schemas/PageToken"
+ page-size:
+ type: integer
+ format: int32
+ description: >
+ The maximum number of events to return in a single response.
+ If not provided, the server may choose a default page size.
+ Servers may return less results than requested for various
reasons, such as
+ server side limits, payload size or processing time.
+ after-timestamp-ms:
+ type: integer
+ format: int64
+ description: >
+ The (server) timestamp in milliseconds to start consuming events
from (inclusive).
+ If not provided, the first available timestamp is used.
+ operation-types:
+ type: array
+ items:
+ $ref: "#/components/schemas/OperationType"
+ description: >
+ Filter events by the type of operation.
+ If not provided, all types are returned.
+ catalog-objects-by-name:
+ type: array
+ items:
+ $ref: "#/components/schemas/CatalogObject"
+ description: >
+ List of catalog objects (namespaces, tables, views) to get events
for.
Review Comment:
> List of catalog objects (namespaces, tables, views) to get events for.
nit: maybe be consistent as other filters, like
```
Filter events by the list of catalog objects referenced by name (namespaces,
tables, views).
```
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3964,6 +4132,332 @@ components:
metadata:
$ref: '#/components/schemas/TableMetadata'
+ EventsResponse:
+ type: object
+ required:
+ - highest-processed-timestamp-ms
+ - events
+ properties:
+ next-page-token:
+ $ref: "#/components/schemas/PageToken"
+ highest-processed-timestamp-ms:
+ description: >
+ The highest event timestamp processed when generating this
response.
+ This may not necessarily appear in the returned changes if it was
filtered out.
+ type: integer
+ format: int64
+ events:
+ type: array
+ items:
+ $ref: "#/components/schemas/Event"
+
+ Event:
+ type: object
+ required:
+ - event-id
+ - request-id
+ - event-count
+ - timestamp-ms
+ - operation
+ properties:
+ event-id:
+ type: string
+ description: Unique ID of this event. Clients should perform
deduplication based on this ID.
+ request-id:
+ description: >
+ Opaque ID of the request this change belongs to.
+ This ID can be used to identify events that were part of the same
request.
+ Servers generate this ID randomly.
+ type: string
+ event-count:
+ type: integer
+ description: >
+ Total number of events in this batch or request.
+ Some endpoints, such as "updateTable" and "commitTransaction", can
perform multiple updates in a single atomic request.
+ Each update is modeled as a separate event. All events generated
by the same request share the same `request-id`.
+ The `event-count` field indicates the total number of events
generated by that request.
+ timestamp-ms:
+ type: integer
+ format: int64
+ description: >
+ Timestamp when this event occurred (epoch milliseconds).
+ Timestamps are not guaranteed to be unique. Typically all events in
+ a transaction will have the same timestamp.
+ actor:
+ type: string
+ description: >
+ The actor who performed the operation, such as a user or service
account.
+ The content of this field is implementation specific.
+ operation:
+ type: object
+ description: >
+ The operation that was performed, such as creating or updating a
table.
+ Clients should discard events with unknown operation types.
+ discriminator:
+ propertyName: operation-type
+ mapping:
+ create-table: "#/components/schemas/CreateTableOperation"
+ register-table: "#/components/schemas/RegisterTableOperation"
+ drop-table: "#/components/schemas/DropTableOperation"
+ update-table: "#/components/schemas/UpdateTableOperation"
+ rename-table: "#/components/schemas/RenameTableOperation"
+ create-view: "#/components/schemas/CreateViewOperation"
+ drop-view: "#/components/schemas/DropViewOperation"
+ update-view: "#/components/schemas/UpdateViewOperation"
+ rename-view: "#/components/schemas/RenameViewOperation"
+ create-namespace: "#/components/schemas/CreateNamespaceOperation"
+ update-namespace-properties:
"#/components/schemas/UpdateNamespacePropertiesOperation"
+ drop-namespace: "#/components/schemas/DropNamespaceOperation"
+ custom: "#/components/schemas/CustomOperation"
+ oneOf:
+ - $ref: "#/components/schemas/CreateTableOperation"
+ - $ref: "#/components/schemas/RegisterTableOperation"
+ - $ref: "#/components/schemas/DropTableOperation"
+ - $ref: "#/components/schemas/UpdateTableOperation"
+ - $ref: "#/components/schemas/RenameTableOperation"
+ - $ref: "#/components/schemas/CreateViewOperation"
+ - $ref: "#/components/schemas/DropViewOperation"
+ - $ref: "#/components/schemas/UpdateViewOperation"
+ - $ref: "#/components/schemas/RenameViewOperation"
+ - $ref: "#/components/schemas/CreateNamespaceOperation"
+ - $ref: "#/components/schemas/UpdateNamespacePropertiesOperation"
+ - $ref: "#/components/schemas/DropNamespaceOperation"
+ - $ref: "#/components/schemas/CustomOperation"
+
+ CreateTableOperation:
+ description: >
+ Operation to create a new table in the catalog.
+ Events for this Operation must be issued when the create is finalized
and committed, not when the create is staged.
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ - updates
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "create-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableUpdate"
Review Comment:
so CreateTable should be modeled as a list of updates (like AddSchemaUpdate,
AddPartitionSpecUpdate, AddSortOrderUpdate)? wondering why do we use
`CreateTableRequest` here like the `RenameTableOperation`?
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -2010,11 +2065,34 @@ components:
Namespace:
description: Reference to one or more levels of a namespace
+ allOf:
Review Comment:
I thought maybe we don't need to modify the `Namespace` at all.
Instead, the new `CataloagObject` can be identified either by name or by id
via `anyOf`. It can have two subtypes `CatalogObjectByName` and
`CatalogObjectByUuid`.
`CatalogObjectByName` can allow all 3 types of objects (namespace, table,
view). `CatalogObjectByUuid` only allows 2 types (table, view).
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3405,6 +3513,66 @@ components:
allOf:
- $ref: '#/components/schemas/ScanTasks'
+ QueryEventsRequest:
+ type: object
+ properties:
+ page-token:
+ $ref: "#/components/schemas/PageToken"
+ page-size:
+ type: integer
+ format: int32
+ description: >
+ The maximum number of events to return in a single response.
+ If not provided, the server may choose a default page size.
+ Servers may return less results than requested for various
reasons, such as
+ server side limits, payload size or processing time.
+ after-timestamp-ms:
+ type: integer
+ format: int64
+ description: >
+ The (server) timestamp in milliseconds to start consuming events
from (inclusive).
+ If not provided, the first available timestamp is used.
+ operation-types:
+ type: array
+ items:
+ $ref: "#/components/schemas/OperationType"
+ description: >
+ Filter events by the type of operation.
+ If not provided, all types are returned.
+ catalog-objects-by-name:
+ type: array
+ items:
+ $ref: "#/components/schemas/CatalogObject"
+ description: >
+ List of catalog objects (namespaces, tables, views) to get events
for.
+ If not provided, events for all objects must be returned subject
to other filters.
+ For specified namespaces, events for the namespaces and all
containing objects
+ (namespaces, tables, views) must be returned (recursively).
+ catalog-objects-by-id:
+ type: array
+ items:
+ $ref: "#/components/schemas/CatalogObjectUuid"
+ description: >
+ List of catalog objects tables and views to get events for by
their UUID.
Review Comment:
> List of catalog objects tables and views to get events for by their UUID.
nit: maybe be consistent as other filters, like
```
Filter events by the list of catalog objects referenced by UUID (tables,
views).
```
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3964,6 +4132,332 @@ components:
metadata:
$ref: '#/components/schemas/TableMetadata'
+ EventsResponse:
+ type: object
+ required:
+ - highest-processed-timestamp-ms
+ - events
+ properties:
+ next-page-token:
+ $ref: "#/components/schemas/PageToken"
+ highest-processed-timestamp-ms:
+ description: >
+ The highest event timestamp processed when generating this
response.
+ This may not necessarily appear in the returned changes if it was
filtered out.
+ type: integer
+ format: int64
+ events:
+ type: array
+ items:
+ $ref: "#/components/schemas/Event"
+
+ Event:
+ type: object
+ required:
+ - event-id
+ - request-id
+ - event-count
+ - timestamp-ms
+ - operation
+ properties:
+ event-id:
+ type: string
+ description: Unique ID of this event. Clients should perform
deduplication based on this ID.
+ request-id:
+ description: >
+ Opaque ID of the request this change belongs to.
+ This ID can be used to identify events that were part of the same
request.
+ Servers generate this ID randomly.
+ type: string
+ event-count:
+ type: integer
+ description: >
+ Total number of events in this batch or request.
+ Some endpoints, such as "updateTable" and "commitTransaction", can
perform multiple updates in a single atomic request.
+ Each update is modeled as a separate event. All events generated
by the same request share the same `request-id`.
+ The `event-count` field indicates the total number of events
generated by that request.
+ timestamp-ms:
+ type: integer
+ format: int64
+ description: >
+ Timestamp when this event occurred (epoch milliseconds).
+ Timestamps are not guaranteed to be unique. Typically all events in
+ a transaction will have the same timestamp.
+ actor:
+ type: string
+ description: >
+ The actor who performed the operation, such as a user or service
account.
+ The content of this field is implementation specific.
+ operation:
+ type: object
+ description: >
+ The operation that was performed, such as creating or updating a
table.
+ Clients should discard events with unknown operation types.
+ discriminator:
+ propertyName: operation-type
+ mapping:
+ create-table: "#/components/schemas/CreateTableOperation"
+ register-table: "#/components/schemas/RegisterTableOperation"
+ drop-table: "#/components/schemas/DropTableOperation"
+ update-table: "#/components/schemas/UpdateTableOperation"
+ rename-table: "#/components/schemas/RenameTableOperation"
+ create-view: "#/components/schemas/CreateViewOperation"
+ drop-view: "#/components/schemas/DropViewOperation"
+ update-view: "#/components/schemas/UpdateViewOperation"
+ rename-view: "#/components/schemas/RenameViewOperation"
+ create-namespace: "#/components/schemas/CreateNamespaceOperation"
+ update-namespace-properties:
"#/components/schemas/UpdateNamespacePropertiesOperation"
+ drop-namespace: "#/components/schemas/DropNamespaceOperation"
+ custom: "#/components/schemas/CustomOperation"
+ oneOf:
+ - $ref: "#/components/schemas/CreateTableOperation"
+ - $ref: "#/components/schemas/RegisterTableOperation"
+ - $ref: "#/components/schemas/DropTableOperation"
+ - $ref: "#/components/schemas/UpdateTableOperation"
+ - $ref: "#/components/schemas/RenameTableOperation"
+ - $ref: "#/components/schemas/CreateViewOperation"
+ - $ref: "#/components/schemas/DropViewOperation"
+ - $ref: "#/components/schemas/UpdateViewOperation"
+ - $ref: "#/components/schemas/RenameViewOperation"
+ - $ref: "#/components/schemas/CreateNamespaceOperation"
+ - $ref: "#/components/schemas/UpdateNamespacePropertiesOperation"
+ - $ref: "#/components/schemas/DropNamespaceOperation"
+ - $ref: "#/components/schemas/CustomOperation"
+
+ CreateTableOperation:
+ description: >
+ Operation to create a new table in the catalog.
+ Events for this Operation must be issued when the create is finalized
and committed, not when the create is staged.
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ - updates
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "create-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableUpdate"
+
+ RegisterTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "register-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableUpdate"
+
+ DropTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "drop-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ purge:
+ type: boolean
+ description: Whether purge flag was set
+
+ UpdateTableOperation:
+ required:
+ - operation-type
+ - identifier
+ - table-uuid
+ - updates
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "update-table"
+ identifier:
+ $ref: "#/components/schemas/TableIdentifier"
+ table-uuid:
+ type: string
+ format: uuid
+ updates:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableUpdate"
+ requirements:
+ type: array
+ items:
+ $ref: "#/components/schemas/TableRequirement"
+
+ RenameTableOperation:
+ allOf:
+ - $ref: "#/components/schemas/RenameTableRequest"
+ required:
+ - operation-type
+ - table-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "rename-table"
+ table-uuid:
+ type: string
+ format: uuid
+
+ RenameViewOperation:
+ allOf:
+ - $ref: "#/components/schemas/RenameTableRequest"
+ required:
+ - operation-type
+ - view-uuid
+ properties:
+ operation-type:
+ $ref: "#/components/schemas/OperationType"
+ const: "rename-view"
+ view-uuid:
+ type: string
+ format: uuid
+
+ CreateViewOperation:
Review Comment:
we don't need to include `CreateViewRequest` info here, like `ViewVersion`?
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3964,6 +4132,332 @@ components:
metadata:
$ref: '#/components/schemas/TableMetadata'
+ EventsResponse:
+ type: object
+ required:
+ - highest-processed-timestamp-ms
+ - events
+ properties:
+ next-page-token:
+ $ref: "#/components/schemas/PageToken"
+ highest-processed-timestamp-ms:
+ description: >
+ The highest event timestamp processed when generating this
response.
+ This may not necessarily appear in the returned changes if it was
filtered out.
+ type: integer
+ format: int64
+ events:
+ type: array
+ items:
+ $ref: "#/components/schemas/Event"
+
+ Event:
+ type: object
+ required:
+ - event-id
+ - request-id
+ - event-count
+ - timestamp-ms
+ - operation
+ properties:
+ event-id:
+ type: string
+ description: Unique ID of this event. Clients should perform
deduplication based on this ID.
+ request-id:
+ description: >
+ Opaque ID of the request this change belongs to.
+ This ID can be used to identify events that were part of the same
request.
+ Servers generate this ID randomly.
+ type: string
+ event-count:
Review Comment:
My initial reaction to `event-count` (without looking through the
description) is the total number of events in the response payload. Maybe this
should be called `request-event-count` to be more clear about its scope.
Can events for the same request be split in two responses? Do we need to
specify that all events for the same request should be contained in one
response? otherwise, the catalog federation/replication may break the
transaction semantics.
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3964,6 +4144,292 @@ components:
metadata:
$ref: '#/components/schemas/TableMetadata'
+ EventsResponse:
+ type: object
+ required:
+ - highest-processed-timestamp-ms
+ - events
+ properties:
+ next-page-token:
+ $ref: "#/components/schemas/PageToken"
+ highest-processed-timestamp-ms:
Review Comment:
Did I misunderstand the purpose of `next-page-token`? Is it only for
pagination of one query checkpoint?
Is the timestamp meant for the query checkpoint? Some catalogs may implement
a [global catalog sequence number
](https://docs.google.com/document/d/1jr4Ah8oceOmo6fwxG_0II4vKDUHUKScb/edit)to
order all catalog changes, which would be a great fit for the query checkpoint
for continuation.
If we use a timestamp as the continuation point, it would requires server to
implement a monotonically increasing clock for this to work correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
