gavin9402 opened a new pull request, #8179:
URL: https://github.com/apache/paimon/pull/8179
### Purpose
This PR introduces **Model Management** and **Resource Management**
capabilities to the Paimon REST Catalog, enabling Paimon to manage machine
learning models and external resources (files, JARs, archives, Python scripts)
as first-class catalog objects.
#### Resource Management
Adds a complete resource abstraction layer for managing external resources
within the Paimon catalog.
**Data Model** (`paimon-api`):
- **`Resource`** interface — core abstraction with `name`, `uri`, `size`,
`resourceType`, `lastModifiedTime`, and a factory method `toResource()` with
type-specific validation (e.g., JAR names must end with `.jar`)
- **`AbstractResource`** — shared base implementation
- **`FileResource`**, **`JarResource`**, **`PyResource`**,
**`ArchiveResource`** — typed resource implementations
- **`ResourceType`** enum — `FILE`, `ARCHIVE`, `JAR`, `PY`
- **`ResourceChange`** — change descriptors for alter operations
**REST API** (`paimon-api`):
- `RESTApi` — resource CRUD endpoints (create, get, list, list details,
alter, drop)
- `ResourcePaths` — REST URL path definitions
- Request/Response: `CreateResourceRequest`, `AlterResourceRequest`,
`GetResourceResponse`, `ListResourcesResponse`, `ListResourceDetailsResponse`
**Catalog Integration** (`paimon-core`):
- `Catalog` — 155 lines of new resource management methods (`listResources`,
`listResourcesPaged`, `getResource`, `createResource`, `alterResource`,
`dropResource`, etc.)
- `AbstractCatalog` — default `UnsupportedOperationException` stubs
- `DelegateCatalog` — delegation wrappers
- `RESTCatalog` — REST client implementation for all resource operations
#### Model Management
Adds a complete model management layer for ML models, model versions, and
model checkpoints.
**Data Model** (`paimon-api`):
- **`Model`** interface — `name`, `fullName`, `description`, `properties`,
`lastModifiedTime`
- **`ModelVersion`** interface — `version`, `alias`, `lastModifiedTime`
- **`ModelCheckpoint`** interface — checkpoint abstraction for model
training state
- **`ModelImpl`**, **`ModelVersionImpl`**, **`ModelCheckpointImpl`** —
concrete implementations
- **`ModelChange`** — change descriptors (set/remove properties, update
description, etc.)
**REST API** (`paimon-api`):
- `RESTApi` — model/version/checkpoint CRUD endpoints
- `ResourcePaths` — REST URL path definitions
- Requests: `CreateModelRequest`, `CreateModelVersionRequest`,
`CreateModelCheckpointRequest`, `AlterModelRequest`,
`SetModelVersionAliasRequest`
- Responses: `GetModelResponse`, `GetModelVersionResponse`,
`GetModelCheckpointResponse`, `ListModelsResponse`, `ListModelDetailsResponse`,
`ListModelVersionsResponse`, `ListModelCheckpointsResponse`
**Catalog Integration** (`paimon-core`):
- `Catalog` — 433 lines of new model management methods (`listModels`,
`getModel`, `createModel`, `alterModel`, `dropModel`, model versions, model
checkpoints, etc.)
- `AbstractCatalog`, `DelegateCatalog`, `RESTCatalog` — corresponding
implementations
#### Summary
| Module | Files | Description |
|--------|-------|-------------|
| `paimon-api` | 35 new files | Data models, REST API definitions,
request/response classes |
| `paimon-core` (main) | 4 modified files | Catalog interface &
implementations |
| `paimon-core` (test) | 4 new/modified files | Mock server, JSON tests,
integration tests |
| **Total** | **43 files** | **+5,594 lines** |
### Tests
- **`RESTCatalogTest`** (+483 lines) — Integration tests covering:
- Resource CRUD: create, get, list, list details, alter, drop
- Model CRUD: create, get, list, list details, alter, drop
- Model version: create, get, list, set alias, drop
- Model checkpoint: create, get, list, drop
- **`RESTApiJsonTest`** (+38 lines) — JSON serialization/deserialization
tests for resource request/response classes
- **`RESTCatalogServer`** (+799 lines) — Mock REST server implementation
supporting all new endpoints
- **`MockRESTMessage`** (+74 lines) — Test data builders for resource and
model objects
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
