richiebachala opened a new pull request, #2376:
URL: https://github.com/apache/iceberg-go/pull/2376
Part of #2044. Thanks @slachiewicz for the proposal.
## What
`catalog/rest/resttest` is an in-memory Iceberg REST catalog `http.Handler`.
The `catalog/rest` client can drive it end to end (create namespace, create
table, commit, load), and it writes metadata files under a warehouse directory.
With it, `catalogtest.RunCatalogTests` runs against REST in plain `go test`.
Today `TestRestCatalogConformance` needs the Docker fixture behind `//go:build
integration`.
```go
srv := resttest.NewServer(t) // warehouse defaults to t.TempDir()
cat, err := rest.NewCatalog(t.Context(), "test", srv.URL)
```
The exported API is only constructors and options, as @laskoviymishka
suggested on #2044: `NewHandler(opts...)`, `NewServer(t, opts...)`,
`WithWarehouse`, `WithPrefix` and `WithProperties`. Nothing in `catalog/rest`
is exported or changed. The server defines its own wire structs. No new
dependencies.
## Scope
- **Config:** `GET /v1/config` returns defaults and overrides, plus
`endpoints`. `endpoints` includes the HEAD routes, so `CheckNamespaceExists`
and `CheckTableExists` use them. `WithPrefix` is returned as the `prefix`
override, so the client picks it up from config.
- **Namespaces:** list (with `parent`), create, load, exists, drop (409 when
not empty) and update properties.
- **Tables:**
- Create: location comes from the namespace `location` property, otherwise
`<warehouse>/<ns>.db/<table>`. `stage-create` is supported.
- Load, exists, list, drop, rename and register are supported.
- **Commit:** runs under a lock. It validates requirements against the
current metadata, writes the next metadata file and swaps the pointer. A
requirement that no longer holds returns 409, so the client reports
`table.ErrCommitFailed`. A commit to a missing table returns 404 unless it
carries `assert-create`, so a commit after a drop doesn't re-create the table.
The server reuses the `catalog/internal` helpers the SQL catalog uses:
`CreateStagedTable`, `UpdateAndStageTable`, `WriteMetadata` and
`GetUpdatedPropsAndUpdateSummary`.
- **Not implemented:** views, `transactions/commit`, scan planning,
credential vending and metrics. They aren't advertised, and they return 501
`NotImplementedException`. Listing returns a single page, and a purge drop
leaves files in place.
## Tests
- `TestConformance` and `TestConformanceWithPrefix` run the full
`catalogtest` suite with namespace properties and rename enabled: 75 subtests,
none skipped.
- Mutation check: I made a stale commit return 500 instead of 409, and
`CommitWithStaleRequirement` failed. That confirms the suite exercises the
server.
- Focused tests:
- conflicting schema commits from the same base;
- 8 concurrent property writers with no lost update;
- HEAD exists-checks;
- register round-trip;
- 501 for unsupported routes;
- create without a warehouse.
- Commands run:
- `go test -race ./catalog/rest/... ./catalog/internal/...
./catalog/sql/...`
- `go vet ./catalog/...` and `go vet -tags integration ./catalog/rest/...`
- golangci-lint v2.14.0 (the Makefile pin): clean.
## Docs
Short "Testing against an in-process REST catalog" section under REST in
`website/src/configuration.md`.
## Follow-ups (separate PRs)
- Views lifecycle and `transactions/commit`.
- Cross-catalog `catalogtest` cases for the #2100 contract:
- a terminal requirement such as a tag target does not match
`ErrCommitFailed`;
- a stale append lands with retries on.
- Pagination and fault injection, such as forcing a 409 or 503 on the next
commit.
Generated with [Snowflake
CoCo](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
