laskoviymishka commented on code in PR #2612:
URL: https://github.com/apache/iceberg-go/pull/2612#discussion_r4596849130


##########
catalog/hive/hive.go:
##########
@@ -784,7 +782,7 @@ func (c *Catalog) DropView(ctx context.Context, identifier 
table.Identifier) err
                return fmt.Errorf("failed to drop view %s.%s: %w", database, 
viewName, err)
        }
 
-       fs, err := io.LoadFS(ctx, c.opts.props, metadataLocation)
+       fs, err := internal.LoadFSWithRegistry(ctx, c.opts.ioRegistry, 
c.opts.props, metadataLocation)

Review Comment:
   Should the registry tests cover 
[`DropView`](https://github.com/apache/iceberg-go/blob/ea12c1758df95793746efce9d5414704498fa494/catalog/hive/hive.go#L785)
 too? The Hive tests only run `LoadTable`, `CreateView` and `UpdateView` with a 
non-nil registry. If a call site like this one lost its registry, IO would 
quietly go to the global factory and the guard test wouldn't catch it. The same 
gap exists for Hive `RegisterTable`/`CommitTable`/`LoadView`, SQL `CommitTable` 
and Glue `RegisterTable`.



##########
catalog/glue/glue.go:
##########
@@ -201,11 +203,12 @@ func NewCatalog(opts ...Option) (*Catalog, error) {
        }
 
        cat := &Catalog{
-               name:      glueOps.name,
-               glueSvc:   glue.NewFromConfig(glueOps.awsConfig),
-               catalogId: catalogId,
-               awsCfg:    &glueOps.awsConfig,
-               props:     iceberg.Properties(glueOps.awsProperties),
+               glueSvc:    glue.NewFromConfig(glueOps.awsConfig),
+               catalogId:  catalogId,
+               awsCfg:     &glueOps.awsConfig,
+               props:      iceberg.Properties(glueOps.awsProperties),
+               ioRegistry: glueOps.ioRegistry,
+               name:       glueOps.name,

Review Comment:
   Could the cosmetic hunks come out so the diff stays on the registry? Here 
`name:` moved from first to last. Same for the removed blank line in hive 
`NewCatalogWithClient`, the reflowed `CreateViewWithIOProperties` signature, 
and the reworded `CreateStagedTableWithNamespaceKey` doc in 
`catalog/internal/utils.go`.



##########
catalog/rest/vended_creds.go:
##########
@@ -545,17 +546,13 @@ func (p *prefixScopedIO) Close() error {
 
 // loadRegistryFS preserves the global registry when none is configured.
 func loadRegistryFS(ctx context.Context, registry *iceio.Registry, props 
iceberg.Properties, location string) (iceio.IO, error) {
-       if registry == nil {
-               return iceio.LoadFS(ctx, props, location)
-       }
-
-       return registry.LoadFS(ctx, props, location)
+       return cataloginternal.LoadFSWithRegistry(ctx, registry, props, 
location)
 }
 
 // loadRegistryFSFunc preserves the global registry for lazy loads when none 
is configured.
 func loadRegistryFSFunc(registry *iceio.Registry, props iceberg.Properties, 
location string) func(context.Context) (iceio.IO, error) {
        return func(ctx context.Context) (iceio.IO, error) {
-               fs, err := loadRegistryFS(ctx, registry, props, location)
+               fs, err := cataloginternal.LoadFSWithRegistry(ctx, registry, 
props, location)

Review Comment:
   `loadRegistryFS` still has two callers (`:222`, `:416`), so this can keep 
going through it as on `main`:
   
   ```suggestion
                fs, err := loadRegistryFS(ctx, registry, props, location)
   ```



##########
view/view.go:
##########
@@ -147,45 +147,66 @@ func CreateView(
        return CreateViewWithIOProperties(ctx, catalogName, viewIdent, schema, 
representation, defaultNS, loc, props, props)
 }
 
+// CreateViewWithIORegistry writes view metadata through the supplied
+// registry. A nil registry keeps the global FileIO behavior. The registry is
+// shared, not copied, so later Register and Unregister calls affect this 
write.
+func CreateViewWithIORegistry(

Review Comment:
   Could these four `...Registry` twins become one functional option on the 
existing functions, e.g. `view.WithIORegistry(r)` passed variadically? Right 
now 
[view.go](https://github.com/apache/iceberg-go/blob/ea12c1758df95793746efce9d5414704498fa494/view/view.go#L150-L215)
 and 
[`WriteMetadataWithRegistry`](https://github.com/apache/iceberg-go/blob/ea12c1758df95793746efce9d5414704498fa494/view/write_metadata.go#L44-L60)
 add four exported names with three suffix styles. After merge, removing any of 
them is a break `apidiff` will flag, and the next knob would need a fifth twin.



##########
website/src/configuration.md:
##########
@@ -458,6 +458,10 @@ fs, err := registry.LoadFS(ctx, props, "myfs://bucket/key")
 
 `io.NewRegistry()` copies schemes already registered globally (including 
blank-imported cloud backends) but keeps later registrations independent. 
`registry.Clone()` makes an isolated copy of another registry. 
`registry.Names()` lists the schemes registered in that instance. 
`registry.LoadFSFunc()` is the lazy factory equivalent. Global `io.LoadFS()` 
and `io.LoadFSFunc()` continue using the default registry.
 
+To use the registry with a catalog, pass `WithIORegistry(registry)` to 
`glue.NewCatalog` or `hive.NewCatalog`; call `hadoop.NewCatalogWithOptions(..., 
hadoop.WithIORegistry(registry))` or `sql.NewCatalogWithOptions(..., 
sql.WithIORegistry(registry))` for Hadoop and SQL. Direct view metadata writes 
can use `view.CreateViewWithIORegistry` or 
`view.CreateViewWithIOPropertiesAndRegistry`. The default remains global when 
no registry is specified. An empty non-nil registry does **not** fall back to 
global schemes. REST catalog support is tracked in 
[#2366](https://github.com/apache/iceberg-go/pull/2366).

Review Comment:
   Could this list REST instead of pointing at #2366? That PR is already merged 
and 
[`rest.WithIORegistry`](https://github.com/apache/iceberg-go/blob/dc09cf00fcdd59caca9d98421693423608d17111/catalog/rest/options.go#L31-L35)
 is on `main`, so "tracked in" reads as pending. The [Hive and Glue option 
lists](https://github.com/apache/iceberg-go/blob/ea12c1758df95793746efce9d5414704498fa494/website/src/configuration.md?plain=1#L227-L252)
 also still omit `WithIORegistry`.
   
   ```suggestion
   To use the registry with a catalog, pass `WithIORegistry(registry)` to 
`glue.NewCatalog` or `hive.NewCatalog`, or `rest.WithIORegistry(registry)` for 
REST; call `hadoop.NewCatalogWithOptions(..., hadoop.WithIORegistry(registry))` 
or `sql.NewCatalogWithOptions(..., sql.WithIORegistry(registry))` for Hadoop 
and SQL. Direct view metadata writes can use `view.CreateViewWithIORegistry` or 
`view.CreateViewWithIOPropertiesAndRegistry`. The default remains global when 
no registry is specified. An empty non-nil registry does **not** fall back to 
global schemes.
   ```



##########
catalog/sql/sql.go:
##########
@@ -423,12 +425,21 @@ func (c *Catalog) isV0() bool { return c.schemaVersion == 
schemaV0 }
 // SQLite callers that allow concurrent writers should configure WAL and a 
busy timeout in the driver DSN so the
 // settings apply to every pooled connection. NewCatalog does not mutate 
connection settings on the caller-owned db.
 func NewCatalog(name string, db *sql.DB, dialect SupportedDialect, props 
iceberg.Properties) (*Catalog, error) {
+       return NewCatalogWithOptions(name, db, dialect, props)
+}
+
+// NewCatalogWithOptions builds a SQL catalog with optional isolated FileIO 
wiring.
+func NewCatalogWithOptions(name string, db *sql.DB, dialect SupportedDialect, 
props iceberg.Properties, opts ...Option) (*Catalog, error) {

Review Comment:
   Why a separate `NewCatalogWithOptions` rather than `NewCatalog(name, db, 
dialect, props, opts ...Option)`? Glue and Hive take `WithIORegistry` through 
their existing variadic `NewCatalog`, while 
[SQL](https://github.com/apache/iceberg-go/blob/ea12c1758df95793746efce9d5414704498fa494/catalog/sql/sql.go#L427-L432)
 and 
[Hadoop](https://github.com/apache/iceberg-go/blob/ea12c1758df95793746efce9d5414704498fa494/catalog/hadoop/hadoop.go#L206-L216)
 end up with two constructors each. Appending the variadic keeps every direct 
call compiling. It only breaks code that holds `sql.NewCatalog` as a typed func 
value.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to