David-Banquet opened a new issue, #2161:
URL: https://github.com/apache/iceberg-go/issues/2161
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
When a table location is a local path that contains a space, iceberg-go
writes the data files, manifests and manifest lists to a sibling directory
where the space is replaced by `%20`. The metadata JSON files go to the real
directory. Readers that follow the recorded paths still work, but a reader that
resolves files relative to the table location does not find them.
`LoadLocationProvider` in `table/locations.go` parses the location with
`url.Parse`, and `NewDataLocation` and `NewMetadataLocation` return
`JoinPath(...).String()`. For a location without a scheme, `String()` escapes
the space. `io.LocalFS` only decodes `file://` URIs, so it then uses the
escaped string as a plain path and creates the `%20` directory.
Reproduced with v0.7.0 and with `main` at `b7232461d6`:
```go
package main
import (
"fmt"
"os"
"path/filepath"
"github.com/apache/iceberg-go/io"
"github.com/apache/iceberg-go/table"
)
func main() {
base, err := os.MkdirTemp("", "iceberg-go-1177")
if err != nil {
panic(err)
}
defer os.RemoveAll(base)
loc := filepath.Join(base, "my table")
lp, err := table.LoadLocationProvider(loc, nil)
if err != nil {
panic(err)
}
dataPath := lp.NewDataLocation("00000.parquet")
fmt.Println("table location:", loc)
fmt.Println("data location: ", dataPath)
f, err := io.LocalFS{}.Create(dataPath)
if err != nil {
panic(err)
}
f.Close()
_, err = os.Stat(filepath.Join(loc, "data", "00000.parquet"))
fmt.Println("file under the table location:", err == nil)
_, err = os.Stat(filepath.Join(base, "my%20table", "data",
"00000.parquet"))
fmt.Println("file under my%20table: ", err == nil)
}
```
Output, with the temp directory shown as `$TMP`:
```
table location: $TMP/my table
data location: $TMP/my%20table/data/00000.parquet
file under the table location: false
file under my%20table: true
```
I expected `NewDataLocation` to return `.../my table/data/00000.parquet` and
the file to be created under the table location.
The same happens with a full write through the Hadoop catalog: the
`metadata.json` files are under the real table directory, while the snapshot's
`manifest-list`, the manifest and the Parquet file are under the `%20`
directory. This came up in duckdb/duckdb-iceberg#1177. With `allow_moved_paths
= true`, DuckDB rebases the manifest list path onto the table location and the
file is not there.
Tested on macOS 26 (arm64) with Go 1.27.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
