C-Loftus opened a new pull request, #1268: URL: https://github.com/apache/iceberg-go/pull/1268
Next step for #1096 This PR allows clients to opt in to use the hadoop catalog with arbitrary schemes. However, they have to explicitly set `allow-unsafe-commits`. I followed the guidance on this as specified here: https://github.com/apache/iceberg-go/issues/1096#issuecomment-4656700687 Note that this allows any arbitrary scheme. It is unclear if this should be constrained to just blob stores? Note, I was going to add an integration test for minio with hadoop but as far as I can tell, I can't get the integration test runner to work for me. Unclear if I am doing something wrong or if we maybe need to make a change to this to use relative paths or something analogous? (I see the Dockerfile is there) Note that this is the current integration test file without any changes to the runner. ``` > make integration-hadoop go test -tags=integration -v -run="^TestHadoopIntegration" ./catalog/hadoop/... === RUN TestHadoopIntegration 2026/06/23 12:18:27 github.com/testcontainers/testcontainers-go - Connected to docker: Server Version: 29.1.3 API Version: 1.52 Operating System: Docker Desktop Total Memory: 7836 MB Labels: com.docker.desktop.address=unix:///Users/cloftus/Library/Containers/com.docker.docker/Data/docker-cli.sock Testcontainers for Go Version: v0.42.0 Resolved Docker Host: unix:///var/run/docker.sock Resolved Docker Socket Path: /var/run/docker.sock Test SessionID: b94c401c54d8169c5101c563021f3b088636ea1765586a79ac332f17f6fff9fc Test ProcessID: 7e1b2a5e-9f59-4ff3-a4bf-7b3afef90979 #1 [internal] load local bake definitions #1 reading from stdin 642B done #1 DONE 0.0s #2 [internal] load build definition from Dockerfile #2 transferring dockerfile: 2B done #2 DONE 0.0s hadoop_integration_test.go:58: Error Trace: /Users/cloftus/github/iceberg-go/catalog/hadoop/hadoop_integration_test.go:58 /Users/cloftus/go/pkg/mod/github.com/stretchr/[email protected]/suite/suite.go:211 /Users/cloftus/github/iceberg-go/catalog/hadoop/hadoop_integration_test.go:315 Error: Received unexpected error: fail to up compose: compose up: failed to solve: failed to read dockerfile: open Dockerfile: no such file or directory View build details: docker-desktop://dashboard/build/default/default/n0tmfi552z84ltchd6clbeftd Test: TestHadoopIntegration --- FAIL: TestHadoopIntegration (0.99s) FAIL FAIL github.com/apache/iceberg-go/catalog/hadoop 2.042s FAIL make: *** [integration-hadoop] Error 1 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
