This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e3fd64b96765 [SPARK-46525][DOCKER][TESTS][FOLLOWUP] Fix
docker-integration-tests on Apple Silicon for db2 and oracle with third-party
docker environments
e3fd64b96765 is described below
commit e3fd64b96765dc04994798e79161026bd1361230
Author: Kent Yao <y...@apache.org>
AuthorDate: Wed Jan 17 07:19:06 2024 -0800
[SPARK-46525][DOCKER][TESTS][FOLLOWUP] Fix docker-integration-tests on
Apple Silicon for db2 and oracle with third-party docker environments
### What changes were proposed in this pull request?
SPARK-46525 makes docker-integration-tests pass most tests on Apple Silicon
except DB2 and Oracle. This PR modifies DockerJDBCIntegrationSuite to make it
compatible with some of the third-party docker environments, such as the Colima
docker environment. Developers can quickly bring up these tests for local
testing after a simple pre-setup process.
### Why are the changes needed?
Make it possible to test and debug locally for developers on Apple Silicon
platforms.
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
- Passed OracleIntegrationSuite on my Apple Silicon Mac locally
```
[info] OracleIntegrationSuite:
[info] - SPARK-33034: ALTER TABLE ... add new columns (21 seconds, 839
milliseconds)
[info] ....
[info] Run completed in 3 minutes, 16 seconds.
[info] Total number of tests run: 26
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 26, failed 0, canceled 0, ignored 11, pending 0
[info] All tests passed.
[success] Total time: 326 s (05:26), completed Jan 5, 2024, 7:10:36 PM
```
- Containers ran normally on my Apple Silicon Mac locally during and after
tests above
```
5ea7cc54165b ibmcom/db2:11.5.6.0a "/var/db2_setup/lib/…" 36
minutes ago Up 36 minutes 22/tcp, 55000/tcp, 60006-60007/tcp,
0.0.0.0:57898->50000/tcp, :::57898->50000/tcp strange_ritchie
d31122b8a504 gvenzl/oracle-free:23.3 "container-entrypoin…" About an
hour ago Up About an hour 0.0.0.0:64193->1521/tcp, :::64193->1521/tcp
priceless_wright
75f9943fd4b6 mariadb:10.5.12 "/docker-entrypoint/…" 2 hours
ago Up 2 hours 0.0.0.0:55052->3306/tcp, :::55052->3306/tcp
angry_ganguly
```
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #44612 from yaooqinn/SPARK-46525-F.
Authored-by: Kent Yao <y...@apache.org>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala | 3 ++-
.../test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala | 2 +-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git
a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala
b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala
index fde228939dd4..4b5820c2302f 100644
---
a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala
+++
b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala
@@ -156,7 +156,7 @@ abstract class DockerJDBCIntegrationSuite
.newHostConfig()
.withNetworkMode("bridge")
.withPrivileged(db.privileged)
-
.withPortBindings(PortBinding.parse(s"$dockerIp:$externalPort:${db.jdbcPort}"))
+ .withPortBindings(PortBinding.parse(s"$externalPort:${db.jdbcPort}"))
if (db.usesIpc) {
hostConfig.withIpcMode("host")
@@ -197,6 +197,7 @@ abstract class DockerJDBCIntegrationSuite
}
} catch {
case NonFatal(e) =>
+ logError(s"Failed to initialize Docker container for
${this.getClass.getName}", e)
try {
afterAll()
} finally {
diff --git
a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala
b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala
index 98d57d6a7475..fa563ae54a96 100644
---
a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala
+++
b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala
@@ -57,7 +57,7 @@ class OracleDatabaseOnDocker extends DatabaseOnDocker with
Logging {
val newBind = new Bind(
dir.getAbsolutePath,
new Volume("/docker-entrypoint-initdb.d"),
- AccessMode.ro)
+ AccessMode.DEFAULT)
hostConfigBuilder.withBinds(hostConfigBuilder.getBinds :+ newBind: _*)
} catch {
case e: Exception =>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
