This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 6b33796d0df [SPARK-42046][TESTS] Add `connect-client-jvm` to `connect`
module and fix port failure
6b33796d0df is described below
commit 6b33796d0df152d4acd52c1237de69d59b94d1fc
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Fri Jan 13 20:13:11 2023 +0900
[SPARK-42046][TESTS] Add `connect-client-jvm` to `connect` module and fix
port failure
### What changes were proposed in this pull request?
This PR aims to add `connect-client-jvm` module to `connect` module.
### Why are the changes needed?
Currently, `connect-client-jvm` are not tested in CI because we only invoke
the following for `connect` module.
https://github.com/apache/spark/blob/5e0a82e590d1c3c3c5fa7f347dddf450fabbf772/.github/workflows/build_and_test.yml#L155
And, it fails in GitHub Action environment.
```
[info] - Check URI: sc://localhost:123/, isCorrect: true *** FAILED *** (6
milliseconds)
[info] java.io.IOException: Failed to bind to address 0.0.0.0/0.0.0.0:123
...
Cause: java.net.SocketException: Permission denied
[info] at sun.nio.ch.Net.bind0(Native Method)
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs and check `SparkConnectClientSuite` from the log.
```
$ dev/run-tests.py -m connect
...
[info] SparkConnectClientSuite:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
[info] - Placeholder test: Create SparkConnectClient (100 milliseconds)
[info] - Test connection (114 milliseconds)
[info] - Test connection string (6 milliseconds)
[info] - Check URI: sc://host, isCorrect: true (0 milliseconds)
[info] - Check URI: sc://localhost/, isCorrect: true (4 milliseconds)
[info] - Check URI: sc://localhost:123/, isCorrect: true (12 milliseconds)
[info] - Check URI: sc://localhost/;, isCorrect: true (4 milliseconds)
[info] - Check URI: sc://host:123, isCorrect: true (1 millisecond)
[info] - Check URI: sc://host:123/;user_id=a94, isCorrect: true (1
millisecond)
[info] - Check URI: scc://host:12, isCorrect: false (1 millisecond)
[info] - Check URI: http://host, isCorrect: false (0 milliseconds)
[info] - Check URI: sc:/host:1234/path, isCorrect: false (0 milliseconds)
[info] - Check URI: sc://host/path, isCorrect: false (1 millisecond)
[info] - Check URI: sc://host/;parm1;param2, isCorrect: false (0
milliseconds)
[info] - Check URI: sc://host:123;user_id=a94, isCorrect: false (0
milliseconds)
[info] - Check URI: sc:///user_id=123, isCorrect: false (0 milliseconds)
[info] - Check URI: sc://host:-4, isCorrect: false (0 milliseconds)
[info] - Check URI: sc://:123/, isCorrect: false (0 milliseconds)
[info] - Non user-id parameters throw unsupported errors (1 millisecond)
[info] Run completed in 1 second, 75 milliseconds.
[info] Total number of tests run: 19
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 19, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 1 s, completed Jan 12, 2023, 10:10:49 PM
```
Closes #39549 from dongjoon-hyun/SPARK-42046.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.../org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala | 4 ++--
dev/sparktestsupport/modules.py | 1 +
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git
a/connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala
b/connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala
index ea810a11272..5dd5f0e2502 100644
---
a/connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala
+++
b/connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala
@@ -98,9 +98,9 @@ class SparkConnectClientSuite
TestPackURI("sc://host", isCorrect = true),
TestPackURI("sc://localhost/", isCorrect = true, client =>
testClientConnection(client)),
TestPackURI(
- "sc://localhost:123/",
+ "sc://localhost:1234/",
isCorrect = true,
- client => testClientConnection(client, 123)),
+ client => testClientConnection(client, 1234)),
TestPackURI("sc://localhost/;", isCorrect = true, client =>
testClientConnection(client)),
TestPackURI("sc://host:123", isCorrect = true),
TestPackURI(
diff --git a/dev/sparktestsupport/modules.py b/dev/sparktestsupport/modules.py
index 9fa9430ba5f..f63e7206599 100644
--- a/dev/sparktestsupport/modules.py
+++ b/dev/sparktestsupport/modules.py
@@ -279,6 +279,7 @@ connect = Module(
],
sbt_test_goals=[
"connect/test",
+ "connect-client-jvm/test",
],
)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
