LuciferYang commented on code in PR #39788:
URL: https://github.com/apache/spark/pull/39788#discussion_r1089908235
##########
project/SparkBuild.scala:
##########
@@ -891,6 +895,30 @@ object SparkConnectClient {
}
}
+object SparkConnectClientTests {
+ lazy val settings = Seq(
+ excludeDependencies ++= {
Review Comment:
`excludeDependencies` does not work. There is a probability that sbt will
fail to compile because `spark-sql` is in the classpath, which will cause
package conflicts as follows:
```
[error]
/${basedir}/spark-mine/connector/connect/client/integration-tests/src/test/scala/org/apache/spark/sql/ClientE2ETestSuite.scala:27:21:
value collectResult is not a member of org.apache.spark.sql.DataFrame
[error] val schema = df.collectResult().schema
[error] ^
[error]
/${basedir}/spark-mine/connector/connect/client/integration-tests/src/test/scala/org/apache/spark/sql/ClientE2ETestSuite.scala:33:21:
value collectResult is not a member of org.apache.spark.sql.DataFrame
[error] val result = df.collectResult()
[error] ^
[error]
/${basedir}/spark-mine/connector/connect/client/integration-tests/src/test/scala/org/apache/spark/sql/ClientE2ETestSuite.scala:43:21:
value collectResult is not a member of org.apache.spark.sql.Dataset[Long]
[error] val result = df.collectResult()
[error] ^
[error]
/${basedir}/spark-mine/connector/connect/client/integration-tests/src/test/scala/org/apache/spark/sql/connect/client/util/RemoteSparkSession.scala:152:36:
value client is not a member of org.apache.spark.sql.SparkSession.Builder
[error] spark =
SparkSession.builder().client(SparkConnectClient.builder().port(port).build()).build()
[error] ^
[error]
/${basedir}/spark-mine/connector/connect/client/integration-tests/src/test/scala/org/apache/spark/sql/connect/client/util/RemoteSparkSession.scala:165:12:
value collectResult is not a member of org.apache.spark.sql.DataFrame
[error] possible cause: maybe a semicolon is missing before `value
collectResult'?
[error] .collectResult()
[error] ^
[error] 5 errors found
[error] (connect-client-integration-tests / Test / compileIncremental)
Compilation failed
[error] Total time: 47 s, completed 2023-1-29 12:41:23
```
I can't solve it temporarily
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
