This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ab7fcacca41d [MINOR][PYTHON][SQL][TESTS] Don't load Python Data Source
when Python executable is not available even for testing
ab7fcacca41d is described below
commit ab7fcacca41dad0ec2334b5d990bf36522fb5c82
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Thu Feb 8 14:19:43 2024 +0800
[MINOR][PYTHON][SQL][TESTS] Don't load Python Data Source when Python
executable is not available even for testing
### What changes were proposed in this pull request?
This PR proposes to don't load Python Data Source Python executable is not
available even for testing
### Why are the changes needed?
Whether if we're in test or not, it can't work loading Python Data Sources
anyway.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Manually tested.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45067 from HyukjinKwon/minor-checking.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
.../org/apache/spark/sql/execution/datasources/DataSourceManager.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceManager.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceManager.scala
index f63157b91efb..1b396675d909 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceManager.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceManager.scala
@@ -98,7 +98,7 @@ object DataSourceManager extends Logging {
private def normalize(name: String): String = name.toLowerCase(Locale.ROOT)
private def initialStaticDataSourceBuilders: Map[String,
UserDefinedPythonDataSource] = {
- if (Utils.isTesting || shouldLoadPythonDataSources) this.synchronized {
+ if (shouldLoadPythonDataSources) this.synchronized {
if (dataSourceBuilders.isEmpty) {
val maybeResult = try {
Some(UserDefinedPythonDataSource.lookupAllDataSourcesInPython())
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
