This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 79b5eed00e46 [MINOR][BUILD] Remove unused Parquet test dependencies
79b5eed00e46 is described below
commit 79b5eed00e46d44506692adae43417d5a431058d
Author: Aaron Niskode-Dossett <[email protected]>
AuthorDate: Thu Jul 23 13:12:29 2026 -0700
[MINOR][BUILD] Remove unused Parquet test dependencies
### What changes were proposed in this pull request?
Remove the `parquet-common` and `parquet-encoding` test-JAR dependencies
from
`sql/core/pom.xml` and their unused managed declarations from the root
`pom.xml`. Retain the
`parquet-column` test JAR using `<type>test-jar</type>`, and update its root
dependency-management entry to match.
Spark SQL uses helper classes from the `parquet-column` test JAR, but does
not reference classes
from the `parquet-common` or `parquet-encoding` test JARs. Removing those
unused artifacts narrows
the test classpath and makes the module's actual dependency explicit.
CI exposed a Maven/SBT interoperability detail: Spark's POM reader treats
`<classifier>tests</classifier>` as a transitive test-configuration
dependency. This revision uses
Maven's `<type>test-jar</type>` representation for the retained
`parquet-column` dependency, which
the POM reader intentionally maps intransitively while Maven continues to
resolve the same
published test JAR.
### Why are the changes needed?
The removed JARs provide no classes referenced by Spark SQL tests. Bytecode
inspection also
confirmed that the retained `MemPageStore` and `Utils` helpers depend only
on Parquet production
classes and other classes in the `parquet-column` test JAR.
Using the intransitive `test-jar` representation also prevents SBT from
resolving the complete
Parquet column test configuration, which includes unrelated common and
encoding test artifacts.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex (GPT-5)
Closes #57432 from dossett/remove-unused-parquet-test-deps.
Authored-by: Aaron Niskode-Dossett <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
pom.xml | 16 +---------------
sql/core/pom.xml | 14 +-------------
2 files changed, 2 insertions(+), 28 deletions(-)
diff --git a/pom.xml b/pom.xml
index 51679825124e..f6354961cf29 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2436,26 +2436,12 @@
<version>${parquet.version}</version>
<scope>${parquet.deps.scope}</scope>
</dependency>
- <dependency>
- <groupId>org.apache.parquet</groupId>
- <artifactId>parquet-encoding</artifactId>
- <version>${parquet.version}</version>
- <scope>${parquet.test.deps.scope}</scope>
- <classifier>tests</classifier>
- </dependency>
- <dependency>
- <groupId>org.apache.parquet</groupId>
- <artifactId>parquet-common</artifactId>
- <version>${parquet.version}</version>
- <scope>${parquet.test.deps.scope}</scope>
- <classifier>tests</classifier>
- </dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-column</artifactId>
<version>${parquet.version}</version>
<scope>${parquet.test.deps.scope}</scope>
- <classifier>tests</classifier>
+ <type>test-jar</type>
</dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
diff --git a/sql/core/pom.xml b/sql/core/pom.xml
index 1c3e510081e9..d96b71a9d805 100644
--- a/sql/core/pom.xml
+++ b/sql/core/pom.xml
@@ -141,23 +141,11 @@
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-column</artifactId>
</dependency>
- <dependency>
- <groupId>org.apache.parquet</groupId>
- <artifactId>parquet-encoding</artifactId>
- <scope>test</scope>
- <classifier>tests</classifier>
- </dependency>
- <dependency>
- <groupId>org.apache.parquet</groupId>
- <artifactId>parquet-common</artifactId>
- <scope>test</scope>
- <classifier>tests</classifier>
- </dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-column</artifactId>
<scope>test</scope>
- <classifier>tests</classifier>
+ <type>test-jar</type>
</dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
