This is an automated email from the ASF dual-hosted git repository.
yma pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new ed60f4998 [GLUTEN-6737]package delta into bundle jar when specify
delta profile (#6738)
ed60f4998 is described below
commit ed60f4998b5bf74e495e2138ec6ec8b03e6c0607
Author: Qian Sun <[email protected]>
AuthorDate: Fri Aug 9 09:24:57 2024 +0800
[GLUTEN-6737]package delta into bundle jar when specify delta profile
(#6738)
* package delta into bundle jar when specify delta profile
* update velox.md
---
docs/get-started/Velox.md | 7 +------
gluten-delta/pom.xml | 4 ----
package/pom.xml | 10 ++++++++++
3 files changed, 11 insertions(+), 10 deletions(-)
diff --git a/docs/get-started/Velox.md b/docs/get-started/Velox.md
index 5f49f542b..776736b2f 100644
--- a/docs/get-started/Velox.md
+++ b/docs/get-started/Velox.md
@@ -314,10 +314,7 @@ First of all, compile gluten-delta module by a `delta`
profile, as follows:
mvn clean package -Pbackends-velox -Pspark-3.3 -Pdelta -DskipTests
```
-Then, put the additional `gluten-delta-XX-SNAPSHOT.jar` to the class path
(usually it's `$SPARK_HOME/jars`).
-The gluten-delta jar is in `gluten-delta/target` directory.
-
-After the two steps, you can query delta table by gluten/velox without scan's
fallback.
+Once built successfully, delta features will be included in
gluten-velox-bundle-X jar. Then you can query delta table by gluten/velox
without scan's fallback.
Gluten with velox backends also support the column mapping of delta tables.
About column mapping, see more
[here](https://docs.delta.io/latest/delta-column-mapping.html).
@@ -336,8 +333,6 @@ mvn clean package -Pbackends-velox -Pspark-3.3 -Piceberg
-DskipTests
Once built successfully, iceberg features will be included in
gluten-velox-bundle-X jar. Then you can query iceberg table by gluten/velox
without scan's fallback.
-After the two steps, you can query iceberg table by gluten/velox without
scan's fallback.
-
# Coverage
Spark3.3 has 387 functions in total. ~240 are commonly used. To get the
support status of all Spark built-in functions, please refer to [Velox
Backend's Supported Operators &
Functions](../velox-backend-support-progress.md).
diff --git a/gluten-delta/pom.xml b/gluten-delta/pom.xml
index 253d5a8e4..08e2060d0 100755
--- a/gluten-delta/pom.xml
+++ b/gluten-delta/pom.xml
@@ -62,10 +62,6 @@
<type>test-jar</type>
<scope>test</scope>
</dependency>
- <dependency>
- <groupId>org.apache.spark</groupId>
- <artifactId>spark-core_${scala.binary.version}</artifactId>
- </dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.binary.version}</artifactId>
diff --git a/package/pom.xml b/package/pom.xml
index 794118f83..ae1432770 100644
--- a/package/pom.xml
+++ b/package/pom.xml
@@ -93,6 +93,16 @@
</dependency>
</dependencies>
</profile>
+ <profile>
+ <id>delta</id>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.gluten</groupId>
+ <artifactId>gluten-delta</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ </dependencies>
+ </profile>
</profiles>
<build>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
