kevinjqliu commented on code in PR #14357:
URL: https://github.com/apache/iceberg/pull/14357#discussion_r2500119529
##########
site/docs/spark-quickstart.md:
##########
@@ -309,27 +309,27 @@ If you already have a Spark environment, you can add
Iceberg, using the `--packa
=== "SparkSQL"
```sh
- spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:{{
icebergVersion }}
+ spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-{{
sparkVersionMajor }}:{{ icebergVersion }}
```
=== "Spark-Shell"
```sh
- spark-shell --packages
org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:{{ icebergVersion }}
+ spark-shell --packages org.apache.iceberg:iceberg-spark-runtime-{{
sparkVersionMajor }}:{{ icebergVersion }}
```
=== "PySpark"
```sh
- pyspark --packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:{{
icebergVersion }}
+ pyspark --packages org.apache.iceberg:iceberg-spark-runtime-{{
sparkVersionMajor }}:{{ icebergVersion }}
```
!!! note
If you want to include Iceberg in your Spark installation, add the Iceberg
Spark runtime to Spark's `jars` folder.
You can download the runtime by visiting to the [Releases](releases.md)
page.
<!-- markdown-link-check-disable-next-line -->
-[spark-runtime-jar]:
https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.5_2.12/{{
icebergVersion }}/iceberg-spark-runtime-3.5_2.12-{{ icebergVersion }}.jar
+[spark-runtime-jar]:
https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-{{
sparkVersionMajor }}/{{ icebergVersion }}/iceberg-spark-runtime-{{
sparkVersionMajor }}-{{ icebergVersion }}.jar
Review Comment:
nit: this isnt rendered
same problem on https://iceberg.apache.org/spark-quickstart/#learn-more
right now
##########
docs/docs/spark-queries.md:
##########
@@ -180,10 +180,6 @@ spark.read
.load("path/to/table")
```
-!!! info
- Spark 3.0 and earlier versions do not support using `option` with `table`
in DataFrameReader commands. All options will be silently
- ignored. Do not use `table` when attempting to time-travel or use other
options. See [SPARK-32592](https://issues.apache.org/jira/browse/SPARK-32592).
-
Review Comment:
nit: i think we should keep this warning
##########
docs/docs/spark-structured-streaming.md:
##########
@@ -76,8 +76,6 @@ data.writeStream
.toTable("database.table_name")
```
-If you're using Spark 3.0 or earlier, you need to use `.option("path",
"database.table_name").start()`, instead of `.toTable("database.table_name")`.
-
Review Comment:
nit: we should keep this warning in case someone is still using spark 3.0 or
earlier
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
