RussellSpitzer commented on issue #1463:
URL: https://github.com/apache/iceberg/issues/1463#issuecomment-693500660
Ok unfortunate thing here.
If you use "table" in your read command it will ignore all other options
passed to the builder.
So
`spark.read.option("snapshot-id",
628886310322778010L).table("hadoop_prod.ice.icetest")` is equivalent to
`spark.table("hadoop_prod.ice.icetest")`
See
https://github.com/apache/spark/blob/v3.0.1/sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala#L795-L798
BUT Good news!
This is fixed in master, the fix version is marked as 3.1.0
https://issues.apache.org/jira/browse/SPARK-32592
https://github.com/apache/spark/blob/56ae95053df4afa9764df3f1d88f300896ca0183/sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala#L832-L838
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
