rdblue commented on a change in pull request #1508:
URL: https://github.com/apache/iceberg/pull/1508#discussion_r698063734
##########
File path:
spark2/src/main/java/org/apache/iceberg/spark/source/IcebergSource.java
##########
@@ -68,8 +68,8 @@ public DataSourceReader createReader(StructType readSchema,
DataSourceOptions op
Reader reader = new Reader(lazySparkSession(), table,
Boolean.parseBoolean(caseSensitive), options);
if (readSchema != null) {
- // convert() will fail if readSchema contains fields not in
table.schema()
- SparkSchemaUtil.convert(table.schema(), readSchema);
+ // convert() will fail if readSchema contains fields not in
reader.snapshotSchema()
+ SparkSchemaUtil.convert(reader.snapshotSchema(), readSchema);
Review comment:
I think it would be better to expose `reader.snapshotId()` and use the
snapshot ID to lookup the snapshot and its schema. This shouldn't be needed in
Spark 3 so I think it is fine if it's a bit uglier here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
