rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot
feature
URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r358524666
##########
File path: spark/src/main/java/org/apache/iceberg/spark/source/Writer.java
##########
@@ -122,9 +124,11 @@ private FileFormat getFileFormat(Map<String, String>
tableProperties, DataSource
return FileFormat.valueOf(formatString.toUpperCase(Locale.ENGLISH));
}
- private boolean isWapTable() {
- return Boolean.parseBoolean(table.properties().getOrDefault(
+ private boolean isWapTable(DataSourceOptions options) {
Review comment:
The original intent was to be able to switch jobs over to WAP with external
properties, which aren't able to configure individual writes. That's why
enabling WAP has two components: a WAP table and a WAP Spark environment. That
way, non-WAP tables don't get accidentally staged commits.
I'm not sure how much sense it makes to be able to override that with write
options baked into the job. And if it is baked in as options on an individual
write, then we don't need to check both the table and the environment. So if
wap-id is set in write options, I think it shouldn't need to validate that it
is a WAP table at all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
