CTTY commented on code in PR #5943:
URL: https://github.com/apache/hudi/pull/5943#discussion_r928198507
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieStorageConfig.java:
##########
@@ -130,6 +130,12 @@ public class HoodieStorageConfig extends HoodieConfig {
.defaultValue("TIMESTAMP_MICROS")
.withDocumentation("Sets spark.sql.parquet.outputTimestampType. Parquet
timestamp type to use when Spark writes data to Parquet files.");
+ public static final ConfigProperty<String> PARQUET_FIELD_ID_WRITE_ENABLED =
ConfigProperty
+ .key("hoodie.parquet.fieldId.write.enabled")
+ .defaultValue("true")
+ .withDocumentation("Sets spark.sql.parquet.fieldId.write.enabled. "
+ + "If enabled, Spark will write out parquet native field ids that
are stored inside StructField's metadata as parquet.field.id to parquet
files.");
Review Comment:
Good point. This field is added in Spark 3.3 . It doesn't break older Spark
but it would not be effective. I'll add this into `withDocumentation`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
