jackye1995 commented on a change in pull request #3810:
URL: https://github.com/apache/iceberg/pull/3810#discussion_r775616550
##########
File path: core/src/main/java/org/apache/iceberg/TableProperties.java
##########
@@ -111,6 +111,12 @@ private TableProperties() {
public static final String DELETE_AVRO_COMPRESSION =
"write.delete.avro.compression-codec";
public static final String AVRO_COMPRESSION_DEFAULT = "gzip";
+ public static final String ORC_STRIPE_SIZE_BYTES =
"write.orc.stripe-size-bytes";
+ public static final String ORC_STRIPE_SIZE_BYTES_DEFAULT = "67108864"; //
64L * 1024 * 1024
Review comment:
nit: can directly be `long`
##########
File path: site/docs/configuration.md
##########
@@ -41,6 +41,8 @@ Iceberg tables support table properties to configure table
behavior, like the de
| write.parquet.compression-codec | gzip | Parquet
compression codec |
| write.parquet.compression-level | null | Parquet
compression level |
| write.avro.compression-codec | gzip | Avro compression
codec |
+| write.orc.stripe-size-bytes | 67108864(64 MB) | Define the default ORC
stripe size, in bytes. |
Review comment:
nit: align `|` for each column unless it's too long.
##########
File path: orc/src/main/java/org/apache/iceberg/orc/ORC.java
##########
@@ -511,4 +513,14 @@ static Reader newFileReader(InputFile file, Configuration
config) {
}
return newFileReader(file.location(), readerOptions);
}
+
+ private static void parseWriterCustomizedConf(Configuration conf) {
+ conf.set(
+ OrcConf.STRIPE_SIZE.name(),
+ conf.get(TableProperties.ORC_STRIPE_SIZE_BYTES,
TableProperties.ORC_STRIPE_SIZE_BYTES_DEFAULT));
+
+ conf.set(OrcConf.BLOCK_SIZE.name(), conf.get(
+ TableProperties.ORC_BLOCK_SIZE_BYTES,
+ TableProperties.ORC_BLOCK_SIZE_BYTES_DEFAULT));
Review comment:
```suggestion
conf.set(
OrcConf.BLOCK_SIZE.name(),
conf.get(TableProperties.ORC_BLOCK_SIZE_BYTES,
TableProperties.ORC_BLOCK_SIZE_BYTES_DEFAULT));
```
##########
File path: orc/src/main/java/org/apache/iceberg/orc/ORC.java
##########
@@ -145,6 +146,7 @@ public WriteBuilder metricsConfig(MetricsConfig
newMetricsConfig) {
public <D> FileAppender<D> build() {
Preconditions.checkNotNull(schema, "Schema is required");
+ parseWriterCustomizedConf(conf);
Review comment:
it seems not necessary to have a separated method for this
##########
File path: core/src/main/java/org/apache/iceberg/TableProperties.java
##########
@@ -111,6 +111,12 @@ private TableProperties() {
public static final String DELETE_AVRO_COMPRESSION =
"write.delete.avro.compression-codec";
public static final String AVRO_COMPRESSION_DEFAULT = "gzip";
+ public static final String ORC_STRIPE_SIZE_BYTES =
"write.orc.stripe-size-bytes";
+ public static final String ORC_STRIPE_SIZE_BYTES_DEFAULT = "67108864"; //
64L * 1024 * 1024
+
+ public static final String ORC_BLOCK_SIZE_BYTES =
"write.orc.block-size-bytes";
+ public static final String ORC_BLOCK_SIZE_BYTES_DEFAULT = "268435456"; //
256L * 1024 * 1024
Review comment:
nit: can directly be `long`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
