clairemcginty commented on code in PR #34105: URL: https://github.com/apache/beam/pull/34105#discussion_r1974222702
########## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java: ########## @@ -121,10 +121,11 @@ class BatchLoads<DestinationT, ElementT> // If user triggering is supplied, we will trigger the file write after this many records are // written. static final int FILE_TRIGGERING_RECORD_COUNT = 500000; + // If user triggering is supplied, we will trigger the file write after this many bytes are // written. static final int DEFAULT_FILE_TRIGGERING_BYTE_COUNT = - AsyncWriteChannelOptions.UPLOAD_CHUNK_SIZE_DEFAULT; // 64MiB as of now + AsyncWriteChannelOptions.DEFAULT.getUploadChunkSize(); // 64MiB as of now Review Comment: we could also just inline the value, though it changes a bit in bigdataoss 2.x vs 3.x [2.x:](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v2.2.26/util/src/main/java/com/google/cloud/hadoop/util/AsyncWriteChannelOptions.java#L74-L77) ```java public static final int UPLOAD_CHUNK_SIZE_GRANULARITY = 8 * 1024 * 1024; public static final int UPLOAD_CHUNK_SIZE_DEFAULT = Runtime.getRuntime().maxMemory() < 512 * 1024 * 1024 ? UPLOAD_CHUNK_SIZE_GRANULARITY : 8 * UPLOAD_CHUNK_SIZE_GRANULARITY; ``` [3.x](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v3.0.5/util/src/main/java/com/google/cloud/hadoop/util/AsyncWriteChannelOptions.java#L72-L75): ```java private static final int UPLOAD_CHUNK_SIZE_GRANULARITY = 8 * 1024 * 1024; private static final int DEFAULT_UPLOAD_CHUNK_SIZE = Runtime.getRuntime().maxMemory() < 512 * 1024 * 1024 ? UPLOAD_CHUNK_SIZE_GRANULARITY : 3 * UPLOAD_CHUNK_SIZE_GRANULARITY; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
