172403678 opened a new issue #1911: URL: https://github.com/apache/iceberg/issues/1911
output file starts with partitionId so file is starts with 00000 and so on but in iceberg on cos scene , cos file name shouldnot begin with the same prefix The cloud maintains the key values of buckets and objects as indexes in each service area of object storage. Object keys are stored in multiple partitions of the index in UTF-8 binary order. For a large number of key values, for example, using timestamp or alphabetical order may exhaust the read and write performance of the partition where the key value is located. Take the bucket path examplebucket-1250000000.cos.ap-beijing.myqcloud.com as an example, as shown below Some cases that may exhaust index performance: such as https://cloud.tencent.com/document/product/436/13653 in file OutputFileFactory private String generateFilename() { return this.format.addExtension(String.format("%05d-%d-%s-%05d", this.partitionId, this.taskId, this.uuid, this.fileCount.incrementAndGet())); }  ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
