qq461613840 opened a new issue, #614: URL: https://github.com/apache/doris-flink-connector/issues/614
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 25.1.0 ### What's Wrong? CSV format with gz compression enabled will freeze after running for a certain period of time 2025-09-09 15:54:51,372 INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad [] - Cache full, waiting for flush, currentBytes: 314572855, maxBlockedBytes: 314572800 2025-09-09 15:54:52,335 INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad [] - bufferMap is empty, no need to flush null 2025-09-09 15:54:52,372 INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad [] - Cache full, waiting for flush, currentBytes: 314572855, maxBlockedBytes: 314572800 2025-09-09 15:54:53,372 INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad [] - Cache full, waiting for flush, currentBytes: 314572855, maxBlockedBytes: 314572800 2025-09-09 15:54:54,335 INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad [] - bufferMap is empty, no need to flush null 2025-09-09 15:54:54,373 INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad [] - Cache full, waiting for flush, currentBytes: 314572855, maxBlockedBytes: 314572800 My configuration is Properties props = new Properties(); props.setProperty("column_separator", ","); props.setProperty("line_delimiter", "\n"); props.setProperty("format", "csv"); props.setProperty("compress_type", "gz"); return DorisExecutionOptions.builder() .setLabelPrefix(tableName + "-" + System.currentTimeMillis()) .setDeletable(false) .setBatchMode(true) .setBufferFlushMaxRows(20000) .setBufferFlushIntervalMs(2000) .setStreamLoadProp(props) .build(); After reviewing the source code, I suspect that the cacheBeforeFlushBytes record represents the size before compression. After sending data, currentCacheBytes.getAndAdd(-respContent.getLoadBytes()); represents the compressed size. This causes the value to not be released. This then leads to the problem of "Cache full, waiting for flush" and a constant locked state. ### What You Expected? Hope to solve this bug ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
