Izeren commented on PR #28268: URL: https://github.com/apache/flink/pull/28268#issuecomment-4633814155
> Yes, you are right. That was initially discussed. what we are observing at the scale of production, users don't really set policies. There are billions of MPU get accumulated and, leading to high cost. Could you please elaborate on how storing subparts in the state is linked to the billing problem. Aren't aborted MPUs introducing all the same dangling S3 objects? My general question was more about why do we store subparts as separate tail files to resume from on S3. Are they as good as the inline Flink state in terms of data corruption risks? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
