kl0u commented on a change in pull request #7639: [FLINK-9920] Only check part
files in BucketingSinkFaultToleranceITCase
URL: https://github.com/apache/flink/pull/7639#discussion_r253410186
##########
File path:
flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/RollingSink.java
##########
@@ -533,26 +533,34 @@ private Method reflectTruncate(FileSystem fs) {
}
// verify that truncate actually works
- Path testPath = new Path(UUID.randomUUID().toString());
- try (FSDataOutputStream outputStream =
fs.create(testPath)) {
- outputStream.writeUTF("hello");
- } catch (IOException e) {
- LOG.error("Could not create file for checking
if truncate works.", e);
- throw new RuntimeException("Could not create
file for checking if truncate works.", e);
- }
-
+ Path testPath = new Path(basePath,
UUID.randomUUID().toString());
try {
- m.invoke(fs, testPath, 2);
- } catch (IllegalAccessException |
InvocationTargetException e) {
- LOG.debug("Truncate is not supported.", e);
- m = null;
- }
+ try (FSDataOutputStream outputStream =
fs.create(testPath)) {
+ outputStream.writeUTF("hello");
+ } catch (IOException e) {
+ LOG.error("Could not create file for
checking if truncate works.", e);
+ throw new RuntimeException(
+ "Could not create file
for checking if truncate works. " +
+ "You
can disable support for truncate() completely via " +
+
"BucketingSink.setUseTruncate(false).",
+ e);
+ }
- try {
- fs.delete(testPath, false);
- } catch (IOException e) {
- LOG.error("Could not delete truncate test
file.", e);
- throw new RuntimeException("Could not delete
truncate test file.", e);
+ try {
+ m.invoke(fs, testPath, 2);
+ } catch (IllegalAccessException |
InvocationTargetException e) {
+ LOG.debug("Truncate is not supported.",
e);
+ m = null;
+ }
+ } finally {
+ try {
+ fs.delete(testPath, false);
+ } catch (IOException e) {
+ LOG.error("Could not delete truncate
test file.", e);
+ throw new RuntimeException("Could not
delete truncate test file. " +
Review comment:
I am not sure if we want to fail the job if we cannot `delete` the `test`
file.
I would say it is enough to just log the fact and move on. If there is a
bigger problem and this was not just a hick-up, the job will fail later when
something that matters fails (e.g. we cannot create a tmp file).
What do you think @aljoscha ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
