steveloughran commented on code in PR #5519:
URL: https://github.com/apache/hadoop/pull/5519#discussion_r1224453999
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/impl/EntryFileIO.java:
##########
@@ -63,7 +63,18 @@ public class EntryFileIO {
private static final Logger LOG = LoggerFactory.getLogger(
EntryFileIO.class);
- public static final int WRITER_SHUTDOWN_TIMEOUT = 60;
+ /**
+ * How long should the writer shutdown take?
+ */
+ public static final int WRITER_SHUTDOWN_TIMEOUT_SECONDS = 60;
+
+ /**
+ * How long should trying to queue a write block before giving up
+ * with an error?
+ * This is a safety feature to ensure that if something has gone wrong
+ * in the queue code the job fails with an error rather than just hangs
+ */
+ public static final int WRITER_QUEUE_PUT_TIMEOUT_MINUTES = 10;
Review Comment:
my view if things are this bad it is a disaster and the job is failing as
either the thread concurrency is broken or the local fs has failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
