[JIRA] (JENKINS-37575) Jenkins node keeps sending the same logs indefinitely
Title: Message Title Jesse Glick updated an issue Jenkins / JENKINS-37575 Jenkins node keeps sending the same logs indefinitely Change By: Jesse Glick Component/s: remoting Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-37575) Jenkins node keeps sending the same logs indefinitely
Title: Message Title
Lutz Neugebauer commented on JENKINS-37575
Re: Jenkins node keeps sending the same logs indefinitely
Reproducing a race condition reliably is usually not that easy. However, it can be done here with a minimally invasive surgery. By adding a 11 seconds sleep right before the return of FileMonitoringTask’s WriteLog.invoke() method the issue can be triggered:
diff --git a/src/main/java/org/jenkinsci/plugins/durabletask/FileMonitoringTask.java b/src/main/java/org/jenkinsci/plugins/durabletask/FileMonitoringTask.java
index 163ca83..d55134c 100644
--- a/src/main/java/org/jenkinsci/plugins/durabletask/FileMonitoringTask.java
+++ b/src/main/java/org/jenkinsci/plugins/durabletask/FileMonitoringTask.java
@@ -137,6 +137,7 @@ public abstract class FileMonitoringTask extends DurableTask {
} finally {
raf.close();
}
+Thread.sleep(11*1000);
return len;
} else {
return null;
For reproducing you may use the following pipeline script with your node:
node('yourNodeHere') {
sh "echo 'This echo infinitely shows up in the log every 10 seconds.'"
}
Add Comment
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
[JIRA] (JENKINS-37575) Jenkins node keeps sending the same logs indefinitely
Title: Message Title Jesse Glick commented on JENKINS-37575 Re: Jenkins node keeps sending the same logs indefinitely Unless it is reproducible from scratch I do not expect to spend time on it since I hope to rewrite this code as part of JENKINS-38381 anyway. Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-37575) Jenkins node keeps sending the same logs indefinitely
Title: Message Title Lutz Neugebauer updated an issue Jenkins / JENKINS-37575 Jenkins node keeps sending the same logs indefinitely Change By: Lutz Neugebauer Component/s: workflow-durable-task-step-plugin Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-37575) Jenkins node keeps sending the same logs indefinitely
Title: Message Title Lutz Neugebauer commented on JENKINS-37575 Re: Jenkins node keeps sending the same logs indefinitely Hello, I’m experiencing this issue, too. My suspicion is that this is caused by the way org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep currently works. In its check() method the call to controller.writeLog(…) is effected by the Timer defined in that method. As far as I understand the Timer interrupts the execution after 10 seconds. If writeLog() has not completed within those 10 seconds it is interrupted. Within those 10 seconds it has already written a certain amount of the log, but at the next attempt it will restart again. If the log on the node has meanwhile grown and machine load or network load conditions don’t allow to complete the writeLog() within 10 seconds this will loop indefinitely until disk space is filled up. The dependency on those conditions makes it that hard to reproduce. I observed the following exception in log approximately every 10 seconds when the problem occurred: Sep 14, 2016 10:26:49 AM FINE org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution check could not check [workspace path here] java.lang.InterruptedException at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:400) at java.util.concurrent.FutureTask.get(FutureTask.java:187) at hudson.remoting.Request.call(Request.java:158) at hudson.remoting.Channel.call(Channel.java:780) at hudson.FilePath.act(FilePath.java:1007) at hudson.FilePath.act(FilePath.java:996) at org.jenkinsci.plugins.durabletask.FileMonitoringTask$FileMonitoringController.writeLog(FileMonitoringTask.java:132) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.check(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.run(DurableTaskStep.java:233) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Note: The line numbers from classes of the durable-task plugin may vary compared to official versions since I have changes in there relating to https://github.com/jenkinsci/durable-task-plugin/pull/28. Regards, Lutz Add Comment
[JIRA] (JENKINS-37575) Jenkins node keeps sending the same logs indefinitely
Title: Message Title Quentin Dufour updated an issue Jenkins / JENKINS-37575 Jenkins node keeps sending the same logs indefinitely Change By: Quentin Dufour Summary: Jenkins node keeps sending the sames same logs indefinitely Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
