Hello,
I am trying to diagnose a few remoting related issues in the unity3d plugin
(https://github.com/jenkinsci/unity3d-plugin/)
I am unable to reproduce the issues so far but would appreciate any insight
into my analysis (and design of the plugin).
Plugin function and design:
The plugins basically executes a program (Unity3d) which happens to write
its output into a file (hereafter called logFile). I pipe this file into
the console as it is written. To achieve this remote piping I used the
following design:
3 elements are created:
* a org.jenkinsci.plugins.unity3d.io.Pipe consisting of a PipedInputStream
and a PipedOutputStream. This outputstream is wrapped into a
RemoteOutputStream if the launcher is to be executed remotely.
* a long running Async Callable is passed to the launcher channel. This
closure, a PipeFileAfterModificationAction instance, detects that the
logFile is started being written and copies from it recursively into the
pipe until interrupted. It then closes the output in a finally
* a org.jenkinsci.plugins.unity3d.io.StreamCopyThread reads from the pipe
on the master and copies it into the job console.
The issues that some users are experiencing:
* the logFile isn't fully pasted into the console
* the pipe is broken:
java.io.IOException: Pipe broken
at java.io.PipedInputStream.read(PipedInputStream.java:322)
at java.io.PipedInputStream.read(PipedInputStream.java:378)
at java.io.InputStream.read(InputStream.java:101)
at
org.jenkinsci.plugins.unity3d.io.StreamCopyThread.run(StreamCopyThread.java:64)
It seems like these errors
* only happen when used in distributed environmenrts
* when the second happens, the first one as well.
Analysis:
I looked into the "Pipe broken" issue and I believe something is fishy in
my design.
Ideally I want to:
* wait for the Unity3d command to finish execution
* stop the thread reading the logFile on the remote side
* pipe everything back into the console
* close things gracefully
To achieve this, I do this in the finally block of the unity3d plugin
execution:
// wait for some time - see
https://github.com/jenkinsci/unity3d-plugin/commit/d792376afd67579f313043e7c701cf2559df02bd
Thread.sleep(1000);
// stop the piping
if (!futureReadBytes.isDone()) {
futureReadBytes.cancel(true);
}
// wait for the copy
try {
copierThread.join();
if (copierThread.getFailure() != null) {
For the "Pipe broken" issue, my clean closing objective isn't reached. The
StreamCopyThread fails while copying, i.e. it detects that the Thread that
writes into the PipedInputStream (named
"Computer.threadPoolForRemoting(...)" is dead (isAlive()==false) before the
read() detects the eof (-1).
Why does it die, I am not sure, this thread is managed by the remoting
framework.
Question #1: futureReadBytes.cancel(true)
This if my understanding is correct should send a Cancel command
asynchronously to the remote end.
I expect this one to interrupt the reading of the
PipeFileAfterModificationAction which would close the stream in its finally
and thus terminate the async operation then the thread that reads into the
pipe.
But could the cancel cause the local thread to die before the remote action
is terminated properly ?
Question #2: lack of flushing could be the cause of the incomplete piping
Am I missing some flushing somewhere ? This could happen at lots of levels.
I don't see much flushing in the jenkins StreamCopyTread either.
Question #3: better design
I believe it would be better if I didn't have a long running async Callable
to pipe the logFile, but something I could signal to stop from the client
side instead of having to use cancel(). Any tip on achieving this in a way
that fits with the remoting library ?
Some ideas I've had:
* get rid of the PipeFileAfterModificationAction action by forcing the
writing of the output into a newly created logFile and use a standard
StreamCopyThread to pipe it. That way I don't have to detect the start of
the operation. This would still require the synchronization of the stop of
the command.
* create a wrapper script that uses OS capabilities to run the command and
pipe the logFile to its standard output, letting the jenkins side of it
execute a single command, and not caring about the piping. I know how to do
this on Unix but not on Windows. Otherwise the script could be a Java
wrapper, but this is a bit overkill it seems.
Thanks for your insights,
Cheers,
Jerome
--
You received this message because you are subscribed to the Google Groups
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
