capturing output of a child process with backticks fails unpredictably
----------------------------------------------------------------------
Key: JRUBY-4909
URL: http://jira.codehaus.org/browse/JRUBY-4909
Project: JRuby
Issue Type: Bug
Affects Versions: JRuby 1.5.1, JRuby 1.4
Environment: Linux 64-bit (Ubuntu 10.04), 12 cores
Reporter: Mark Longair
Assignee: Thomas E Enebo
I noticed when running a script that invokes many child processes via backticks
(from up to 24 concurrent threads) that a small proportion of these fail
unpredictably. Sometimes the output appears to be empty when it shouldn't be
and sometimes the standard output of backticks seems to miss being caught and
goes straight to stdout of the parent. In addition, with version 1.4 I
sometimes see these errors:
{noformat}
<script>:1: Cannot run program "/bin/sh" (in directory
"/home/mark/jruby-bug/"): java.io.IOException: error=9, Bad file descriptor
(IOError)
{noformat}
... or:
{noformat}
<script>:1: Cannot run program "/bin/sh" (in directory "/home/mark/jruby-bug"):
java.io.IOException: error=2, No such file or directory (IOError)
{noformat}
As a simplified test case, I'm using the following script:
{noformat}
require 'thread'
max_threads = 24
mutex = Mutex.new
condition = ConditionVariable.new
threads = []
running_threads = 0
count = 0
loop do
# Only start a new thread if there are less than max_threads
# running:
mutex.synchronize {
condition.wait(mutex) while running_threads >= max_threads
running_threads += 1
condition.signal if running_threads < max_threads
if (count % 10000) == 0
puts "#### count: #{count}"
end
}
thread = Thread.new {
Thread.current[:output] = `echo echoed-back count is: #{count}`
unless Thread.current[:output] =~ /count is: \d+/
puts "Unexpected output: '#{Thread.current[:output]}'"
end
# Reduce the running threads by one and signal:
mutex.synchronize {
running_threads -= 1
condition.signal if running_threads < max_threads
}
}
# Go through the list of threads, find any that have finished, join
# them, and remove all the finished threads from the array afterwards:
mutex.synchronize {
indices_to_remove = []
threads.each_index do |i|
if threads[i]
status = threads[i].status
if status == false or status == nil
threads[i].join
indices_to_remove << i
end
end
end
indices_to_remove.reverse.each do |i|
threads.delete_at i
end
threads << thread
}
count += 1
end
{noformat}
I see the following output with JRuby 1.5.1 (release tarball) and JRuby 1.4
(Ubuntu package version 1.4.0-2) respectively:
{noformat}
$ ~/Downloads/jruby-1.5.1/bin/jruby --version
jruby 1.5.1 (ruby 1.8.7 patchlevel 249) (2010-06-06 f3a3480) (Java HotSpot(TM)
64-Bit Server VM 1.6.0_20) [amd64-java]
$ ~/Downloads/jruby-1.5.1/bin/jruby jruby-test-case
#### count: 0
#### count: 10000
#### count: 20000
#### count: 30000
#### count: 40000
#### count: 50000
#### count: 60000
Unexpected output: ''Unexpected output: ''
#### count: 70000
#### count: 80000
echoed-back count is: 83905
Unexpected output: ''
#### count: 90000
Unexpected output: ''
Unexpected output: ''
Unexpected output: ''
#### count: 100000
Unexpected output: ''
#### count: 110000
#### count: 120000
#### count: 130000
#### count: 140000
Unexpected output: ''
#### count: 150000
Unexpected output: ''
#### count: 160000
Unexpected output: ''
#### count: 170000
#### count: 180000
#### count: 190000
#### count: 200000
Unexpected output: ''
#### count: 210000
#### count: 220000
Unexpected output: ''
#### count: 230000
#### count: 240000
#### count: 250000
#### count: 260000
{noformat}
And with JRuby 1.4:
{noformat}
$ jruby --version
jruby 1.4.0 (ruby 1.8.7 patchlevel 174) (2010-02-11 6586) (Java HotSpot(TM)
64-Bit Server VM 1.6.0_20) [amd64-java]
$ jruby jruby-test-case
#### count: 0
#### count: 10000
#### count: 20000
#### count: 30000
Unexpected output: ''
#### count: 40000
#### count: 50000
Unexpected output: ''
Unexpected output: ''
#### count: 60000
#### count: 70000
#### count: 80000
#### count: 90000
#### count: 100000
#### count: 110000
Unexpected output: ''
#### count: 120000
{noformat}
I guess this might be related to http://jira.codehaus.org/browse/JRUBY-4626 but
the workaround suggested there isn't ideal since $? doesn't seem to be set by
IO.popen() with JRuby, so I'm not sure how to get at the return code.
Many thanks for all your work on JRuby,
Mark
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
