Re: ReviewBoard 1.5 under Apache: IOError when posting review from command line post-review tool
You're welcome. I hope that helps. In the interest of accuracy, I need to modify my post earlier regarding the problems with platform.py. After more thorough testing, I do not believe that these exceptions are directly caused by any of the functions defined in platform.py. Rather, I think they are all instances of race conditions between subprocess creation and attempts to use the resulting subprocesses. Note that platform.py (at least the version I have), uses os.popen() to create subprocesses to make the system calls, not subprocess.Popen(). It seems that in certain circumstances, with certain processors, etc., these same race condition issues can affect os.popen() calls as well as subprocess.Popen(). Stubbing out the _cleanup() function to be a no-op, as recommended in the bug thread to which I sent the link, dramatically reduced the number of exceptions during diff viewing. It didn't totally eliminate them. So, as also suggested in that thread, I've added a time.sleep(0.1) line in perforce.py, between the call to open the perforce cmdline subprocess and the line to communicate with it. I'm hoping 100ms will provide enough time for Popen to win the race every time. A similar small wait may need to be added between platform.py module's calls to os.popen and calls to read(). Good luck, Dennis On Oct 31, 1:28 pm, Rob wrote: > Thank you very much for the reference and tips. I'll give that a > try. > > On Oct 29, 11:54 am, Dennis wrote: > > > > > > > > > Rob, have you seen this? > > >http://bugs.python.org/issue1731717 > > > Implementing the proposed workaround in the latest entry of that > > thread fixed a lot of the [Errno 10] problems for me--all of the ones > > from the diff viewer, in fact. > > > I may have fixed the remaining ones from that 'uname -p' function call > > in platform.py that you're seeing, by inserting the return string from > > 'uname -o' on my system to the output value of syscmd_uname. > > > It looks to me like the intent was to use 'uname -o' in the first > > place, since that returns the operating system name, while uname -p > > returns the processor type. Somehow, this function ends up calling > > uname -p. > > > I assigned 'GNU/Linux' to the output of syscmd_uname in platform.py, > > and so far I've not seen the [Errno 10] in 3 consecutive post-review > > submissions. > > > I'm not certain that it's fixed completely, but it looks good so far. > > > Dennis > > > On Oct 13, 12:30 pm, Rob wrote: > > > > Starting in ReviewBoard 1.5 (Python 2.5.1 linux64 compiled > > > mod_python.so) the changes to use Paramiko result in this exception > > > when posting a review changelist (from Perforce) via the commandline > > > post-review tool. > > > > HTTP POSTing tohttp://sw-web3.altera.com/review/api/json/accounts/login/: > > > {'username': 'rromano', 'password': '**'} > > > > Traceback (most recent call last): > > > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > > > mod_python/importer.py", line 1537, in HandlerDispatch > > > default=default_handler, arg=req, silent=hlist.silent) > > > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > > > mod_python/importer.py", line 1229, in _process_target > > > result = _execute_target(config, req, object, arg) > > > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > > > mod_python/importer.py", line 1128, in _execute_target > > > result = object(arg) > > > > File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", > > > line 228, in handler > > > return ModPythonHandler()(req) > > > > File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", > > > line 191, in __call__ > > > self.load_middleware() > > > > File "/tools/django/trunk/1.2.3/django/core/handlers/base.py", line > > > 40, in load_middleware > > > mod = import_module(mw_module) > > > > File "/tools/django/trunk/1.2.3/django/utils/importlib.py", line 35, > > > in import_module > > > __import__(name) > > > > File "/usr/local/altera/django/reviewboard/admin/middleware.py", > > > line 13, in> > > from reviewboard.webapi.json import service_not_configured > > > > File "/usr/local/altera/django/reviewboard/webapi/json.py", line 29, > > > in > > > from reviewboard.diffviewer.forms import EmptyDiffError > > > > File "/usr/local/altera/django/reviewboard/diffviewer/forms.py", > > > line 7, in > > > from reviewboard.diffviewer.diffutils import > > > DEFAULT_DIFF_COMPAT_VERSION > > > > File "/usr/local/altera/django/reviewboard/diffviewer/diffutils.py", > > > line 29, in > > > from reviewboard.scmtools.core import PRE_CREATION, HEAD > > > > File "/usr/local/altera/django/reviewboard/scmtools/core.py", line > > > 5, in > > > from reviewboard.scmtools import sshutils > > > > File "/usr/local/altera/django/reviewboard/scmtools/sshutils.py", > > > line 5, in > > > import parami
Re: ReviewBoard 1.5 under Apache: IOError when posting review from command line post-review tool
Thank you very much for the reference and tips. I'll give that a try. On Oct 29, 11:54 am, Dennis wrote: > Rob, have you seen this? > > http://bugs.python.org/issue1731717 > > Implementing the proposed workaround in the latest entry of that > thread fixed a lot of the [Errno 10] problems for me--all of the ones > from the diff viewer, in fact. > > I may have fixed the remaining ones from that 'uname -p' function call > in platform.py that you're seeing, by inserting the return string from > 'uname -o' on my system to the output value of syscmd_uname. > > It looks to me like the intent was to use 'uname -o' in the first > place, since that returns the operating system name, while uname -p > returns the processor type. Somehow, this function ends up calling > uname -p. > > I assigned 'GNU/Linux' to the output of syscmd_uname in platform.py, > and so far I've not seen the [Errno 10] in 3 consecutive post-review > submissions. > > I'm not certain that it's fixed completely, but it looks good so far. > > Dennis > > On Oct 13, 12:30 pm, Rob wrote: > > > > > > > > > Starting in ReviewBoard 1.5 (Python 2.5.1 linux64 compiled > > mod_python.so) the changes to use Paramiko result in this exception > > when posting a review changelist (from Perforce) via the commandline > > post-review tool. > > > HTTP POSTing tohttp://sw-web3.altera.com/review/api/json/accounts/login/: > > {'username': 'rromano', 'password': '**'} > > > Traceback (most recent call last): > > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > > mod_python/importer.py", line 1537, in HandlerDispatch > > default=default_handler, arg=req, silent=hlist.silent) > > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > > mod_python/importer.py", line 1229, in _process_target > > result = _execute_target(config, req, object, arg) > > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > > mod_python/importer.py", line 1128, in _execute_target > > result = object(arg) > > > File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", > > line 228, in handler > > return ModPythonHandler()(req) > > > File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", > > line 191, in __call__ > > self.load_middleware() > > > File "/tools/django/trunk/1.2.3/django/core/handlers/base.py", line > > 40, in load_middleware > > mod = import_module(mw_module) > > > File "/tools/django/trunk/1.2.3/django/utils/importlib.py", line 35, > > in import_module > > __import__(name) > > > File "/usr/local/altera/django/reviewboard/admin/middleware.py", > > line 13, in> > from reviewboard.webapi.json import service_not_configured > > > File "/usr/local/altera/django/reviewboard/webapi/json.py", line 29, > > in > > from reviewboard.diffviewer.forms import EmptyDiffError > > > File "/usr/local/altera/django/reviewboard/diffviewer/forms.py", > > line 7, in > > from reviewboard.diffviewer.diffutils import > > DEFAULT_DIFF_COMPAT_VERSION > > > File "/usr/local/altera/django/reviewboard/diffviewer/diffutils.py", > > line 29, in > > from reviewboard.scmtools.core import PRE_CREATION, HEAD > > > File "/usr/local/altera/django/reviewboard/scmtools/core.py", line > > 5, in > > from reviewboard.scmtools import sshutils > > > File "/usr/local/altera/django/reviewboard/scmtools/sshutils.py", > > line 5, in > > import paramiko > > > File "/usr/local/altera/django/reviewboard/paramiko/__init__.py", > > line 69, in > > from transport import randpool, SecurityOptions, Transport > > > File "/usr/local/altera/django/reviewboard/paramiko/transport.py", > > line 32, in > > from paramiko import util > > > File "/usr/local/altera/django/reviewboard/paramiko/util.py", line > > 32, in > > from paramiko.common import * > > > File "/usr/local/altera/django/reviewboard/paramiko/common.py", line > > 98, in > > from rng import StrongLockingRandomPool > > > File "/usr/local/altera/django/reviewboard/paramiko/rng.py", line > > 44, in > > if ((platform is not None and platform.system().lower() == > > 'windows') or > > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > > 1042, in system > > return uname()[0] > > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > > 1014, in uname > > processor = _syscmd_uname('-p','') > > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > > 801, in _syscmd_uname > > rc = f.close() > > > IOError: [Errno 10] No child processes > > > This looks very fundamental. Python's platform.uname() fn does a > > subprocess call to `uname` and it's not liked. > > > Incidentally, under ReviewBoard 1.0.x, I had similar issues in > > ReviewBoard's own code using subprocess.POpen to extract diffs from > > `p4`. I simply changed the code to use os.s
Re: ReviewBoard 1.5 under Apache: IOError when posting review from command line post-review tool
Rob, have you seen this? http://bugs.python.org/issue1731717 Implementing the proposed workaround in the latest entry of that thread fixed a lot of the [Errno 10] problems for me--all of the ones from the diff viewer, in fact. I may have fixed the remaining ones from that 'uname -p' function call in platform.py that you're seeing, by inserting the return string from 'uname -o' on my system to the output value of syscmd_uname. It looks to me like the intent was to use 'uname -o' in the first place, since that returns the operating system name, while uname -p returns the processor type. Somehow, this function ends up calling uname -p. I assigned 'GNU/Linux' to the output of syscmd_uname in platform.py, and so far I've not seen the [Errno 10] in 3 consecutive post-review submissions. I'm not certain that it's fixed completely, but it looks good so far. Dennis On Oct 13, 12:30 pm, Rob wrote: > Starting in ReviewBoard 1.5 (Python 2.5.1 linux64 compiled > mod_python.so) the changes to use Paramiko result in this exception > when posting a review changelist (from Perforce) via the commandline > post-review tool. > > HTTP POSTing tohttp://sw-web3.altera.com/review/api/json/accounts/login/: > {'username': 'rromano', 'password': '**'} > > Traceback (most recent call last): > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > mod_python/importer.py", line 1537, in HandlerDispatch > default=default_handler, arg=req, silent=hlist.silent) > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > mod_python/importer.py", line 1229, in _process_target > result = _execute_target(config, req, object, arg) > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > mod_python/importer.py", line 1128, in _execute_target > result = object(arg) > > File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", > line 228, in handler > return ModPythonHandler()(req) > > File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", > line 191, in __call__ > self.load_middleware() > > File "/tools/django/trunk/1.2.3/django/core/handlers/base.py", line > 40, in load_middleware > mod = import_module(mw_module) > > File "/tools/django/trunk/1.2.3/django/utils/importlib.py", line 35, > in import_module > __import__(name) > > File "/usr/local/altera/django/reviewboard/admin/middleware.py", > line 13, in> from reviewboard.webapi.json import service_not_configured > > File "/usr/local/altera/django/reviewboard/webapi/json.py", line 29, > in > from reviewboard.diffviewer.forms import EmptyDiffError > > File "/usr/local/altera/django/reviewboard/diffviewer/forms.py", > line 7, in > from reviewboard.diffviewer.diffutils import > DEFAULT_DIFF_COMPAT_VERSION > > File "/usr/local/altera/django/reviewboard/diffviewer/diffutils.py", > line 29, in > from reviewboard.scmtools.core import PRE_CREATION, HEAD > > File "/usr/local/altera/django/reviewboard/scmtools/core.py", line > 5, in > from reviewboard.scmtools import sshutils > > File "/usr/local/altera/django/reviewboard/scmtools/sshutils.py", > line 5, in > import paramiko > > File "/usr/local/altera/django/reviewboard/paramiko/__init__.py", > line 69, in > from transport import randpool, SecurityOptions, Transport > > File "/usr/local/altera/django/reviewboard/paramiko/transport.py", > line 32, in > from paramiko import util > > File "/usr/local/altera/django/reviewboard/paramiko/util.py", line > 32, in > from paramiko.common import * > > File "/usr/local/altera/django/reviewboard/paramiko/common.py", line > 98, in > from rng import StrongLockingRandomPool > > File "/usr/local/altera/django/reviewboard/paramiko/rng.py", line > 44, in > if ((platform is not None and platform.system().lower() == > 'windows') or > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > 1042, in system > return uname()[0] > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > 1014, in uname > processor = _syscmd_uname('-p','') > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > 801, in _syscmd_uname > rc = f.close() > > IOError: [Errno 10] No child processes > > This looks very fundamental. Python's platform.uname() fn does a > subprocess call to `uname` and it's not liked. > > Incidentally, under ReviewBoard 1.0.x, I had similar issues in > ReviewBoard's own code using subprocess.POpen to extract diffs from > `p4`. I simply changed the code to use os.system() and it worked fine > thereafter. > > Are there known issues with the subprocess.POpen module when running > under Apache? -- Want to help the Review Board project? Donate today at http://www.reviewboard.org/donate/ Happy user? Let us know at http://www.reviewboard.org/users/ -~--~~~~--~-
Re: ReviewBoard 1.5 under Apache: IOError when posting review from command line post-review tool
I'm not really sure. This is the first I've heard of this issue. The call actually comes from paramiko, a Python module we use for the SSH support. Might be worth seeing if other people outside Review Board have hit that issue. Christian On Wednesday, October 13, 2010, Rob wrote: > Starting in ReviewBoard 1.5 (Python 2.5.1 linux64 compiled > mod_python.so) the changes to use Paramiko result in this exception > when posting a review changelist (from Perforce) via the commandline > post-review tool. > > HTTP POSTing to http://sw-web3.altera.com/review/api/json/accounts/login/: > {'username': 'rromano', 'password': '**'} > > Traceback (most recent call last): > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > mod_python/importer.py", line 1537, in HandlerDispatch > default=default_handler, arg=req, silent=hlist.silent) > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > mod_python/importer.py", line 1229, in _process_target > result = _execute_target(config, req, object, arg) > > File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ > mod_python/importer.py", line 1128, in _execute_target > result = object(arg) > > File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", > line 228, in handler > return ModPythonHandler()(req) > > File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", > line 191, in __call__ > self.load_middleware() > > File "/tools/django/trunk/1.2.3/django/core/handlers/base.py", line > 40, in load_middleware > mod = import_module(mw_module) > > File "/tools/django/trunk/1.2.3/django/utils/importlib.py", line 35, > in import_module > __import__(name) > > File "/usr/local/altera/django/reviewboard/admin/middleware.py", > line 13, in> from reviewboard.webapi.json import service_not_configured > > File "/usr/local/altera/django/reviewboard/webapi/json.py", line 29, > in > from reviewboard.diffviewer.forms import EmptyDiffError > > File "/usr/local/altera/django/reviewboard/diffviewer/forms.py", > line 7, in > from reviewboard.diffviewer.diffutils import > DEFAULT_DIFF_COMPAT_VERSION > > File "/usr/local/altera/django/reviewboard/diffviewer/diffutils.py", > line 29, in > from reviewboard.scmtools.core import PRE_CREATION, HEAD > > File "/usr/local/altera/django/reviewboard/scmtools/core.py", line > 5, in > from reviewboard.scmtools import sshutils > > File "/usr/local/altera/django/reviewboard/scmtools/sshutils.py", > line 5, in > import paramiko > > File "/usr/local/altera/django/reviewboard/paramiko/__init__.py", > line 69, in > from transport import randpool, SecurityOptions, Transport > > File "/usr/local/altera/django/reviewboard/paramiko/transport.py", > line 32, in > from paramiko import util > > File "/usr/local/altera/django/reviewboard/paramiko/util.py", line > 32, in > from paramiko.common import * > > File "/usr/local/altera/django/reviewboard/paramiko/common.py", line > 98, in > from rng import StrongLockingRandomPool > > File "/usr/local/altera/django/reviewboard/paramiko/rng.py", line > 44, in > if ((platform is not None and platform.system().lower() == > 'windows') or > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > 1042, in system > return uname()[0] > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > 1014, in uname > processor = _syscmd_uname('-p','') > > File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line > 801, in _syscmd_uname > rc = f.close() > > IOError: [Errno 10] No child processes > > > This looks very fundamental. Python's platform.uname() fn does a > subprocess call to `uname` and it's not liked. > > Incidentally, under ReviewBoard 1.0.x, I had similar issues in > ReviewBoard's own code using subprocess.POpen to extract diffs from > `p4`. I simply changed the code to use os.system() and it worked fine > thereafter. > > Are there known issues with the subprocess.POpen module when running > under Apache? > > -- > Want to help the Review Board project? Donate today at > http://www.reviewboard.org/donate/ > Happy user? Let us know at http://www.reviewboard.org/users/ > -~--~~~~--~~--~--~--- > To unsubscribe from this group, send email to > reviewboard+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/reviewboard?hl=en -- -- Christian Hammond - chip...@chipx86.com Review Board - http://www.reviewboard.org VMware, Inc. - http://www.vmware.com -- Want to help the Review Board project? Donate today at http://www.reviewboard.org/donate/ Happy user? Let us know at http://www.reviewboard.org/users/ -~--~~~~--~~--~--~--- To unsubscribe from this group, send email to reviewboard+unsubscr...@goo
ReviewBoard 1.5 under Apache: IOError when posting review from command line post-review tool
Starting in ReviewBoard 1.5 (Python 2.5.1 linux64 compiled mod_python.so) the changes to use Paramiko result in this exception when posting a review changelist (from Perforce) via the commandline post-review tool. HTTP POSTing to http://sw-web3.altera.com/review/api/json/accounts/login/: {'username': 'rromano', 'password': '**'} Traceback (most recent call last): File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ mod_python/importer.py", line 1537, in HandlerDispatch default=default_handler, arg=req, silent=hlist.silent) File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ mod_python/importer.py", line 1229, in _process_target result = _execute_target(config, req, object, arg) File "/tools/python/2.5.1/linux64/lib/python2.5/site-packages/ mod_python/importer.py", line 1128, in _execute_target result = object(arg) File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", line 228, in handler return ModPythonHandler()(req) File "/tools/django/trunk/1.2.3/django/core/handlers/modpython.py", line 191, in __call__ self.load_middleware() File "/tools/django/trunk/1.2.3/django/core/handlers/base.py", line 40, in load_middleware mod = import_module(mw_module) File "/tools/django/trunk/1.2.3/django/utils/importlib.py", line 35, in import_module __import__(name) File "/usr/local/altera/django/reviewboard/admin/middleware.py", line 13, infrom reviewboard.webapi.json import service_not_configured File "/usr/local/altera/django/reviewboard/webapi/json.py", line 29, in from reviewboard.diffviewer.forms import EmptyDiffError File "/usr/local/altera/django/reviewboard/diffviewer/forms.py", line 7, in from reviewboard.diffviewer.diffutils import DEFAULT_DIFF_COMPAT_VERSION File "/usr/local/altera/django/reviewboard/diffviewer/diffutils.py", line 29, in from reviewboard.scmtools.core import PRE_CREATION, HEAD File "/usr/local/altera/django/reviewboard/scmtools/core.py", line 5, in from reviewboard.scmtools import sshutils File "/usr/local/altera/django/reviewboard/scmtools/sshutils.py", line 5, in import paramiko File "/usr/local/altera/django/reviewboard/paramiko/__init__.py", line 69, in from transport import randpool, SecurityOptions, Transport File "/usr/local/altera/django/reviewboard/paramiko/transport.py", line 32, in from paramiko import util File "/usr/local/altera/django/reviewboard/paramiko/util.py", line 32, in from paramiko.common import * File "/usr/local/altera/django/reviewboard/paramiko/common.py", line 98, in from rng import StrongLockingRandomPool File "/usr/local/altera/django/reviewboard/paramiko/rng.py", line 44, in if ((platform is not None and platform.system().lower() == 'windows') or File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line 1042, in system return uname()[0] File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line 1014, in uname processor = _syscmd_uname('-p','') File "/tools/python/2.5.1/linux64/lib/python2.5/platform.py", line 801, in _syscmd_uname rc = f.close() IOError: [Errno 10] No child processes This looks very fundamental. Python's platform.uname() fn does a subprocess call to `uname` and it's not liked. Incidentally, under ReviewBoard 1.0.x, I had similar issues in ReviewBoard's own code using subprocess.POpen to extract diffs from `p4`. I simply changed the code to use os.system() and it worked fine thereafter. Are there known issues with the subprocess.POpen module when running under Apache? -- Want to help the Review Board project? Donate today at http://www.reviewboard.org/donate/ Happy user? Let us know at http://www.reviewboard.org/users/ -~--~~~~--~~--~--~--- To unsubscribe from this group, send email to reviewboard+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/reviewboard?hl=en