Well, I just got the __import__ error but with a different file and module
using assert=plain and not running the doctest module.
The subsequent runs could not find a certain c extension function. I
removed all the __pytcache__ and .pyc files with git clean -fxd -e "**.so"
and it claimed it can't import charm.core despite the fact that it exists
and can by imported via python3.2 -c "import charm.core" (which is
imported from the expected location even ). The next test pass worked.
No I am not using any plugins.
Interestingly, I just got the error on a run with the unittest module.
Though it maybe because something was corrupted since clearing the pyc
files fixed it.
I am now completely baffled.
It looks like something is , for lack of a better term, corrupting the
running copy of the interpreted code. Do you know anything that could
possibly do that? Poorly written c extensions perhaps ?
I think I am going to set up some script and just see if I can collect
errors with pytest and unittest
Ian
ython3.2 -m pytest --assert=plain
=============================================== test session starts
================================================
platform darwin -- Python 3.2.3 -- pytest-2.2.4
collected 98 items / 1 errors
schemes/test/chamhash_test.py ..
schemes/test/commit_test.py ..
schemes/test/dabenc_test.py ..
schemes/test/encap_bchk05_test.py .
schemes/test/grpsig_test.py ..
schemes/test/hibenc_test.py .
schemes/test/ibenc_test.py .........
schemes/test/pk_vrf_test.py .
schemes/test/pkenc_test.py ...........
schemes/test/pksig_test.py .................
schemes/test/rsa_alg_test.py ....
charm/test/toolbox/conversion_test.py ...
charm/test/toolbox/paddingschemes_test.py s..........
charm/test/toolbox/secretshare_test.py .
charm/test/toolbox/symcrypto_test.py ..........
charm/toolbox/paddingschemes_test.py s..........
charm/toolbox/symcrypto_test.py ..........
====================================================== ERRORS
======================================================
___________________________________ ERROR collecting
schemes/test/abenc_test.py ____________________________________
schemes/test/abenc_test.py:1: in <module>
> from schemes.abenc.abenc_adapt_hybrid import HybridABEnc as HybridABEnc
schemes/abenc/abenc_adapt_hybrid.py:6: in <module>
> from charm.toolbox.symcrypto import AuthenticatedCryptoAbstraction
charm/toolbox/symcrypto.py:9: in <module>
> import hmac
E TypeError: __import__() argument 1 must be str without null bytes, not
str
On Sat, Jun 16, 2012 at 12:30 PM, holger krekel <hol...@merlinux.eu> wrote:
> On Sat, Jun 16, 2012 at 11:59 -0400, Ian Miers wrote:
> > So these issues only happen on OSX and as I said,they come and go and
> > change,which is why I am skeptical it's our code. Actually if I had to
> > guess I'd say its something with python3 on OSX Lion, but so far we've
> only
> > seen the issues in test runs, not when attempting to manually recreate
> the
> > issue.
> >
> > Regarding import os.path error, I got it again. Trying to import the
> module
> > containing that import after that error caused a bus error. However all
> > subsequent imports worked fine. The strange thing is that the pyc file
> > didn't change ( or at least its md5 sum didn't) and the modification date
> > for all of the pyc files for that package remain unchanged.
> >
> >
> > python3.2 -m pytest --assert=plain seemed to work for a little bit, but
> > then we started getting the same intermittent changing errors.
>
> With --assert=plain pytest should not interfere with anything related
> to importing. Did you get the strange error with "import os.path"
> in this configuration? Are you using any plugins (see output of
> py.test --version)?
>
> The below error may in principle be a doctest/pytest bug with
> python3.2 although it would be strange if it only happened sometimes.
>
> Holger
>
> > Allmost all runs with assert=reinterp , including after removing all
> > __pychache__ and recompiling the c extensions produce the INTERNALERROR
> > below.
> >
> >
> >
> > Thanks again,
> > Ian
> >
> > python3.2 -m pytest --assert=reinterp
> > ============================= test session starts
> > ==============================
> > platform darwin -- Python 3.2.3 -- pytest-2.2.4
> > collected 232 items
> >
> > schemes/__init__.py .
> > schemes/chamhash_adm05.py .
> > schemes/chamhash_rsa_hw09.py .
> > schemes/encap_bchk05.py .
> > schemes/pk_fre_ccv11.py .
> > schemes/pk_vrf.py .
> > schemes/protocol_cns07.py .
> > schemes/protocol_schnorr91.py .
> > schemes/sigma1.py .
> > schemes/sigma2.py .
> > schemes/sigma3.py .
> > schemes/abenc/__init__.py .
> > schemes/abenc/abenc_adapt_hybrid.py F
> > schemes/abenc/abenc_bsw07.py .
> > schemes/abenc/abenc_lsw08.py .
> > schemes/abenc/abenc_waters09.py .
> > schemes/abenc/kpabenc_adapt_hybrid.py F
> > schemes/commit/__init__.py .
> > schemes/commit/commit_gs08.py .
> > schemes/commit/commit_pedersen92.py .
> > schemes/dabenc/__init__.py .
> > schemes/dabenc/dabe_aw11.py
> > INTERNALERROR> Traceback (most recent call last):
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> > line 74, in wrap_session
> > INTERNALERROR> doit(config, session)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> > line 106, in _main
> > INTERNALERROR> config.hook.pytest_runtestloop(session=session)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 421, in __call__
> > INTERNALERROR> return self._docall(methods, kwargs)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 432, in _docall
> > INTERNALERROR> res = mc.execute()
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 350, in execute
> > INTERNALERROR> res = method(**kwargs)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> > line 119, in pytest_runtestloop
> > INTERNALERROR> item.config.hook.pytest_runtest_protocol(item=item,
> > nextitem=nextitem)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 421, in __call__
> > INTERNALERROR> return self._docall(methods, kwargs)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 432, in _docall
> > INTERNALERROR> res = mc.execute()
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 350, in execute
> > INTERNALERROR> res = method(**kwargs)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> > line 61, in pytest_runtest_protocol
> > INTERNALERROR> runtestprotocol(item, nextitem=nextitem)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> > line 68, in runtestprotocol
> > INTERNALERROR> reports.append(call_and_report(item, "call", log))
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> > line 99, in call_and_report
> > INTERNALERROR> report = hook.pytest_runtest_makereport(item=item,
> > call=call)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> > line 141, in call_matching_hooks
> > INTERNALERROR> return hookmethod.pcall(plugins, **kwargs)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 425, in pcall
> > INTERNALERROR> return self._docall(methods, kwargs)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 432, in _docall
> > INTERNALERROR> res = mc.execute()
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 350, in execute
> > INTERNALERROR> res = method(**kwargs)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/capture.py",
> > line 173, in pytest_runtest_makereport
> > INTERNALERROR> rep = __multicall__.execute()
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> > line 350, in execute
> > INTERNALERROR> res = method(**kwargs)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> > line 187, in pytest_runtest_makereport
> > INTERNALERROR> longrepr = item.repr_failure(excinfo)
> > INTERNALERROR> File
> >
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/doctest.py",
> > line 44, in repr_failure
> > INTERNALERROR> lineno = test.lineno + example.lineno + 1
> > INTERNALERROR> TypeError: unsupported operand type(s) for +: 'NoneType'
> and
> > 'int'
> >
> > ===================== 2 failed, 19 passed in 3.42 seconds
> > ======================
> >
> >
> >
> >
> > On Sat, Jun 16, 2012 at 4:19 AM, holger krekel <hol...@merlinux.eu>
> wrote:
> >
> > > Hi Ian,
> > >
> > > On Fri, Jun 15, 2012 at 20:32 -0400, Ian Miers wrote:
> > > > Hi, I just started using pytest. It's lovely.
> > > > The TLDR on this is we are getting intermittent non reproducible
> errors
> > > > that change and sometimes disappear between test runs like the
> > > following :
> > > > > import os.path
> > > > E TypeError: __import__() argument 1 must be str without null
> bytes,
> > > not
> > > > str
> > >
> > > Looks odd. pytest does not override __import__ but it does by default
> > > use a PEP302 compliant module loader. If you use
> > >
> > > py.test --assert=reinterpret # or --assert=plain
> > >
> > > do the errors go away? If so that points to a problem in pytest's
> module
> > > loader or some interaction problem with python3.2.
> > >
> > > If changing the assertion mode still leads to errors then i strongly
> > > suspect it's other parts of the code you are running. I'd then suggest
> > > to check if something in your environment modifies __import__
> > > e.g. by writing a test that checks/prints out __import__?
> > >
> > > best,
> > > holger
> > >
> > > > Longer version:
> > > > We've been getting buggy results out of test runs on OSX Lion with
> python
> > > > 3.2.3 and py.test version 2.2.4. Specifically we've been getting
> what
> > > > appear to be false-positive test failures that change from run to
> run
> > > and
> > > > cannot be reproduced by running some code ourselves or in the case of
> > > > doctest manually running python3.2 -m doctest file.
> > > >
> > > > Some of these errors will stay around for multiple test runs even
> after
> > > > make clean, etc. Some will change from run to run. All of them
> > > > eventually disappeared temporarily and we got a clean test pass,
> though
> > > > how I don't know. Morever, the problems cropped up again. All of
> this was
> > > > with no code changes and code known to work on ubuntu and partially
> > > > manually tested on OSX.
> > > >
> > > > Ordinarly I'd say there was something wrong with our code. However,
> some
> > > of
> > > > the errors are vanishingly unlikely. Claims that modules don't exist
> when
> > > > they do and are importable via python3.2 -c "from foo.bar.baz import
> > > narf"
> > > > and such.
> > > >
> > > > The most glaringly, however, is this gem:
> > > > charm/toolbox/pairinggroup.py:3: in <module>
> > > > > import os.path
> > > > E TypeError: __import__() argument 1 must be str without null
> bytes,
> > > not
> > > > str
> > > >
> > > > We also got a lovely bug where it appeared __pycache__ was corrupted
> > > > during test runs. On an initial run, we could import a function
> from a
> > > > python c extension. On subsequent runs, it didn't exist. The
> function was
> > > > still in the .so file, as shown by nm, however help(module) returned
> > > > function_name#$@^%#$% Function description. Deleting __pychache__
> folders
> > > > resolved it for the next test run but then it came back. It too
> > > > disappeared after a couple of test runs never to be seen since.
> > > >
> > > > Has anyone seen anyhting like this? Are their known issues on OSX
> with
> > > > python ? With pytest? Does anyone have any idea how I might get a
> better
> > > > idea whats going on?
> > > >
> > > > As an addendum, the latest error I am getting is now :
> > > > > test = [hashFn(struct.pack(">%dsI" % (len(seed)), seed, i)) for
> i in
> > > > ran]
> > > > E UnicodeEncodeError: 'ascii' codec can't encode character '\x9e'
> in
> > > > position 0: ordinal not in range(128
> > > > This actually might be in our code,though again it works on ubuntu
> and at
> > > > one point on OSX. Given that its a pattern that points to python or
> > > pytest
> > > > doing something to binaries, I'm including it anyway.
> > > >
> > > > The project is charm, you can see the code on this branch here via
> > > > https://github.com/JHUISI/charm/tree/dev. Python was installed via
> fenc
> > > ( I
> > > > think, its not my box). The errors happened both with python3.2 -m
> pytest
> > > > and with python3.2 setup.py test. Though it appears more so with
> the
> > > > later.
> > > >
> > > > Thanks,
> > > >
> > > > Ian
> > >
> > > > _______________________________________________
> > > > py-dev mailing list
> > > > py-dev@codespeak.net
> > > > http://codespeak.net/mailman/listinfo/py-dev
> > >
> > >
>
_______________________________________________
py-dev mailing list
py-dev@codespeak.net
http://codespeak.net/mailman/listinfo/py-dev
