On 09/12/2014 01:14 PM, Doug Hellmann wrote: > > On Sep 12, 2014, at 12:03 PM, Sean Dague <s...@dague.net> wrote: > >> On 09/12/2014 11:52 AM, Doug Hellmann wrote: >>> >>> On Sep 12, 2014, at 11:21 AM, Mike Bayer <mba...@redhat.com> wrote: >>> >>>> >>>> On Sep 12, 2014, at 7:39 AM, Sean Dague <s...@dague.net> wrote: >>>> >>>>> I assume you, gentle OpenStack developers, often find yourself in a hair >>>>> tearing out moment of frustration about why local unit tests are doing >>>>> completely insane things. The code that it is stack tracing on is no >>>>> where to be found, and yet it fails. >>>>> >>>>> And then you realize.... that part of oslo doesn't exist any more.... >>>>> except there are still pyc files laying around. Gah! >>>>> >>>>> I've proposed the following to Nova and Python novaclient - >>>>> https://review.openstack.org/#/c/121044/ >>>>> >>>>> Which sets PYTHONDONTWRITEBYTECODE=true in the unit tests. >>>> >>>> my VPN was down and I didn’t get this thread just now, but I am strongly >>>> -1 on this as added to tox.ini, my response is >>>> http://lists.openstack.org/pipermail/openstack-dev/2014-September/045873.html. >>>> >>>> Short answer: if you want this feature, put PYTHONDONTWRITEBYTECODE into >>>> *your* environment. Don’t force it on our automated tests or on my >>>> environment. .pyc files make a difference in behavior, and if we banish >>>> them from all testing, then our code is never tested within the >>>> environment that it will normally be run in after shipment. >>>> >>>> I’d far prefer a simple script added to tox.ini which deletes orphaned >>>> .pyc files only, if a change to tox.ini must be made. >>> >>> I have to agree with Mike here. Cleaning up our dev environments using a >>> little automation is better than disabling a feature of the interpreter >>> that may have unforeseen consequences in behavior or performance. The more >>> we introduce unusual settings like this into our environments and tools, >>> the more edge cases and weirdness we’re going to find in those tools that >>> keep us from doing the work we really want to be doing. >>> >>> We could use a git hook (see my earlier message in this thread) or we could >>> add a command to tox to remove them before starting the tests. Neither of >>> those solutions would affect the runtime behavior in a way that makes our >>> dev environments fundamentally different from a devstack or production >>> deployment. >> >> You believe that unit tests are going to change in the way they run so >> dramatically with this change that it invalidates their use? >> >> Do we have examples of what changes if you do and don't have pyc files >> there? >> >> Remember, we're not changing integration testing with this. This is >> solely unit testing. >> >> The reason I don't like "just fix it in your local env" is you are then >> exporting the complexity to developers. For something that they should >> really not have to get bitten by... a lot. > > Adding a command to tox to remove the files would be less intrusive than > disabling their creation. > > We have had bad experiences mixing features to produce unusual dev > environments that are different from the way the software really runs. All of > the issues we had with namespace packages were caused by a bug in that > implementation exposed because we were doing something unusual in devstack, > for example. > > Adding some variation of “find $(python setup.py --name) --name ‘*.pyc’ | > xargs rm -f” to tox.ini before running testr solves the problem you have > identified without introducing any side-effects.
Ok, while I'm not actually convinced that PYTHONDONTWRITEBYTECODE=true is a problem (especially after looking at the actual source of the python interpreter, where it's pretty clear everything is abstracted through a set of AST classes regardless of whether these files are there or not), I changed my upstream proposal to just the same purge line we'd be using in Nova run_tests.sh forever before every tox run. -Sean -- Sean Dague http://dague.net _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev