2 new commits in tox: https://bitbucket.org/hpk42/tox/commits/11c13b4313a4/ Changeset: 11c13b4313a4 User: hpk42 Date: 2016-06-25 11:06:55+00:00 Summary: addresses issue66: add --workdir option to override where tox stores its ".tox" directory to "pip freeze" to obtain the list of installed packages. Thanks Ted Shaw, Holger Krekel. close issue66: add documentation to jenkins page on how to avoid "too long shebang" lines when calling pip from tox. Note that we can not use "python -m pip install X" by default because the latter adds the CWD and pip will think X is installed if it is there. "pip install X" does not do that.
(experimental) New feature: When a search for a config file fails, tox tries loading Affected #: 7 files diff -r cb509b4c25c3dee2be78c39591f65de25e1058be -r 11c13b4313a44a7872761bd02b031b6a1728369c CHANGELOG --- a/CHANGELOG +++ b/CHANGELOG @@ -5,21 +5,23 @@ chars "{" and "}" to appear in your commands or other ini values. Thanks John Vandenberg. -- add --workdir option to override where tox stores its ".tox" directory +- addresses issue66: add --workdir option to override where tox stores its ".tox" directory and all of the virtualenv environment. Thanks Danring. - introduce per-venv list_dependencies_command which defaults - to "python -m pip freeze" to obtain the list of installed packages. - If you need to run python2.6 you need to configure it to - something like "pip freeze". Thanks Ted Shaw, Holger Krekel. + to "pip freeze" to obtain the list of installed packages. + Thanks Ted Shaw, Holger Krekel. -- fix issue66, issue121: change install_command to use "python -m pip" - by default instead of "pip ..." directly which avoids long shebang - issues. If you need to run python2.6 you need to configure it to - something like "pip install {opts} {packages}". Thanks Ted Shaw, - Holger Krekel. +- close issue66: add documentation to jenkins page on how to avoid + "too long shebang" lines when calling pip from tox. Note that we + can not use "python -m pip install X" by default because the latter + adds the CWD and pip will think X is installed if it is there. + "pip install X" does not do that. -- New feature: When a search for a config file fails, tox tries loading +- new list_dependencies_command to influence how tox determines + which dependencies are installed in a testenv. + +- (experimental) New feature: When a search for a config file fails, tox tries loading setup.cfg with a section prefix of "tox". - fix issue275: Introduce hooks ``tox_runtest_pre``` and diff -r cb509b4c25c3dee2be78c39591f65de25e1058be -r 11c13b4313a44a7872761bd02b031b6a1728369c doc/config.txt --- a/doc/config.txt +++ b/doc/config.txt @@ -105,7 +105,7 @@ **default**:: - python -m pip install {opts} {packages} + pip install {opts} {packages} .. confval:: list_dependencies_command @@ -117,7 +117,7 @@ **default**:: - python -m pip freeze + pip freeze .. confval:: ignore_errors=True|False(default) diff -r cb509b4c25c3dee2be78c39591f65de25e1058be -r 11c13b4313a44a7872761bd02b031b6a1728369c doc/example/jenkins.txt --- a/doc/example/jenkins.txt +++ b/doc/example/jenkins.txt @@ -44,7 +44,7 @@ .. note:: This feature is broken currently because "toxbootstrap.py" - has been removed. Please file an issue if you'd like to + has been removed. Please file an issue if you'd like to see it back. If you manage many Jenkins slaves and want to use the latest officially @@ -151,6 +151,24 @@ same ``tox.ini`` file and make use of automatic sharing of your artifacts between runs or Jenkins jobs. + +Avoiding the "path too long" error with long shebang lines +--------------------------------------------------------------- + +If you are using Jenkins builds you might run into the issue +that tox can not call ``pip`` because the so called "shebang" +line is too long. There is a limit of 127 chars on some systems. +Probably the best way to fix the problem is to use the +new ``--workdir`` option which tells tox to use a specific +directory for its virtualenvironments. Set it to some unique +enough short path. If somebody is interested to do a PR +you could add a new option to tox which uses a random +directory for storing its workdir results and removes +it after the tox run finishes. This could be used +from CI environments where you probably anyway want +to recreate everything on new runs. + + .. include:: ../links.txt diff -r cb509b4c25c3dee2be78c39591f65de25e1058be -r 11c13b4313a44a7872761bd02b031b6a1728369c doc/install.txt --- a/doc/install.txt +++ b/doc/install.txt @@ -4,7 +4,7 @@ Install info in a nutshell ---------------------------------- -**Pythons**: CPython 2.6-3.3, Jython-2.5.1, pypy-1.9ff +**Pythons**: CPython 2.6-3.5, Jython-2.5.1, pypy-1.9ff **Operating systems**: Linux, Windows, OSX, Unix diff -r cb509b4c25c3dee2be78c39591f65de25e1058be -r 11c13b4313a44a7872761bd02b031b6a1728369c setup.py --- a/setup.py +++ b/setup.py @@ -36,7 +36,7 @@ def main(): version = sys.version_info[:2] - install_requires = ['virtualenv>=1.11.2', 'py>=1.4.17', 'pluggy>=0.3.0,<0.4.0'] + install_requires = ['virtualenv>=1.11.2', 'py>=1.4.17', 'pluggy>=0.3.0,<0.5.0'] extras_require = {} if has_environment_marker_support(): extras_require[':python_version=="2.6"'] = ['argparse'] diff -r cb509b4c25c3dee2be78c39591f65de25e1058be -r 11c13b4313a44a7872761bd02b031b6a1728369c tests/test_venv.py --- a/tests/test_venv.py +++ b/tests/test_venv.py @@ -138,8 +138,8 @@ assert len(l) == 2 args = l[-1].args assert l[-1].cwd == venv.envconfig.config.toxinidir - assert "pip" in str(args[2]) - assert args[3] == "install" + assert "pip" in str(args[0]) + assert args[1] == "install" # arg = "--download-cache=" + str(venv.envconfig.downloadcache) # assert arg in args[2:] args = [arg for arg in args if str(arg).endswith("dep1-1.1.zip")] @@ -169,7 +169,7 @@ args = l[-1].args assert l[-1].cwd == venv.envconfig.config.toxinidir assert "pip" in str(args) - assert args[3] == "install" + assert args[1] == "install" assert "dep1" in args assert "dep2" in args deps = list(filter(None, [x[1] for x in venv._getliveconfig().deps])) @@ -366,7 +366,7 @@ venv._install(["hello"], action=action) assert len(l) == 1 args = l[0].args - assert "pip" in [str(x) for x in args] + assert "pip" in args[0] for x in args: assert "--download-cache" not in args, args @@ -598,7 +598,7 @@ venv.run_install_command(packages=["whatever"], action=action) l = mocksession._pcalls assert len(l) == 1 - assert 'pip' in l[0].args[2] + assert 'pip' in l[0].args[0] assert 'install' in l[0].args env = l[0].env assert env is not None diff -r cb509b4c25c3dee2be78c39591f65de25e1058be -r 11c13b4313a44a7872761bd02b031b6a1728369c tox/config.py --- a/tox/config.py +++ b/tox/config.py @@ -182,7 +182,7 @@ class InstallcmdOption: name = "install_command" type = "argv" - default = "python -m pip install {opts} {packages}" + default = "pip install {opts} {packages}" help = "install command for dependencies and package under test." def postprocess(self, testenv_config, value): @@ -529,7 +529,7 @@ parser.add_testenv_attribute( name="list_dependencies_command", type="argv", - default="python -m pip freeze", + default="pip freeze", help="list dependencies for a virtual environment") parser.add_testenv_attribute_obj(DepOption()) https://bitbucket.org/hpk42/tox/commits/1f8ef875278c/ Changeset: 1f8ef875278c User: hpk42 Date: 2016-06-25 11:08:56+00:00 Summary: Merged hpk42/tox into default Affected #: 2 files diff -r 11c13b4313a44a7872761bd02b031b6a1728369c -r 1f8ef875278c3a773812f3ebc3dba8ba0dc6b1e2 doc/drafts/extend-envs-and-packagebuilds.md --- a/doc/drafts/extend-envs-and-packagebuilds.md +++ b/doc/drafts/extend-envs-and-packagebuilds.md @@ -1,5 +1,7 @@ # Extension of environment handling and building packages +Issue reference: #338 + *Notes from a discussion at the pytest sprint 2016* Goal: drive building of packages and the environments needed to test them, exercising the tests and report the results for more than just virtualenvs and python virtualenvs @@ -64,3 +66,19 @@ * Floris: metadata driven. Package has metadata to the env with what env it is compatible * Holger: configuration driven. explicitly configuring which packages should be used (default sdist to be used, overridable by concrete env) * Ronny: "package definitions" (this package, this setup command) + matching definitions (matching packages (with wildcards) for environments) + +### Feature - builddef + +This feature shall allow to specify how to build an artifact in a specific build definition (builddef). + +Currently tox uses the current python interpreter to build the artifact (python package) and thus +does not allow to freely choose the interpreter to build with. +This means that as of now build environment and test environment are different by design. + +Support for different build definitions is implemented by individual tox plugins. +This would result in a collection of plugins supporting different build definitions (e.g. conda, pyenv, docker, rpm) + +Default behavior: + +To keep backwards-compatibility, a python package is built with the python interpreter tox is executed with, +using sdist. This does not require any builddef specification in tox.ini. diff -r 11c13b4313a44a7872761bd02b031b6a1728369c -r 1f8ef875278c3a773812f3ebc3dba8ba0dc6b1e2 doc/drafts/tox_conda_notes_niccodemus.md --- /dev/null +++ b/doc/drafts/tox_conda_notes_niccodemus.md @@ -0,0 +1,84 @@ +[tox] +envlist=py27,py35 + +[testenv] +commands= py.test --timeout=180 {posargs:tests} +deps=pytest>=2.3.5 + pytest-timeout + +# USE CASE 1: plain conda, with deps on tox.ini +create_env_command = conda create --prefix {envdir} python={python_version} +install_command = conda install --prefix {envdir} {opts} {packages} +list_dependencies_command = conda list --prefix {envdir} + +# deprecated: see tox_create_popen hook +linux:env_activate_command=source activate {envdir} +win:env_activate_command=activate.bat {envdir} + +# USE CASE 2: plain conda, using requirements.txt +install_command = conda install --prefix {envdir} {opts} --file requirements.txt + +# USE CASE 3: conda env +create_env_command = conda env create --prefix {envdir} python={python_version} --file environment.yml +install_command = + +[testenv] +type=virtualenv +type=venv +type=conda +type=conda-reqs +type=conda-env + +1. Create a new ``create_env_command`` option. +;2. Create a new ``env_activate_command`` option (also consider how to make that platform dependent). +2. New substitution variable: {python_version} ('3.4', '2.7', etc') +3. env type concept: different types change the default options. + +1. tox_addoption can now add new "testenv" sections to tox.ini: + +[virtualenv] +[conda] +[venv] + +2. extend hooks: + + * tox_addoption + * tox_configure + for each requested env in config: + tox_testenv_up_to_date(envmeta) + tox_testenv_create(envmeta) + tox_testenv_install_deps(envmeta, env) + tox_runtest_pre(envmeta, env) + tox_runtest(envmeta, env, popen) + tox_runtest_post(envmeta, env) + +3. separate virtualenv details from "VirtualEnv" class into a plugin. + +[tox] +envlist={py27,py35}-{sdist,wheel,conda} + +[package-sdist] +command = python setup.py sdist + +[package-wheel] +command = python setup.py bdist_wheel + +[package-conda] +command = conda build ./conda-recipe + +[testenv:{sdist,wheel}] +commands = py.test + +[testenv:conda] +packages = sdist,wheel +commands = py.test --conda-only + +* tox_addoption +* tox_get_python_executable +* tox_configure +for each requested env in config: + tox_testenv_create(envmeta) + tox_testenv_install_deps(envmeta, env) + tox_runtest_pre(envmeta, env) + tox_runtest(envmeta, env, popen) + tox_runtest_post(envmeta, env) Repository URL: https://bitbucket.org/hpk42/tox/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. _______________________________________________ pytest-commit mailing list pytest-commit@python.org https://mail.python.org/mailman/listinfo/pytest-commit