We've talked in the past about switching build tools. Just to keep that in context - how would you weigh this effort against a tool change?
-=Bill On Thu, Jul 30, 2015 at 4:00 PM, Kevin Sweeney <[email protected]> wrote: > I propose a simplification of the Python BUILD layout as follows and a set > of new conventions, as follows: > > 1) 1 BUILD per 3rd level directory. These are currently > > ``` > % find src/main/python -maxdepth 3 -mindepth 3 -type d |while read dirname; > do echo $dirname | sed 's@src/main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@ > '; > done > apache.aurora.client > apache.aurora.common > apache.aurora.tools > apache.aurora.admin > apache.aurora.executor > apache.aurora.config > apache.thermos.monitoring > apache.thermos.common > apache.thermos.cli > apache.thermos.testing > apache.thermos.core > apache.thermos.runner > apache.thermos.observer > apache.thermos.config > ``` > > 2) Each BUILD file exports 1 python_library that provides a setup_py > containing each python_binary in the BUILD file, named the same as the > directory it's in so that it can be referenced without a ':' character. The > sources field in the python_binary will always be rglobs('*.py'). > > 3) Other BUILD files may only depend on this single public python_library > target. Any other target is considered a private implementation detail and > should be prefixed with an _. > > 4) python_binary targets are always named the same as the exported console > script. > > 5) python_binary targets must have identical dependencies to the > python_library exported by the package and must use entry_point. > > The advantage of this change is that a PEX file generated by pants will > contain exactly the same files that will be available on the PYTHONPATH in > the case of pip-installation of the corresponding library target. This will > help our migration off pants in the future. > > Annotated example: apache.thermos.runner (renamed thermos/bin -> > thermos/runner) > > ``` > % find src/main/python/apache/thermos/runner > src/main/python/apache/thermos/runner > src/main/python/apache/thermos/runner/__init__.py > src/main/python/apache/thermos/runner/thermos_runner.py > src/main/python/apache/thermos/runner/BUILD > % cat src/main/python/apache/thermos/runner/BUILD > # License boilerplate omitted > import os > > > # Private target so that a setup_py can exist without a circular > dependency. Only targets within this file should depend on > # this. > python_library( > name = '_runner', > # The target covers every python file under this directory and > subdirectories. > sources = rglobs('*.py'), > dependencies = [ > '3rdparty/python:twitter.common.app', > '3rdparty/python:twitter.common.log', > # Source dependencies are always referenced without a ':'. > 'src/main/python/apache/thermos/common', > 'src/main/python/apache/thermos/config', > 'src/main/python/apache/thermos/core', > ], > ) > > # Binary target for thermos_runner.pex. Nothing should depend on this - > it's only used as an argument to ./pants binary. > python_binary( > name = 'thermos_runner', > # Use entry_point, not source so the files used here are the same ones > tests see. > entry_point = 'apache.thermos.bin.thermos_runner', > dependencies = [ > # Notice that we depend only on the single private target from this > BUILD file here. > ':_runner', > ], > ) > > # The public library that everyone importing the runner symbols uses. > # The test targets and any other dependent source code should depend on > this. > python_library( > name = 'runner', > dependencies = [ > # Again, notice that we depend only on the single private target from > this BUILD file here. > ':_runner', > ], > # We always provide a setup_py. This will cause any dependee libraries to > automatically reference this library > # in their requirements.txt rather than copy the source files into their > sdist. > provides = setup_py( > # Conventionally named and versioned. > name = 'apache.thermos.runner', > version = open(os.path.join(get_buildroot(), > '.auroraversion')).read().strip().upper(), > ).with_binaries({ > # Every binary in this file should also be repeated here. > # Always use the dict-form of .with_binaries so that commands with > dashes in their names are supported. > # The console script name is always the same as the PEX with .pex > stripped. > 'thermos_runner': ':thermos_runner', > }), > ) > ``` > > Let me know what you think, if y'all agree I'll prepare a patch shortly. >
