This is an automated email from the ASF dual-hosted git repository. bbannier pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 37d76fff124d28a0281b9231058bb1b92fc65abe Author: Benjamin Bannier <[email protected]> AuthorDate: Wed Sep 18 11:37:15 2019 +0200 Removed old mesos-style and references. This patch removes references to `support/mesos-style.py` which was replaced with a pre-commit setup in a previous commit. We also remove the tool itself. Review: https://reviews.apache.org/r/71206/ --- docs/c++-style-guide.md | 2 +- support/build-virtualenv | 72 ------ support/hooks/commit-msg | 46 ---- support/hooks/post-rewrite | 40 ---- support/hooks/pre-commit | 41 ---- support/mesos-split.py | 2 - support/mesos-style.py | 557 ++------------------------------------------- 7 files changed, 17 insertions(+), 743 deletions(-) diff --git a/docs/c++-style-guide.md b/docs/c++-style-guide.md index 8a48afe..8df99c8 100644 --- a/docs/c++-style-guide.md +++ b/docs/c++-style-guide.md @@ -7,7 +7,7 @@ layout: documentation The Mesos codebase follows the [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html) with some notable differences, as described below. -Note that the [clang-format](clang-format.md) tool can be helpful to ensure that some of the mechanical style rules are obeyed. Commits should be checked with the script `support/mesos-style.py` for high-level conformance, or with `support/mesos-tidy.sh` for conformance to low-level expectations. +Note that the [clang-format](clang-format.md) tool can be helpful to ensure that some of the mechanical style rules are obeyed. Commits should be checked with the command `pre-commit run cpplint` for high-level conformance, or with `support/mesos-tidy.sh` for conformance to low-level expectations. ## Scoping diff --git a/support/build-virtualenv b/support/build-virtualenv deleted file mode 100755 index 7dc03b0..0000000 --- a/support/build-virtualenv +++ /dev/null @@ -1,72 +0,0 @@ -#!/usr/bin/env bash - -# This script sets up a Python virtualenv for the Web UI. This creates -# a new virtualenv and installs nodeenv inside the virtualenv. - -set -e -trap "exit 1" INT - -CURRDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" - -: ${VIRTUALENV_NAME:="linters"} -: ${VIRTUALENV_DIRECTORY:="${CURRDIR}/.virtualenv"} - -: ${PYTHON:="$(which python)"} -: ${VIRTUALENV:="$(which virtualenv)"} - -OLD_PYTHONPATH="${PYTHONPATH}" -PYTHONPATH="" - -# If we already have a virtual environment activated, -# bail out and advise the user to deactivate. -OLD_VIRTUAL_ENV="${VIRTUAL_ENV}" -if [ "${OLD_VIRTUAL_ENV}" != "" ]; then - echo "Please deactivate your current virtual environment in order to continue." - echo "source deactivate" - exit 1 -fi - -# Verify that python is installed. -if [ "${PYTHON}" = "" ]; then - echo "You must have python installed in order to continue." - exit 1 -fi - -# Old versions of virtualenv do not remove the bin directory in the -# virtual environment even when using `--clear`. We thus remove the -# entire directory in case the virtual environment already exists. -# See https://github.com/pypa/virtualenv/issues/2 for more info. -rm -rf ${VIRTUALENV_DIRECTORY} - -PYTHON_MAJOR=$(${PYTHON} -c 'import sys; print(sys.version_info[0])') -PYTHON_MINOR=$(${PYTHON} -c 'import sys; print(sys.version_info[1])') - -if [ "${PYTHON_MAJOR}" = "3" ]; then - if [ "${PYTHON_MINOR}" -lt "6" ]; then - echo "You must be running python 3.6 or newer in order to continue." - echo "Consider running as 'PYTHON=python3 ${0}' or similar." - exit 1 - else - # Set up a virtual environment for the linters. - ${PYTHON} -m venv --prompt="${VIRTUALENV_NAME}" ${VIRTUALENV_DIRECTORY} - fi -else - echo "You must be running python 3.6 or newer in order to continue." - echo "Consider running as 'PYTHON=python3 ${0}' or similar." - exit 1 -fi - -source ${VIRTUALENV_DIRECTORY}/bin/activate -pip install --upgrade pip -pip install -r ${CURRDIR}/pip-requirements.txt - -# Add Node.js virtual environment to the existing virtual environment. -nodeenv -p - -# Restart the virtual environment to then have npm available. -deactivate -source ${VIRTUALENV_DIRECTORY}/bin/activate - -# Install the JavaScript linter in the virtual environment. -npm install -g [email protected] -deactivate diff --git a/support/hooks/commit-msg b/support/hooks/commit-msg deleted file mode 100755 index a0c218d..0000000 --- a/support/hooks/commit-msg +++ /dev/null @@ -1,46 +0,0 @@ -#!/usr/bin/env bash -# -# A hook script to verify commit message format. Called by "git commit" -# with one argument, the name of the file that has the commit message. -# The hook exits with non-zero status after issuing an appropriate -# message if it wants to stop the commit. The hook is allowed to edit the -# commit message file. -# -# To enable this hook, run `bootstrap` or do this from the root of the repo: -# -# $ ln -s ../../support/hooks/commit-msg .git/hooks/commit-msg - -COMMIT_MESSAGE=$(cat "$1") -FIRST_LINE=$(head -n 1 "$1") -LAST_CHAR=$(echo -n "$FIRST_LINE" | tail -c 1) -FIRST_CHAR=$(echo -n "$FIRST_LINE" | head -c 1) - -if [[ "$FIRST_LINE" =~ ^(fixup|squash)\! ]]; then - # If the commit starts with fixup! or squash! we ignore everything. - # The message will be checked when the final commit is made. - exit 0 -fi - -while read -r LINE -do - # In verbose mode, the diff of the commit is included in the commit message. - # Since git looks for the following line and skips everything after it, - # we also skip everything once this line is observed. - if [ "$LINE" = "# ------------------------ >8 ------------------------" ]; then break; fi - if [ "$(echo -n "$LINE" | head -c 1)" = "#" ]; then continue; fi - LENGTH=$(echo -n "$LINE" | wc -c) - if [ "$LENGTH" -gt "72" ]; then - echo >&2 "Error: No line in the commit message summary may exceed 72 characters." - exit 1 - fi -done <<< "$COMMIT_MESSAGE" - -if [[ ! "$FIRST_CHAR" =~ [A-Z] ]]; then - echo >&2 "Error: Commit message summary (the first line) must start with a capital letter." - exit 1 -fi - -if [ "$LAST_CHAR" != "." ]; then - echo >&2 "Error: Commit message summary (the first line) must end in a period." - exit 1 -fi diff --git a/support/hooks/post-rewrite b/support/hooks/post-rewrite deleted file mode 100755 index 1ab14ab..0000000 --- a/support/hooks/post-rewrite +++ /dev/null @@ -1,40 +0,0 @@ -#!/bin/sh -# -# A hook script to verify what is about to be committed. -# Called by "git commit --amend" or "git rebase". The hook exits with -# non-zero status and warning messages if the files being rewritten do -# not conform to the Mesos style. -# -# To enable this hook, do this from the root of the repo: -# -# $ ln -s ../../support/hooks/post_rewrite .git/hooks/post_rewrite - -# Redirect output to stderr. -exec 1>&2 - -# If there are whitespace errors, print the offending file names. -## In git, '@~' represent previous commit. We check the whitespace between -## current head and previous commit after a commit is rewritten. -git diff-index --check @~ -- || exit 1 - -# Check Mesos style. -## In git, '@' represent current head, '@~' represent previous commit. We check -## the style of changes between current head and previous commit after a commit -## is rewritten. -ADDED_OR_MODIFIED=$(git diff --name-only --diff-filter=AM @~..@) -if [ "$ADDED_OR_MODIFIED" ]; then - # NOTE: We choose to implement this as a conditional rather than a call to - # `xargs` on purpose. Some implementations of `xargs` will call your script - # even if the arguments you pass in are empty. In our case, this would - # cause us to erroneously lint every file in the repository. Additionally, - # many implementations do not support the `-r` flag, (which instructs - # `xargs` to not run the script if the arguments are empty), so we also - # cannot use that. - ./support/mesos-style.py $ADDED_OR_MODIFIED || exit 1 -fi - -# Check that the commits are properly split between mesos, libprocess and stout. -## In git, '@' represent current head, '@~' represent previous commit. We check -## the style of changes between current head and previous commit after a commit -## is rewritten. -git diff --name-only --diff-filter=AMD @~..@ | xargs ./support/mesos-split.py || exit 1 diff --git a/support/hooks/pre-commit b/support/hooks/pre-commit deleted file mode 100755 index 519567b..0000000 --- a/support/hooks/pre-commit +++ /dev/null @@ -1,41 +0,0 @@ -#!/bin/sh -# -# A hook script to verify what is about to be committed. -# Called by "git commit" with no arguments. The hook exits with -# non-zero status if the files being committed do not conform to -# the Mesos style. -# -# To enable this hook, do this from the root of the repo: -# -# $ ln -s ../../support/hooks/pre-commit .git/hooks/pre-commit - -if git rev-parse --verify HEAD >/dev/null 2>&1 -then - against=HEAD -else - # Initial commit: diff against an empty tree object - against=4b825dc642cb6eb9a060e54bf8d69288fbee4904 -fi - -# Redirect output to stderr. -exec 1>&2 - -# If there are whitespace errors, print the offending file names and fail. -git diff-index --check --cached $against -- || exit 1 - -# Check Mesos style. -ADDED_OR_MODIFIED=$(git diff --cached --name-only --diff-filter=AM) -if [ -n "$ADDED_OR_MODIFIED" ]; then - # NOTE: We choose to implement this as a conditional rather than a call to - # `xargs` on purpose. Some implementations of `xargs` will call your script - # even if the arguments you pass in are empty. In our case, this would - # cause us to erroneously lint every file in the repository. Additionally, - # many implementations do not support the `-r` flag, (which instructs - # `xargs` to not run the script if the arguments are empty), so we also - # cannot use that. - ./support/mesos-style.py $ADDED_OR_MODIFIED || exit 1 -fi - -# Check that the commits are properly split between mesos, libprocess and stout. -# TODO(ArmandGrillet): Remove the if to really switch to Python 3. -git diff --cached --name-only --diff-filter=AMD | xargs ./support/mesos-split.py || exit 1 diff --git a/support/mesos-split.py b/support/mesos-split.py index 0a77c25..aecad3f 100755 --- a/support/mesos-split.py +++ b/support/mesos-split.py @@ -58,8 +58,6 @@ def find_project(filename): def main(): """ Expects a list of filenames on the command line. - - See `support/hooks/pre-commit` for the canonical usage of this method. """ touched_projects = defaultdict(list) for filename in sys.argv[1:]: diff --git a/support/mesos-style.py b/support/mesos-style.py index b3e20fd..4d9d3b4 100755 --- a/support/mesos-style.py +++ b/support/mesos-style.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -# + # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information @@ -15,551 +15,26 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +"""This is a dummy script to ease the transition from mesos-style to pre-commit +hooks. This script can be removed after we have given contributors enough time +to adjust their checkouts.""" -"""Runs checks for mesos style.""" - -import os -import platform -import re -import string -import subprocess import sys -from pathlib import PurePath - - -class LinterBase(): - """ - This is an abstract class that provides the base functionality for - linting files in the mesos project. Its 'main()' function - walks through the set of files passed to it and runs some - standard linting over them. This includes checking for license headers - and checking for non-supported characters. From there it calls a - 'run_lint()' function that can be overridden to provide - customizable style checks for a specific class of files (e.g. C++, - Python, etc.). - - Any class that extends from 'LinterBase' should override the - following class variables / functions: - - linter_type - source_dirs - exclude_files - source_files - comment_prefix - - run_lint() - - Please see the comments below for details on how to override each - variable. - """ - # The name of the linter to help with printing which linter files - # are currently being processed by. - linter_type = '' - - # Root source paths (will be traversed recursively). - source_dirs = [] - - # Add file paths and patterns which should not be checked - # This should include 3rdparty libraries, includes and machine generated - # source. - exclude_files = '' - - # A regex of possible matches for your source files. - source_files = '' - - # A prefix at the beginning of the line to demark comments (e.g. '//') - comment_prefix = '' - - def check_encoding(self, source_paths): - """ - Checks for encoding errors in the given files. Source - code files must contain only printable ascii characters. - This excludes the extended ascii characters 128-255. - http://www.asciitable.com/ - """ - error_count = 0 - for path in source_paths: - with open(path) as source_file: - for line_number, line in enumerate(source_file): - # If we find an error, add 1 to both the character and - # the line offset to give them 1-based indexing - # instead of 0 (as is common in most editors). - char_errors = [offset for offset, char in enumerate(line) - if char not in string.printable] - if char_errors: - sys.stderr.write( - "{path}:{line_number}: Non-printable characters" - " found at [{offsets}]: \"{line}\"\n".format( - path=path, - line_number=line_number + 1, - offsets=', '.join([str(offset + 1) for offset - in char_errors]), - line=line.rstrip())) - error_count += 1 - - return error_count - - def check_license_header(self, source_paths): - """Checks the license headers of the given files.""" - error_count = 0 - for path in source_paths: - with open(path) as source_file: - # We read the three first lines of the file as the - # first line could be a shebang and the second line empty. - head = "".join([next(source_file) for _ in range(3)]) - - # TODO(bbannier) We allow `Copyright` for - # currently deviating files. This should be - # removed one we have a uniform license format. - regex = r'^{comment_prefix} [Licensed|Copyright]'.format( - comment_prefix=self.comment_prefix) - # pylint: disable=no-member - regex = re.compile(regex, re.MULTILINE) - - if not regex.search(head): - sys.stderr.write( - "{path}:1: A license header should appear's on one of" - " the first line of the file starting with" - " '{comment_prefix} Licensed'.: {head}".format( - path=path, - head=head, - comment_prefix=self.comment_prefix)) - error_count += 1 - - return error_count - - def find_candidates(self, root_dir): - """ - Search through the all files rooted at 'root_dir' and compare - them against 'self.exclude_files' and 'self.source_files' to - come up with a set of candidate files to lint. - """ - exclude_file_regex = re.compile(self.exclude_files) - source_criteria_regex = re.compile(self.source_files) - for root, _, files in os.walk(root_dir): - for name in files: - path = os.path.join(root, name) - if exclude_file_regex.search(path) is not None: - continue - - if source_criteria_regex.search(name) is not None: - yield path - - def run_command_in_virtualenv(self, command): - """ - Activate the virtual environment, run the - given command and return its output. - """ - virtualenv = os.path.join('support', '.virtualenv') - - if platform.system() == 'Windows': - command = r'{virtualenv_path}\Scripts\activate.bat & {cmd}'.format( - virtualenv_path=virtualenv, cmd=command) - else: - command = '. {virtualenv_path}/bin/activate; {cmd}'.format( - virtualenv_path=virtualenv, cmd=command) - - return subprocess.Popen(command, shell=True, stdout=subprocess.PIPE) - - # pylint: disable=unused-argument - def run_lint(self, source_paths): - """ - A custom function to provide linting for 'linter_type'. - It takes a list of source files to lint and returns the number - of errors found during the linting process. - - It should print any errors as it encounters them to provide - feedback to the caller. - """ - return 0 - - def main(self, modified_files): - """ - This function takes a list of files and lints them for the - class of files defined by 'linter_type'. - """ - - # Verify that source roots are accessible from current - # working directory. A common error could be to call - # the style checker from other (possibly nested) paths. - for source_dir in self.source_dirs: - if not os.path.exists(source_dir): - print("Could not find '{dir}'".format(dir=source_dir)) - print('Please run from the root of the mesos source directory') - exit(1) - - # Add all source file candidates to candidates list. - candidates = [] - for source_dir in self.source_dirs: - for candidate in self.find_candidates(source_dir): - candidates.append(candidate) - - # Normalize paths of any files given. - modified_files = [os.fspath(PurePath(f)) for f in modified_files] - - # If file paths are specified, check all file paths that are - # candidates; else check all candidates. - file_paths = modified_files if modified_files else candidates - - # Compute the set intersect of the input file paths and candidates. - # This represents the reduced set of candidates to run lint on. - candidates_set = set(candidates) - clean_file_paths_set = set(path.rstrip() for path in file_paths) - filtered_candidates_set = clean_file_paths_set.intersection( - candidates_set) - - if filtered_candidates_set: - plural = '' if len(filtered_candidates_set) == 1 else 's' - print('Checking {num_files} {linter} file{plural}'.format( - num_files=len(filtered_candidates_set), - linter=self.linter_type, - plural=plural)) - - license_errors = self.check_license_header(filtered_candidates_set) - encoding_errors = self.check_encoding(list(filtered_candidates_set)) - lint_errors = self.run_lint(list(filtered_candidates_set)) - total_errors = license_errors + encoding_errors + lint_errors - - if total_errors > 0: - sys.stderr.write('Total errors found: {num_errors}\n'.format( - num_errors=total_errors)) - - return total_errors - - return 0 - - -class CppLinter(LinterBase): - """The linter for C++ files, uses cpplint.""" - linter_type = 'C++' - - source_dirs = ['src', - 'include', - os.path.join('3rdparty', 'libprocess'), - os.path.join('3rdparty', 'stout')] - - exclude_files = '(' \ - r'elfio\-3\.2|' \ - r'protobuf\-2\.4\.1|' \ - r'googletest\-release\-1\.8\.0|' \ - r'glog\-0\.3\.3|' \ - r'boost\-1\.53\.0|' \ - r'libev\-4\.15|' \ - r'\.pb\.cc|\.pb\.h|\.md|\.virtualenv' \ - ')' - - source_files = r'\.(cpp|hpp|cc|h)$' - - comment_prefix = r'\/\/' - - def run_lint(self, source_paths): - """ - Runs cpplint over given files. - - http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py - """ - # We do not use a version of cpplint available through pip as - # we use a custom version (see cpplint.path) to lint C++ files. - process = subprocess.Popen( - [sys.executable, 'support/cpplint.py', '--extensions=hpp,cpp'] + - source_paths, - stderr=subprocess.PIPE, - close_fds=True) - - # Lines are stored and filtered, only showing found errors instead - # of e.g., 'Done processing XXX.' which tends to be dominant output. - for line in process.stderr: - line = line.decode(sys.stdout.encoding) - if re.match('^(Done processing |Total errors found: )', line): - continue - sys.stderr.write(line) - - process.wait() - return process.returncode - - -class JsLinter(LinterBase): - """The linter for JavaScript files, uses eslint.""" - linter_type = 'JavaScript' - - source_dirs = [os.path.join('src', 'webui')] - - exclude_files = '(' \ - r'angular\-1\.2\.32|' \ - r'angular\-route\-1\.2\.32|' \ - r'bootstrap\-table\-1\.11\.1|' \ - r'clipboard\-1\.5\.16|' \ - r'jquery\-3\.2\.1|' \ - r'relative\-date|' \ - r'ui\-bootstrap\-tpls\-0\.9\.0|' \ - r'angular\-route\-1\.2\.32|' \ - r'underscore\-1\.4\.3' \ - ')' - - source_files = r'\.(js)$' - - comment_prefix = '//' - - def run_lint(self, source_paths): - """ - Runs eslint over given files. - - https://eslint.org/docs/user-guide/configuring - """ - - num_errors = 0 - - source_files = ' '.join(source_paths) - config_path = os.path.join('support', '.eslintrc.js') - - process = self.run_command_in_virtualenv( - 'eslint {files} -c {config} -f compact'.format( - files=source_files, - config=config_path - ) - ) - - for line in process.stdout: - line = line.decode(sys.stdout.encoding) - if "Error -" in line or "Warning -" in line: - sys.stderr.write(line) - if "Error -" in line: - num_errors += 1 - - return num_errors - - -class PyLinter(LinterBase): - """The linter for Python files, uses pylint.""" - linter_type = 'Python' - - cli_dir = os.path.join('src', 'python', 'cli_new') - lib_dir = os.path.join('src', 'python', 'lib') - support_dir = os.path.join('support') - source_dirs_to_lint_with_venv = [support_dir] - source_dirs_to_lint_with_tox = [cli_dir, lib_dir] - source_dirs = source_dirs_to_lint_with_tox + source_dirs_to_lint_with_venv - - exclude_files = '(' \ - r'protobuf\-2\.4\.1|' \ - r'googletest\-release\-1\.8\.0|' \ - r'glog\-0\.3\.3|' \ - r'boost\-1\.53\.0|' \ - r'libev\-4\.15|' \ - r'\.virtualenv|' \ - r'\.tox' \ - ')' - - source_files = r'\.(py)$' - - comment_prefix = '#' - - pylint_config = os.path.abspath(os.path.join('support', 'pylint.config')) - - def run_tox(self, configfile, args, tox_env=None, recreate=False): - """ - Runs tox with given configfile and args. Optionally set tox env - and/or recreate the tox-managed virtualenv. - """ - support_dir = os.path.dirname(__file__) - bin_dir = 'Script' if platform.system() == 'Windows' else 'bin' - - cmd = [os.path.join(support_dir, '.virtualenv', bin_dir, 'tox')] - cmd += ['-qq'] - cmd += ['-c', configfile] - if tox_env is not None: - cmd += ['-e', tox_env] - if recreate: - cmd += ['--recreate'] - cmd += ['--'] - cmd += args - - # We do not use `run_command_in_virtualenv()` here, as we - # directly call `tox` from inside the virtual environment bin - # directory without activating the virtualenv. - return subprocess.Popen(cmd, stdout=subprocess.PIPE) - - def filter_source_files(self, source_dir, source_files): - """ - Filters out files starting with source_dir. - """ - return [f for f in source_files if f.startswith(source_dir)] - - def lint_source_files_under_source_dir(self, source_dir, source_files): - """ - Runs pylint directly or indirectly throgh tox on source_files which - are under source_dir. If tox is to be used, it must be configured - in source_dir, i.e. a tox.ini must be present. - """ - filtered_source_files = self.filter_source_files( - source_dir, source_files) - - if not filtered_source_files: - return 0 - - if source_dir in self.source_dirs_to_lint_with_tox: - process = self.run_tox( - configfile=os.path.join(source_dir, 'tox.ini'), - args=['--rcfile='+self.pylint_config] + filtered_source_files, - tox_env='py3-lint') - else: - process = self.run_command_in_virtualenv( - 'pylint --score=n --rcfile={rcfile} {files}'.format( - rcfile=self.pylint_config, - files=' '.join(filtered_source_files))) - - num_errors = 0 - for line in process.stdout: - line = line.decode(sys.stdout.encoding) - if re.search(r'[RCWEF][0-9]{4}:', line): - num_errors += 1 - sys.stderr.write(line) - - return num_errors - - def run_lint(self, source_paths): - """ - Runs pylint over given files. - - https://google.github.io/styleguide/pyguide.html - """ - num_errors = 0 - - for source_dir in self.source_dirs: - num_errors += self.lint_source_files_under_source_dir( - source_dir, source_paths) - - return num_errors - - -def should_build_virtualenv(modified_files): - """ - Check if we should build the virtual environment required. - This is the case if the requirements of the environment - have changed or if the support script is run with no - arguments (meaning that the entire codebase should be linted). - """ - # NOTE: If the file list is empty, we are linting the entire test - # codebase. We should always rebuild the virtualenv in this case. - if not modified_files: - return True - - support_dir = os.path.dirname(__file__) - bin_dir = 'Script' if platform.system() == 'Windows' else 'bin' - - interpreter = os.path.basename(sys.executable) - interpreter = os.path.join(support_dir, '.virtualenv', bin_dir, interpreter) - if not os.path.isfile(interpreter): - return True - - basenames = [os.path.basename(path) for path in modified_files] - - if 'pip-requirements.txt' in basenames: - print('The "pip-requirements.txt" file has changed.') - return True - - if 'build-virtualenv' in basenames: - print('The "build-virtualenv" file has changed.') - return True - - # The JS and Python linters require a virtual environment. - # If all the files modified are not JS or Python files, - # we do not need to build the virtual environment. - # TODO(ArmandGrillet): There should be no duplicated logic to know - # which linters to instantiate depending on the files to analyze. - if not os.path.isdir(os.path.join('support', '.virtualenv')): - js_and_python_files = [JsLinter().source_files, PyLinter().source_files] - js_and_python_files_regex = re.compile('|'.join(js_and_python_files)) - - for basename in basenames: - if js_and_python_files_regex.search(basename) is not None: - print('Virtualenv not detected and required... building') - return True - - return False - - -def build_virtualenv(): - """ - Rebuild the virtualenv by running a bootstrap script. - This will exit the program if there is a failure. - """ - print('Rebuilding virtualenv...') - - python3_env = os.environ.copy() - python3_env["PYTHON"] = sys.executable - - build_virtualenv_file = [os.path.join('support', 'build-virtualenv')] - - if platform.system() == 'Windows': - # TODO(andschwa): Port more of the `build-virtualenv` Bash script. - python_dir = os.path.dirname(sys.executable) - virtualenv = os.path.join(python_dir, 'Scripts', 'virtualenv.exe') - build_virtualenv_file = [virtualenv, - '--no-site-packages', - 'support/.virtualenv'] - - process = subprocess.Popen( - build_virtualenv_file, - env=python3_env, - stdout=subprocess.PIPE) - - output = '' - for line in process.stdout: - output += line.decode(sys.stdout.encoding) - - process.wait() - - if process.returncode != 0: - sys.stderr.write(output) - sys.exit(1) - - # TODO(andschwa): Move this into a script like above. - if platform.system() == 'Windows': - def run_command_in_virtualenv(command): - """ - Stolen from `PyLinter`, runs command in virtualenv. - """ - virtualenv = os.path.join('support', - '.virtualenv', - 'Scripts', - 'activate.bat') - command = '{virtualenv_path} & {cmd}'.format( - virtualenv_path=virtualenv, cmd=command) - - return subprocess.Popen(command, shell=True, stdout=subprocess.PIPE) - - pip_install_pip = 'python.exe -m pip install --upgrade pip' - process = run_command_in_virtualenv(pip_install_pip) - for line in process.stdout: - output += line.decode(sys.stdout.encoding) - process.wait() +error = """\ +'mesos-style.py' was removed in favor of hooks managed by pre-commit. - if process.returncode != 0: - sys.stderr.write(output) - sys.exit(1) +Linting requires an installation of pre-commit, see +https://pre-commit.com/#install, e.g., - pip_reqs = 'python.exe -m pip install -r support/pip-requirements.txt' - process = run_command_in_virtualenv(pip_reqs) - for line in process.stdout: - output += line.decode(sys.stdout.encoding) - process.wait() + $ pip3 install pre-commit - if process.returncode != 0: - sys.stderr.write(output) - sys.exit(1) +After installing pre-commit, remove existing hooks in '.git/hooks' +and reinstall hooks with './support/setup-dev.sh'. -if __name__ == '__main__': - if should_build_virtualenv(sys.argv[1:]): - build_virtualenv() + $ rm -rfv .git/hooks/* + $ ./support/setup-dev.sh +""" - # TODO(ArmandGrillet): We should only instantiate the linters - # required to lint the files to analyze. See MESOS-8351. - CPP_LINTER = CppLinter() - CPP_ERRORS = CPP_LINTER.main(sys.argv[1:]) - JS_LINTER = JsLinter() - JS_ERRORS = JS_LINTER.main(sys.argv[1:]) - PY_LINTER = PyLinter() - PY_ERRORS = PY_LINTER.main(sys.argv[1:]) - sys.exit(CPP_ERRORS + JS_ERRORS + PY_ERRORS) +print(error, file=sys.stderr) +sys.exit(1)
