driftx commented on code in PR #2351: URL: https://github.com/apache/cassandra/pull/2351#discussion_r1232406233
########## .build/build-artifacts.sh: ########## @@ -0,0 +1,29 @@ +#!/bin/sh -e +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# variables, with defaults +[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)" + +# pre-conditions +command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; } +command -v pip >/dev/null 2>&1 || { echo >&2 "pip needs to be installed"; exit 1; } Review Comment: I don't think pip is required here, I don't use it to build C* and I commented it out just fine. ########## .build/build-redhat.sh: ########## @@ -0,0 +1,123 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +################################ +# +# Prep +# +################################ + +# variables, w/ defaults, w/ checks +[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)" +[ "x${DIST_DIR}" != "x" ] || DIST_DIR="${CASSANDRA_DIR}/build" +[ "x${RPM_BUILD_DIR}" != "x" ] || RPM_BUILD_DIR="$(mktemp -d /tmp/rpmbuild.XXXXXX)" + +# pre-conditions +command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; } +command -v pip >/dev/null 2>&1 || { echo >&2 "pip needs to be installed"; exit 1; } Review Comment: Pip again, but like debs I question when we would run this outside of docker. ########## .build/run-tests.sh: ########## @@ -0,0 +1,220 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Wrapper script for running a split chunk of an ant test target +# +# Usage: run-tests.sh target [split_chunk] +# split_chunk formatted as "K/N" for the Kth chunk of N chunks + +set -o errexit +set -o pipefail + +# variables, with defaults +[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)" +[ "x${DIST_DIR}" != "x" ] || DIST_DIR="${CASSANDRA_DIR}/build" + +# pre-conditions +command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; } +command -v git >/dev/null 2>&1 || { echo >&2 "git needs to be installed"; exit 1; } +[ -d "${CASSANDRA_DIR}" ] || { echo >&2 "Directory ${CASSANDRA_DIR} must exist"; exit 1; } +[ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; } +[ -d "${DIST_DIR}" ] || { mkdir -p "${DIST_DIR}" ; } + +# print debug information on versions +ant -version +git --version +pip --version Review Comment: There is no check for pip before this, but I question if pip is required. ########## .build/docker/centos7-build.docker: ########## @@ -0,0 +1,85 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM centos:7 +MAINTAINER Apache Cassandra <[email protected]> + +# CONTEXT is expected to be cassandra/.build + +ENV BUILD_HOME=/home/build Review Comment: That's not what docker-compose does and I don't think it's worthy DRYing this up, it's not that much repetition and each docker file stands alone well; different distros may diverge. ########## redhat/noboolean/cassandra.spec: ########## @@ -69,7 +69,7 @@ Cassandra is a distributed (peer-to-peer) system for the management and storage %build export LANG=en_US.UTF-8 export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF-8" -ant clean jar -Dversion=%{upstream_version} +ant jar -Dversion=%{upstream_version} -Dno-checkstyle=true -Drat.skip=true -Dant.gen-doc.skip=true Review Comment: why not clean? shouldn't be needed but won't hurt? ########## .build/run-python-dtests.sh: ########## @@ -0,0 +1,160 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Wrapper script for running a split chunk of a pytest run of cassandra-dtest +# +# Usage: dtest-python.sh target split_chunk +# split_chunk formatted as "K/N" for the Kth chunk of N chunks + +################################ +# +# Prep +# +################################ + +# Pass in target to run, defaults to dtest +DTEST_TARGET="${1:-dtest}" +# Optional: pass in chunk to test, formatted as "K/N" for the Kth chunk of N chunks +DTEST_SPLIT_CHUNK="$2" + +# variables, with defaults +[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)" +[ "x${CASSANDRA_DTEST_DIR}" != "x" ] || CASSANDRA_DTEST_DIR="${CASSANDRA_DIR}/../cassandra-dtest" +[ "x${DIST_DIR}" != "x" ] || DIST_DIR="${CASSANDRA_DIR}/build" + +export CASSANDRA_HOME=${DIST_DIR}/dist +export PYTHONIOENCODING="utf-8" +export PYTHONUNBUFFERED=true +export CASS_DRIVER_NO_EXTENSIONS=true +export CASS_DRIVER_NO_CYTHON=true +export CCM_MAX_HEAP_SIZE="1024M" +export CCM_HEAP_NEWSIZE="512M" +export CCM_CONFIG_DIR=${CASSANDRA_DIR}/build/.ccm +export NUM_TOKENS="16" +#Have Cassandra skip all fsyncs to improve test performance and reliability +export CASSANDRA_SKIP_SYNC=true +export TMPDIR="${DIST_DIR}/tmp" + +# pre-conditions +command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; } +command -v pip3 >/dev/null 2>&1 || { echo >&2 "pip3 needs to be installed"; exit 1; } Review Comment: I don't think pip3 is needed since we're going to activate a virtualenv that has it. ########## .build/build-debian.sh: ########## @@ -0,0 +1,127 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +################################ +# +# Prep +# +################################ + +# variables, with defaults +[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)" +[ "x${DIST_DIR}" != "x" ] || DIST_DIR="${CASSANDRA_DIR}/build" + +# pre-conditions +command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; } +command -v pip >/dev/null 2>&1 || { echo >&2 "pip needs to be installed"; exit 1; } Review Comment: You don't need pip here... but are we going to build packages outside of docker? I think this should be in docker/build-debian.sh only so nobody tries to run this outside of docker. ########## .build/build-owasp.xml: ########## @@ -17,24 +17,26 @@ ~ limitations under the License. --> <project basedir="." name="apache-cassandra-owasp-tasks"> - <property name="dependency-check.version" value="6.3.2"/> - <property name="dependency-check.home" value="${build.dir}/dependency-check-ant-${dependency-check.version}"/> + <property name="dependency-check.version" value="6.4.1"/> + <property name="dependency-check.home.base" value="${build.dir}"/> + <property name="dependency-check.home" value="${dependency-check.home.base}/dependency-check-ant-${dependency-check.version}"/> <condition property="is.dependency.check.jar"> <available file="${dependency-check.home}/dependency-check-ant/dependency-check-ant.jar" type="file" /> </condition> <target name="dependency-check-download" - depends="build" description="Fetch OWASP Dependency checker" unless="is.dependency.check.jar"> <echo>Downloading OWASP Dependency checks ...</echo> <mkdir dir="${dependency-check.home}"/> - <get src="https://github.com/jeremylong/DependencyCheck/releases/download/v${dependency-check.version}/dependency-check-ant-${dependency-check.version}-release.zip"; - dest="${dependency-check.home}/dependency-check-ant-${dependency-check.version}-release.zip"/> + <retry retrycount="3" retrydelay="10" > Review Comment: I think 3 is reasonable before giving up. Owasp fails on downloading stuff fairly often so retrying makes sense. ########## .build/build-jars.sh: ########## @@ -0,0 +1,31 @@ +#!/bin/sh -e +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# temporary between CASSANDRA-18133 and CASSANDRA-18594 + +# variables, with defaults +[ "x${CASSANDRA_DIR}" != "x" ] || CASSANDRA_DIR="$(readlink -f $(dirname "$0")/..)" + +# pre-conditions +command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1; } +command -v pip >/dev/null 2>&1 || { echo >&2 "pip needs to be installed"; exit 1; } Review Comment: No pip needed again. ########## .build/docker/build-debian.sh: ########## @@ -0,0 +1,35 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +if [ "$1" == "-h" ]; then + echo "$0 [-h] [<java_version>]" + echo " build debian packages" + exit 1 +fi + +echo +echo "===" +echo "WARNING: this script modifies local versioned files" Review Comment: What is this referring to? This warning exists for running in docker but not outside of it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
