This is an automated email from the ASF dual-hosted git repository. wesm pushed a commit to branch maint-0.14.x in repository https://gitbox.apache.org/repos/asf/arrow.git
commit 297c953139e0df8fc25446e4a2b1e1eb4c6b766a Author: Yosuke Shiro <[email protected]> AuthorDate: Fri Jul 5 16:38:14 2019 +0900 ARROW-5784: [Release][GLib] Replace c_glib/ after running c_glib/autogen.sh in dev/release/02-source.sh c_glib/ source archive is generated by `make dist` because includes configure script. The current `dev/release/02-source.sh` build Arrow C++ and Arrow GLib to include the artifacts of GTK-Doc and then run `make dist`. But it is slow. So this PR run only `c_glib/autogen.sh` and then replace c_glib/. Author: Yosuke Shiro <[email protected]> Author: Sutou Kouhei <[email protected]> Closes #4749 from shiro615/release-replace-c-glib-after-running-autogen and squashes the following commits: 9a69f8edc <Yosuke Shiro> Remove an unnecessary environment variable 3a2550fbd <Yosuke Shiro> Remove omit from 02-source-test.rb 501a2dd97 <Yosuke Shiro> Remove autom4te.cache after running autogen.sh 46a4f8995 <Sutou Kouhei> Use docker-compose e357a88b7 <Yosuke Shiro> Exclude c_glib/autom4te.cache/* from RAT check 70cb4a762 <Yosuke Shiro> Remove an unnecessary diff aa786804d <Yosuke Shiro> Enable test test_glib_configure on Travis CI e04276e33 <Yosuke Shiro> Remove libraries for C++ build 56098ae87 <Yosuke Shiro> Replace c_glib/ by c_glib/ after running autogen.sh --- dev/release/02-source-test.rb | 4 ---- dev/release/02-source.sh | 26 +++++++++++--------------- dev/release/source/Dockerfile | 25 ++----------------------- dev/release/source/build.sh | 33 ++++++--------------------------- docker-compose.yml | 10 ++++++++++ 5 files changed, 29 insertions(+), 69 deletions(-) diff --git a/dev/release/02-source-test.rb b/dev/release/02-source-test.rb index e777c75..cf3c887 100644 --- a/dev/release/02-source-test.rb +++ b/dev/release/02-source-test.rb @@ -55,10 +55,6 @@ class SourceTest < Test::Unit::TestCase end def test_glib_configure - unless ENV["ARROW_TEST_SOURCE_GLIB"] == "yes" - omit("This takes a long time. " + - "Set ARROW_TEST_SOURCE_GLIB=yes environment variable to test this.") - end source("GLIB") Dir.chdir("#{@tag_name}/c_glib") do assert_equal([ diff --git a/dev/release/02-source.sh b/dev/release/02-source.sh index edaeec9..ca192a8 100755 --- a/dev/release/02-source.sh +++ b/dev/release/02-source.sh @@ -61,26 +61,22 @@ rm -rf ${tag} git archive ${release_hash} --prefix ${tag}/) | \ tar xf - -# replace c_glib/ by tar.gz generated by "make dist" +# Replace c_glib/ after running c_glib/autogen.sh to create c_gilb/ source archive containing the configure script if [ ${SOURCE_GLIB} -gt 0 ]; then archive_name=tmp-apache-arrow (cd "${SOURCE_TOP_DIR}" && \ git archive ${release_hash} --prefix ${archive_name}/) \ - > ${archive_name}.tar - dist_c_glib_tar_gz=c_glib.tar.gz - docker_image_name=apache-arrow/release-source - DEBUG=yes docker build -t ${docker_image_name} "${SOURCE_DIR}/source" - docker \ - run \ - --rm \ - --interactive \ - --volume "$PWD":/host \ - ${docker_image_name} \ - /build.sh ${archive_name} ${dist_c_glib_tar_gz} - rm -f ${archive_name}.tar + > "${SOURCE_TOP_DIR}/${archive_name}.tar" + c_glib_including_configure_tar_gz=c_glib.tar.gz + "${SOURCE_TOP_DIR}/dev/run_docker_compose.sh" \ + release-source \ + /arrow/dev/release/source/build.sh \ + ${archive_name} \ + ${c_glib_including_configure_tar_gz} + rm -f "${SOURCE_TOP_DIR}/${archive_name}.tar" rm -rf ${tag}/c_glib - tar xf ${dist_c_glib_tar_gz} -C ${tag} - rm -f ${dist_c_glib_tar_gz} + tar xf "${SOURCE_TOP_DIR}/${c_glib_including_configure_tar_gz}" -C ${tag} + rm -f "${SOURCE_TOP_DIR}/${c_glib_including_configure_tar_gz}" fi # Resolve all hard and symbolic links diff --git a/dev/release/source/Dockerfile b/dev/release/source/Dockerfile index 9085cef..7d5453b 100644 --- a/dev/release/source/Dockerfile +++ b/dev/release/source/Dockerfile @@ -15,38 +15,17 @@ # specific language governing permissions and limitations # under the License. -FROM ubuntu:18.04 +FROM debian:buster ENV DEBIAN_FRONTEND noninteractive -RUN apt-get update -y -q && \ - apt-get install -y -q --no-install-recommends wget software-properties-common gpg-agent && \ - wget --quiet -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - && \ - apt-add-repository -y "deb http://apt.llvm.org/bionic llvm-toolchain-bionic-7 main" && \ - apt-get -y install clang-7 - RUN apt update && \ apt install -y -V \ autoconf-archive \ - bison \ - cmake \ - flex \ - g++ \ - gcc \ gtk-doc-tools \ - libboost-filesystem-dev \ - libboost-regex-dev \ - libboost-system-dev \ libgirepository1.0-dev \ libglib2.0-doc \ - libprotobuf-dev \ - libprotoc-dev \ libtool \ - lsb-release \ - make \ - pkg-config \ - protobuf-compiler && \ + pkg-config && \ apt clean && \ rm -rf /var/lib/apt/lists/* - -COPY build.sh /build.sh diff --git a/dev/release/source/build.sh b/dev/release/source/build.sh index 20c9724..558600e 100755 --- a/dev/release/source/build.sh +++ b/dev/release/source/build.sh @@ -20,35 +20,14 @@ set -e archive_name=$1 -dist_c_glib_tar_gz=$2 +c_glib_including_configure_tar_gz=$2 -tar xf /host/${archive_name}.tar +tar xf /arrow/${archive_name}.tar -# build Apache Arrow C++ before building Apache Arrow GLib because -# Apache Arrow GLib requires Apache Arrow C++. -mkdir -p ${archive_name}/cpp/build -cpp_install_dir=${PWD}/${archive_name}/cpp/install -cd ${archive_name}/cpp/build -cmake .. \ - -DCMAKE_INSTALL_PREFIX=${cpp_install_dir} \ - -DCMAKE_INSTALL_LIBDIR=lib \ - -DARROW_PLASMA=yes \ - -DARROW_GANDIVA=yes \ - -DARROW_PARQUET=yes -make -j8 -make install -cd - - -# build source archive for Apache Arrow GLib by "make dist". +# Run autogen.sh to create c_glib/ source archive containing the configure script cd ${archive_name}/c_glib ./autogen.sh -./configure \ - PKG_CONFIG_PATH=${cpp_install_dir}/lib/pkgconfig \ - --enable-gtk-doc -LD_LIBRARY_PATH=${cpp_install_dir}/lib make -j8 -make dist -tar xzf *.tar.gz -rm *.tar.gz +rm -rf autom4te.cache cd - -mv ${archive_name}/c_glib/apache-arrow-glib-* c_glib/ -tar czf /host/${dist_c_glib_tar_gz} c_glib +mv ${archive_name}/c_glib/ c_glib/ +tar czf /arrow/${c_glib_including_configure_tar_gz} c_glib diff --git a/docker-compose.yml b/docker-compose.yml index 2fa5ab4..c141a70 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -597,6 +597,16 @@ services: # TODO(kszucs): hive-integration + ################################# Release ################################### + + release-source: + image: arrow:release-source + build: + context: . + dockerfile: dev/release/source/Dockerfile + volumes: + - .:/arrow:delegated + ######################## Verification Containers ############################ debian-stretch:
