[Impala-ASF-CR](branch-3.4.2) IMPALA-10262: RPM/DEB Packaging Support

2024-04-06 Thread Zihao Ye (Code Review) 
Zihao Ye has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/21130 )

Change subject: IMPALA-10262: RPM/DEB Packaging Support
..

IMPALA-10262: RPM/DEB Packaging Support

This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/

It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.

Tests:
 - Built on Ubuntu 18.04/20.04 and CentOS 7 using
   ./buildall.sh -noclean -skiptests -release -package
 - Deployed the RPM package on a CDP cluster. Verifed the scripts.
 - Deployed the DEB package on a docker container. Verified the scripts.

Resolved trivial backport conflicts in:
 - CMakeLists.txt
 - bin/bootstrap_system.sh
 - bin/jenkins/build-all-flag-combinations.sh
 - buildall.sh
 - docker/install_os_packages.sh

Non-trivial backport notes:
CMake function cmake_host_system_information does not recognize keys of
DISTRIB_ID and DISTRIB_VERSION_ID (required version >= 3.22). Currently
version used in branch-3.4 is 3.14.3. Details to remove using them:
 - One usage of DISTRIB_ID is to skip packaging impala-shell on redhat8.
   Removes it since redhat8 is not supported on branch-3.4.
 - Another usage of DISTRIB_ID is to determine the package file type
   (DEB vs RPM) based on the OS. Replaces it with content of the
   os-release files.
 - Removes the usage of OS_DISTRIB_VERSION_ID in the package file name

Tests:
 - Built on Ubuntu 18.04

Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Reviewed-on: http://gerrit.cloudera.org:8080/18939
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
Reviewed-on: http://gerrit.cloudera.org:8080/21130
Tested-by: Quanlong Huang 
Reviewed-by: Zihao Ye 
---
M CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
A docker/install_os_packages.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestored.sh
A package/bin/stop-catalogd.sh
A package/bin/stop-impalad.sh
A package/bin/stop-statestored.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
23 files changed, 745 insertions(+), 1 deletion(-)

Approvals:
  Quanlong Huang: Verified
  Zihao Ye: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/21130
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-3.4.2
Gerrit-MessageType: merged
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 21130
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Zihao Ye 
Gerrit-Reviewer: ttz <2433038...@qq.com>

[Impala-ASF-CR](branch-3.4.2) IMPALA-10262: RPM/DEB Packaging Support

2024-04-06 Thread Zihao Ye (Code Review) 
Zihao Ye has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21130 )

Change subject: IMPALA-10262: RPM/DEB Packaging Support
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/21130
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-3.4.2
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 21130
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Zihao Ye 
Gerrit-Reviewer: ttz <2433038...@qq.com>
Gerrit-Comment-Date: Sun, 07 Apr 2024 05:03:03 +
Gerrit-HasComments: No

[Impala-ASF-CR](branch-3.4.2) IMPALA-10262: RPM/DEB Packaging Support

2024-04-06 Thread Quanlong Huang (Code Review) 
Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21130 )

Change subject: IMPALA-10262: RPM/DEB Packaging Support
..


Patch Set 2: Verified+1

Verified by https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/19980/


--
To view, visit http://gerrit.cloudera.org:8080/21130
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-3.4.2
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 21130
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Zihao Ye 
Gerrit-Reviewer: ttz <2433038...@qq.com>
Gerrit-Comment-Date: Sat, 06 Apr 2024 22:20:56 +
Gerrit-HasComments: No

[Impala-ASF-CR](branch-3.4.2) IMPALA-10262: RPM/DEB Packaging Support

2024-04-06 Thread Zihao Ye (Code Review) 
Zihao Ye has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21130 )

Change subject: IMPALA-10262: RPM/DEB Packaging Support
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/21130
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-3.4.2
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 21130
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Zihao Ye 
Gerrit-Reviewer: ttz <2433038...@qq.com>
Gerrit-Comment-Date: Sat, 06 Apr 2024 06:58:57 +
Gerrit-HasComments: No

[Impala-ASF-CR](branch-3.4.2) IMPALA-10262: RPM/DEB Packaging Support

2024-03-10 Thread Quanlong Huang (Code Review) 
Hello Impala Public Jenkins,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/21130

to review the following change.


Change subject: IMPALA-10262: RPM/DEB Packaging Support
..

IMPALA-10262: RPM/DEB Packaging Support

This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/

It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.

Tests:
 - Built on Ubuntu 18.04/20.04 and CentOS 7 using
   ./buildall.sh -noclean -skiptests -release -package
 - Deployed the RPM package on a CDP cluster. Verifed the scripts.
 - Deployed the DEB package on a docker container. Verified the scripts.

Resolved trivial backport conflicts in:
 - CMakeLists.txt
 - bin/bootstrap_system.sh
 - bin/jenkins/build-all-flag-combinations.sh
 - buildall.sh
 - docker/install_os_packages.sh

Non-trivial backport notes:
CMake function cmake_host_system_information does not recognize keys of
DISTRIB_ID and DISTRIB_VERSION_ID (required version >= 3.22). Currently
version used in branch-3.4 is 3.14.3. Details to remove using them:
 - One usage of DISTRIB_ID is to skip packaging impala-shell on redhat8.
   Removes it since redhat8 is not supported on branch-3.4.
 - Another usage of DISTRIB_ID is to determine the package file type
   (DEB vs RPM) based on the OS. Replaces it with content of the
   os-release files.
 - Removes the usage of OS_DISTRIB_VERSION_ID in the package file name

Tests:
 - Built on Ubuntu 18.04

Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Reviewed-on: http://gerrit.cloudera.org:8080/18939
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
A docker/install_os_packages.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestored.sh
A package/bin/stop-catalogd.sh
A package/bin/stop-impalad.sh
A package/bin/stop-statestored.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
23 files changed, 745 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/21130/1
--
To view, visit http://gerrit.cloudera.org:8080/21130
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-3.4.2
Gerrit-MessageType: newchange
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 21130
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins