Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 )
Change subject: IMPALA-7698: Add centos support to bootstrap_system. ...................................................................... Patch Set 1: (10 comments) Thanks for the reviews! http://gerrit.cloudera.org:8080/#/c/11731/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11731/1//COMMIT_MSG@29 PS1, Line 29: I'm not yet clear on why the ORC library is choosing to open that time zone : file (which doesn't exist), but this change seems workable even with the test : failures. > At a short glance, it looks like the writer(hive) writes an abnormal timezo Hi Quanlong! The ORC files themselves are small (only 50KB) and I pushed them to https://github.com/philz/tmp/blob/master/orc.zip . I've attached them to the JIRA as well. The schema is as follows: [localhost:21000] default> show create table functional_orc_def.alltypes; Query: show create table functional_orc_def.alltypes +-------------------------------------------------------------------+ | result | +-------------------------------------------------------------------+ | CREATE EXTERNAL TABLE functional_orc_def.alltypes ( | | id INT COMMENT 'Add a comment', | | bool_col BOOLEAN, | | tinyint_col TINYINT, | | smallint_col SMALLINT, | | int_col INT, | | bigint_col BIGINT, | | float_col FLOAT, | | double_col DOUBLE, | | date_string_col STRING, | | string_col STRING, | | timestamp_col TIMESTAMP | | ) | | PARTITIONED BY ( | | year INT, | | month INT | | ) | | STORED AS ORC | | LOCATION 'hdfs://localhost:20500/test-warehouse/alltypes_orc_def' | | | +-------------------------------------------------------------------+ Fetched 1 row(s) in 0.02s http://gerrit.cloudera.org:8080/#/c/11731/1//COMMIT_MSG@33 PS1, Line 33: centos:7 is not addressed by this change. The move to systemd makes "service > nit: long lines Done http://gerrit.cloudera.org:8080/#/c/11731/1/bin/bootstrap_system.sh File bin/bootstrap_system.sh: http://gerrit.cloudera.org:8080/#/c/11731/1/bin/bootstrap_system.sh@104 PS1, Line 104: REAL_APT_GET=$(ubuntu which apt-get) > Do you want to do something similar for yum? IIRC, i wrote this block becau Per https://linux.die.net/man/5/yum.conf, yum seems to have a default of 10. In practice, I've seen apt-get fail mostly when a machine is booting and some Ubuntu auto-upgrade thing has the apt lock. I've not yet experienced yum failing similarly. Anyway, I just added this as a comment. http://gerrit.cloudera.org:8080/#/c/11731/1/bin/bootstrap_system.sh@131 PS1, Line 131: psmisc lsof sudo openssh-server redhat-lsb java-1.8.0-openjdk-devel \ > tab used for whitespace Done http://gerrit.cloudera.org:8080/#/c/11731/1/bin/bootstrap_system.sh@143 PS1, Line 143: redhat sudo wget -nv http://www-us.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz http://www-us.apache.org/dist/ant/binaries/apache-ant-1.9.13-bin.tar.gz > httpS and check signatures and checksums? Done http://gerrit.cloudera.org:8080/#/c/11731/1/bin/bootstrap_system.sh@143 PS1, Line 143: redhat sudo wget -nv http://www-us.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz http://www-us.apache.org/dist/ant/binaries/apache-ant-1.9.13-bin.tar.gz > line too long (181 > 90) I'm ignoring a certain amount of long line stuff here because shortening sha512sums and long urls isn't particularly useful. http://gerrit.cloudera.org:8080/#/c/11731/1/bin/bootstrap_system.sh@194 PS1, Line 194: # redhat sudo bash -c "echo host all all 127.0.0.1/32 md5 >> /var/lib/pgsql/data/pg_hba.conf" > I think this line was accidentally left in. It's similar. I updated the comment about security implications, which I think is now truer than the current TODO. http://gerrit.cloudera.org:8080/#/c/11731/1/bin/bootstrap_system.sh@257 PS1, Line 257: redhat sudo sed -i 's,\*\s*soft\s*nproc\s*1024,* soft nproc unlimited,' /etc/security/limits.d/90-nproc.conf > line too long (108 > 90) Done http://gerrit.cloudera.org:8080/#/c/11731/1/bin/bootstrap_system.sh@274 PS1, Line 274: SET_JAVA_HOME="export JAVA_HOME=$(echo /usr/lib/jvm/java-1.8.0-openjdk-* | head -n 1)" > Does this not work on Ubuntu, too? The paths are slightly different. I could glob it so that "1.8.0" and "-8-" both match, but it doesn't seem right. I added an assertion that there's only one here. http://gerrit.cloudera.org:8080/#/c/11731/1/docker/entrypoint.sh File docker/entrypoint.sh: http://gerrit.cloudera.org:8080/#/c/11731/1/docker/entrypoint.sh@81 PS1, Line 81: adduser --uid $1 impdev > Do you want to add a comment here about why the command is different for Ub Done. I just said they're different. I can't actually say "why" they're different meaningfully :) -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Jim Apple <jbapple-imp...@apache.org> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Laszlo Gaal <laszlo.g...@cloudera.com> Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Comment-Date: Wed, 24 Oct 2018 00:13:37 +0000 Gerrit-HasComments: Yes
