[Impala-ASF-CR] IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.
Jim Apple has uploaded a new patch set (#4). Change subject: IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs. .. IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs. While I'm here, fix as many of the warnings as possible. The instructions on how to run this are in check-rat-report.py: 1. Either download a release candidate or create an archive from your git repo using `git archive --prefix=Foo/ -o baz.tar.gz HEAD` 2. Untar and run `java -jar apache-rat-0.12.jar -x . >baz.xml`. Only RAT version 0.12 is known to work at this time. 3. `bin/check-rat-report.py bin/rat_exclude_files.txt baz.xml` Change-Id: I5bfe77f9a871018e7a67553ed270e2df53006962 --- M LICENSE.txt M be/.astylerc M be/.impala.doxy M be/src/benchmarks/expr-benchmark.cc M be/src/experiments/data-provider.cc M be/src/experiments/hashing/cache-hash-test.cc M be/src/experiments/hashing/growing-test.cc M be/src/experiments/hashing/interface/cache-hash-test.cc M be/src/experiments/hashing/interface/growing-test.cc M be/src/experiments/hashing/multilevel/cache-hash-test.cc M be/src/experiments/hashing/multilevel/growing-test.cc M be/src/experiments/hashing/prefetch/cache-hash-test.cc M be/src/experiments/hashing/prefetch/growing-test.cc M be/src/experiments/hashing/split-benchmarks/cache-hash-test.cc M be/src/experiments/hashing/split-benchmarks/growing-test.cc M be/src/experiments/hashing/split-benchmarks/partitioning-throughput-test.cc M be/src/experiments/hashing/streaming/cache-hash-test.cc M be/src/experiments/hashing/streaming/growing-test.cc M be/src/testutil/certificates-info.txt M be/src/util/asan.h M bin/README-RUNNING-BENCHMARKS D bin/cpplint.py M bin/impala-ipython M bin/impala-py.test M bin/impala-python M cmake_modules/FindAvro.cmake M cmake_modules/FindBreakpad.cmake M cmake_modules/FindBzip2.cmake M cmake_modules/FindGFlags.cmake M cmake_modules/FindGLog.cmake M cmake_modules/FindHDFS.cmake M cmake_modules/FindJNI.cmake M cmake_modules/FindLdap.cmake M cmake_modules/FindLlvm.cmake M cmake_modules/FindLlvmBinaries.cmake M cmake_modules/FindLz4.cmake M cmake_modules/FindOpenSSL.cmake M cmake_modules/FindPProf.cmake M cmake_modules/FindRapidJson.cmake M cmake_modules/FindRe2.cmake M cmake_modules/FindSnappy.cmake M cmake_modules/FindThrift.cmake M cmake_modules/FindZlib.cmake M fe/src/main/java/com/cloudera/impala/analysis/AggregateInfoBase.java M fe/src/main/java/com/cloudera/impala/analysis/ExprSubstitutionMap.java M fe/src/main/java/com/cloudera/impala/analysis/PartitionSpec.java M fe/src/main/java/com/cloudera/impala/catalog/ArrayType.java M fe/src/main/java/com/cloudera/impala/catalog/MapType.java M fe/src/main/java/com/cloudera/impala/catalog/StructType.java M fe/src/main/java/com/cloudera/impala/planner/Planner.java M fe/src/test/resources/authz-policy.ini.template M fe/src/test/resources/hive-log4j.properties.template M fe/src/test/resources/log4j.properties.template M infra/python/deps/download_requirements M infra/python/deps/requirements.txt M testdata/avro_schema_resolution/create_table.sql M testdata/bin/cache_tables.py M testdata/bin/create-mini.sql M testdata/bin/load-dependent-tables.sql M testdata/bin/load-hive-builtins.sh M testdata/bin/run-hive.sh M testdata/cluster/node_templates/cdh5/etc/init.d/kms M testdata/cluster/node_templates/cdh5/etc/init.d/kudu-common M testdata/cluster/node_templates/cdh5/etc/init.d/kudu-master M testdata/cluster/node_templates/cdh5/etc/init.d/kudu-tserver M testdata/cluster/node_templates/cdh5/etc/init.d/llama-application M testdata/cluster/node_templates/common/etc/init.d/common.tmpl M testdata/cluster/node_templates/common/etc/init.d/hdfs-common M testdata/cluster/node_templates/common/etc/init.d/hdfs-datanode M testdata/cluster/node_templates/common/etc/init.d/hdfs-namenode M testdata/cluster/node_templates/common/etc/init.d/yarn-common M testdata/cluster/node_templates/common/etc/init.d/yarn-nodemanager M testdata/cluster/node_templates/common/etc/init.d/yarn-resourcemanager D testdata/data/mstr/eatwh1/EATWH1-DDL.sql D testdata/data/mstr/eatwh1/LoadTablesEATWH1.sql D testdata/data/mstr/eatwh1/csv/COST_MARKET_CLASS.TXT D testdata/data/mstr/eatwh1/csv/COST_MARKET_DEP.TXT D testdata/data/mstr/eatwh1/csv/COST_MARKET_DIV.TXT D testdata/data/mstr/eatwh1/csv/COST_REGION_CLASS.TXT D testdata/data/mstr/eatwh1/csv/COST_REGION_ITEM.TXT D testdata/data/mstr/eatwh1/csv/COST_STORE_DEP.TXT D testdata/data/mstr/eatwh1/csv/COST_STORE_ITEM.TXT D testdata/data/mstr/eatwh1/csv/FT1.TXT D testdata/data/mstr/eatwh1/csv/FT10.TXT D testdata/data/mstr/eatwh1/csv/FT11.TXT D testdata/data/mstr/eatwh1/csv/FT12.TXT D testdata/data/mstr/eatwh1/csv/FT13.TXT D testdata/data/mstr/eatwh1/csv/FT14.TXT D testdata/data/mstr/eatwh1/csv/FT15.TXT D testdata/data/mstr/eatwh1/csv/FT17.TXT D testdata/data/mstr/eatwh1/csv/FT2.TXT D testdata/data/mstr/eatwh1/csv/FT3.TXT D testdata/data/mstr/eatwh1/csv/FT4.TXT D testdata/data/mstr/eatwh1/csv/FT5
[Impala-ASF-CR] IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.
Alex Behm has posted comments on this change. Change subject: IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs. .. Patch Set 3: (3 comments) Looks good to me. Can we please separate the tool integration from the fixes in two separate patches? http://gerrit.cloudera.org:8080/#/c/4361/3/bin/check-rat-report.py File bin/check-rat-report.py: Line 21: # Usage: Can you describe the purpose of this tool? Where did this script come from? Line 24: # `git archive --prefix=Foo/ -o baz.tar.gz HEAD` can we make the examples more concrete? http://gerrit.cloudera.org:8080/#/c/4361/3/testdata/bin/cache_tables.py File testdata/bin/cache_tables.py: Line 22: # query. This only works on a mini-dfs cluster. This is remarkably difficult to do fix whitespace while here -- To view, visit http://gerrit.cloudera.org:8080/4361 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I5bfe77f9a871018e7a67553ed270e2df53006962 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.
Jim Apple has posted comments on this change.
Change subject: IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.
..
Patch Set 1:
(3 comments)
> (5 comments)
>
> looks pretty reasonable. Might want to do a surface level
> understanding of why you need this ratcursive workaround and file a
> JIRA upstream.
Looks like it was some 0.11 problems; 0.12 worked without ratcursive.
http://gerrit.cloudera.org:8080/#/c/4361/1/bin/rat.sh
File bin/rat.sh:
PS1, Line 38: RAT_JAR=$1
: # The location of the Impala directory (or subdirectory within
the Impala tree) to check:
: START=$2
> rather than relying just on set -u, i think it's nicer to print a real 'usa
File removed.
PS1, Line 51: PN
> what's PN?
PseudoNym
Line 63: echo "Failed to RAT ${FN}" >&2
> should set the exit code in this case, no? otherwise people might miss this
I thinkk RAT exits with 0 even when the license isn't OK. I've changed the
checker python script I borrowed from Kudu to be a little stricter in what goes
to stderr.
--
To view, visit http://gerrit.cloudera.org:8080/4361
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5bfe77f9a871018e7a67553ed270e2df53006962
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple
Gerrit-Reviewer: Henry Robinson
Gerrit-Reviewer: Jim Apple
Gerrit-Reviewer: Todd Lipcon
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.
Jim Apple has uploaded a new patch set (#2). Change subject: IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs. .. IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs. While I'm here, fix as many of the warnings as possible. The instructions on how to run this are in check-rat-report.py: 1. Either download a release candidate or create an archive from your git repo using `git archive --prefix=Foo/ -o baz.tar.gz HEAD` 2. Untar and run `java -jar apache-rat-0.12.jar -x . >baz.xml`. Only RAT version 0.12 is known to work at this time. 3. `bin/check-rat-report.py bin/rat_exclude_files.txt baz.xml` Change-Id: I5bfe77f9a871018e7a67553ed270e2df53006962 --- M LICENSE.txt M be/.astylerc M be/.impala.doxy M be/src/benchmarks/expr-benchmark.cc M be/src/experiments/data-provider.cc M be/src/experiments/hashing/cache-hash-test.cc M be/src/experiments/hashing/growing-test.cc M be/src/experiments/hashing/interface/cache-hash-test.cc M be/src/experiments/hashing/interface/growing-test.cc M be/src/experiments/hashing/multilevel/cache-hash-test.cc M be/src/experiments/hashing/multilevel/growing-test.cc M be/src/experiments/hashing/prefetch/cache-hash-test.cc M be/src/experiments/hashing/prefetch/growing-test.cc M be/src/experiments/hashing/split-benchmarks/cache-hash-test.cc M be/src/experiments/hashing/split-benchmarks/growing-test.cc M be/src/experiments/hashing/split-benchmarks/partitioning-throughput-test.cc M be/src/experiments/hashing/streaming/cache-hash-test.cc M be/src/experiments/hashing/streaming/growing-test.cc M be/src/testutil/certificates-info.txt M be/src/util/asan.h M bin/README-RUNNING-BENCHMARKS A bin/check-rat-report.py D bin/cpplint.py M bin/impala-ipython M bin/impala-py.test M bin/impala-python A bin/rat_exclude_files.txt M cmake_modules/FindAvro.cmake M cmake_modules/FindBreakpad.cmake M cmake_modules/FindBzip2.cmake M cmake_modules/FindGFlags.cmake M cmake_modules/FindGLog.cmake M cmake_modules/FindHDFS.cmake M cmake_modules/FindJNI.cmake M cmake_modules/FindLdap.cmake M cmake_modules/FindLlvm.cmake M cmake_modules/FindLlvmBinaries.cmake M cmake_modules/FindLz4.cmake M cmake_modules/FindOpenSSL.cmake M cmake_modules/FindPProf.cmake M cmake_modules/FindRapidJson.cmake M cmake_modules/FindRe2.cmake M cmake_modules/FindSnappy.cmake M cmake_modules/FindThrift.cmake M cmake_modules/FindZlib.cmake M fe/src/main/java/com/cloudera/impala/analysis/AggregateInfoBase.java M fe/src/main/java/com/cloudera/impala/analysis/ExprSubstitutionMap.java M fe/src/main/java/com/cloudera/impala/analysis/PartitionSpec.java M fe/src/main/java/com/cloudera/impala/catalog/ArrayType.java M fe/src/main/java/com/cloudera/impala/catalog/MapType.java M fe/src/main/java/com/cloudera/impala/catalog/StructType.java M fe/src/main/java/com/cloudera/impala/planner/Planner.java M fe/src/test/resources/authz-policy.ini.template M fe/src/test/resources/hive-log4j.properties.template M fe/src/test/resources/log4j.properties.template M infra/python/deps/download_requirements M infra/python/deps/requirements.txt M testdata/avro_schema_resolution/create_table.sql M testdata/bin/cache_tables.py M testdata/bin/create-mini.sql M testdata/bin/load-dependent-tables.sql M testdata/bin/load-hive-builtins.sh M testdata/bin/run-hive.sh M testdata/cluster/node_templates/cdh5/etc/init.d/kms M testdata/cluster/node_templates/cdh5/etc/init.d/kudu-common M testdata/cluster/node_templates/cdh5/etc/init.d/kudu-master M testdata/cluster/node_templates/cdh5/etc/init.d/kudu-tserver M testdata/cluster/node_templates/cdh5/etc/init.d/llama-application M testdata/cluster/node_templates/common/etc/init.d/common.tmpl M testdata/cluster/node_templates/common/etc/init.d/hdfs-common M testdata/cluster/node_templates/common/etc/init.d/hdfs-datanode M testdata/cluster/node_templates/common/etc/init.d/hdfs-namenode M testdata/cluster/node_templates/common/etc/init.d/yarn-common M testdata/cluster/node_templates/common/etc/init.d/yarn-nodemanager M testdata/cluster/node_templates/common/etc/init.d/yarn-resourcemanager M tests/comparison/conftest.py M tests/test-hive-udfs/pom.xml 77 files changed, 1,473 insertions(+), 4,809 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/4361/2 -- To view, visit http://gerrit.cloudera.org:8080/4361 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5bfe77f9a871018e7a67553ed270e2df53006962 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Todd Lipcon
