METRON-1091 STELLAR Shell: Stand Alone installation (ottobackwards) closes apache/metron#690
Project: http://git-wip-us.apache.org/repos/asf/metron/repo Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/124becd1 Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/124becd1 Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/124becd1 Branch: refs/heads/feature/METRON-1344-test-infrastructure Commit: 124becd1a35d00e620d4eaed5d5d84c0abbe8ba9 Parents: 644e951 Author: ottobackwards <ottobackwa...@gmail.com> Authored: Tue Jan 30 16:34:16 2018 -0500 Committer: otto <o...@apache.org> Committed: Tue Jan 30 16:34:16 2018 -0500 ---------------------------------------------------------------------- metron-stellar/stellar-common/NOTICE | 6 +++ metron-stellar/stellar-common/README.md | 43 +++++++++++++++ metron-stellar/stellar-common/pom.xml | 1 + .../src/main/assembly/assembly.xml | 2 +- .../src/main/assembly/stand-alone-assembly.xml | 57 ++++++++++++++++++++ .../src/main/scripts/deployed/stellar | 36 +++++++++++++ .../src/main/scripts/stand-alone/stellar | 21 ++++++++ .../stellar-common/src/main/scripts/stellar | 36 ------------- 8 files changed, 165 insertions(+), 37 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/metron/blob/124becd1/metron-stellar/stellar-common/NOTICE ---------------------------------------------------------------------- diff --git a/metron-stellar/stellar-common/NOTICE b/metron-stellar/stellar-common/NOTICE new file mode 100644 index 0000000..1a19f3d --- /dev/null +++ b/metron-stellar/stellar-common/NOTICE @@ -0,0 +1,6 @@ + Apache Metron + Copyright 2015-2018 The Apache Software Foundation + + This product includes software developed at + The Apache Software Foundation (http://www.apache.org/). + http://git-wip-us.apache.org/repos/asf/metron/blob/124becd1/metron-stellar/stellar-common/README.md ---------------------------------------------------------------------- diff --git a/metron-stellar/stellar-common/README.md b/metron-stellar/stellar-common/README.md index 98aea77..a2f1242 100644 --- a/metron-stellar/stellar-common/README.md +++ b/metron-stellar/stellar-common/README.md @@ -1422,6 +1422,49 @@ Please note that functions are loading lazily in the background and will be unav %functions ABS, APPEND_IF_MISSING, BIN, BLOOM_ADD, BLOOM_EXISTS, BLOOM_INIT, BLOOM_MERGE, CHOMP, CHOP, COUNT_MATCHES, DAY_OF_MONTH, DAY_OF_WEEK, DAY_OF_YEAR, DOMAIN_REMOVE_SUBDOMAINS, DOMAIN_REMOVE_TLD, DOMAIN_TO_TLD, ENDS_WITH, FILL_LEFT, FILL_RIGHT, FILTER, FORMAT, GET, GET_FIRST, GET_LAST, HLLP_ADD, HLLP_CARDINALITY, HLLP_INIT, HLLP_MERGE, IN_SUBNET, IS_DATE, IS_DOMAIN, IS_EMAIL, IS_EMPTY, IS_INTEGER, IS_IP, IS_URL, JOIN, LENGTH, LIST_ADD, MAAS_GET_ENDPOINT, MAAS_MODEL_APPLY, MAP, MAP_EXISTS, MAP_GET, MONTH, OUTLIER_MAD_ADD, OUTLIER_MAD_SCORE, OUTLIER_MAD_STATE_MERGE, PREPEND_IF_MISSING, PROFILE_FIXED, PROFILE_GET, PROFILE_WINDOW, PROTOCOL_TO_NAME, REDUCE, REGEXP_MATCH, SPLIT, STARTS_WITH, STATS_ADD, STATS_BIN, STATS_COUNT, STATS_GEOMETRIC_MEAN, STATS_INIT, STATS_KURTOSIS, STATS_MAX, STATS_MEAN, STATS_MERGE, STATS_MIN, STATS_PERCENTILE, STATS_POPULATION_VARIANCE, STATS_QUADRATIC_MEAN, STATS_SD, STATS_SKEWNESS, STATS_SUM, STATS_SUM_LOGS, STATS_SUM_SQUARES, STATS_VARIANCE, STRING_ENTROPY, SYS TEM_ENV_GET, SYSTEM_PROPERTY_GET, TO_DOUBLE, TO_EPOCH_TIMESTAMP, TO_FLOAT, TO_INTEGER, TO_LONG, TO_LOWER, TO_STRING, TO_UPPER, TRIM, URL_TO_HOST, URL_TO_PATH, URL_TO_PORT, URL_TO_PROTOCOL, WEEK_OF_MONTH, WEEK_OF_YEAR, YEAR ``` +## Stellar Shell Stand Alone + +The Stellar Shell is also packaged as a stand alone application. It can be unpacked on any supported +operating system. + +> Only the base Stellar functions are available as packaged. Other functions, such as those in metron-profiler and metron-management are not available. + + +```bash +metron-stellar/stellar-common/target/stellar-common-0.4.3-stand-alone.tar.gz +``` + +When unpacked, the following structure will be created: + +```bash +. +âââ bin +â  âââ stellar +âââ lib + âââ stellar-common-0.4.3-uber.jar +``` + +To run the Stellar Shell run the following from the directory you unpacked to: + +```bash +bin/stellar +``` + +```bash +-> % bin/stellar +Stellar, Go! +Please note that functions are loading lazily in the background and will be unavailable until loaded fully. +[Stellar]>>> Functions loaded, you may refer to functions now... + +[Stellar]>>> %functions +ABS, APPEND_IF_MISSING, BLOOM_ADD, BLOOM_EXISTS, BLOOM_INIT, BLOOM_MERGE, CEILING, CHOMP, CHOP, COS, COUNT_MATCHES, DAY_OF_MONTH, DAY_OF_WEEK, DAY_OF_YEAR, DECODE, DOMAIN_REMOVE_SUBDOMAINS, DOMAIN_REMOVE_TLD, DOMAIN_TO_TLD, ENCODE, ENDS_WITH, EXP, FILL_LEFT, FILL_RIGHT, FILTER, FLOOR, FORMAT, GET, GET_FIRST, GET_LAST, GET_SUPPORTED_ENCODINGS, IN_SUBNET, IS_EMPTY, IS_ENCODING, JOIN, LENGTH, LIST_ADD, LN, LOG10, LOG2, MAP, MAP_EXISTS, MAP_GET, MONTH, PREPEND_IF_MISSING, REDUCE, REGEXP_GROUP_VAL, REGEXP_MATCH, ROUND, SIN, SPLIT, SQRT, STARTS_WITH, STRING_ENTROPY, SYSTEM_ENV_GET, SYSTEM_PROPERTY_GET, TAN, TO_DOUBLE, TO_EPOCH_TIMESTAMP, TO_FLOAT, TO_INTEGER, TO_LONG, TO_LOWER, TO_STRING, TO_UPPER, TRIM, URL_TO_HOST, URL_TO_PATH, URL_TO_PORT, URL_TO_PROTOCOL, WEEK_OF_MONTH, WEEK_OF_YEAR, YEAR, ZIP, ZIP_LONGEST +[Stellar]>>> +``` + +By default the shell will have the base Stellar Language commands available. Any jars in the lib directory +that contain Stellar functions will also be loaded, and their commands will be available to shell, as long +as their dependencies are satisfied. + ### Implementation http://git-wip-us.apache.org/repos/asf/metron/blob/124becd1/metron-stellar/stellar-common/pom.xml ---------------------------------------------------------------------- diff --git a/metron-stellar/stellar-common/pom.xml b/metron-stellar/stellar-common/pom.xml index a5dd20b..b43fcc1 100644 --- a/metron-stellar/stellar-common/pom.xml +++ b/metron-stellar/stellar-common/pom.xml @@ -348,6 +348,7 @@ <artifactId>maven-assembly-plugin</artifactId> <configuration> <descriptor>src/main/assembly/assembly.xml</descriptor> + <descriptor>src/main/assembly/stand-alone-assembly.xml</descriptor> </configuration> <executions> <execution> http://git-wip-us.apache.org/repos/asf/metron/blob/124becd1/metron-stellar/stellar-common/src/main/assembly/assembly.xml ---------------------------------------------------------------------- diff --git a/metron-stellar/stellar-common/src/main/assembly/assembly.xml b/metron-stellar/stellar-common/src/main/assembly/assembly.xml index 0b5dfb8..b50be00 100644 --- a/metron-stellar/stellar-common/src/main/assembly/assembly.xml +++ b/metron-stellar/stellar-common/src/main/assembly/assembly.xml @@ -19,7 +19,7 @@ <includeBaseDirectory>false</includeBaseDirectory> <fileSets> <fileSet> - <directory>${project.basedir}/src/main/scripts</directory> + <directory>${project.basedir}/src/main/scripts/deployed</directory> <outputDirectory>bin</outputDirectory> <useDefaultExcludes>true</useDefaultExcludes> <excludes> http://git-wip-us.apache.org/repos/asf/metron/blob/124becd1/metron-stellar/stellar-common/src/main/assembly/stand-alone-assembly.xml ---------------------------------------------------------------------- diff --git a/metron-stellar/stellar-common/src/main/assembly/stand-alone-assembly.xml b/metron-stellar/stellar-common/src/main/assembly/stand-alone-assembly.xml new file mode 100644 index 0000000..490f02e --- /dev/null +++ b/metron-stellar/stellar-common/src/main/assembly/stand-alone-assembly.xml @@ -0,0 +1,57 @@ +<!-- + Licensed to the Apache Software + Foundation (ASF) under one or more contributor license agreements. See the + NOTICE file distributed with this work for additional information regarding + copyright ownership. The ASF licenses this file to You under the Apache License, + Version 2.0 (the "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software distributed + under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES + OR CONDITIONS OF ANY KIND, either express or implied. See the License for + the specific language governing permissions and limitations under the License. + --> + +<assembly> + <id>stand-alone</id> + <formats> + <format>tar.gz</format> + </formats> + <includeBaseDirectory>false</includeBaseDirectory> + <fileSets> + <fileSet> + <directory>${project.basedir}/src/main/scripts/stand-alone</directory> + <outputDirectory>bin</outputDirectory> + <useDefaultExcludes>true</useDefaultExcludes> + <excludes> + <exclude>**/*.formatted</exclude> + <exclude>**/*.filtered</exclude> + </excludes> + <fileMode>0755</fileMode> + <lineEnding>unix</lineEnding> + <filtered>true</filtered> + </fileSet> + <fileSet> + <directory>${project.basedir}/target</directory> + <includes> + <include>${project.artifactId}-${project.version}-uber.jar</include> + </includes> + <outputDirectory>lib</outputDirectory> + <useDefaultExcludes>true</useDefaultExcludes> + </fileSet> + <fileSet> + <directory>${project.basedir}</directory> + <includes> + <include>README.md</include> + </includes> + <outputDirectory></outputDirectory> + </fileSet> + <fileSet> + <directory>${project.basedir}/src/main/resources/META-INF</directory> + <includes> + <include>LICENSE</include> + <include>NOTICE</include> + </includes> + <outputDirectory></outputDirectory> + </fileSet> + </fileSets> +</assembly> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/metron/blob/124becd1/metron-stellar/stellar-common/src/main/scripts/deployed/stellar ---------------------------------------------------------------------- diff --git a/metron-stellar/stellar-common/src/main/scripts/deployed/stellar b/metron-stellar/stellar-common/src/main/scripts/deployed/stellar new file mode 100644 index 0000000..7b0f06d --- /dev/null +++ b/metron-stellar/stellar-common/src/main/scripts/deployed/stellar @@ -0,0 +1,36 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + + +BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default} +[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hbase ] && . ${BIGTOP_DEFAULTS_DIR}/hbase + +# Autodetect JAVA_HOME if not defined +if [ -e /usr/libexec/bigtop-detect-javahome ]; then + . /usr/libexec/bigtop-detect-javahome +elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ]; then + . /usr/lib/bigtop-utils/bigtop-detect-javahome +fi + +export HBASE_CONFIGS=$(hbase classpath) +export METRON_VERSION=${project.version} +export METRON_HOME=/usr/metron/$METRON_VERSION +export STELLAR_LIB=$(find $METRON_HOME/lib/ -name metron-parsers*.jar) +export MANAGEMENT_LIB=$(find $METRON_HOME/lib/ -name metron-management*.jar) +java $JVMFLAGS -cp "${CONTRIB:-$METRON_HOME/contrib}:$STELLAR_LIB:$MANAGEMENT_LIB:$HBASE_CONFIGS" org.apache.metron.stellar.common.shell.cli.StellarShell "$@" http://git-wip-us.apache.org/repos/asf/metron/blob/124becd1/metron-stellar/stellar-common/src/main/scripts/stand-alone/stellar ---------------------------------------------------------------------- diff --git a/metron-stellar/stellar-common/src/main/scripts/stand-alone/stellar b/metron-stellar/stellar-common/src/main/scripts/stand-alone/stellar new file mode 100644 index 0000000..7906a32 --- /dev/null +++ b/metron-stellar/stellar-common/src/main/scripts/stand-alone/stellar @@ -0,0 +1,21 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +binDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +java $JVMFLAGS -cp "${binDir}/../lib/*" org.apache.metron.stellar.common.shell.cli.StellarShell "$@" http://git-wip-us.apache.org/repos/asf/metron/blob/124becd1/metron-stellar/stellar-common/src/main/scripts/stellar ---------------------------------------------------------------------- diff --git a/metron-stellar/stellar-common/src/main/scripts/stellar b/metron-stellar/stellar-common/src/main/scripts/stellar deleted file mode 100644 index 7b0f06d..0000000 --- a/metron-stellar/stellar-common/src/main/scripts/stellar +++ /dev/null @@ -1,36 +0,0 @@ -#!/bin/bash -# -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - - -BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default} -[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hbase ] && . ${BIGTOP_DEFAULTS_DIR}/hbase - -# Autodetect JAVA_HOME if not defined -if [ -e /usr/libexec/bigtop-detect-javahome ]; then - . /usr/libexec/bigtop-detect-javahome -elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ]; then - . /usr/lib/bigtop-utils/bigtop-detect-javahome -fi - -export HBASE_CONFIGS=$(hbase classpath) -export METRON_VERSION=${project.version} -export METRON_HOME=/usr/metron/$METRON_VERSION -export STELLAR_LIB=$(find $METRON_HOME/lib/ -name metron-parsers*.jar) -export MANAGEMENT_LIB=$(find $METRON_HOME/lib/ -name metron-management*.jar) -java $JVMFLAGS -cp "${CONTRIB:-$METRON_HOME/contrib}:$STELLAR_LIB:$MANAGEMENT_LIB:$HBASE_CONFIGS" org.apache.metron.stellar.common.shell.cli.StellarShell "$@"
