AMBARI-22357. Log Search server should be able to run locally with dockerized dev env by default (oleewere)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/68bc38eb Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/68bc38eb Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/68bc38eb Branch: refs/heads/branch-feature-AMBARI-21674 Commit: 68bc38eb241b9f8bd3a31b81f8db0ada5e5229f7 Parents: 1b020cd Author: Oliver Szabo <[email protected]> Authored: Fri Nov 3 14:33:29 2017 +0100 Committer: Oliver Szabo <[email protected]> Committed: Fri Nov 3 14:36:06 2017 +0100 ---------------------------------------------------------------------- ambari-logsearch/README.md | 4 ++ .../ambari-logsearch-server/.gitignore | 1 + .../ambari-logsearch-server/README.md | 44 +++++----------- ambari-logsearch/ambari-logsearch-server/run.sh | 2 +- .../src/main/resources/logsearch.properties | 54 ++++++-------------- 5 files changed, 36 insertions(+), 69 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ambari/blob/68bc38eb/ambari-logsearch/README.md ---------------------------------------------------------------------- diff --git a/ambari-logsearch/README.md b/ambari-logsearch/README.md index 43878f2..33952a2 100644 --- a/ambari-logsearch/README.md +++ b/ambari-logsearch/README.md @@ -31,6 +31,10 @@ docker exec -it logsearch bash ``` In case if you started the containers separately and if you would like to access Solr locally with through your external ZooKeeper container, then point `solr` to `localhost` in your `/etc/hosts` file. +### Run applications from IDE / maven + +- [Start Log Search locally](ambari-logsearch-server/README.md) + ## Package build process http://git-wip-us.apache.org/repos/asf/ambari/blob/68bc38eb/ambari-logsearch/ambari-logsearch-server/.gitignore ---------------------------------------------------------------------- diff --git a/ambari-logsearch/ambari-logsearch-server/.gitignore b/ambari-logsearch/ambari-logsearch-server/.gitignore index 07e0389..7ea6a1f 100644 --- a/ambari-logsearch/ambari-logsearch-server/.gitignore +++ b/ambari-logsearch/ambari-logsearch-server/.gitignore @@ -6,4 +6,5 @@ target node_modules/ logs/ node/ +*.pid http://git-wip-us.apache.org/repos/asf/ambari/blob/68bc38eb/ambari-logsearch/ambari-logsearch-server/README.md ---------------------------------------------------------------------- diff --git a/ambari-logsearch/ambari-logsearch-server/README.md b/ambari-logsearch/ambari-logsearch-server/README.md index 126f651..26b1f73 100644 --- a/ambari-logsearch/ambari-logsearch-server/README.md +++ b/ambari-logsearch/ambari-logsearch-server/README.md @@ -17,39 +17,23 @@ limitations under the License. {% endcomment %} --> -#Compilation -mvn clean compile package +# Log Search Server -#Deploy -##Copy to remote -copy target/logsearch-portal.tar.gz to host machine -##Setup environment -```bash -mkdir /opt/logsearch -cd /opt/logsearch -tar xfz ~/logsearch-portal.tar.gz -``` -#Create Solr Collection -*Edit for log retention days (default is 7 days)* -```bash -vi solr_configsets/hadoop_logs/conf/solrconfig.xml -``` -``` - <processor class="solr.DefaultValueUpdateProcessorFactory"> - <str name="fieldName">_ttl_</str> - <str name="value">+7DAYS</str> - </processor> -``` -```bash -./create_collections.sh $SOLR_HOME $NUM_SHARDS $NUM_OF_REPLICATIONS `pwd`/solr_configsets -``` +## Start locally from maven / IDE + +Other services (like zookeeper, solr, logfeeder) can be started with `docker-compose` ```bash -vi classes/logsearch.properties -``` -``` -solr.zkhosts=$ZK1:2181,$ZK2:2181,$ZK3:2181/solr +cd ambari/ambari-logsearch/docker +docker-compose up -d zookeeper solr logfeeder ``` -*This script will stop logsearch if it is running and restart it* + +Then you can start Log Search server from maven + ```bash +cd ambari/ambari-logsearch/ambari-logsearch-server ./run.sh +# or +mvn clean spring-boot:run ``` + +You can also start Log Search server from an IDE as well. One thing is important: the config set location that the server tries to upload to ZooKeeper. By default config sets are located at `${LOGSEARCH_SERVER_RELATIVE_LOCATION:}src/main/configsets` in `logsearch.properties`. Based or from where you run `LogSearch.java`, you need to set `LOGSEARCH_SERVER_RELATIVE_LOCATION` env variable properly. http://git-wip-us.apache.org/repos/asf/ambari/blob/68bc38eb/ambari-logsearch/ambari-logsearch-server/run.sh ---------------------------------------------------------------------- diff --git a/ambari-logsearch/ambari-logsearch-server/run.sh b/ambari-logsearch/ambari-logsearch-server/run.sh index 765fe6c..f245930 100755 --- a/ambari-logsearch/ambari-logsearch-server/run.sh +++ b/ambari-logsearch/ambari-logsearch-server/run.sh @@ -17,4 +17,4 @@ echo " ââââââââââââââââââââââââââ âââââââââââââââââââ ââââââ ââââââââââââââ âââ ââââââââ âââââââ âââââââ âââââââââââââââââââ ââââââ âââ ââââââââââ âââ " -mvn spring-boot:run +mvn clean spring-boot:run http://git-wip-us.apache.org/repos/asf/ambari/blob/68bc38eb/ambari-logsearch/ambari-logsearch-server/src/main/resources/logsearch.properties ---------------------------------------------------------------------- diff --git a/ambari-logsearch/ambari-logsearch-server/src/main/resources/logsearch.properties b/ambari-logsearch/ambari-logsearch-server/src/main/resources/logsearch.properties index db8a0a2..3a0f460 100755 --- a/ambari-logsearch/ambari-logsearch-server/src/main/resources/logsearch.properties +++ b/ambari-logsearch/ambari-logsearch-server/src/main/resources/logsearch.properties @@ -12,49 +12,27 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - -#logsearch.solr.url=http://host:port/solr -logsearch.solr.url= - -#Solr Core +logsearch.solr.zk_connect_string=localhost:2181 logsearch.solr.collection.service.logs=hadoop_logs -#logsearch.solr.collection.service.logs=ranger_audits +logsearch.service.logs.split.interval.mins=15 +logsearch.collection.service.logs.numshards=3 +logsearch.collection.service.logs.replication.factor=2 +logsearch.solr.audit.logs.zk_connect_string=localhost:2181 +logsearch.solr.collection.audit.logs=audit_logs +logsearch.audit.logs.split.interval.mins=15 +logsearch.collection.audit.logs.numshards=2 +logsearch.collection.audit.logs.replication.factor=2 +logsearch.solr.config_set.folder=${LOGSEARCH_SERVER_RELATIVE_LOCATION:}src/main/configsets +logsearch.solr.audit.logs.config_set.folder=${LOGSEARCH_SERVER_RELATIVE_LOCATION:}src/main/configsets logsearch.solr.collection.history=history -logsearch.service.logs.split.interval.mins=none -logsearch.collection.service.logs.numshards=1 -logsearch.collection.service.logs.replication.factor=1 - -#If set, metrics will be sent to Ambari -#logsearch.solr.metrics.collector.hosts=example.com -logsearch.solr.metrics.collector.hosts= -logsearch.solr.audit.logs.url= -#logsearch.solr.collection.audit.logs=collection_name -logsearch.solr.collection.audit.logs=ranger_audits -logsearch.audit.logs.split.interval.mins=none -logsearch.collection.audit.logs.numshards=1 -logsearch.collection.audit.logs.replication.factor=1 - -#Authentication settings -#Note: Simple will be supported only if file ,ldap and external_auth all three are disabled. +logsearch.solr.history.config.name=history +logsearch.collection.history.replication.factor=1 logsearch.auth.file.enable=true +logsearch.login.credentials.file=user_pass.json + logsearch.auth.ldap.enable=false logsearch.auth.simple.enable=false logsearch.auth.external_auth.enable=false -logsearch.auth.external_auth.host_url=http://ip:port -logsearch.auth.external_auth.login_url=/api/v1/users/$USERNAME/privileges?fields=* -#Note: Use comma(,) for separation of multiple roles -logsearch.roles.allowed=AMBARI.ADMINISTRATOR - -logsearch.http.port=61888 -logsearch.https.port=61889 logsearch.protocol=http - -logsearch.solr.kerberos.enable=false -logsearch.solr.jaas.file=/usr/lib/ambari-logsearch-portal/logsearch_solr_jaas.conf - -#portal Kerberos -logsearch.spnego.kerberos.enabled=false -logsearch.spnego.kerberos.keytab= -logsearch.spnego.kerberos.principal= -logsearch.spnego.kerberos.host= \ No newline at end of file +logsearch.config.zk_connect_string=localhost:2181 \ No newline at end of file
