[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350547#comment-16350547 ] ASF GitHub Bot commented on METRON-1436: Github user mmiklavc closed the pull request at: https://github.com/apache/metron/pull/918 > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350393#comment-16350393 ] ASF GitHub Bot commented on METRON-1436: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/918 Thanks for the updates. I'm +1 on including this in the feature branch. > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349287#comment-16349287 ] ASF GitHub Bot commented on METRON-1436: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/918 I got full dev spinning spun up with this, and it looks good. I think the latest couple comments are still to be addressed, but otherwise good to go. > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349160#comment-16349160 ] ASF GitHub Bot commented on METRON-1436: Github user justinleet commented on a diff in the pull request: https://github.com/apache/metron/pull/918#discussion_r165468451 --- Diff: metron-deployment/packaging/docker/rpm-docker/SPECS/metron.spec --- @@ -225,6 +225,7 @@ This package installs the Metron Solr files %dir %{metron_home}/bin %dir %{metron_home}/config %dir %{metron_home}/lib +%{metron_home}/bin/install_solr.sh --- End diff -- There should also be an entry at the bottom for the changelog. > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349161#comment-16349161 ] ASF GitHub Bot commented on METRON-1436: Github user justinleet commented on a diff in the pull request: https://github.com/apache/metron/pull/918#discussion_r165466486 --- Diff: metron-platform/metron-solr/README.md --- @@ -0,0 +1,61 @@ + +# Solr in Metron + +## Table of Contents + +* [Introduction](#introduction) +* [Installing](#installing) + +## Introduction + +Metron ships with Solr 6.6.2 support. Solr Cloud can be used as the real-time portion of the datastore resulting from [metron-indexing](../metron-indexing/README.md). + +## Installing + +A script is provided in the installation for installing Solr Cloud in quick-start mode in the [full dev environment for CentOS](../../metron-deployment/development/centos6). +The script is installed as part of the Solr RPM. Metron's Ambari MPack does not currently manage Solr installation in any way, so +you must first install the RPM in order to run the Solr setup script. + +The script performs the following tasks + +* Stops ES and Kibana +* Downloads Solr +* Installs Solr +* Starts Solr Cloud + +_Note: for details on setting up Solr Cloud in production mode, see https://lucene.apache.org/solr/guide/6_6/taking-solr-to-production.html_ + +Login to the full dev environment as root, navigate to the Metron rpm repo, and get a listing of the RPM files. + +``` +cd /localrepo +ls -al +``` + +Copy the name of the Solr RPM and install it via the following command. + +``` +# Install Solr RPM - replace the Metron version and appropriate timestamp on the rpm +rpm -ivh metron-solr-0.4.3-201801301626.noarch.rpm --- End diff -- Could this just be the following? ``` cd /localrepo rpm -ivh metron-solr*.rpm ``` > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347759#comment-16347759 ] ASF GitHub Bot commented on METRON-1436: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/918 I tested it out and worked fine. I think it's a good start. +1 as long as other commenters are satisfied. > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347639#comment-16347639 ] ASF GitHub Bot commented on METRON-1436: Github user justinleet commented on a diff in the pull request: https://github.com/apache/metron/pull/918#discussion_r165192051 --- Diff: metron-platform/metron-solr/README.md --- @@ -0,0 +1,52 @@ + +# Solr in Metron + +## Table of Contents + +* [Introduction](#introduction) +* [Installing](#installing) + +## Introduction + +Metron ships with Solr 6.6.2. Solr Cloud can be used as the real-time portion of the datastore resulting from [metron-indexing](../metron-indexing/README.md). --- End diff -- Can you make this "... Solr 6.6.2 support", so it's obvious we're not actually including Solr? I'd also add the blurb / link from the script about production use here as well. > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346068#comment-16346068 ] ASF GitHub Bot commented on METRON-1436: Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/metron/pull/918#discussion_r164925950 --- Diff: metron-platform/metron-solr/src/main/scripts/install_solr.sh --- @@ -0,0 +1,62 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# This is provided for development purposes + +# Full dev env setup script for Solr Cloud 6.6.2 +# - Stops ES and Kibana +# - Downloads Solr +# - Installs Solr +# - Starts Solr Cloud + +# Note: for production mode, see https://lucene.apache.org/solr/guide/6_6/taking-solr-to-production.html + +service kibana stop +service elasticsearch stop + +SOLR_VERSION=6.6.2 +SOLR_USER=solr +SOLR_SERVICE=$SOLR_USER +SOLR_VAR_DIR="/var/$SOLR_SERVICE" + +# create user if not exists +solr_uid="`id -u "$SOLR_USER"`" +if [ $? -ne 0 ]; then + echo "Creating new user: $SOLR_USER" + adduser --system -U -m --home-dir "$SOLR_VAR_DIR" "$SOLR_USER" +fi +cd $SOLR_VAR_DIR +wget http://archive.apache.org/dist/lucene/solr/${SOLR_VERSION}/solr-${SOLR_VERSION}.tgz +tar zxvf solr-${SOLR_VERSION}.tgz +chown -R solr:solr solr-${SOLR_VERSION} +cd solr-${SOLR_VERSION} +su solr -c "bin/solr -e cloud -noprompt" --- End diff -- `su $SOLR_USER -c "bin/solr -e cloud -noprompt"` > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346067#comment-16346067 ] ASF GitHub Bot commented on METRON-1436: Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/metron/pull/918#discussion_r164925885 --- Diff: metron-platform/metron-solr/src/main/scripts/install_solr.sh --- @@ -0,0 +1,62 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# This is provided for development purposes + +# Full dev env setup script for Solr Cloud 6.6.2 +# - Stops ES and Kibana +# - Downloads Solr +# - Installs Solr +# - Starts Solr Cloud + +# Note: for production mode, see https://lucene.apache.org/solr/guide/6_6/taking-solr-to-production.html + +service kibana stop +service elasticsearch stop + +SOLR_VERSION=6.6.2 +SOLR_USER=solr +SOLR_SERVICE=$SOLR_USER +SOLR_VAR_DIR="/var/$SOLR_SERVICE" + +# create user if not exists +solr_uid="`id -u "$SOLR_USER"`" +if [ $? -ne 0 ]; then + echo "Creating new user: $SOLR_USER" + adduser --system -U -m --home-dir "$SOLR_VAR_DIR" "$SOLR_USER" +fi +cd $SOLR_VAR_DIR +wget http://archive.apache.org/dist/lucene/solr/${SOLR_VERSION}/solr-${SOLR_VERSION}.tgz +tar zxvf solr-${SOLR_VERSION}.tgz +chown -R solr:solr solr-${SOLR_VERSION} --- End diff -- `chown -R $SOLR_USER:$SOLR_USER solr-${SOLR_VERSION}` > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1436) Manually Install Solr Cloud in Full Dev
[ https://issues.apache.org/jira/browse/METRON-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346056#comment-16346056 ] ASF GitHub Bot commented on METRON-1436: GitHub user mmiklavc opened a pull request: https://github.com/apache/metron/pull/918 METRON-1436: Manually Install Solr Cloud in Full Dev https://issues.apache.org/jira/browse/METRON-1436 Provide script for starting Solr Cloud in quickstart mode for testing purposes ## Contributor Comments Adds a solr installation script to the solr RPM. New Solr README includes documentation for manually installing the RPM and running the script to spin up Solr Cloud. Expect follow-on work to get a more production-ready deployment version (e.g. with system service scripts). Preliminary notes are provided in the README and installation script. ## Pull Request Checklist In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [x ] Have you included steps or a guide to how the change may be verified and tested manually? - [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mmiklavc/metron METRON-1436 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/918.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #918 commit 3cc9897f09dfa84957525da8f14d18c4f8a97e2d Author: Michael Miklavcic Date: 2018-01-30T22:11:54Z Provide script for starting Solr Cloud in quickstart mode for testing purposes > Manually Install Solr Cloud in Full Dev > --- > > Key: METRON-1436 > URL: https://issues.apache.org/jira/browse/METRON-1436 > Project: Metron > Issue Type: Sub-task >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > > Script and documentation for getting Solr setup in full dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)