Today, I tried to install graylog 2.1.1 in new Amazon Instance to test
features of new graylog. After I installed elastic search 2.4.0, mongodb
3.2.9 and graylog 2.1.1, I configured elasticsearch.yml and graylog config
as below. Then, even though graylog server is up, running and elastic
search added graylog node in logs, I encountered weird problem. Then I
typed the graylog server ip ( <"my amazon instance public ip">:9000 ) in
chrome and safari. However, when I entered my credentials ( admin/graylog
password) and clicked signin, nothing was fired. Then 15-30 seconds later,
graylog web interface gave an error as below:
We are experiencing problems connecting to the Graylog server running on
*http://172.31.29.124:12900/api/*. Please verify that the server is healthy
and working correctly..
My graylog config looks like this (/etc/graylog/server/server.conf):
is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = SECRECT
root_password_sha2 = SECRET
root_timezone = GMT
plugin_dir = /usr/share/graylog-server/plugin
rest_listen_uri = http://0.0.0.0:12900/api/
external_rest_uri: http://MY_AWS_RT53_DNS/api
web_listen_uri = http://0.0.0.0:9000/
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 1
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_cluster_name = graylog
elasticsearch_discovery_zen_ping_unicast_hosts = 172.31.29.124:9300
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load = grok-patterns.json
proxied_requests_thread_pool_size = 32
My elasticsearch yml file looks like this
(/etc/elasticsearch/elasticsearch.yml):
cluster.name: graylog
path.data: /elasticsearch/data/
path.logs: /var/log/elasticsearch/
script.inline: false
script.indexed: false
script.file: false
network.host: 172.31.29.124
discovery.zen.ping.timeout: 10s
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["172.31.29.124:9300"]
This is the graylog server log (/var/log/graylog-server/server.log):
2016-09-21T08:45:59.563Z INFO [CmdLineTool] Loaded plugin: Elastic Beats
Input 1.1.1 [org.graylog.plugins.beats.BeatsInputPlugin]
2016-09-21T08:45:59.564Z INFO [CmdLineTool] Loaded plugin: Collector 1.1.1
[org.graylog.plugins.collector.CollectorPlugin]
2016-09-21T08:45:59.565Z INFO [CmdLineTool] Loaded plugin: Enterprise
Integration Plugin 1.1.1
[org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2016-09-21T08:45:59.565Z INFO [CmdLineTool] Loaded plugin: MapWidgetPlugin
1.1.1 [org.graylog.plugins.map.MapWidgetPlugin]
2016-09-21T08:45:59.565Z INFO [CmdLineTool] Loaded plugin: Pipeline
Processor Plugin 1.1.1
[org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2016-09-21T08:45:59.566Z INFO [CmdLineTool] Loaded plugin: Anonymous Usage
Statistics 2.1.1 [org.graylog.plugins.usagestatistics.UsageStatsPlugin]
2016-09-21T08:45:59.676Z INFO [CmdLineTool] Running with JVM arguments:
-Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC
-XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC
-XX:-OmitStackTraceInFastThrow
-Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml
-Djava.library.path=/usr/share/graylog-server/lib/sigar
-Dgraylog2.installation_source=rpm
2016-09-21T08:46:01.979Z INFO [InputBufferImpl] Message journal is enabled.
2016-09-21T08:46:02.003Z INFO [NodeId] Node ID:
a5e73742-5454-49d7-a089-eb3beb6443b8
2016-09-21T08:46:02.202Z INFO [LogManager] Loading logs.
2016-09-21T08:46:02.257Z INFO [LogManager] Logs loading complete.
2016-09-21T08:46:02.257Z INFO [KafkaJournal] Initialized Kafka based
journal at /var/lib/graylog-server/journal
2016-09-21T08:46:02.274Z INFO [InputBufferImpl] Initialized
InputBufferImpl with ring size <65536> and wait strategy
<BlockingWaitStrategy>, running 2 parallel message handlers.
2016-09-21T08:46:02.300Z INFO [cluster] Cluster created with settings
{hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN,
serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000}
2016-09-21T08:46:02.355Z INFO [cluster] No server chosen by
ReadPreferenceServerSelector{readPreference=primary} from cluster
description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE,
serverDescriptions=[ServerDescription{address=localhost:27017,
type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2016-09-21T08:46:02.377Z INFO [connection] Opened connection
[connectionId{localValue:1, serverValue:89}] to localhost:27017
2016-09-21T08:46:02.379Z INFO [cluster] Monitor thread successfully
connected to server with description
ServerDescription{address=localhost:27017, type=STANDALONE,
state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 2, 9]},
minWireVersion=0, maxWireVersion=4, maxDocumentSize=16777216,
roundTripTimeNanos=1058001}
2016-09-21T08:46:02.393Z INFO [connection] Opened connection
[connectionId{localValue:2, serverValue:90}] to localhost:27017
2016-09-21T08:46:02.643Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] version[2.3.5], pid[22078],
build[90f439f/2016-07-27T10:36:52Z]
2016-09-21T08:46:02.643Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] initializing ...
2016-09-21T08:46:02.648Z INFO [plugins]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] modules [], plugins
[graylog-monitor], sites []
2016-09-21T08:46:04.189Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] initialized
2016-09-21T08:46:04.305Z INFO [Version] HV000001: Hibernate Validator
5.2.4.Final
2016-09-21T08:46:04.466Z INFO [ProcessBuffer] Initialized ProcessBuffer
with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2016-09-21T08:46:06.532Z INFO [RulesEngineProvider] No static rules file
loaded.
2016-09-21T08:46:06.678Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:06.684Z INFO [OutputBuffer] Initialized OutputBuffer with
ring size <65536> and wait strategy <BlockingWaitStrategy>.
2016-09-21T08:46:06.737Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:06.794Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:06.841Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:06.883Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:07.328Z INFO [ServerBootstrap] Graylog server
2.1.1+01d50e5 starting up
2016-09-21T08:46:07.328Z INFO [ServerBootstrap] JRE: Oracle Corporation
1.8.0_102 on Linux 3.10.0-327.10.1.el7.x86_64
2016-09-21T08:46:07.329Z INFO [ServerBootstrap] Deployment: rpm
2016-09-21T08:46:07.329Z INFO [ServerBootstrap] OS: CentOS Linux 7 (Core)
(centos)
2016-09-21T08:46:07.329Z INFO [ServerBootstrap] Arch: amd64
2016-09-21T08:46:07.345Z WARN [DeadEventLoggingListener] Received
unhandled event of type <org.graylog2.plugin.lifecycles.Lifecycle> from
event bus <AsyncEventBus{graylog-eventbus}>
2016-09-21T08:46:07.364Z INFO [PeriodicalsService] Starting 25 periodicals
...
2016-09-21T08:46:07.364Z INFO [Periodicals] Starting
[org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling
every [1s].
2016-09-21T08:46:07.367Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] starting ...
2016-09-21T08:46:07.368Z INFO [Periodicals] Starting
[org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling
every [60s].
2016-09-21T08:46:07.369Z INFO [Periodicals] Starting
[org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical
in [0s], polling every [1s].
2016-09-21T08:46:07.370Z INFO [Periodicals] Starting
[org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s],
polling every [20s].
2016-09-21T08:46:07.372Z INFO [Periodicals] Starting
[org.graylog2.periodical.ContentPackLoaderPeriodical] periodical, running
forever.
2016-09-21T08:46:07.372Z INFO [Periodicals] Starting
[org.graylog2.periodical.GarbageCollectionWarningThread] periodical,
running forever.
2016-09-21T08:46:07.374Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s],
polling every [30s].
2016-09-21T08:46:07.375Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling
every [300s].
2016-09-21T08:46:07.377Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling
every [10s].
2016-09-21T08:46:07.377Z INFO [IndexRetentionThread] Elasticsearch cluster
not available, skipping index retention checks.
2016-09-21T08:46:07.377Z INFO [Periodicals] Starting
[org.graylog2.periodical.NodePingThread] periodical in [0s], polling every
[1s].
2016-09-21T08:46:07.378Z INFO [Periodicals] Starting
[org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling
every [1800s].
2016-09-21T08:46:07.380Z INFO [Periodicals] Starting
[org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s],
polling every [1s].
2016-09-21T08:46:07.381Z INFO [Periodicals] Starting
[org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling
every [1s].
2016-09-21T08:46:07.384Z INFO [Periodicals] Starting
[org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s],
polling every [86400s].
2016-09-21T08:46:07.384Z INFO [Periodicals] Starting
[org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running
forever.
2016-09-21T08:46:07.385Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical,
running forever.
2016-09-21T08:46:07.385Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s],
polling every [3600s].
2016-09-21T08:46:07.398Z INFO [connection] Opened connection
[connectionId{localValue:4, serverValue:92}] to localhost:27017
2016-09-21T08:46:07.417Z INFO [IndexerClusterCheckerThread] Indexer not
fully initialized yet. Skipping periodic cluster check.
2016-09-21T08:46:07.450Z INFO [connection] Opened connection
[connectionId{localValue:7, serverValue:95}] to localhost:27017
2016-09-21T08:46:07.454Z INFO [connection] Opened connection
[connectionId{localValue:8, serverValue:96}] to localhost:27017
2016-09-21T08:46:07.460Z INFO [connection] Opened connection
[connectionId{localValue:5, serverValue:93}] to localhost:27017
2016-09-21T08:46:07.463Z INFO [connection] Opened connection
[connectionId{localValue:3, serverValue:91}] to localhost:27017
2016-09-21T08:46:07.484Z INFO [connection] Opened connection
[connectionId{localValue:6, serverValue:94}] to localhost:27017
2016-09-21T08:46:07.580Z INFO [PeriodicalsService] Not starting
[org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not
configured to run on this node.
2016-09-21T08:46:07.580Z INFO [Periodicals] Starting
[org.graylog2.periodical.AlarmCallbacksMigrationPeriodical] periodical,
running forever.
2016-09-21T08:46:07.585Z INFO [Periodicals] Starting
[org.graylog2.periodical.ConfigurationManagementPeriodical] periodical,
running forever.
2016-09-21T08:46:07.608Z INFO [Periodicals] Starting
[org.graylog2.periodical.LdapGroupMappingMigration] periodical, running
forever.
2016-09-21T08:46:07.610Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexFailuresPeriodical] periodical, running
forever.
2016-09-21T08:46:07.624Z INFO [Periodicals] Starting
[org.graylog.plugins.usagestatistics.UsageStatsNodePeriodical] periodical
in [300s], polling every [21600s].
2016-09-21T08:46:07.630Z INFO [Periodicals] Starting
[org.graylog.plugins.usagestatistics.UsageStatsClusterPeriodical]
periodical in [300s], polling every [21600s].
2016-09-21T08:46:07.631Z INFO [Periodicals] Starting
[org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread]
periodical in [0s], polling every [3600s].
2016-09-21T08:46:07.713Z INFO [transport]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] publish_address
{127.0.0.1:9350}, bound_addresses {[::1]:9350}, {127.0.0.1:9350}
2016-09-21T08:46:07.720Z INFO [discovery]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8]
graylog/Aw20hwsCTyqUhHY9L9Ebog
2016-09-21T08:46:07.918Z INFO [JerseyService] Enabling CORS for HTTP
endpoint
2016-09-21T08:46:10.724Z WARN [discovery]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] waited for 3s and no initial
state was set by the discovery
2016-09-21T08:46:10.724Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] started
2016-09-21T08:46:10.802Z INFO [service]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] detected_master {Captain
Fate}{fZ8u-PvuQOWerE29DTVrfA}{172.31.29.124}{172.31.29.124:9300}, added
{{Captain
Fate}{fZ8u-PvuQOWerE29DTVrfA}{172.31.29.124}{172.31.29.124:9300},}, reason:
zen-disco-receive(from master [{Captain
Fate}{fZ8u-PvuQOWerE29DTVrfA}{172.31.29.124}{172.31.29.124:9300}])
2016-09-21T08:46:14.505Z INFO [NetworkListener] Started listener bound to
[0.0.0.0:12900]
2016-09-21T08:46:14.506Z INFO [HttpServer] [HttpServer] Started.
2016-09-21T08:46:14.507Z INFO [JerseyService] Started REST API at
<http://0.0.0.0:12900/api/>
2016-09-21T08:46:16.393Z INFO [NetworkListener] Started listener bound to
[0.0.0.0:9000]
2016-09-21T08:46:16.393Z INFO [HttpServer] [HttpServer-1] Started.
2016-09-21T08:46:16.394Z INFO [JerseyService] Started Web Interface at
<http://0.0.0.0:9000/>
2016-09-21T08:46:16.395Z INFO [ServiceManagerListener] Services are healthy
2016-09-21T08:46:16.396Z INFO [ServerBootstrap] Services started, startup
times in ms: {InputSetupService [RUNNING]=47, JournalReader [RUNNING]=53,
OutputSetupService [RUNNING]=84, BufferSynchronizerService [RUNNING]=96,
KafkaJournal [RUNNING]=140, PeriodicalsService [RUNNING]=321,
IndexerSetupService [RUNNING]=3463, JerseyService [RUNNING]=9034}
2016-09-21T08:46:16.399Z INFO [ServerBootstrap] Graylog server up and
running.
2016-09-21T08:46:16.400Z INFO [InputSetupService] Triggering launching
persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running
[LB:ALIVE]
Does anyone know what the issue is in the configuration that I have
missed/set incorrectly?
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/b27d5e8d-9c49-4f5b-a893-66548c722eb7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
