Today, I tried to install graylog 2.1.1 in new Amazon Instance to test
features of new graylog. After I installed elastic search 2.4.0, mongodb
3.2.9 and graylog 2.1.1, I configured elasticsearch.yml and graylog config
as below. Then, even though graylog server is up, running and elastic
search added graylog node in logs, I encountered weird problem. Then I
typed the graylog server ip ( <"my amazon instance public ip">:9000 ) in
chrome and safari. However, when I entered my credentials ( admin/graylog
password) and clicked signin, nothing was fired. Then 15-30 seconds later,
graylog web interface gave an error as below:
We are experiencing problems connecting to the Graylog server running on
*http://172.31.29.124:12900/api/*. Please verify that the server is healthy
and working correctly..
My graylog config looks like this (/etc/graylog/server/server.conf):
is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = SECRECT
root_password_sha2 = SECRET
root_timezone = GMT
plugin_dir = /usr/share/graylog-server/plugin
rest_listen_uri = http://0.0.0.0:12900/api/
external_rest_uri: http://MY_AWS_RT53_DNS/api
web_listen_uri = http://0.0.0.0:9000/
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 1
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_cluster_name = graylog
elasticsearch_discovery_zen_ping_unicast_hosts = 172.31.29.124:9300
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load = grok-patterns.json
proxied_requests_thread_pool_size = 32
My elasticsearch yml file looks like this
(/etc/elasticsearch/elasticsearch.yml):
cluster.name: graylog
path.data: /elasticsearch/data/
path.logs: /var/log/elasticsearch/
script.inline: false
script.indexed: false
script.file: false
network.host: 172.31.29.124
discovery.zen.ping.timeout: 10s
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["172.31.29.124:9300"]
This is the graylog server log (/var/log/graylog-server/server.log):
2016-09-21T08:45:59.563Z INFO [CmdLineTool] Loaded plugin: Elastic Beats
Input 1.1.1 [org.graylog.plugins.beats.BeatsInputPlugin]
2016-09-21T08:45:59.564Z INFO [CmdLineTool] Loaded plugin: Collector 1.1.1
[org.graylog.plugins.collector.CollectorPlugin]
2016-09-21T08:45:59.565Z INFO [CmdLineTool] Loaded plugin: Enterprise
Integration Plugin 1.1.1
[org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2016-09-21T08:45:59.565Z INFO [CmdLineTool] Loaded plugin: MapWidgetPlugin
1.1.1 [org.graylog.plugins.map.MapWidgetPlugin]
2016-09-21T08:45:59.565Z INFO [CmdLineTool] Loaded plugin: Pipeline
Processor Plugin 1.1.1
[org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2016-09-21T08:45:59.566Z INFO [CmdLineTool] Loaded plugin: Anonymous Usage
Statistics 2.1.1 [org.graylog.plugins.usagestatistics.UsageStatsPlugin]
2016-09-21T08:45:59.676Z INFO [CmdLineTool] Running with JVM arguments:
-Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC
-XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC
-XX:-OmitStackTraceInFastThrow
-Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml
-Djava.library.path=/usr/share/graylog-server/lib/sigar
-Dgraylog2.installation_source=rpm
2016-09-21T08:46:01.979Z INFO [InputBufferImpl] Message journal is enabled.
2016-09-21T08:46:02.003Z INFO [NodeId] Node ID:
a5e73742-5454-49d7-a089-eb3beb6443b8
2016-09-21T08:46:02.202Z INFO [LogManager] Loading logs.
2016-09-21T08:46:02.257Z INFO [LogManager] Logs loading complete.
2016-09-21T08:46:02.257Z INFO [KafkaJournal] Initialized Kafka based
journal at /var/lib/graylog-server/journal
2016-09-21T08:46:02.274Z INFO [InputBufferImpl] Initialized
InputBufferImpl with ring size <65536> and wait strategy
<BlockingWaitStrategy>, running 2 parallel message handlers.
2016-09-21T08:46:02.300Z INFO [cluster] Cluster created with settings
{hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN,
serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000}
2016-09-21T08:46:02.355Z INFO [cluster] No server chosen by
ReadPreferenceServerSelector{readPreference=primary} from cluster
description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE,
serverDescriptions=[ServerDescription{address=localhost:27017,
type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2016-09-21T08:46:02.377Z INFO [connection] Opened connection
[connectionId{localValue:1, serverValue:89}] to localhost:27017
2016-09-21T08:46:02.379Z INFO [cluster] Monitor thread successfully
connected to server with description
ServerDescription{address=localhost:27017, type=STANDALONE,
state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 2, 9]},
minWireVersion=0, maxWireVersion=4, maxDocumentSize=16777216,
roundTripTimeNanos=1058001}
2016-09-21T08:46:02.393Z INFO [connection] Opened connection
[connectionId{localValue:2, serverValue:90}] to localhost:27017
2016-09-21T08:46:02.643Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] version[2.3.5], pid[22078],
build[90f439f/2016-07-27T10:36:52Z]
2016-09-21T08:46:02.643Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] initializing ...
2016-09-21T08:46:02.648Z INFO [plugins]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] modules [], plugins
[graylog-monitor], sites []
2016-09-21T08:46:04.189Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] initialized
2016-09-21T08:46:04.305Z INFO [Version] HV000001: Hibernate Validator
5.2.4.Final
2016-09-21T08:46:04.466Z INFO [ProcessBuffer] Initialized ProcessBuffer
with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2016-09-21T08:46:06.532Z INFO [RulesEngineProvider] No static rules file
loaded.
2016-09-21T08:46:06.678Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:06.684Z INFO [OutputBuffer] Initialized OutputBuffer with
ring size <65536> and wait strategy <BlockingWaitStrategy>.
2016-09-21T08:46:06.737Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:06.794Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:06.841Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:06.883Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T08:46:07.328Z INFO [ServerBootstrap] Graylog server
2.1.1+01d50e5 starting up
2016-09-21T08:46:07.328Z INFO [ServerBootstrap] JRE: Oracle Corporation
1.8.0_102 on Linux 3.10.0-327.10.1.el7.x86_64
2016-09-21T08:46:07.329Z INFO [ServerBootstrap] Deployment: rpm
2016-09-21T08:46:07.329Z INFO [ServerBootstrap] OS: CentOS Linux 7 (Core)
(centos)
2016-09-21T08:46:07.329Z INFO [ServerBootstrap] Arch: amd64
2016-09-21T08:46:07.345Z WARN [DeadEventLoggingListener] Received
unhandled event of type <org.graylog2.plugin.lifecycles.Lifecycle> from
event bus <AsyncEventBus{graylog-eventbus}>
2016-09-21T08:46:07.364Z INFO [PeriodicalsService] Starting 25 periodicals
...
2016-09-21T08:46:07.364Z INFO [Periodicals] Starting
[org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling
every [1s].
2016-09-21T08:46:07.367Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] starting ...
2016-09-21T08:46:07.368Z INFO [Periodicals] Starting
[org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling
every [60s].
2016-09-21T08:46:07.369Z INFO [Periodicals] Starting
[org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical
in [0s], polling every [1s].
2016-09-21T08:46:07.370Z INFO [Periodicals] Starting
[org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s],
polling every [20s].
2016-09-21T08:46:07.372Z INFO [Periodicals] Starting
[org.graylog2.periodical.ContentPackLoaderPeriodical] periodical, running
forever.
2016-09-21T08:46:07.372Z INFO [Periodicals] Starting
[org.graylog2.periodical.GarbageCollectionWarningThread] periodical,
running forever.
2016-09-21T08:46:07.374Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s],
polling every [30s].
2016-09-21T08:46:07.375Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling
every [300s].
2016-09-21T08:46:07.377Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling
every [10s].
2016-09-21T08:46:07.377Z INFO [IndexRetentionThread] Elasticsearch cluster
not available, skipping index retention checks.
2016-09-21T08:46:07.377Z INFO [Periodicals] Starting
[org.graylog2.periodical.NodePingThread] periodical in [0s], polling every
[1s].
2016-09-21T08:46:07.378Z INFO [Periodicals] Starting
[org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling
every [1800s].
2016-09-21T08:46:07.380Z INFO [Periodicals] Starting
[org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s],
polling every [1s].
2016-09-21T08:46:07.381Z INFO [Periodicals] Starting
[org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling
every [1s].
2016-09-21T08:46:07.384Z INFO [Periodicals] Starting
[org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s],
polling every [86400s].
2016-09-21T08:46:07.384Z INFO [Periodicals] Starting
[org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running
forever.
2016-09-21T08:46:07.385Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical,
running forever.
2016-09-21T08:46:07.385Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s],
polling every [3600s].
2016-09-21T08:46:07.398Z INFO [connection] Opened connection
[connectionId{localValue:4, serverValue:92}] to localhost:27017
2016-09-21T08:46:07.417Z INFO [IndexerClusterCheckerThread] Indexer not
fully initialized yet. Skipping periodic cluster check.
2016-09-21T08:46:07.450Z INFO [connection] Opened connection
[connectionId{localValue:7, serverValue:95}] to localhost:27017
2016-09-21T08:46:07.454Z INFO [connection] Opened connection
[connectionId{localValue:8, serverValue:96}] to localhost:27017
2016-09-21T08:46:07.460Z INFO [connection] Opened connection
[connectionId{localValue:5, serverValue:93}] to localhost:27017
2016-09-21T08:46:07.463Z INFO [connection] Opened connection
[connectionId{localValue:3, serverValue:91}] to localhost:27017
2016-09-21T08:46:07.484Z INFO [connection] Opened connection
[connectionId{localValue:6, serverValue:94}] to localhost:27017
2016-09-21T08:46:07.580Z INFO [PeriodicalsService] Not starting
[org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not
configured to run on this node.
2016-09-21T08:46:07.580Z INFO [Periodicals] Starting
[org.graylog2.periodical.AlarmCallbacksMigrationPeriodical] periodical,
running forever.
2016-09-21T08:46:07.585Z INFO [Periodicals] Starting
[org.graylog2.periodical.ConfigurationManagementPeriodical] periodical,
running forever.
2016-09-21T08:46:07.608Z INFO [Periodicals] Starting
[org.graylog2.periodical.LdapGroupMappingMigration] periodical, running
forever.
2016-09-21T08:46:07.610Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexFailuresPeriodical] periodical, running
forever.
2016-09-21T08:46:07.624Z INFO [Periodicals] Starting
[org.graylog.plugins.usagestatistics.UsageStatsNodePeriodical] periodical
in [300s], polling every [21600s].
2016-09-21T08:46:07.630Z INFO [Periodicals] Starting
[org.graylog.plugins.usagestatistics.UsageStatsClusterPeriodical]
periodical in [300s], polling every [21600s].
2016-09-21T08:46:07.631Z INFO [Periodicals] Starting
[org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread]
periodical in [0s], polling every [3600s].
2016-09-21T08:46:07.713Z INFO [transport]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] publish_address
{127.0.0.1:9350}, bound_addresses {[::1]:9350}, {127.0.0.1:9350}
2016-09-21T08:46:07.720Z INFO [discovery]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8]
graylog/Aw20hwsCTyqUhHY9L9Ebog
2016-09-21T08:46:07.918Z INFO [JerseyService] Enabling CORS for HTTP
endpoint
2016-09-21T08:46:10.724Z WARN [discovery]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] waited for 3s and no initial
state was set by the discovery
2016-09-21T08:46:10.724Z INFO [node]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] started
2016-09-21T08:46:10.802Z INFO [service]
[graylog-a5e73742-5454-49d7-a089-eb3beb6443b8] detected_master {Captain
Fate}{fZ8u-PvuQOWerE29DTVrfA}{172.31.29.124}{172.31.29.124:9300}, added
{{Captain
Fate}{fZ8u-PvuQOWerE29DTVrfA}{172.31.29.124}{172.31.29.124:9300},}, reason:
zen-disco-receive(from master [{Captain
Fate}{fZ8u-PvuQOWerE29DTVrfA}{172.31.29.124}{172.31.29.124:9300}])
2016-09-21T08:46:14.505Z INFO [NetworkListener] Started listener bound to
[0.0.0.0:12900]
2016-09-21T08:46:14.506Z INFO [HttpServer] [HttpServer] Started.
2016-09-21T08:46:14.507Z INFO [JerseyService] Started REST API at
<http://0.0.0.0:12900/api/>
2016-09-21T08:46:16.393Z INFO [NetworkListener] Started listener bound to
[0.0.0.0:9000]
2016-09-21T08:46:16.393Z INFO [HttpServer] [HttpServer-1] Started.
2016-09-21T08:46:16.394Z INFO [JerseyService] Started Web Interface at
<http://0.0.0.0:9000/>
2016-09-21T08:46:16.395Z INFO [ServiceManagerListener] Services are healthy
2016-09-21T08:46:16.396Z INFO [ServerBootstrap] Services started, startup
times in ms: {InputSetupService [RUNNING]=47, JournalReader [RUNNING]=53,
OutputSetupService [RUNNING]=84, BufferSynchronizerService [RUNNING]=96,
KafkaJournal [RUNNING]=140, PeriodicalsService [RUNNING]=321,
IndexerSetupService [RUNNING]=3463, JerseyService [RUNNING]=9034}
2016-09-21T08:46:16.399Z INFO [ServerBootstrap] Graylog server up and
running.
2016-09-21T08:46:16.400Z INFO [InputSetupService] Triggering launching
persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running
[LB:ALIVE]
Does anyone know what the issue is in the configuration that I have
missed/set incorrectly?
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/b27d5e8d-9c49-4f5b-a893-66548c722eb7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
