Today, I tried to install graylog 2.1.1 in new Amazon Instance to test
features of new graylog. After I installed elastic search 2.3.5, mongodb
3.2.8 and graylog 2.0.3, I configured elasticsearch.yml and graylog config
as below. Then, even though graylog server is up, running and elastic
search added graylog node in logs, I encountered weird problem. Then I
typed the graylog server ip ( <"my amazon instance public ip">:9000 ) in
chrome and safari. However, when I entered my credentials ( admin/graylog
password) and clicked signin, nothing was fired. Then 15-30 seconds later,
graylog web interface gave an error as below:
We are experiencing problems connecting to the Graylog server running on
*http://MY_AWS_RT53_DNS:12900/api/
<http://172.31.29.124:12900/api/>*. Please verify that the server is
healthy and working correctly..
My graylog config looks like this (/etc/graylog/server/server.conf):
is_master = true
node_id_file = /etc/graylog/server/node-id
root_password_sha2 = SECRETKEY
plugin_dir = /usr/share/graylog-server/plugin
rest_listen_uri = http://0.0.0.0:12900/api/
external_rest_uri: http://MY_AWS_RT53_DNS/api
rest_enable_cors = true
rest_enable_tls = true
web_listen_uri = http://0.0.0.0:9000/
web_endpoint_uri = http://MY_AWS_RT53_DNS:12900/api/
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 1
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_discovery_zen_ping_unicast_hosts = 127.0.0.1:9300
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load = grok-patterns.json
My elasticsearch yml file looks like this (/etc/elasticsearch/
elasticsearch.yml):
cluster.name: graylog
network.host: 127.0.0.1
This time the system has some json files as well
(/etc/graylog/graylog-settings.json):
{
"timezone": "Etc/UTC",
"smtp_server": "",
"smtp_port": 587,
"smtp_user": "",
"smtp_password": "",
"smtp_from_email": null,
"smtp_web_url": null,
"smtp_no_tls": false,
"smtp_no_ssl": false,
"master_node": "MY_PRIVATE_IP",
"local_connect": false,
"current_address": "MY_PRIVATE_IP",
"last_address": "MY_PRIVATE_IP",
"enforce_ssl": false,
"journal_size": 1,
"internal_logging": true,
"external_rest_uri": "http://MY_AWS_RT53_DNS/api";,
"custom_attributes": {
}
}
This is the graylog server log (/var/log/graylog-server/server.log):
2016-09-21T15:18:41.127Z INFO [CmdLineTool] Loaded plugin: Collector 1.0.3
[org.graylog.plugins.collector.CollectorPlugin]
2016-09-21T15:18:41.128Z INFO [CmdLineTool] Loaded plugin: Enterprise
Integration Plugin 1.0.3
[org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2016-09-21T15:18:41.128Z INFO [CmdLineTool] Loaded plugin: MapWidgetPlugin
1.0.3 [org.graylog.plugins.map.MapWidgetPlugin]
2016-09-21T15:18:41.128Z INFO [CmdLineTool] Loaded plugin: Pipeline
Processor Plugin 1.0.0-beta.5
[org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2016-09-21T15:18:41.128Z INFO [CmdLineTool] Loaded plugin: Anonymous Usage
Statistics 2.0.3 [org.graylog.plugins.usagestatistics.UsageStatsPlugin]
2016-09-21T15:18:41.199Z INFO [CmdLineTool] Running with JVM arguments:
-Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC
-XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC
-XX:-OmitStackTraceInFastThrow
-Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml
-Djava.library.path=/usr/share/graylog-server/lib/sigar
-Dgraylog2.installation_source=rpm
2016-09-21T15:18:43.203Z INFO [InputBufferImpl] Message journal is enabled.
2016-09-21T15:18:43.371Z INFO [LogManager] Loading logs.
2016-09-21T15:18:43.429Z INFO [LogManager] Logs loading complete.
2016-09-21T15:18:43.429Z INFO [KafkaJournal] Initialized Kafka based
journal at /var/lib/graylog-server/journal
2016-09-21T15:18:43.440Z INFO [InputBufferImpl] Initialized
InputBufferImpl with ring size <65536> and wait strategy
<BlockingWaitStrategy>, running 2 parallel message handlers.
2016-09-21T15:18:43.467Z INFO [cluster] Cluster created with settings
{hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN,
serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000}
2016-09-21T15:18:43.503Z INFO [cluster] No server chosen by
ReadPreferenceServerSelector{readPreference=primary} from cluster
description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE,
all=[ServerDescription{address=localhost:27017, type=UNKNOWN,
state=CONNECTING}]}. Waiting for 30000 ms before timing out
2016-09-21T15:18:43.522Z INFO [connection] Opened connection
[connectionId{localValue:1, serverValue:351}] to localhost:27017
2016-09-21T15:18:43.524Z INFO [cluster] Monitor thread successfully
connected to server with description
ServerDescription{address=localhost:27017, type=STANDALONE,
state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 2, 8]},
minWireVersion=0, maxWireVersion=4, maxDocumentSize=16777216,
roundTripTimeNanos=1093959}
2016-09-21T15:18:43.532Z INFO [connection] Opened connection
[connectionId{localValue:2, serverValue:352}] to localhost:27017
2016-09-21T15:18:43.707Z INFO [NodeId] Node ID:
da3b9920-8b26-4162-b35d-29e263626505
2016-09-21T15:18:43.777Z INFO [node]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] version[2.3.2], pid[23338],
build[b9e4a6a/2016-04-21T16:03:47Z]
2016-09-21T15:18:43.777Z INFO [node]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] initializing ...
2016-09-21T15:18:43.781Z INFO [plugins]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] modules [], plugins
[graylog-monitor], sites []
2016-09-21T15:18:45.164Z INFO [node]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] initialized
2016-09-21T15:18:45.228Z INFO [Version] HV000001: Hibernate Validator
5.2.4.Final
2016-09-21T15:18:45.339Z INFO [ProcessBuffer] Initialized ProcessBuffer
with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2016-09-21T15:18:47.079Z INFO [RulesEngineProvider] No static rules file
loaded.
2016-09-21T15:18:47.176Z INFO [connection] Opened connection
[connectionId{localValue:3, serverValue:353}] to localhost:27017
2016-09-21T15:18:47.330Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T15:18:47.381Z INFO [OutputBuffer] Initialized OutputBuffer with
ring size <65536> and wait strategy <BlockingWaitStrategy>.
2016-09-21T15:18:47.504Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T15:18:47.538Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T15:18:47.581Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T15:18:47.655Z WARN [GeoIpResolverEngine] GeoIP database file
does not exist: /tmp/GeoLite2-City.mmdb
2016-09-21T15:18:48.342Z INFO [ServerBootstrap] Graylog server 2.0.3
(f07c170) starting up
2016-09-21T15:18:48.342Z INFO [ServerBootstrap] JRE: Oracle Corporation
1.8.0_91 on Linux 3.10.0-327.22.2.el7.x86_64
2016-09-21T15:18:48.342Z INFO [ServerBootstrap] Deployment: rpm
2016-09-21T15:18:48.342Z INFO [ServerBootstrap] OS: CentOS Linux 7 (Core)
(centos)
2016-09-21T15:18:48.342Z INFO [ServerBootstrap] Arch: amd64
2016-09-21T15:18:48.347Z WARN [DeadEventLoggingListener] Received
unhandled event of type <org.graylog2.plugin.lifecycles.Lifecycle> from
event bus <AsyncEventBus{graylog-eventbus}>
2016-09-21T15:18:48.373Z INFO [PeriodicalsService] Starting 24 periodicals
...
2016-09-21T15:18:48.374Z INFO [Periodicals] Starting
[org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling
every [1s].
2016-09-21T15:18:48.376Z INFO [node]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] starting ...
2016-09-21T15:18:48.377Z INFO [Periodicals] Starting
[org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling
every [60s].
2016-09-21T15:18:48.377Z INFO [Periodicals] Starting
[org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical
in [0s], polling every [1s].
2016-09-21T15:18:48.378Z INFO [Periodicals] Starting
[org.graylog2.periodical.ClusterHealthCheckThread] periodical in [0s],
polling every [20s].
2016-09-21T15:18:48.380Z INFO [Periodicals] Starting
[org.graylog2.periodical.ContentPackLoaderPeriodical] periodical, running
forever.
2016-09-21T15:18:48.381Z INFO [Periodicals] Starting
[org.graylog2.periodical.GarbageCollectionWarningThread] periodical,
running forever.
2016-09-21T15:18:48.382Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s],
polling every [30s].
2016-09-21T15:18:48.384Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling
every [300s].
2016-09-21T15:18:48.384Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling
every [10s].
2016-09-21T15:18:48.385Z INFO [IndexRetentionThread] Elasticsearch cluster
not available, skipping index retention checks.
2016-09-21T15:18:48.387Z INFO [Periodicals] Starting
[org.graylog2.periodical.NodePingThread] periodical in [0s], polling every
[1s].
2016-09-21T15:18:48.387Z INFO [Periodicals] Starting
[org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling
every [1800s].
2016-09-21T15:18:48.388Z INFO [Periodicals] Starting
[org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s],
polling every [1s].
2016-09-21T15:18:48.390Z INFO [Periodicals] Starting
[org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling
every [1s].
2016-09-21T15:18:48.391Z INFO [Periodicals] Starting
[org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s],
polling every [300s].
2016-09-21T15:18:48.392Z INFO [Periodicals] Starting
[org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running
forever.
2016-09-21T15:18:48.393Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical,
running forever.
2016-09-21T15:18:48.393Z INFO [Periodicals] Starting
[org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s],
polling every [3600s].
2016-09-21T15:18:48.403Z INFO [connection] Opened connection
[connectionId{localValue:4, serverValue:354}] to localhost:27017
2016-09-21T15:18:48.405Z INFO [connection] Opened connection
[connectionId{localValue:5, serverValue:355}] to localhost:27017
2016-09-21T15:18:48.405Z INFO [connection] Opened connection
[connectionId{localValue:6, serverValue:356}] to localhost:27017
2016-09-21T15:18:48.426Z INFO [connection] Opened connection
[connectionId{localValue:7, serverValue:357}] to localhost:27017
2016-09-21T15:18:48.434Z INFO [connection] Opened connection
[connectionId{localValue:8, serverValue:358}] to localhost:27017
2016-09-21T15:18:48.435Z INFO [IndexerClusterCheckerThread] Indexer not
fully initialized yet. Skipping periodic cluster check.
2016-09-21T15:18:48.517Z INFO [PeriodicalsService] Not starting
[org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not
configured to run on this node.
2016-09-21T15:18:48.517Z INFO [Periodicals] Starting
[org.graylog2.periodical.AlarmCallbacksMigrationPeriodical] periodical,
running forever.
2016-09-21T15:18:48.519Z INFO [Periodicals] Starting
[org.graylog2.periodical.ConfigurationManagementPeriodical] periodical,
running forever.
2016-09-21T15:18:48.528Z INFO [Periodicals] Starting
[org.graylog2.periodical.LdapGroupMappingMigration] periodical, running
forever.
2016-09-21T15:18:48.543Z INFO [Periodicals] Starting
[org.graylog.plugins.usagestatistics.UsageStatsNodePeriodical] periodical
in [300s], polling every [21600s].
2016-09-21T15:18:48.543Z INFO [Periodicals] Starting
[org.graylog.plugins.usagestatistics.UsageStatsClusterPeriodical]
periodical in [300s], polling every [21600s].
2016-09-21T15:18:48.543Z INFO [Periodicals] Starting
[org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread]
periodical in [0s], polling every [3600s].
2016-09-21T15:18:48.616Z INFO [transport]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] publish_address
{127.0.0.1:9350}, bound_addresses {[::1]:9350}, {127.0.0.1:9350}
2016-09-21T15:18:48.647Z INFO [discovery]
[graylog-da3b9920-8b26-4162-b35d-29e263626505]
graylog/CY5XabKHTNG5qTj3rqHSIg
2016-09-21T15:18:48.820Z INFO [AbstractJerseyService] Enabling CORS for
HTTP endpoint
2016-09-21T15:18:48.823Z WARN [AbstractJerseyService] Private key or
certificate is empty. Using self-signed certificates for 0.0.0.0 instead.
2016-09-21T15:18:51.397Z INFO [NetworkListener] Started listener bound to
[0.0.0.0:9000]
2016-09-21T15:18:51.399Z INFO [HttpServer] [HttpServer] Started.
2016-09-21T15:18:51.400Z INFO [WebInterfaceService] Started Web Interface
at <http://0.0.0.0:9000/>
2016-09-21T15:18:51.654Z WARN [discovery]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] waited for 3s and no initial
state was set by the discovery
2016-09-21T15:18:51.654Z INFO [node]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] started
2016-09-21T15:18:51.771Z INFO [service]
[graylog-da3b9920-8b26-4162-b35d-29e263626505] detected_master
{Hephaestus}{tHeyGC4xSky6uyJ1XSFKrw}{127.0.0.1}{127.0.0.1:9300}, added
{{Hephaestus}{tHeyGC4xSky6uyJ1XSFKrw}{127.0.0.1}{127.0.0.1:9300},}, reason:
zen-disco-receive(from master
[{Hephaestus}{tHeyGC4xSky6uyJ1XSFKrw}{127.0.0.1}{127.0.0.1:9300}])
2016-09-21T15:18:55.308Z INFO [NetworkListener] Started listener bound to
[0.0.0.0:12900]
2016-09-21T15:18:55.308Z INFO [HttpServer] [HttpServer-1] Started.
2016-09-21T15:18:55.308Z INFO [RestApiService] Started REST API at
<https://0.0.0.0:12900/api/>
2016-09-21T15:18:55.309Z INFO [ServiceManagerListener] Services are healthy
2016-09-21T15:18:55.310Z INFO [ServerBootstrap] Services started, startup
times in ms: {OutputSetupService [RUNNING]=44, MetricsReporterService
[RUNNING]=45, JournalReader [RUNNING]=45, BufferSynchronizerService
[RUNNING]=45, InputSetupService [RUNNING]=45, KafkaJournal [RUNNING]=51,
PeriodicalsService [RUNNING]=178, WebInterfaceService [RUNNING]=3028,
IndexerSetupService [RUNNING]=3424, RestApiService [RUNNING]=6939}
2016-09-21T15:18:55.313Z INFO [ServerBootstrap] Graylog server up and
running.
2016-09-21T15:18:55.314Z INFO [InputSetupService] Triggering launching
persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running
[LB:ALIVE]
Does anyone know what the issue is in the configuration that I have
missed/set incorrectly? This is the second GrayLog system I have worked on
with this issue but previous settings matching has not helped.
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/0839c8fb-4d2e-4a0c-a555-7f5e3bb0df8c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
