Hi Chris, what's the output of the following curl commands if you run it on the machine your web browser is running on?
curl -v -X GET http://10.18.16.15:9000/ curl -v -X GET -H 'Accept: application/json' http://10.18.16.15:9000/api/ The long GC pauses (over 1 minute is really bad and unusual) are also strange. Try giving Graylog more memory (currently it's 1 GB according to the logs, try 2 GB), see http://docs.graylog.org/en/2.1/pages/faq.html#raise-the-java-heap. Cheers, Jochen On Wednesday, 21 September 2016 15:59:35 UTC+2, Chris Call wrote: > > I had a great experience with Graylog before version 2.0 but I started > fresh with a 2.0 install and had issues (streams kept stopping) and then > upgraded to 2.0.1 I believe and now finally to 2.1. > > Right now, I can't open the web interface when I browse to > 10.18.16.15:9000 (IP address of the host). I get nothing in the browser. > Chrome gives me this "The 10.18.16.15 page isn’t working. 10.18.16.15 > didn’t send any data." and no errors or messages in the developer java > console. > > This is a single system running everything for Graylog and here are my > config files and output: > > etc/graylog/server/server.conf: > is_master = true > node_id_file = /etc/graylog/server/node-id > password_secret = <secret> > root_password_sha2 = <secret> > root_email = <email> > plugin_dir = plugin > rest_listen_uri = http://10.18.16.15:9000/api/ > web_enable = true > web_listen_uri = http://10.18.16.15:9000/ > rotation_strategy = count > elasticsearch_max_docs_per_index = 20000000 > elasticsearch_max_number_of_indices = 20 > retention_strategy = delete > elasticsearch_shards = 4 > elasticsearch_replicas = 0 > elasticsearch_index_prefix = graylog > allow_leading_wildcard_searches = false > allow_highlighting = false > elasticsearch_analyzer = standard > output_batch_size = 500 > output_flush_interval = 1 > output_fault_count_threshold = 5 > output_fault_penalty_seconds = 30 > processbuffer_processors = 5 > outputbuffer_processors = 3 > processor_wait_strategy = blocking > ring_size = 65536 > inputbuffer_ring_size = 65536 > inputbuffer_processors = 2 > inputbuffer_wait_strategy = blocking > message_journal_enabled = true > message_journal_dir = /var/lib/graylog-server/journal > lb_recognition_period_seconds = 3 > mongodb_uri = mongodb://localhost/graylog > mongodb_max_connections = 1000 > mongodb_threads_allowed_to_block_multiplier = 5 > content_packs_loader_enabled = false > content_packs_auto_load = grok-patterns.json > proxied_requests_thread_pool_size = 32 > > > /etc/elasticsearch/elasticsearch.yml > # ======================== Elasticsearch Configuration > ========================= > cluster.name: graylog > > result of " curl -XGET 'http://localhost:9200/_cluster/health?pretty=true > ' ": > { > "cluster_name" : "graylog", > "status" : "green", > "timed_out" : false, > "number_of_nodes" : 1, > "number_of_data_nodes" : 1, > "active_primary_shards" : 20, > "active_shards" : 20, > "relocating_shards" : 0, > "initializing_shards" : 0, > "unassigned_shards" : 0, > "delayed_unassigned_shards" : 0, > "number_of_pending_tasks" : 0, > "number_of_in_flight_fetch" : 0, > "task_max_waiting_in_queue_millis" : 0, > "active_shards_percent_as_number" : 100.0 > } > > /var/log/graylog-server/server.log: > 2016-09-21T09:41:42.573-04:00 WARN [PluginLoader] Plugin directory > /plugin does not exist, not loading plugins. > 2016-09-21T09:41:43.030-04:00 INFO [CmdLineTool] Running with JVM > arguments: -Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB > -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled > -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC > -XX:-OmitStackTraceInFastThrow > -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml > -Djava.library.path=/usr/share/graylog-server/lib/sigar > -Dgraylog2.installation_source=deb > 2016-09-21T09:41:47.611-04:00 INFO [InputBufferImpl] Message journal is > enabled. > 2016-09-21T09:41:47.678-04:00 INFO [NodeId] Node ID: > ecdff2ab-d0a2-4ddb-975e-d2379fb3625d > 2016-09-21T09:41:48.054-04:00 INFO [LogManager] Loading logs. > 2016-09-21T09:41:48.184-04:00 WARN [Log] Found a corrupted index file, > /var/lib/graylog-server/journal/messagejournal-0/00000000000167286888.index, > deleting and rebuilding index... > 2016-09-21T09:41:49.740-04:00 INFO [LogManager] Logs loading complete. > 2016-09-21T09:41:49.740-04:00 INFO [KafkaJournal] Initialized Kafka based > journal at /var/lib/graylog-server/journal > 2016-09-21T09:41:49.779-04:00 INFO [InputBufferImpl] Initialized > InputBufferImpl with ring size <65536> and wait strategy > <BlockingWaitStrategy>, running 2 parallel message handlers. > 2016-09-21T09:41:49.825-04:00 INFO [cluster] Cluster created with > settings {hosts=[localhost:27017], mode=SINGLE, > requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', > maxWaitQueueSize=5000} > 2016-09-21T09:41:49.946-04:00 INFO [cluster] No server chosen by > ReadPreferenceServerSelector{readPreference=primary} from cluster > description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, > serverDescriptions=[ServerDescription{address=localhost:27017, > type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out > 2016-09-21T09:41:50.002-04:00 INFO [connection] Opened connection > [connectionId{localValue:1, serverValue:78}] to localhost:27017 > 2016-09-21T09:41:50.007-04:00 INFO [cluster] Monitor thread successfully > connected to server with description > ServerDescription{address=localhost:27017, type=STANDALONE, > state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 2, 9]}, > minWireVersion=0, maxWireVersion=4, maxDocumentSize=16777216, > roundTripTimeNanos=3105180} > 2016-09-21T09:41:50.028-04:00 INFO [connection] Opened connection > [connectionId{localValue:2, serverValue:79}] to localhost:27017 > 2016-09-21T09:41:50.737-04:00 INFO [node] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] version[2.3.5], pid[3698], > build[90f439f/2016-07-27T10:36:52Z] > 2016-09-21T09:41:50.741-04:00 INFO [node] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] initializing ... > 2016-09-21T09:41:50.751-04:00 INFO [plugins] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] modules [], plugins > [graylog-monitor], sites [] > 2016-09-21T09:41:54.439-04:00 INFO [node] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] initialized > 2016-09-21T09:41:54.726-04:00 INFO [Version] HV000001: Hibernate > Validator 5.2.4.Final > 2016-09-21T09:41:55.085-04:00 INFO [ProcessBuffer] Initialized > ProcessBuffer with ring size <65536> and wait strategy > <BlockingWaitStrategy>. > 2016-09-21T09:41:58.911-04:00 INFO [RulesEngineProvider] No static rules > file loaded. > 2016-09-21T09:41:59.117-04:00 INFO [OutputBuffer] Initialized > OutputBuffer with ring size <65536> and wait strategy > <BlockingWaitStrategy>. > 2016-09-21T09:42:01.395-04:00 INFO [ServerBootstrap] Graylog server > 2.1.1+01d50e5 starting up > 2016-09-21T09:42:01.396-04:00 INFO [ServerBootstrap] JRE: Oracle > Corporation 1.8.0_101 on Linux 3.16.0-77-generic > 2016-09-21T09:42:01.396-04:00 INFO [ServerBootstrap] Deployment: deb > 2016-09-21T09:42:01.396-04:00 INFO [ServerBootstrap] OS: Ubuntu 14.04.5 > LTS (trusty) > 2016-09-21T09:42:01.396-04:00 INFO [ServerBootstrap] Arch: amd64 > 2016-09-21T09:42:01.409-04:00 WARN [DeadEventLoggingListener] Received > unhandled event of type <org.graylog2.plugin.lifecycles.Lifecycle> from > event bus <AsyncEventBus{graylog-eventbus}> > 2016-09-21T09:42:01.539-04:00 INFO [node] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] starting ... > 2016-09-21T09:42:01.537-04:00 INFO [PeriodicalsService] Starting 22 > periodicals ... > 2016-09-21T09:42:01.549-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling > every [1s]. > 2016-09-21T09:42:01.556-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling > every [60s]. > 2016-09-21T09:42:01.559-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical > in [0s], polling every [1s]. > 2016-09-21T09:42:01.560-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s], > polling every [20s]. > 2016-09-21T09:42:01.560-04:00 INFO [PeriodicalsService] Not starting > [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical. Not > configured to run on this node. > 2016-09-21T09:42:01.561-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, > running forever. > 2016-09-21T09:42:01.562-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], > polling every [30s]. > 2016-09-21T09:42:01.565-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling > every [300s]. > 2016-09-21T09:42:01.566-04:00 INFO [IndexRetentionThread] Elasticsearch > cluster not available, skipping index retention checks. > 2016-09-21T09:42:01.571-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling > every [10s]. > 2016-09-21T09:42:01.571-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every > [1s]. > 2016-09-21T09:42:01.572-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling > every [1800s]. > 2016-09-21T09:42:01.573-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s], > polling every [1s]. > 2016-09-21T09:42:01.573-04:00 INFO [Periodicals] Starting > [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling > every [1s]. > 2016-09-21T09:42:01.574-04:00 INFO [Periodicals] Starting > [org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s], > polling every [86400s]. > 2016-09-21T09:42:01.574-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running > forever. > 2016-09-21T09:42:01.575-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical, > running forever. > 2016-09-21T09:42:01.575-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s], > polling every [3600s]. > 2016-09-21T09:42:01.615-04:00 INFO [connection] Opened connection > [connectionId{localValue:3, serverValue:80}] to localhost:27017 > 2016-09-21T09:42:01.633-04:00 INFO [connection] Opened connection > [connectionId{localValue:5, serverValue:82}] to localhost:27017 > 2016-09-21T09:42:01.634-04:00 INFO [connection] Opened connection > [connectionId{localValue:4, serverValue:81}] to localhost:27017 > 2016-09-21T09:42:01.642-04:00 INFO [connection] Opened connection > [connectionId{localValue:6, serverValue:83}] to localhost:27017 > 2016-09-21T09:42:01.654-04:00 INFO [connection] Opened connection > [connectionId{localValue:7, serverValue:84}] to localhost:27017 > 2016-09-21T09:42:01.674-04:00 INFO [IndexerClusterCheckerThread] Indexer > not fully initialized yet. Skipping periodic cluster check. > 2016-09-21T09:42:01.747-04:00 INFO [PeriodicalsService] Not starting > [org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not > configured to run on this node. > 2016-09-21T09:42:01.747-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.AlarmCallbacksMigrationPeriodical] periodical, > running forever. > 2016-09-21T09:42:01.752-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.ConfigurationManagementPeriodical] periodical, > running forever. > 2016-09-21T09:42:01.763-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.LdapGroupMappingMigration] periodical, running > forever. > 2016-09-21T09:42:01.782-04:00 INFO [Periodicals] Starting > [org.graylog2.periodical.IndexFailuresPeriodical] periodical, running > forever. > 2016-09-21T09:42:02.449-04:00 INFO [JerseyService] Enabling CORS for HTTP > endpoint > 2016-09-21T09:42:02.503-04:00 INFO [transport] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] publish_address { > 127.0.0.1:9350}, bound_addresses {[::1]:9350}, {127.0.0.1:9350} > 2016-09-21T09:42:02.529-04:00 INFO [discovery] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] > graylog/LSqj-br4QV2viKzC5-4cjA > 2016-09-21T09:42:05.544-04:00 WARN [discovery] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] waited for 3s and no initial > state was set by the discovery > 2016-09-21T09:42:05.544-04:00 INFO [node] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] started > 2016-09-21T09:42:06.919-04:00 INFO [service] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] detected_master > {Magdalena}{jtlOAsLnQ0GHZ6E_FRpKZA}{127.0.0.1}{127.0.0.1:9300}, added > {{Magdalena}{jtlOAsLnQ0GHZ6E_FRpKZA}{127.0.0.1}{127.0.0.1:9300},}, > reason: zen-disco-receive(from master > [{Magdalena}{jtlOAsLnQ0GHZ6E_FRpKZA}{127.0.0.1}{127.0.0.1:9300}]) > 2016-09-21T09:42:20.153-04:00 INFO [NetworkListener] Started listener > bound to [10.18.16.15:9000] > 2016-09-21T09:42:20.171-04:00 INFO [HttpServer] [HttpServer] Started. > 2016-09-21T09:42:20.177-04:00 INFO [JerseyService] Started REST API at < > http://10.18.16.15:9000/api/> > 2016-09-21T09:42:20.177-04:00 INFO [JerseyService] Started Web Interface > at <http://10.18.16.15:9000/> > 2016-09-21T09:42:20.178-04:00 INFO [ServiceManagerListener] Services are > healthy > 2016-09-21T09:42:20.194-04:00 INFO [ServerBootstrap] Services started, > startup times in ms: {OutputSetupService [RUNNING]=15, > BufferSynchronizerService [RUNNING]=17, KafkaJournal [RUNNING]=58, > InputSetupService [RUNNING]=81, JournalReader [RUNNING]=82, > PeriodicalsService [RUNNING]=271, IndexerSetupService [RUNNING]=5681, > JerseyService [RUNNING]=18654} > 2016-09-21T09:42:20.193-04:00 INFO [InputSetupService] Triggering > launching persisted inputs, node transitioned from Uninitialized?[LB:DEAD] > to Running?[LB:ALIVE] > 2016-09-21T09:42:20.261-04:00 INFO [ServerBootstrap] Graylog server up > and running. > 2016-09-21T09:42:20.299-04:00 INFO [InputStateListener] Input [Syslog > UDP/573241bbf6bc790854fe5ec4] is now STARTING > 2016-09-21T09:42:20.301-04:00 INFO [InputStateListener] Input [Syslog > UDP/57324191f6bc790854fe5e94] is now STARTING > 2016-09-21T09:42:20.303-04:00 INFO [InputStateListener] Input [Syslog > UDP/573241e0f6bc790854fe5eef] is now STARTING > 2016-09-21T09:42:20.461-04:00 WARN [NettyTransport] receiveBufferSize > (SO_RCVBUF) for input SyslogUDPInput{title=Clearpass, > type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=null} should be > 262144 but is 212992. > 2016-09-21T09:42:20.463-04:00 WARN [NettyTransport] receiveBufferSize > (SO_RCVBUF) for input SyslogUDPInput{title=Brocade Syslogs - Port 1516, > type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=null} should be > 262144 but is 212992. > 2016-09-21T09:42:20.461-04:00 WARN [NettyTransport] receiveBufferSize > (SO_RCVBUF) for input SyslogUDPInput{title=Syslog UDP 1514, > type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=null} should be > 1048576 but is 212992. > 2016-09-21T09:42:20.464-04:00 INFO [InputStateListener] Input [Syslog > UDP/573241bbf6bc790854fe5ec4] is now RUNNING > 2016-09-21T09:42:20.465-04:00 INFO [InputStateListener] Input [Syslog > UDP/57324191f6bc790854fe5e94] is now RUNNING > 2016-09-21T09:42:20.466-04:00 INFO [InputStateListener] Input [Syslog > UDP/573241e0f6bc790854fe5eef] is now RUNNING > 2016-09-21T09:42:30.533-04:00 WARN [jvm] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][25][4] duration > [3.7s], collections [1]/[4.4s], total [3.7s]/[4.6s], memory > [477.5mb]->[154.3mb]/[972.8mb], all_pools {[young] > [387.6mb]->[20.7mb]/[409.6mb]}{[survivor] > [49.6mb]->[51.1mb]/[51.1mb]}{[old] [40.3mb]->[94.3mb]/[512mb]} > 2016-09-21T09:42:41.544-04:00 WARN [jvm] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][30][5] duration > [6s], collections [1]/[6.8s], total [6s]/[10.7s], memory > [506.7mb]->[210.4mb]/[972.8mb], all_pools {[young] > [361.2mb]->[19.1mb]/[409.6mb]}{[survivor] > [51.1mb]->[51.1mb]/[51.1mb]}{[old] [94.3mb]->[140.1mb]/[512mb]} > 2016-09-21T09:43:02.982-04:00 WARN [jvm] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][35][6] duration > [15.4s], collections [1]/[16.6s], total [15.4s]/[26.1s], memory > [566.6mb]->[274.8mb]/[972.8mb], all_pools {[young] > [375.2mb]->[13.3mb]/[409.6mb]}{[survivor] > [51.1mb]->[51.1mb]/[51.1mb]}{[old] [140.1mb]->[210.7mb]/[512mb]} > 2016-09-21T09:44:13.861-04:00 WARN [jvm] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][48][7] duration > [55.9s], collections [1]/[58.2s], total [55.9s]/[1.3m], memory > [668.5mb]->[342.8mb]/[972.8mb], all_pools {[young] > [406.6mb]->[24.1mb]/[409.6mb]}{[survivor] > [51.1mb]->[51.1mb]/[51.1mb]}{[old] [210.7mb]->[269.2mb]/[512mb]} > 2016-09-21T09:45:53.548-04:00 WARN [jvm] > [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][71][8] duration > [1.2m], collections [1]/[1.2m], total [1.2m]/[2.6m], memory > [724.3mb]->[379.1mb]/[972.8mb], all_pools {[young] > [403.9mb]->[9.3mb]/[409.6mb]}{[survivor] [51.1mb]->[51.1mb]/[51.1mb]}{[old] > [269.2mb]->[319.9mb]/[512mb]} > > > > I really don't care about historical data so I'm willing to start fresh > with a new install of 2.1. I just want my performance back from Graylog2 > (version 1). > > Any help is greatly appreciated! > > Chris > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/dd4e3eb0-d1fb-4fcd-982f-69bc6115550a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
