Hi Chris, what kind of clients are sending syslog messages into Graylog and how are they formatted?
Cheers, Jochen On Wednesday, 21 September 2016 17:34:45 UTC+2, Chris Call wrote: > > Thanks for the reply! > > I did not realize when I built the new VM I only gave it 1GB. Just > increased it to 16 and added 10GB for Java with a max of 14GB. Now I can > log in fine but no messages are appearing. I go to the inputs and click > 'show received messages' on a syslog input running on port 1516 and it > comes up with "Nothing found." > > When I run "sudo TCPDUMP 'port 1516'" I see messages coming in to the box. > Any clues? > > Thanks again for your help! > > > > On Wednesday, September 21, 2016 at 10:09:53 AM UTC-4, Jochen Schalanda > wrote: >> >> Hi Chris, >> >> what's the output of the following curl commands if you run it on the >> machine your web browser is running on? >> >> curl -v -X GET http://10.18.16.15:9000/ >> curl -v -X GET -H 'Accept: application/json' http://10.18.16.15:9000/api/ >> >> >> The long GC pauses (over 1 minute is really bad and unusual) are also >> strange. Try giving Graylog more memory (currently it's 1 GB according to >> the logs, try 2 GB), see >> http://docs.graylog.org/en/2.1/pages/faq.html#raise-the-java-heap. >> >> >> Cheers, >> Jochen >> >> On Wednesday, 21 September 2016 15:59:35 UTC+2, Chris Call wrote: >>> >>> I had a great experience with Graylog before version 2.0 but I started >>> fresh with a 2.0 install and had issues (streams kept stopping) and then >>> upgraded to 2.0.1 I believe and now finally to 2.1. >>> >>> Right now, I can't open the web interface when I browse to >>> 10.18.16.15:9000 (IP address of the host). I get nothing in the >>> browser. Chrome gives me this "The 10.18.16.15 page isn’t working. >>> 10.18.16.15 didn’t send any data." and no errors or messages in the >>> developer java console. >>> >>> This is a single system running everything for Graylog and here are my >>> config files and output: >>> >>> etc/graylog/server/server.conf: >>> is_master = true >>> node_id_file = /etc/graylog/server/node-id >>> password_secret = <secret> >>> root_password_sha2 = <secret> >>> root_email = <email> >>> plugin_dir = plugin >>> rest_listen_uri = http://10.18.16.15:9000/api/ >>> web_enable = true >>> web_listen_uri = http://10.18.16.15:9000/ >>> rotation_strategy = count >>> elasticsearch_max_docs_per_index = 20000000 >>> elasticsearch_max_number_of_indices = 20 >>> retention_strategy = delete >>> elasticsearch_shards = 4 >>> elasticsearch_replicas = 0 >>> elasticsearch_index_prefix = graylog >>> allow_leading_wildcard_searches = false >>> allow_highlighting = false >>> elasticsearch_analyzer = standard >>> output_batch_size = 500 >>> output_flush_interval = 1 >>> output_fault_count_threshold = 5 >>> output_fault_penalty_seconds = 30 >>> processbuffer_processors = 5 >>> outputbuffer_processors = 3 >>> processor_wait_strategy = blocking >>> ring_size = 65536 >>> inputbuffer_ring_size = 65536 >>> inputbuffer_processors = 2 >>> inputbuffer_wait_strategy = blocking >>> message_journal_enabled = true >>> message_journal_dir = /var/lib/graylog-server/journal >>> lb_recognition_period_seconds = 3 >>> mongodb_uri = mongodb://localhost/graylog >>> mongodb_max_connections = 1000 >>> mongodb_threads_allowed_to_block_multiplier = 5 >>> content_packs_loader_enabled = false >>> content_packs_auto_load = grok-patterns.json >>> proxied_requests_thread_pool_size = 32 >>> >>> >>> /etc/elasticsearch/elasticsearch.yml >>> # ======================== Elasticsearch Configuration >>> ========================= >>> cluster.name: graylog >>> >>> result of " curl -XGET ' >>> http://localhost:9200/_cluster/health?pretty=true' ": >>> { >>> "cluster_name" : "graylog", >>> "status" : "green", >>> "timed_out" : false, >>> "number_of_nodes" : 1, >>> "number_of_data_nodes" : 1, >>> "active_primary_shards" : 20, >>> "active_shards" : 20, >>> "relocating_shards" : 0, >>> "initializing_shards" : 0, >>> "unassigned_shards" : 0, >>> "delayed_unassigned_shards" : 0, >>> "number_of_pending_tasks" : 0, >>> "number_of_in_flight_fetch" : 0, >>> "task_max_waiting_in_queue_millis" : 0, >>> "active_shards_percent_as_number" : 100.0 >>> } >>> >>> /var/log/graylog-server/server.log: >>> 2016-09-21T09:41:42.573-04:00 WARN [PluginLoader] Plugin directory >>> /plugin does not exist, not loading plugins. >>> 2016-09-21T09:41:43.030-04:00 INFO [CmdLineTool] Running with JVM >>> arguments: -Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB >>> -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled >>> -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC >>> -XX:-OmitStackTraceInFastThrow >>> -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml >>> -Djava.library.path=/usr/share/graylog-server/lib/sigar >>> -Dgraylog2.installation_source=deb >>> 2016-09-21T09:41:47.611-04:00 INFO [InputBufferImpl] Message journal is >>> enabled. >>> 2016-09-21T09:41:47.678-04:00 INFO [NodeId] Node ID: >>> ecdff2ab-d0a2-4ddb-975e-d2379fb3625d >>> 2016-09-21T09:41:48.054-04:00 INFO [LogManager] Loading logs. >>> 2016-09-21T09:41:48.184-04:00 WARN [Log] Found a corrupted index file, >>> /var/lib/graylog-server/journal/messagejournal-0/00000000000167286888.index, >>> >>> deleting and rebuilding index... >>> 2016-09-21T09:41:49.740-04:00 INFO [LogManager] Logs loading complete. >>> 2016-09-21T09:41:49.740-04:00 INFO [KafkaJournal] Initialized Kafka >>> based journal at /var/lib/graylog-server/journal >>> 2016-09-21T09:41:49.779-04:00 INFO [InputBufferImpl] Initialized >>> InputBufferImpl with ring size <65536> and wait strategy >>> <BlockingWaitStrategy>, running 2 parallel message handlers. >>> 2016-09-21T09:41:49.825-04:00 INFO [cluster] Cluster created with >>> settings {hosts=[localhost:27017], mode=SINGLE, >>> requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', >>> maxWaitQueueSize=5000} >>> 2016-09-21T09:41:49.946-04:00 INFO [cluster] No server chosen by >>> ReadPreferenceServerSelector{readPreference=primary} from cluster >>> description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, >>> serverDescriptions=[ServerDescription{address=localhost:27017, >>> type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out >>> 2016-09-21T09:41:50.002-04:00 INFO [connection] Opened connection >>> [connectionId{localValue:1, serverValue:78}] to localhost:27017 >>> 2016-09-21T09:41:50.007-04:00 INFO [cluster] Monitor thread >>> successfully connected to server with description >>> ServerDescription{address=localhost:27017, type=STANDALONE, >>> state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 2, 9]}, >>> minWireVersion=0, maxWireVersion=4, maxDocumentSize=16777216, >>> roundTripTimeNanos=3105180} >>> 2016-09-21T09:41:50.028-04:00 INFO [connection] Opened connection >>> [connectionId{localValue:2, serverValue:79}] to localhost:27017 >>> 2016-09-21T09:41:50.737-04:00 INFO [node] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] version[2.3.5], pid[3698], >>> build[90f439f/2016-07-27T10:36:52Z] >>> 2016-09-21T09:41:50.741-04:00 INFO [node] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] initializing ... >>> 2016-09-21T09:41:50.751-04:00 INFO [plugins] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] modules [], plugins >>> [graylog-monitor], sites [] >>> 2016-09-21T09:41:54.439-04:00 INFO [node] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] initialized >>> 2016-09-21T09:41:54.726-04:00 INFO [Version] HV000001: Hibernate >>> Validator 5.2.4.Final >>> 2016-09-21T09:41:55.085-04:00 INFO [ProcessBuffer] Initialized >>> ProcessBuffer with ring size <65536> and wait strategy >>> <BlockingWaitStrategy>. >>> 2016-09-21T09:41:58.911-04:00 INFO [RulesEngineProvider] No static >>> rules file loaded. >>> 2016-09-21T09:41:59.117-04:00 INFO [OutputBuffer] Initialized >>> OutputBuffer with ring size <65536> and wait strategy >>> <BlockingWaitStrategy>. >>> 2016-09-21T09:42:01.395-04:00 INFO [ServerBootstrap] Graylog server >>> 2.1.1+01d50e5 starting up >>> 2016-09-21T09:42:01.396-04:00 INFO [ServerBootstrap] JRE: Oracle >>> Corporation 1.8.0_101 on Linux 3.16.0-77-generic >>> 2016-09-21T09:42:01.396-04:00 INFO [ServerBootstrap] Deployment: deb >>> 2016-09-21T09:42:01.396-04:00 INFO [ServerBootstrap] OS: Ubuntu 14.04.5 >>> LTS (trusty) >>> 2016-09-21T09:42:01.396-04:00 INFO [ServerBootstrap] Arch: amd64 >>> 2016-09-21T09:42:01.409-04:00 WARN [DeadEventLoggingListener] Received >>> unhandled event of type <org.graylog2.plugin.lifecycles.Lifecycle> from >>> event bus <AsyncEventBus{graylog-eventbus}> >>> 2016-09-21T09:42:01.539-04:00 INFO [node] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] starting ... >>> 2016-09-21T09:42:01.537-04:00 INFO [PeriodicalsService] Starting 22 >>> periodicals ... >>> 2016-09-21T09:42:01.549-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling >>> every [1s]. >>> 2016-09-21T09:42:01.556-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling >>> every [60s]. >>> 2016-09-21T09:42:01.559-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical >>> in [0s], polling every [1s]. >>> 2016-09-21T09:42:01.560-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s], >>> polling every [20s]. >>> 2016-09-21T09:42:01.560-04:00 INFO [PeriodicalsService] Not starting >>> [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical. Not >>> configured to run on this node. >>> 2016-09-21T09:42:01.561-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, >>> running forever. >>> 2016-09-21T09:42:01.562-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], >>> polling every [30s]. >>> 2016-09-21T09:42:01.565-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling >>> every [300s]. >>> 2016-09-21T09:42:01.566-04:00 INFO [IndexRetentionThread] Elasticsearch >>> cluster not available, skipping index retention checks. >>> 2016-09-21T09:42:01.571-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling >>> every [10s]. >>> 2016-09-21T09:42:01.571-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every >>> [1s]. >>> 2016-09-21T09:42:01.572-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling >>> every [1800s]. >>> 2016-09-21T09:42:01.573-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s], >>> polling every [1s]. >>> 2016-09-21T09:42:01.573-04:00 INFO [Periodicals] Starting >>> [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling >>> every [1s]. >>> 2016-09-21T09:42:01.574-04:00 INFO [Periodicals] Starting >>> [org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s], >>> polling every [86400s]. >>> 2016-09-21T09:42:01.574-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running >>> forever. >>> 2016-09-21T09:42:01.575-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical, >>> running forever. >>> 2016-09-21T09:42:01.575-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s], >>> polling every [3600s]. >>> 2016-09-21T09:42:01.615-04:00 INFO [connection] Opened connection >>> [connectionId{localValue:3, serverValue:80}] to localhost:27017 >>> 2016-09-21T09:42:01.633-04:00 INFO [connection] Opened connection >>> [connectionId{localValue:5, serverValue:82}] to localhost:27017 >>> 2016-09-21T09:42:01.634-04:00 INFO [connection] Opened connection >>> [connectionId{localValue:4, serverValue:81}] to localhost:27017 >>> 2016-09-21T09:42:01.642-04:00 INFO [connection] Opened connection >>> [connectionId{localValue:6, serverValue:83}] to localhost:27017 >>> 2016-09-21T09:42:01.654-04:00 INFO [connection] Opened connection >>> [connectionId{localValue:7, serverValue:84}] to localhost:27017 >>> 2016-09-21T09:42:01.674-04:00 INFO [IndexerClusterCheckerThread] >>> Indexer not fully initialized yet. Skipping periodic cluster check. >>> 2016-09-21T09:42:01.747-04:00 INFO [PeriodicalsService] Not starting >>> [org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not >>> configured to run on this node. >>> 2016-09-21T09:42:01.747-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.AlarmCallbacksMigrationPeriodical] periodical, >>> running forever. >>> 2016-09-21T09:42:01.752-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.ConfigurationManagementPeriodical] periodical, >>> running forever. >>> 2016-09-21T09:42:01.763-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.LdapGroupMappingMigration] periodical, running >>> forever. >>> 2016-09-21T09:42:01.782-04:00 INFO [Periodicals] Starting >>> [org.graylog2.periodical.IndexFailuresPeriodical] periodical, running >>> forever. >>> 2016-09-21T09:42:02.449-04:00 INFO [JerseyService] Enabling CORS for >>> HTTP endpoint >>> 2016-09-21T09:42:02.503-04:00 INFO [transport] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] publish_address { >>> 127.0.0.1:9350}, bound_addresses {[::1]:9350}, {127.0.0.1:9350} >>> 2016-09-21T09:42:02.529-04:00 INFO [discovery] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] >>> graylog/LSqj-br4QV2viKzC5-4cjA >>> 2016-09-21T09:42:05.544-04:00 WARN [discovery] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] waited for 3s and no initial >>> state was set by the discovery >>> 2016-09-21T09:42:05.544-04:00 INFO [node] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] started >>> 2016-09-21T09:42:06.919-04:00 INFO [service] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] detected_master >>> {Magdalena}{jtlOAsLnQ0GHZ6E_FRpKZA}{127.0.0.1}{127.0.0.1:9300}, added >>> {{Magdalena}{jtlOAsLnQ0GHZ6E_FRpKZA}{127.0.0.1}{127.0.0.1:9300},}, >>> reason: zen-disco-receive(from master >>> [{Magdalena}{jtlOAsLnQ0GHZ6E_FRpKZA}{127.0.0.1}{127.0.0.1:9300}]) >>> 2016-09-21T09:42:20.153-04:00 INFO [NetworkListener] Started listener >>> bound to [10.18.16.15:9000] >>> 2016-09-21T09:42:20.171-04:00 INFO [HttpServer] [HttpServer] Started. >>> 2016-09-21T09:42:20.177-04:00 INFO [JerseyService] Started REST API at < >>> http://10.18.16.15:9000/api/> >>> 2016-09-21T09:42:20.177-04:00 INFO [JerseyService] Started Web >>> Interface at <http://10.18.16.15:9000/> >>> 2016-09-21T09:42:20.178-04:00 INFO [ServiceManagerListener] Services >>> are healthy >>> 2016-09-21T09:42:20.194-04:00 INFO [ServerBootstrap] Services started, >>> startup times in ms: {OutputSetupService [RUNNING]=15, >>> BufferSynchronizerService [RUNNING]=17, KafkaJournal [RUNNING]=58, >>> InputSetupService [RUNNING]=81, JournalReader [RUNNING]=82, >>> PeriodicalsService [RUNNING]=271, IndexerSetupService [RUNNING]=5681, >>> JerseyService [RUNNING]=18654} >>> 2016-09-21T09:42:20.193-04:00 INFO [InputSetupService] Triggering >>> launching persisted inputs, node transitioned from Uninitialized?[LB:DEAD] >>> to Running?[LB:ALIVE] >>> 2016-09-21T09:42:20.261-04:00 INFO [ServerBootstrap] Graylog server up >>> and running. >>> 2016-09-21T09:42:20.299-04:00 INFO [InputStateListener] Input [Syslog >>> UDP/573241bbf6bc790854fe5ec4] is now STARTING >>> 2016-09-21T09:42:20.301-04:00 INFO [InputStateListener] Input [Syslog >>> UDP/57324191f6bc790854fe5e94] is now STARTING >>> 2016-09-21T09:42:20.303-04:00 INFO [InputStateListener] Input [Syslog >>> UDP/573241e0f6bc790854fe5eef] is now STARTING >>> 2016-09-21T09:42:20.461-04:00 WARN [NettyTransport] receiveBufferSize >>> (SO_RCVBUF) for input SyslogUDPInput{title=Clearpass, >>> type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=null} should be >>> 262144 but is 212992. >>> 2016-09-21T09:42:20.463-04:00 WARN [NettyTransport] receiveBufferSize >>> (SO_RCVBUF) for input SyslogUDPInput{title=Brocade Syslogs - Port 1516, >>> type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=null} should be >>> 262144 but is 212992. >>> 2016-09-21T09:42:20.461-04:00 WARN [NettyTransport] receiveBufferSize >>> (SO_RCVBUF) for input SyslogUDPInput{title=Syslog UDP 1514, >>> type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=null} should be >>> 1048576 but is 212992. >>> 2016-09-21T09:42:20.464-04:00 INFO [InputStateListener] Input [Syslog >>> UDP/573241bbf6bc790854fe5ec4] is now RUNNING >>> 2016-09-21T09:42:20.465-04:00 INFO [InputStateListener] Input [Syslog >>> UDP/57324191f6bc790854fe5e94] is now RUNNING >>> 2016-09-21T09:42:20.466-04:00 INFO [InputStateListener] Input [Syslog >>> UDP/573241e0f6bc790854fe5eef] is now RUNNING >>> 2016-09-21T09:42:30.533-04:00 WARN [jvm] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][25][4] duration >>> [3.7s], collections [1]/[4.4s], total [3.7s]/[4.6s], memory >>> [477.5mb]->[154.3mb]/[972.8mb], all_pools {[young] >>> [387.6mb]->[20.7mb]/[409.6mb]}{[survivor] >>> [49.6mb]->[51.1mb]/[51.1mb]}{[old] [40.3mb]->[94.3mb]/[512mb]} >>> 2016-09-21T09:42:41.544-04:00 WARN [jvm] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][30][5] duration >>> [6s], collections [1]/[6.8s], total [6s]/[10.7s], memory >>> [506.7mb]->[210.4mb]/[972.8mb], all_pools {[young] >>> [361.2mb]->[19.1mb]/[409.6mb]}{[survivor] >>> [51.1mb]->[51.1mb]/[51.1mb]}{[old] [94.3mb]->[140.1mb]/[512mb]} >>> 2016-09-21T09:43:02.982-04:00 WARN [jvm] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][35][6] duration >>> [15.4s], collections [1]/[16.6s], total [15.4s]/[26.1s], memory >>> [566.6mb]->[274.8mb]/[972.8mb], all_pools {[young] >>> [375.2mb]->[13.3mb]/[409.6mb]}{[survivor] >>> [51.1mb]->[51.1mb]/[51.1mb]}{[old] [140.1mb]->[210.7mb]/[512mb]} >>> 2016-09-21T09:44:13.861-04:00 WARN [jvm] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][48][7] duration >>> [55.9s], collections [1]/[58.2s], total [55.9s]/[1.3m], memory >>> [668.5mb]->[342.8mb]/[972.8mb], all_pools {[young] >>> [406.6mb]->[24.1mb]/[409.6mb]}{[survivor] >>> [51.1mb]->[51.1mb]/[51.1mb]}{[old] [210.7mb]->[269.2mb]/[512mb]} >>> 2016-09-21T09:45:53.548-04:00 WARN [jvm] >>> [graylog-ecdff2ab-d0a2-4ddb-975e-d2379fb3625d] [gc][young][71][8] duration >>> [1.2m], collections [1]/[1.2m], total [1.2m]/[2.6m], memory >>> [724.3mb]->[379.1mb]/[972.8mb], all_pools {[young] >>> [403.9mb]->[9.3mb]/[409.6mb]}{[survivor] [51.1mb]->[51.1mb]/[51.1mb]}{[old] >>> [269.2mb]->[319.9mb]/[512mb]} >>> >>> >>> >>> I really don't care about historical data so I'm willing to start fresh >>> with a new install of 2.1. I just want my performance back from Graylog2 >>> (version 1). >>> >>> Any help is greatly appreciated! >>> >>> Chris >>> >> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/a07db7bd-5647-4885-ad17-f14acdb50900%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
