Repository: incubator-senssoft Updated Branches: refs/heads/master 460766793 -> c4b63e90b
Setup system and container service monitoring for SensSoft stack. Project: http://git-wip-us.apache.org/repos/asf/incubator-senssoft/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-senssoft/commit/ae3ebdfd Tree: http://git-wip-us.apache.org/repos/asf/incubator-senssoft/tree/ae3ebdfd Diff: http://git-wip-us.apache.org/repos/asf/incubator-senssoft/diff/ae3ebdfd Branch: refs/heads/master Commit: ae3ebdfd2381be1f710d960bc8ef59e877e78245 Parents: 4607667 Author: msbeard <msbe...@apache.org> Authored: Fri May 11 12:24:24 2018 -0400 Committer: msbeard <msbe...@apache.org> Committed: Fri May 11 12:24:24 2018 -0400 ---------------------------------------------------------------------- docker/README.md | 44 ++++++-- docker/beats/metricbeat/metricbeat.yml | 24 +++++ docker/beats/metricbeat/modules.d/docker.yml | 6 ++ .../metricbeat/modules.d/elasticsearch.yml | 6 ++ docker/beats/metricbeat/modules.d/kibana.yml | 6 ++ .../beats/metricbeat/modules.d/kubernetes.yml | 29 ++++++ docker/beats/metricbeat/modules.d/logstash.yml | 6 ++ .../beats/metricbeat/modules.d/postgresql.yml | 25 +++++ docker/beats/metricbeat/modules.d/system.yml | 24 +++++ docker/docker-compose.cluster.yml | 85 +++++++++++----- docker/docker-compose.single-5.6.3.yml | 28 ++--- docker/docker-compose.yml | 102 ++++++++++--------- 12 files changed, 278 insertions(+), 107 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/README.md ---------------------------------------------------------------------- diff --git a/docker/README.md b/docker/README.md index 42d9856..3b7f51d 100644 --- a/docker/README.md +++ b/docker/README.md @@ -8,13 +8,25 @@ Prerequisites 1. Install docker-compose. Full instructions can be found [``here``](https://docs.docker.com/compose/install/). +1. Create docker-machine instance + ```bash + docker-machine create --virtualbox-memory 3072 --virtualbox-cpu-count 2 senssoft + ``` + 1. Before launching the Docker containers, ensure your ``vm_max_map_count`` kernel setting is set to at least 262144. Visit [``Running Elasticsearch in Production mode``](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/docker.html#docker-cli-run-prod-mode) for OS specific instructions. ```bash # Example for Linux systems - $ sysctl -w vm.max_map_count=262144 + $ docker-machine ssh senssoft sudo sysctl -w vm.max_map_count=262144 + ``` + +1. Create externel docker network to enable system monitoring. Only enable if running + the Elasticsearch 6.2.2 configuration (single and cluster mode) + + ```bash + $ docker network create esnet ``` Single Node Deployment @@ -24,16 +36,16 @@ Single Node Deployment its state. ```bash - $ docker-compose -f docker-compose.single-5.6.3.yml up -d loadbalancer + $ docker-compose -f docker-compose.single-5.6.3.yml up -d elasticsearch or - $ docker-compose up -d loadbalancer + $ docker-compose up -d elasticsearch ``` 1. Confirm state: ```bash - $ curl -XGET http://localhost:9200/_cluster/health?pretty + $ docker-machine ssh senssoft curl -XGET http://localhost:9200/_cluster/health?pretty { "cluster_name" : "SensSoft", "status" : "yellow", @@ -58,7 +70,7 @@ Single Node Deployment ```bash $ docker-compose up -d logstash - $ curl -XGET http://localhost:8100 + $ docker-machine ssh senssoft curl -XGET http://localhost:8100 ok ``` @@ -67,6 +79,7 @@ Single Node Deployment ```bash $ docker-compose up -d site + $ ssh docker@$(docker-machine ip senssoft) -L 8080:localhost:8080 ``` Visit `http://localhost:8080` and you will see Apache SensSoft's home page. @@ -76,6 +89,7 @@ Single Node Deployment ```bash $ docker-compose up -d kibana + $ ssh docker@$(docker-machine ip senssoft) -L 5601:localhost:5601 ``` 1. Register an index in Kibana to see the logs: @@ -118,20 +132,32 @@ Multi-Node Deployment on a Single Machine 1. Install ``docker-compose`` in an virtual environment. Full instructions can be found [``here``](https://docs.docker.com/compose/install/). +1. Create docker-machine instance + ```bash + docker-machine create --virtualbox-memory 2048 --virtualbox-cpu-count 2 senssoft + ``` + 1. Before launching the Docker containers, ensure your ``vm_max_map_count`` kernel setting is set to at least 262144. Visit [``Running Elasticsearch in Production mode``](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/docker.html#docker-cli-run-prod-mode) for OS specific instructions. ```bash # Example for Linux systems - $ sysctl -w vm.max_map_count=262144 + $ docker-machine ssh senssoft sudo sysctl -w vm.max_map_count=262144 + ``` + +1. Create externel docker network to enable system monitoring. Only enable if running + the Elasticsearch 6.2.2 configuration (single and cluster mode) + + ```bash + $ docker network create esnet ``` 1. Start Elasticsearch cluster: ```bash - $ docker-compose -f docker-compose.cluster.yaml up -d --scale elasticsearch=3 elasticsearch - $ docker-compose -f docker-compose.cluster.yaml up -d loadbalancer + $ docker-compose -f docker-compose.cluster.yml up -d --scale elasticsearch=3 elasticsearch + $ docker-compose -f docker-compose.cluster.yml up -d loadbalancer ``` The loadbalancer node exposes port 9200 on localhost and is the only node @@ -148,7 +174,7 @@ Multi-Node Deployment on a Single Machine 1. Confirm cluster state: ```bash - $ curl -XGET http://localhost:9200/_cluster/health?pretty + $ docker-machine ssh senssoft curl -XGET http://localhost:9200/_cluster/health\?pretty { "cluster_name" : "SensSoft", "status" : "green", http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/beats/metricbeat/metricbeat.yml ---------------------------------------------------------------------- diff --git a/docker/beats/metricbeat/metricbeat.yml b/docker/beats/metricbeat/metricbeat.yml new file mode 100644 index 0000000..3c61d77 --- /dev/null +++ b/docker/beats/metricbeat/metricbeat.yml @@ -0,0 +1,24 @@ +metricbeat.config.modules: + + # Glob pattern for configuration reloading + path: ${path.config}/modules.d/*.yml + + # Period on which files under path should be checked for changes + reload.period: 10s + + # Set to true to enable config reloading + reload.enabled: true + +# Maximum amount of time to randomly delay the start of a metricset. Use 0 to +# disable startup delay. +metricbeat.max_start_delay: 10s + +output.elasticsearch: + hosts: ["${ELASTICSEARCH_URL}"] + +setup.dashboards.enabled: true + +setup.kibana: + host: "${KIBANA_URL}" + +logging.to_files: false \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/beats/metricbeat/modules.d/docker.yml ---------------------------------------------------------------------- diff --git a/docker/beats/metricbeat/modules.d/docker.yml b/docker/beats/metricbeat/modules.d/docker.yml new file mode 100644 index 0000000..87a18f0 --- /dev/null +++ b/docker/beats/metricbeat/modules.d/docker.yml @@ -0,0 +1,6 @@ +#------------------------------- Docker Module ------------------------------- +- module: docker + metricsets: ["container", "cpu", "diskio", "healthcheck", "info", "memory", "network"] + hosts: ["unix:///var/run/docker.sock"] + enabled: true + period: 10s http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/beats/metricbeat/modules.d/elasticsearch.yml ---------------------------------------------------------------------- diff --git a/docker/beats/metricbeat/modules.d/elasticsearch.yml b/docker/beats/metricbeat/modules.d/elasticsearch.yml new file mode 100644 index 0000000..62f7dcb --- /dev/null +++ b/docker/beats/metricbeat/modules.d/elasticsearch.yml @@ -0,0 +1,6 @@ +#---------------------------- Elasticsearch Module --------------------------- +- module: elasticsearch + metricsets: ["node", "node_stats"] + period: 10s + enabled: true + hosts: ["${ELASTICSEARCH_URL}"] \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/beats/metricbeat/modules.d/kibana.yml ---------------------------------------------------------------------- diff --git a/docker/beats/metricbeat/modules.d/kibana.yml b/docker/beats/metricbeat/modules.d/kibana.yml new file mode 100644 index 0000000..2134a62 --- /dev/null +++ b/docker/beats/metricbeat/modules.d/kibana.yml @@ -0,0 +1,6 @@ +#------------------------------- Kibana Module ------------------------------- +- module: kibana + metricsets: ["status"] + period: 10s + enabled: true + hosts: ["${KIBANA_URL}"] \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/beats/metricbeat/modules.d/kubernetes.yml ---------------------------------------------------------------------- diff --git a/docker/beats/metricbeat/modules.d/kubernetes.yml b/docker/beats/metricbeat/modules.d/kubernetes.yml new file mode 100644 index 0000000..b3e018d --- /dev/null +++ b/docker/beats/metricbeat/modules.d/kubernetes.yml @@ -0,0 +1,29 @@ +#----------------------------- Kubernetes Module ----------------------------- +# Node metrics, from kubelet: +#- module: kubernetes +# metricsets: +# - node +# - system +# - pod +# - container +# - volume +# period: 10s +# hosts: ["localhost:10255"] + +# State metrics from kube-state-metrics service: +#- module: kubernetes +# enabled: false +# metricsets: +# - state_node +# - state_deployment +# - state_replicaset +# - state_pod +# - state_container +# period: 10s +# hosts: ["kube-state-metrics:8080"] + +# Kubernetes events +#- module: kubernetes +# enabled: false +# metricsets: +# - event \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/beats/metricbeat/modules.d/logstash.yml ---------------------------------------------------------------------- diff --git a/docker/beats/metricbeat/modules.d/logstash.yml b/docker/beats/metricbeat/modules.d/logstash.yml new file mode 100644 index 0000000..55a67ff --- /dev/null +++ b/docker/beats/metricbeat/modules.d/logstash.yml @@ -0,0 +1,6 @@ +#------------------------------ Logstash Module ------------------------------ +- module: logstash + metricsets: ["node", "node_stats"] + enabled: true + period: 10s + hosts: ["${LOGSTASH_URL}"] \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/beats/metricbeat/modules.d/postgresql.yml ---------------------------------------------------------------------- diff --git a/docker/beats/metricbeat/modules.d/postgresql.yml b/docker/beats/metricbeat/modules.d/postgresql.yml new file mode 100644 index 0000000..2b02360 --- /dev/null +++ b/docker/beats/metricbeat/modules.d/postgresql.yml @@ -0,0 +1,25 @@ +#----------------------------- PostgreSQL Module ----------------------------- +#- module: postgresql +# metricsets: +# # Stats about every PostgreSQL database +# - database +# +# # Stats about the background writer process's activity +# - bgwriter +# +# # Stats about every PostgreSQL process +# - activity +# +# period: 10s + + # The host must be passed as PostgreSQL URL. Example: + # postgres://localhost:5432?sslmode=disable + # The available parameters are documented here: + # https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters +# hosts: ["postgres://localhost:5432"] + + # Username to use when connecting to PostgreSQL. Empty by default. + #username: user + + # Password to use when connecting to PostgreSQL. Empty by default. + #password: pass \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/beats/metricbeat/modules.d/system.yml ---------------------------------------------------------------------- diff --git a/docker/beats/metricbeat/modules.d/system.yml b/docker/beats/metricbeat/modules.d/system.yml new file mode 100644 index 0000000..f2d6bb2 --- /dev/null +++ b/docker/beats/metricbeat/modules.d/system.yml @@ -0,0 +1,24 @@ +#------------------------------- System Module ------------------------------- +- module: system + metricsets: + - core + - cpu + - diskio + - filesystem + - fsstat + - load + - memory + - network + - process + - process_summary + - socket + - uptime + enabled: true + period: 10s + processes: ['.*'] + process.include_top_n: + by_cpu: 5 # include top 5 processes by CPU + by_memory: 5 # incude top 5 processes by memory + core.metrics: [percentages, ticks] + cpu.metrics: [percentages, normalized_percentages, ticks] + process.cgroups.enabled: true \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/docker-compose.cluster.yml ---------------------------------------------------------------------- diff --git a/docker/docker-compose.cluster.yml b/docker/docker-compose.cluster.yml index 5835716..96f6cc4 100644 --- a/docker/docker-compose.cluster.yml +++ b/docker/docker-compose.cluster.yml @@ -21,8 +21,6 @@ services: # Startup startup: build: ./startup - image: senssoft/startup:latest - container_name: senssoft-start depends_on: - "loadbalancer" command: bash /opt/entrypoint.sh @@ -30,8 +28,6 @@ services: # # Site site: build: ./site - image: senssoft/website:latest - container_name: senssoft-app ports: - 8080:8080 depends_on: @@ -40,8 +36,7 @@ services: # Elasticsearch Swarm Workers elasticsearch: - build: ./elasticsearch - image: senssoft/elasticsearch:6.2.2 + build: ./elasticsearch/6.2.2 environment: - NODE_MASTER=true - NUMBER_OF_MASTERS=3 @@ -54,15 +49,19 @@ services: - HTTP_ENABLE=false - HTTP_CORS_ENABLE=false - HTTP_CORS_ALLOW_ORIGIN=* - - "ES_JAVA_OPTS=-Xms512m -Xmx512m" + - "ES_JAVA_OPTS=-Xms256m -Xmx256m" + ulimits: + memlock: + soft: -1 + hard: -1 + healthcheck: + test: ["CMD", "curl","-s" ,"-f", "-u", "http://localhost:9200/_cluster/health"] networks: - esnet - # Elasticsearch Loadbalancer/Entry + # Elasticsearch Loadbalancer loadbalancer: - build: ./elasticsearch - image: senssoft/elasticsearch:6.2.2 - container_name: senssoft-loadbalancer + build: ./elasticsearch/6.2.2 environment: - NODE_MASTER=false - NUMBER_OF_MASTERS=1 @@ -75,7 +74,13 @@ services: - HTTP_ENABLE=true - HTTP_CORS_ENABLE=true - HTTP_CORS_ALLOW_ORIGIN=* - - "ES_JAVA_OPTS=-Xms512m -Xmx512m" + - "ES_JAVA_OPTS=-Xms256m -Xmx256m" + ulimits: + memlock: + soft: -1 + hard: -1 + healthcheck: + test: ["CMD", "curl","-s" ,"-f", "-u", "http://localhost:9200/_cluster/health"] ports: - "9200:9200" - "9300" @@ -84,11 +89,9 @@ services: # Kibana kibana: - build: ./kibana - image: senssoft/kibana:6.2.2 - container_name: senssoft-kibana + build: ./kibana/6.2.2 environment: - - "ES_JAVA_OPTS=-Xms2g -Xmx2g" + - "ES_JAVA_OPTS=-Xms256m -Xmx256m" - "ELASTICSEARCH_URL=http://loadbalancer:9200" ports: - "5601:5601" @@ -99,16 +102,14 @@ services: # Logstash logstash: - build: ./logstash - image: senssoft/logstash:6.2.2 - container_name: senssoft-logstash + build: ./logstash/6.2.2 environment: - "ELASTICSEARCH_URL=loadbalancer:9200" ports: - 8100:8100 volumes: - - ./logstash/pipeline/logstash-userale.conf:/usr/share/logstash/pipeline/logstash-userale.conf - - ./logstash/templates/userale.json:/usr/share/logstash/templates/userale.json + - ./logstash/6.2.2/pipeline/logstash-userale.conf:/usr/share/logstash/pipeline/logstash-userale.conf + - ./logstash/6.2.2/templates/userale.json:/usr/share/logstash/templates/userale.json depends_on: - "loadbalancer" networks: @@ -117,8 +118,6 @@ services: # Distill distill: build: ./distill - image: senssoft/distill:0.1.0 - container_name: senssoft-distill ports: - 8090:8090 depends_on: @@ -130,16 +129,12 @@ services: db: build: ./db image: senssoft/postgres:latest - container_name: senssoft-postgres ports: - "5432:5432" # TAP tap: build: ./tap - image: senssoft/tap:0.1.0 - container_name: senssoft-tap - # command: ["bash", "./wait-for-postgres.sh"] command: bash -c "/usr/src/app/wait-for-postgres.sh" ports: - "8010:8010" @@ -148,6 +143,40 @@ services: links: - db:db + # Specific container and configuration to monitor the HOST (filesystem, memory, processes,...) + metricbeat: + image: docker.elastic.co/beats/metricbeat:6.2.4 + restart: unless-stopped + user: root + pid: host + volumes: + #The commented sections below enable Metricbeat to monitor the Docker host + #rather than the Metricbeat container. These are used by the system module. + - /proc:/hostfs/proc:ro + - /sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro + #We mount the host filesystem so we can report on disk usage with the system module + - /:/hostfs:ro + #Mount the metricbeat configuration so users can make edit + - ./beats/metricbeat/metricbeat.yml:/usr/share/metricbeat/metricbeat.yml + #Mount the modules.d directory into the container. This allows user to + #potentially make changes to the modules and they will be dynamically loaded. + - ./beats/metricbeat/modules.d/:/usr/share/metricbeat/modules.d/ + #Allows us to report on docker from the hosts information + - /var/run/docker.sock:/var/run/docker.sock + depends_on: + - loadbalancer + environment: + - ELASTICSEARCH_URL=http://loadbalancer:9200 + - KIBANA_URL=http://kibana:5601 + - LOGSTASH_URL=http://logstash:9600 + extra_hosts: + - "loadbalancer:172.17.0.1" # The IP of docker0 interface to access host from container + - "kibana:172.17.0.1" # The IP of docker0 interface to access host from container + - "logstash:172.17.0.1" + command: metricbeat -e -system.hostfs=/hostfs -strict.perms=false + network_mode: host # Mandatory to monitor HOST filesystem, memory, processes,... + networks: esnet: - driver: bridge + external: + name: esnet http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/docker-compose.single-5.6.3.yml ---------------------------------------------------------------------- diff --git a/docker/docker-compose.single-5.6.3.yml b/docker/docker-compose.single-5.6.3.yml index e8e333f..0198644 100644 --- a/docker/docker-compose.single-5.6.3.yml +++ b/docker/docker-compose.single-5.6.3.yml @@ -21,8 +21,6 @@ services: # Startup startup: build: ./startup - image: senssoft/startup:latest - container_name: senssoft-start depends_on: - "loadbalancer" command: bash /opt/entrypoint.sh @@ -30,24 +28,20 @@ services: # Site site: build: ./site - image: senssoft/website:latest - container_name: senssoft-app ports: - 8080:8080 depends_on: - "logstash" command: python -m SimpleHTTPServer 8080 - # Elasticsearch Loadbalancer/Entry + # Elasticsearch loadbalancer: build: ./elasticsearch/5.6.3 - image: senssoft/elasticsearch:5.6.3 - container_name: senssoft-loadbalancers environment: - NODE_MASTER=true - NUMBER_OF_MASTERS=1 - NODE_COORDINATE=true - - "DISCOVERY_SERVICE=elasticsearch" + - "DISCOVERY_SERVICE=0.0.0.0" - NODE_DATA=true - NODE_INGEST=true - MAX_LOCAL_STORAGE_NODES=1 @@ -64,37 +58,31 @@ services: # Kibana kibana: build: ./kibana/5.6.3 -# image: senssoft/kibana:5.6.3 - container_name: senssoft-kibana environment: - "ES_JAVA_OPTS=-Xms2g -Xmx2g" - - "ELASTICSEARCH_URL=http://loadbalancer:9200" + - "ELASTICSEARCH_URL=http://elasticsearch:9200" ports: - "5601:5601" depends_on: - - "loadbalancer" + - "elasticsearch" networks: - esnet # Logstash logstash: build: ./logstash/5.6.3 - image: senssoft/logstash:5.6.3 - container_name: senssoft-logstash environment: - - "ELASTICSEARCH_URL=loadbalancer:9200" + - "ELASTICSEARCH_URL=elasticsearch:9200" ports: - 8100:8100 depends_on: - - "loadbalancer" + - "elasticsearch" networks: - esnet # Distill distill: build: ./distill - image: senssoft/distill:0.1.0 - container_name: senssoft-distill ports: - 8090:8090 depends_on: @@ -106,15 +94,12 @@ services: db: build: ./db image: senssoft/postgres:latest - container_name: senssoft-postgres ports: - "5432:5432" # TAP tap: build: ./tap - image: senssoft/tap:0.1.0 - container_name: senssoft-tap command: bash -c "/usr/src/app/wait-for-postgres.sh" ports: - "8010:8010" @@ -123,6 +108,7 @@ services: links: - db:db + networks: esnet: driver: bridge http://git-wip-us.apache.org/repos/asf/incubator-senssoft/blob/ae3ebdfd/docker/docker-compose.yml ---------------------------------------------------------------------- diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index a675f7b..30507c6 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -21,54 +21,27 @@ services: # Startup startup: build: ./startup - image: senssoft/startup:latest - container_name: senssoft-start depends_on: - - "loadbalancer" + - "elasticsearch" command: bash /opt/entrypoint.sh # Site site: build: ./site - image: senssoft/website:latest - container_name: senssoft-app ports: - 8080:8080 depends_on: - "logstash" command: python -m SimpleHTTPServer 8080 + # Elasticsearch elasticsearch: build: ./elasticsearch/6.2.2 - image: senssoft/elasticsearch:6.2.2 environment: - NODE_MASTER=true - NUMBER_OF_MASTERS=1 - - NODE_COORDINATE=false - - "DISCOVERY_SERVICE=elasticsearch" - - NODE_DATA=false - - NODE_INGEST=false - - MAX_LOCAL_STORAGE_NODES=1 - - NETWORK_HOST=0.0.0.0 - - HTTP_ENABLE=false - - HTTP_CORS_ENABLE=false - - HTTP_CORS_ALLOW_ORIGIN=* - - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - ports: - - "9300" - networks: - - esnet - - # Elasticsearch Loadbalancer/Entry - loadbalancer: - build: ./elasticsearch/6.2.2 - image: senssoft/elasticsearch:6.2.2 - container_name: senssoft-loadbalancer - environment: - - NODE_MASTER=false - - NUMBER_OF_MASTERS=1 - NODE_COORDINATE=true - - "DISCOVERY_SERVICE=elasticsearch" + - "DISCOVERY_SERVICE=0.0.0.0" - NODE_DATA=true - NODE_INGEST=true - MAX_LOCAL_STORAGE_NODES=1 @@ -77,54 +50,55 @@ services: - HTTP_CORS_ENABLE=true - HTTP_CORS_ALLOW_ORIGIN=* - "ES_JAVA_OPTS=-Xms512m -Xmx512m" + ulimits: + memlock: + soft: -1 + hard: -1 + healthcheck: + test: ["CMD", "curl","-s" ,"-f", "-u", "http://localhost:9200/_cluster/health"] + # Mount volumes to backup elasticsearch data +# volumes: +# - esdata:/usr/share/elasticsaerch/data ports: - "9200:9200" networks: - esnet - depends_on: - - elasticsearch # Kibana kibana: build: ./kibana/6.2.2 - image: senssoft/kibana:6.2.2 - container_name: senssoft-kibana environment: - - "ES_JAVA_OPTS=-Xms2g -Xmx2g" - - "ELASTICSEARCH_URL=http://loadbalancer:9200" + - "ES_JAVA_OPTS=-Xms512m -Xmx512m" + - "ELASTICSEARCH_URL=http://elasticsearch:9200" ports: - "5601:5601" depends_on: - - "loadbalancer" + - "elasticsearch" networks: - esnet # Logstash logstash: build: ./logstash/6.2.2 - image: senssoft/logstash:6.2.2 - container_name: senssoft-logstash environment: - - "ELASTICSEARCH_URL=loadbalancer:9200" + - "ELASTICSEARCH_URL=elasticsearch:9200" ports: - 8100:8100 volumes: - ./logstash/6.2.2/pipeline/logstash-userale.conf:/usr/share/logstash/pipeline/logstash-userale.conf - ./logstash/6.2.2/templates/userale.json:/usr/share/logstash/templates/userale.json depends_on: - - "loadbalancer" + - "elasticsearch" networks: - esnet # Distill distill: build: ./distill - image: senssoft/distill:0.1.0 - container_name: senssoft-distill ports: - 8090:8090 depends_on: - - "loadbalancer" + - "elasticsearch" networks: - esnet @@ -132,16 +106,12 @@ services: db: build: ./db image: senssoft/postgres:latest - container_name: senssoft-postgres ports: - "5432:5432" # TAP tap: build: ./tap - image: senssoft/tap:0.1.0 - container_name: senssoft-tap - # command: ["bash", "./wait-for-postgres.sh"] command: bash -c "/usr/src/app/wait-for-postgres.sh" ports: - "8010:8010" @@ -150,6 +120,40 @@ services: links: - db:db + # Specific container and configuration to monitor the HOST (filesystem, memory, processes,...) + metricbeat: + image: docker.elastic.co/beats/metricbeat:6.2.4 + restart: unless-stopped + user: root + pid: host + volumes: + #The commented sections below enable Metricbeat to monitor the Docker host + #rather than the Metricbeat container. These are used by the system module. + - /proc:/hostfs/proc:ro + - /sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro + #We mount the host filesystem so we can report on disk usage with the system module + - /:/hostfs:ro + #Mount the metricbeat configuration so users can make edit + - ./beats/metricbeat/metricbeat.yml:/usr/share/metricbeat/metricbeat.yml + #Mount the modules.d directory into the container. This allows user to + #potentially make changes to the modules and they will be dynamically loaded. + - ./beats/metricbeat/modules.d/:/usr/share/metricbeat/modules.d/ + #Allows us to report on docker from the hosts information + - /var/run/docker.sock:/var/run/docker.sock + depends_on: + - elasticsearch + environment: + - ELASTICSEARCH_URL=http://elasticsearch:9200 + - KIBANA_URL=http://kibana:5601 + - LOGSTASH_URL=http://logstash:9600 + extra_hosts: + - "elasticsearch:172.17.0.1" # The IP of docker0 interface to access host from container + - "kibana:172.17.0.1" # The IP of docker0 interface to access host from container + - "logstash:172.17.0.1" + command: metricbeat -e -system.hostfs=/hostfs -strict.perms=false + network_mode: host # Mandatory to monitor HOST filesystem, memory, processes,... + networks: esnet: - driver: bridge + external: + name: esnet