This is an automated email from the ASF dual-hosted git repository. heneveld pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/brooklyn-docs.git
commit 703983f7d1b70d2d17e219f62a9ce68abdff7916 Author: Juan Cabrerizo <[email protected]> AuthorDate: Fri Sep 24 13:36:57 2021 +0100 Segreate logging docs --- Gemfile.lock | 4 + guide/ops/index.md | 2 +- guide/ops/logging/index.md | 115 +++++++++++++++++++++++ guide/ops/{logging.md => logging/logbook.md} | 121 +++---------------------- guide/ops/production-installation.md | 2 +- guide/ops/requirements.md | 4 +- guide/ops/troubleshooting/fails-to-start.md | 2 +- guide/ops/troubleshooting/overview.md | 2 +- guide/ops/troubleshooting/slow-unresponsive.md | 2 +- 9 files changed, 140 insertions(+), 114 deletions(-) diff --git a/Gemfile.lock b/Gemfile.lock index 85873ad..9a141b8 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -53,6 +53,7 @@ GEM kramdown-parser-gfm (1.1.0) kramdown (~> 2.0) libv8 (3.16.14.19) + libv8 (3.16.14.19-x86_64-linux) liquid (4.0.3) listen (3.5.1) rb-fsevent (~> 0.10, >= 0.10.3) @@ -64,6 +65,8 @@ GEM jekyll-seo-tag (~> 2.1) nokogiri (1.11.5-x86_64-darwin) racc (~> 1.4) + nokogiri (1.11.5-x86_64-linux) + racc (~> 1.4) nokogumbo (2.0.5) nokogiri (~> 1.8, >= 1.8.4) parallel (1.20.1) @@ -94,6 +97,7 @@ GEM PLATFORMS x86_64-darwin-20 + x86_64-linux DEPENDENCIES html-proofer diff --git a/guide/ops/index.md b/guide/ops/index.md index 06853ca..20077a7 100644 --- a/guide/ops/index.md +++ b/guide/ops/index.md @@ -12,7 +12,7 @@ children: - configuration/ - persistence/ - high-availability/ -- logging.md +- logging/ - externalized-configuration.md - requirements.md - paths.md diff --git a/guide/ops/logging/index.md b/guide/ops/logging/index.md new file mode 100644 index 0000000..b920967 --- /dev/null +++ b/guide/ops/logging/index.md @@ -0,0 +1,115 @@ +--- +title: Logging +layout: website-normal +check_directory_for_children: true +--- + +Brooklyn uses the SLF4J logging facade, which allows use of many popular frameworks including `logback`, +`java.util.logging` and `log4j`. + +The convention for log levels is as follows: + +* `ERROR` and above: exceptional situations which indicate that something has unexpectedly failed or + some other problem has occurred which the user is expected to attend to +* `WARN`: exceptional situations which the user may which to know about but which do not necessarily indicate failure or require a response. +* `INFO`: a synopsis of activity, but which should not generate large volumes of events nor overwhelm a human observer. +* `DEBUG`: detail of activity which might merit closer inspection under certain circumstances. +* `TRACE` and lower: detail of activity which is not normally of interest, but which might merit closer inspection under certain circumstances including sensitive information (e.g. secrets) that should not be exposed in higher lover levels. A configuration example for TRACE level is present in the log configuration file, but is commented because of security concerns. + +Loggers follow the ``package.ClassName`` naming standard. + +The default logging is to write INFO+ messages to `brooklyn.info.log`, +and DEBUG+ to `brooklyn.debug.log`. Each is a rolling log file, +where the past 10 files will be kept. INFO level, and above, messages +will be logged to the karaf console. Use the `log:` commands in the +karaf client, e.g. `log:tail`, to read these messages. + + +## Using Logback through OSGi Pax Logging + +In the OSGi based Apache Brooklyn logging is configured from ops4j pax logging. + +See: [Logging - OSGi based Apache Brooklyn](/guide/dev/tips/logging/index.md#osgi-based-apache-brooklyn) <br/> +[https://ops4j1.jira.com/wiki/display/paxlogging/Configuration](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration) + +## Standard Configuration + +A `org.ops4j.pax.logging.cfg` file is included in the `etc/` directly of the Brooklyn distro; +this is read by `brooklyn` at launch time. Changes to the logging configuration, +such as new appenders or different log levels, can be made directly in this file. + +Karaf logging is highly configurable. For example enable the sift appender to log to separate log files for +each bundle as described here: [Advanced configuration](https://karaf.apache.org/manual/latest/#_advanced_configuration) + +Using the default configuration the log entries are reported in UTC time. If you want the logging to be reported using the server local time you can replace the `log4j2.pattern` removing the UTC flag and the Z suffix: +```properties +log4j2.pattern = %d{ISO8601}Z %X{task.id}-%X{entity.ids} %-5.5p %3X{bundle.id} %c{1.} [%.16t] %m%n +``` + +## Advanced Configuration + +The default `logback.xml` file references a collection of other log configuration files +included in the Brooklyn jars. It is necessary to understand the source structure +in the [logback-includes]({{ site.brooklyn.url.git }}/logging/logback-includes) project. + +For example, to change the debug log inclusions, create a folder `brooklyn` under `conf` +and create a file `logback-debug.xml` based on the +[brooklyn/logback-debug.xml]({{ site.brooklyn.url.git }}/logging/logback-includes/src/main/resources/brooklyn/logback-debug.xml) +from that project. + +A full explanation of logging in karaf is available [here](https://karaf.apache.org/manual/latest/#_log). + +## Karaf Log commands + +Logging commands are available through the karaf console. These let you interact with the logs and dynamically change +logging configuration in a running application. + +Some useful log: commands are: + +log:display mylogger -p "%d - %c - %m%n" - Show the log entries for a specific logger with a different pattern. + +log:get/set - Show / set the currently configured log levels + +log:tail - As display but will show continuously + +log:exception-display - Display the last exception + +## Log File Backup + +*This sub-section is a work in progress; feedback from the community is extremely welcome.* + +The default rolling log files can be backed up periodically, e.g. using a CRON job. + +Note however that the rolling log file naming scheme will rename the historic zipped log files +such that `brooklyn.debug-1.log.zip` is the most recent zipped log file. When the current +`brooklyn.debug.log` is to be zipped, the previous zip file will be renamed +`brooklyn.debug-2.log.zip`. This renaming of files can make RSYNC or backups tricky. + +An option is to covert/move the file to a name that includes the last-modified timestamp. +For example (on mac): + +```shell + LOG_FILE=brooklyn.debug-1.log.zip + TIMESTAMP=`stat -f '%Um' $LOG_FILE` + mv $LOG_FILE /path/to/archive/brooklyn.debug-$TIMESTAMP.log.zip +``` + +## Logging Aggregators + +Integration with systems like Logstash and Splunk is possible using standard log4j configuration. +Log4j can be configured to write to syslog using the SyslogAppender +which can then [feed its logs to Logstash](http://www.logstash.net/docs/1.4.2/inputs/syslog). + +## For More Information + +The following resources may be useful when configuring logging: + +* The [logback-includes]({{ site.brooklyn.url.git }}/usage/logback-includes) project +* [Brooklyn Developer Guide](/guide/dev/tips/logging/) logging tips +* The [Logback Project](http://logback.qos.ch/) home page +* [OPS4J Pax Logging](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration) +* [Elasticsearch Best Practices](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/best_practices.html) +* [Elasticsearch Memory Usage](https://www.elastic.co/blog/significantly-decrease-your-elasticsearch-heap-memory-usage) +* [OpenSearch Index Management](https://opensearch.org/docs/im-plugin/ism/index/) and [policies](https://opensearch.org/docs/im-plugin/ism/policies/) + +* {% child_content %} \ No newline at end of file diff --git a/guide/ops/logging.md b/guide/ops/logging/logbook.md similarity index 56% rename from guide/ops/logging.md rename to guide/ops/logging/logbook.md index 9089543..4398e90 100644 --- a/guide/ops/logging.md +++ b/guide/ops/logging/logbook.md @@ -1,101 +1,21 @@ --- -title: Logging -layout: website-normal +section: Logbook +section_type: inline +section_position: 2 --- -Brooklyn uses the SLF4J logging facade, which allows use of many popular frameworks including `logback`, -`java.util.logging` and `log4j`. - -The convention for log levels is as follows: - -* `ERROR` and above: exceptional situations which indicate that something has unexpectedly failed or -some other problem has occurred which the user is expected to attend to -* `WARN`: exceptional situations which the user may which to know about but which do not necessarily indicate failure or require a response. -* `INFO`: a synopsis of activity, but which should not generate large volumes of events nor overwhelm a human observer. -* `DEBUG`: detail of activity which might merit closer inspection under certain circumstances. -* `TRACE` and lower: detail of activity which is not normally of interest, but which might merit closer inspection under certain circumstances including sensitive information (e.g. secrets) that should not be exposed in higher lover levels. A configuration example for TRACE level is present in the log configuration file, but is commented because of security concerns. - -Loggers follow the ``package.ClassName`` naming standard. - -The default logging is to write INFO+ messages to `brooklyn.info.log`, -and DEBUG+ to `brooklyn.debug.log`. Each is a rolling log file, -where the past 10 files will be kept. INFO level, and above, messages -will be logged to the karaf console. Use the `log:` commands in the -karaf client, e.g. `log:tail`, to read these messages. - - -## Using Logback through OSGi Pax Logging - -In the OSGi based Apache Brooklyn logging is configured from ops4j pax logging. - -See: [Logging - OSGi based Apache Brooklyn](/guide/dev/tips/logging.md#osgi-based-apache-brooklyn) <br/> -[https://ops4j1.jira.com/wiki/display/paxlogging/Configuration](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration) - -## Standard Configuration - -A `org.ops4j.pax.logging.cfg` file is included in the `etc/` directly of the Brooklyn distro; -this is read by `brooklyn` at launch time. Changes to the logging configuration, -such as new appenders or different log levels, can be made directly in this file. - -Karaf logging is highly configurable. For example enable the sift appender to log to separate log files for -each bundle as described here: [Advanced configuration](https://karaf.apache.org/manual/latest/#_advanced_configuration) - -Using the default configuration the log entries are reported in UTC time. If you want the logging to be reported using the server local time you can replace the `log4j2.pattern` removing the UTC flag and the Z suffix: -```properties -log4j2.pattern = %d{ISO8601} %X{task.id}-%X{entity.ids} %-5.5p %3X{bundle.id} %c{1.} [%.16t] %m%n -``` - -## Advanced Configuration - -The default `logback.xml` file references a collection of other log configuration files -included in the Brooklyn jars. It is necessary to understand the source structure -in the [logback-includes]({{ site.brooklyn.url.git }}/logging/logback-includes) project. - -For example, to change the debug log inclusions, create a folder `brooklyn` under `conf` -and create a file `logback-debug.xml` based on the -[brooklyn/logback-debug.xml]({{ site.brooklyn.url.git }}/logging/logback-includes/src/main/resources/brooklyn/logback-debug.xml) -from that project. - -A full explanation of logging in karaf is available [here](https://karaf.apache.org/manual/latest/#_log). - - -## Log File Backup - -*This sub-section is a work in progress; feedback from the community is extremely welcome.* - -The default rolling log files can be backed up periodically, e.g. using a CRON job. - -Note however that the rolling log file naming scheme will rename the historic zipped log files -such that `brooklyn.debug-1.log.zip` is the most recent zipped log file. When the current -`brooklyn.debug.log` is to be zipped, the previous zip file will be renamed -`brooklyn.debug-2.log.zip`. This renaming of files can make RSYNC or backups tricky. - -An option is to covert/move the file to a name that includes the last-modified timestamp. -For example (on mac): - - LOG_FILE=brooklyn.debug-1.log.zip - TIMESTAMP=`stat -f '%Um' $LOG_FILE` - mv $LOG_FILE /path/to/archive/brooklyn.debug-$TIMESTAMP.log.zip - - -## Logging Aggregators - -Integration with systems like Logstash and Splunk is possible using standard log4j configuration. -Log4j can be configured to write to syslog using the SyslogAppender -which can then [feed its logs to Logstash](http://www.logstash.net/docs/1.4.2/inputs/syslog). - - ## Logbook The logbook offers the possibility to query and view logs in the UI. By default, logs are stored in files as per configuration in `etc/org.ops4j.pax.logging.cfg`. The logbook can be configured against different log aggregation sources by setting the `brooklyn.logbook.logStore` parameter in `brooklyn.cfg`, and depending which backend is used, other parameters. -For example to use the local log files written by Apache Brooklyn in a production environment, assuming `/var/logs`, +For example to use the local log files written by Apache Brooklyn in a production environment, assuming `/var/logs`, you could configure: - +```properties brooklyn.logbook.logStore=org.apache.brooklyn.util.core.logbook.file.FileLogStore brooklyn.logbook.fileLogStore.path=/var/logs/brooklyn/brooklyn.debug.log +``` The default mode is to use the local log file in `data/log/` relative to the launch directory. @@ -104,13 +24,13 @@ The `FileLogStore` implementation is not compatible with multiline logs, only th In production environments where log data is desired to be retained, Apache Brooklyn supports Elasticsearch backends. This can be a dedicated ES environment for use by Apache Brooklyn or a shared/managed ES facility that handles many logs, or -- for lightweight usage -- a simple local ES server running on the same instance as Apache Brooklyn. -As with any log storage requirement, the sizing, scaling, backup and maintenance of the logging environment +As with any log storage requirement, the sizing, scaling, backup and maintenance of the logging environment requires careful attention. Elasticsearch includes numerous options to configure these, with one suggested configuration outlined in more detail below. By default, only users with the `root`, `powerUser`, or an explicit `logViewer` entitlement are able to see log info through Apache Brooklyn. -The Logbook UI widget can be found throughout the product: +The Logbook UI widget can be found throughout the product: in the About section, where all logs can be viewed; on the App Inspector Entity Summary view, and on the App Inspector Activity view, where logs filtered by entity ID and activity ID respectively are shown. @@ -118,21 +38,21 @@ on the App Inspector Activity view, where logs filtered by entity ID and activit ### Suggested Elasticsearch Setup -[OpenSearch (OpenDistro for Elasticsearch)](https://opendistro.github.io/for-elasticsearch) is an Apache-licensed open-source +[OpenSearch (OpenDistro for Elasticsearch)](https://opendistro.github.io/for-elasticsearch) is an Apache-licensed open-source backend that works well with Apache Brooklyn, with this configuration in `brooklyn.cfg`: - +```properties brooklyn.logbook.logStore=org.apache.brooklyn.util.core.logbook.opensearch.OpenSearchLogStore brooklyn.logbook.openSearchLogStore.host=https://localhost:9200 brooklyn.logbook.openSearchLogStore.index=brooklyn brooklyn.logbook.openSearchLogStore.user=admin brooklyn.logbook.openSearchLogStore.password=admin brooklyn.logbook.openSearchLogStore.verifySsl=false - +``` #### Routing Logs to Elastic Search There are many solutions to routing log messages from Apache Brooklyn to Elasticsearch, either plugging in to the log4j subsystem -or routing the log files from disk. [Fluentd](https://www.fluentd.org/download), with the following configuration in `td-agent.conf`, +or routing the log files from disk. [Fluentd](https://www.fluentd.org/download), with the following configuration in `td-agent.conf`, is a good simple way to forward content added to the info and debug log files: ``` @@ -241,7 +161,7 @@ Apache Brooklyn can be configured to use an index _pattern_ for querying, eg: Policies also allow handling the lifecycle of the indexes. For example, to delete debug indexes after a period of time based on the index naming pattern used in this page: -``` +```yaml { "policy": { "description": "Delete workflow", @@ -276,20 +196,7 @@ For example, to delete debug indexes after a period of time based on the index n } ``` -With these building blocks, and others linked below, you can configure the retention policy that suits your environment, +With these building blocks, and others linked below, you can configure the retention policy that suits your environment, balancing the trade-off between data availability and resource usage. - -## For More Information - -The following resources may be useful when configuring logging: - -* The [logback-includes]({{ site.brooklyn.url.git }}/usage/logback-includes) project -* [Brooklyn Developer Guide](/guide/dev/tips/logging.md) logging tips -* The [Logback Project](http://logback.qos.ch/) home page -* [OPS4J Pax Logging](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration) -* [Elasticsearch Best Practices](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/best_practices.html) -* [Elasticsearch Memory Usage](https://www.elastic.co/blog/significantly-decrease-your-elasticsearch-heap-memory-usage) -* [OpenSearch Index Management](https://opensearch.org/docs/im-plugin/ism/index/) and [policies](https://opensearch.org/docs/im-plugin/ism/policies/) - diff --git a/guide/ops/production-installation.md b/guide/ops/production-installation.md index b6aecc5..57cefa0 100644 --- a/guide/ops/production-installation.md +++ b/guide/ops/production-installation.md @@ -16,7 +16,7 @@ To install Apache Brooklyn on a production server: This guide covers the basics. You may also wish to configure: -* [Logging](/guide/ops/logging.md) +* [Logging](/guide/ops/logging/index.md) * [Persistence](persistence) * [High availability](high-availability) diff --git a/guide/ops/requirements.md b/guide/ops/requirements.md index 90a6242..b470da6 100644 --- a/guide/ops/requirements.md +++ b/guide/ops/requirements.md @@ -36,7 +36,7 @@ There are three main consumers of disk space: * **Log files**: Brooklyn writes info and debug log files. By default, these are written to the local filesystem. This can be reconfigured to set the destination and to increase or decrease the detail in the logs. See the - [Logging](/guide/ops/logging.md) section for more details. + [Logging](/guide/ops/logging/index.md) section for more details. The Apache Brooklyn distribution itself, when unpacked, consumes approximately 75MB of disk space. This includes everything needed to run Brooklyn except for a @@ -57,7 +57,7 @@ therefore, can consume up to 2GB of disk space. In the default configuration of Brooklyn's `.tar.gz` and `.zip` distributions, logs are saved to the Brooklyn installation directory. You will most likely want -to [reconfigure Brooklyn's logging](/guide/ops/logging.md) to save logs to a location +to [reconfigure Brooklyn's logging](/guide/ops/logging/index.md) to save logs to a location elsewhere. In the `.rpm` and `.deb` packaging, logging files will be located under `/var/log`. You can further reconfiguring the logging detail level and log rotation according to your organisation's policy. diff --git a/guide/ops/troubleshooting/fails-to-start.md b/guide/ops/troubleshooting/fails-to-start.md index 14544d4..035721e 100644 --- a/guide/ops/troubleshooting/fails-to-start.md +++ b/guide/ops/troubleshooting/fails-to-start.md @@ -5,7 +5,7 @@ toc: /guide/toc.json --- If Apache Brooklyn does not start, or starts with errors, the problem is usually easy to resolve. -The first place to look is the [logs](/guide/ops/logging.md): `grep` for the first `ERROR`, +The first place to look is the [logs](/guide/ops/logging/index.md): `grep` for the first `ERROR`, and sometimes look backwards for the first `WARN` message. There are a handful of common causes. diff --git a/guide/ops/troubleshooting/overview.md b/guide/ops/troubleshooting/overview.md index ab7e42f..720dd89 100644 --- a/guide/ops/troubleshooting/overview.md +++ b/guide/ops/troubleshooting/overview.md @@ -78,7 +78,7 @@ For example, it can show the exception stack trace in the thread that was execut ## Log Files Brooklyn's logging is configurable, for the files created, the logging levels, etc. -See [Logging docs](/guide/ops/logging.md). +See [Logging docs](/guide/ops/logging/index.md). With out-of-the-box logging, `brooklyn.info.log` and `brooklyn.debug.log` files are created. These are by default rolling log files: when the log reaches a given size, it is compressed and a new log file is started. diff --git a/guide/ops/troubleshooting/slow-unresponsive.md b/guide/ops/troubleshooting/slow-unresponsive.md index 7eb9a0c..93b5e5e 100644 --- a/guide/ops/troubleshooting/slow-unresponsive.md +++ b/guide/ops/troubleshooting/slow-unresponsive.md @@ -164,7 +164,7 @@ and the [IDE docs](/guide/dev/env/ide) for more information. ## Log Files Apache Brooklyn will by default create brooklyn.info.log and brooklyn.debug.log files. See the -[Logging](/guide/ops/logging.md) docs for more information. +[Logging](/guide/ops/logging/index.md) docs for more information. The following are useful log messages to search for (e.g. using `grep`). Note the wording of these messages (or their very presence) may change in future version of Brooklyn.
