Added: qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Memory.html URL: http://svn.apache.org/viewvc/qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Memory.html?rev=1722689&view=auto ============================================================================== --- qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Memory.html (added) +++ qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Memory.html Sat Jan 2 23:59:48 2016 @@ -0,0 +1,266 @@ +<!DOCTYPE html> +<!-- + - + - Licensed to the Apache Software Foundation (ASF) under one + - or more contributor license agreements. See the NOTICE file + - distributed with this work for additional information + - regarding copyright ownership. The ASF licenses this file + - to you under the Apache License, Version 2.0 (the + - "License"); you may not use this file except in compliance + - with the License. You may obtain a copy of the License at + - + - http://www.apache.org/licenses/LICENSE-2.0 + - + - Unless required by applicable law or agreed to in writing, + - software distributed under the License is distributed on an + - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + - KIND, either express or implied. See the License for the + - specific language governing permissions and limitations + - under the License. + - +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en"> + <head> + <title>9.10. Memory - Apache Qpid™</title> + <meta http-equiv="X-UA-Compatible" content="IE=edge"/> + <meta name="viewport" content="width=device-width, initial-scale=1.0"/> + <link rel="stylesheet" href="/site.css" type="text/css" async="async"/> + <link rel="stylesheet" href="/deferred.css" type="text/css" defer="defer"/> + <script type="text/javascript">var _deferredFunctions = [];</script> + <script type="text/javascript" src="/deferred.js" defer="defer"></script> + <!--[if lte IE 8]> + <link rel="stylesheet" href="/ie.css" type="text/css"/> + <script type="text/javascript" src="/html5shiv.js"></script> + <![endif]--> + + <!-- Redirects for `go get` and godoc.org --> + <meta name="go-import" + content="qpid.apache.org git https://git-wip-us.apache.org/repos/asf/qpid-proton.git"/> + <meta name="go-source" + content="qpid.apache.org +https://github.com/apache/qpid-proton/blob/go1/README.md +https://github.com/apache/qpid-proton/tree/go1{/dir} +https://github.com/apache/qpid-proton/blob/go1{/dir}/{file}#L{line}"/> + </head> + <body> + <div id="-content"> + <div id="-top" class="panel"> + <a id="-menu-link"><img width="16" height="16" src="" alt="Menu"/></a> + + <a id="-search-link"><img width="22" height="16" src="" alt="Search"/></a> + + <ul id="-global-navigation"> + <li><a id="-logotype" href="/index.html">Apache Qpid<sup>™</sup></a></li> + <li><a href="/documentation.html">Documentation</a></li> + <li><a href="/download.html">Download</a></li> + <li><a href="/discussion.html">Discussion</a></li> + </ul> + </div> + + <div id="-menu" class="panel" style="display: none;"> + <div class="flex"> + <section> + <h3>Project</h3> + + <ul> + <li><a href="/overview.html">Overview</a></li> + <li><a href="/components/index.html">Components</a></li> + <li><a href="/releases/index.html">Releases</a></li> + </ul> + </section> + + <section> + <h3>Messaging APIs</h3> + + <ul> + <li><a href="/proton/index.html">Qpid Proton</a></li> + <li><a href="/components/jms/index.html">Qpid JMS</a></li> + <li><a href="/components/messaging-api/index.html">Qpid Messaging API</a></li> + </ul> + </section> + + <section> + <h3>Servers and tools</h3> + + <ul> + <li><a href="/components/java-broker/index.html">Java broker</a></li> + <li><a href="/components/cpp-broker/index.html">C++ broker</a></li> + <li><a href="/components/dispatch-router/index.html">Dispatch router</a></li> + </ul> + </section> + + <section> + <h3>Resources</h3> + + <ul> + <li><a href="/dashboard.html">Dashboard</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/qpid/Index";>Wiki</a></li> + <li><a href="/resources.html">More resources</a></li> + </ul> + </section> + </div> + </div> + + <div id="-search" class="panel" style="display: none;"> + <form action="http://www.google.com/search"; method="get"> + <input type="hidden" name="sitesearch" value="qpid.apache.org"/> + <input type="text" name="q" maxlength="255" autofocus="autofocus" tabindex="1"/> + <button type="submit">Search</button> + <a href="/search.html">More ways to search</a> + </form> + </div> + + <div id="-middle" class="panel"> + <ul id="-path-navigation"><li><a href="/index.html">Home</a></li><li><a href="/releases/index.html">Releases</a></li><li><a href="/releases/qpid-java-trunk/index.html">Qpid Java Trunk</a></li><li><a href="/releases/qpid-java-trunk/java-broker/book/index.html">AMQP Messaging Broker (Java)</a></li><li>9.10. Memory</li></ul> + + <div id="-middle-content"> + <div class="docbook"><div class="navheader"><table summary="Navigation header" width="100%"><tr><th align="center" colspan="3">9.10. Memory</th></tr><tr><td align="left" width="20%"><a accesskey="p" href="Java-Broker-Runtime-Connection-Limit.html">Prev</a> </td><th align="center" width="60%">Chapter 9. Runtime</th><td align="right" width="20%"> <a accesskey="n" href="Java-Broker-High-Availability.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Java-Broker-Runtime-Memory"></a>9.10. Memory</h2></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Introduction"></a>9.10.1. Introduction</h3></div></div></div><p> + Understanding how the Qpid broker uses memory is essential to running a high performing and reliable service. + A wrongly configured broker can exhibit poor performance or even crash with an <code class="literal">OutOfMemoryError</code>. + Unfortunately, memory usage is not a simple topic and thus requires some in depth explanations. + This page should give the required background information to make informed decisions on how to configure your broker. + </p><p> + <a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Types" title="9.10.2. Types of Memory">Section 9.10.2, “Types of Memory”</a> explains the two different kinds of Java memory most relevant to the broker. + <a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Usage" title="9.10.3. Memory Usage in the Broker">Section 9.10.3, “Memory Usage in the Broker”</a> goes on to explain which parts of the broker use what kind of memory. + <a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory" title="9.10.4. Low Memory Conditions">Section 9.10.4, “Low Memory Conditions”</a> explains what happens when the system runs low on memory. + <a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Defaults" title="9.10.5. Defaults">Section 9.10.5, “Defaults”</a> lays out the default settings of the Qpid broker. + Finally, <a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Tuning" title="9.10.6. Memory Tuning the Broker">Section 9.10.6, “Memory Tuning the Broker”</a> gives some advice on tuning your broker. + </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Types"></a>9.10.2. Types of Memory</h3></div></div></div><p> + While Java has a couple of different internal memory types we will focus on the two types that are relevant to the Qpid broker. + Both of these memory types are taken from the same physical memory (RAM). + </p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5806"></a>9.10.2.1. Heap</h4></div></div></div><p> + Normally, all objects are allocated from Java's heap memory. + Once, nothing references an object it is cleaned up by the Java Garbage Collector and it's memory returned to the heap. + This works fine for most use cases. + However, when interacting with other parts of the operating system using Java's heap is not ideal. + This is where the so called direct memory comes into play. + </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5811"></a>9.10.2.2. Direct</h4></div></div></div><p> + The world ouside of the JVM, in particular the operating system (OS), does not know about Java heap memory and uses other structures like C arrays. + In order to interact with these systems Java needs to copy data between its own heap memory and these native structures. + This can become a bottle neck when there is a lot of exchange between Java and the OS like in I/O (both disk and network) heavy applications. + Java's solution to this is to allow programmers to request <code class="literal">ByteBuffer</code>s from so called direct memory. + This is an opaque structure that <span class="emphasis"><em>might</em></span> have an underlying implementation that makes it efficient to interact with the OS. + Unfortunately, the GC is not good at tracking direct memory and in general it is inadvisable to use direct memory for regular objects. + </p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Usage"></a>9.10.3. Memory Usage in the Broker</h3></div></div></div><p> + This section lists some note worthy users of memory within the broker and where possible lists their usage of heap and direct memory. + Note that to ensure smooth performance some heap memory should remain unused by the application and be reserved for the JVM to do house keeping and garbage collection. + <a class="link" href="https://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/util/DbCacheSize.html"; target="_top">Some guides</a> advise to reserve up to 30% of heap memory for the JVM. + </p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5830"></a>9.10.3.1. Broker</h4></div></div></div><p> + The broker itself uses a moderate amount of heap memory (≈15 MB). + However, each connection and session comes with a heap overhead of about 17 kB and 15 kB respectively. + In addition, each connection reserves 512 kB direct memory for network I/O. + </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5835"></a>9.10.3.2. Virtual Hosts</h4></div></div></div><p> + The amount of memory a Virtual Host uses depends on its type. + For a JSON Virtual Host Node with a BDB Virtual Host the heap memory usage is approximately 2 MB. + However, each BDB Virtual Hosts has a mandatory cache in heap memory which has an impact on performance. + See <a class="link" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Usage-BDB" title="9.10.3.4. Message Store">below</a> for more information. + </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5843"></a>9.10.3.3. Messages</h4></div></div></div><p> + Messages and their headers are kept in direct memory and have an additional overhead of approximately 1 kB heap memory each. + This means that most brokers will want to have more direct memory than heap memory. + When many small messages accumulate on the broker the 1 kB heap memory overhead can become a <a class="link" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory-Heap" title="9.10.4.1. Low on Heap Memory">limiting factor</a>. + </p><p> + When the broker is <a class="link" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory-Direct" title="9.10.4.2. Low on Direct Memory">running low on direct memory</a> + it will evict messages from memory and <a class="link" href="Java-Broker-Runtime-Flow-To-Disk.html" title="9.6. Flow to Disk">flow them to disk</a>. + For persistent messages this only means freeing the direct memory representation because they always have an on-disk representation to guard against unexpected failure (e.g., a power cut). + For transient messages this implies additional disk I/O. + After being flown to disk messages need to be re-read from disk before delivery. + </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Memory-Usage-BDB"></a>9.10.3.4. Message Store</h4></div></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="d0e5862"></a>Berkeley DB (BDB)</h5></div></div></div><p> + The broker can use Oracle's BDB JE (BDB) as a message store to persist messages by writing them to a database. + BDB uses a mandatory cache for navigating and organising its database structure. + Sizing and tuning this cache is a topic of its own and would go beyond the scope of this guide. + Suffice to say that by default Qpid uses 5% of heap memory for BDB caches (each Virtual Host uses a separate cache) or 10 MB per BDB store, whichever is greater. + See the <a class="link" href="http://www.oracle.com/us/products/database/berkeley-db/je"; target="_top">official webpage</a> especially <a class="link" href="http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/util/DbCacheSize.html"; target="_top">this page</a> for more information. + For those interested, Qpid uses <a class="link" href="http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/CacheMode.html#EVICT_LN"; target="_top">EVICT_LN</a> as its default JE cacheMode. + </p><p> + Note that due to licensing concerns Qpid does not ship the BDB JE jar files. + </p></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="d0e5878"></a>Derby</h5></div></div></div><p> + TODO + </p></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5883"></a>9.10.3.5. HTTP Management</h4></div></div></div><p> + Qpid uses Jetty for the HTTP Management (both REST and Web Management Console). + When the management plugin is loaded it will allocate the memory it needs and should not require more memory during operation and can thus be largely ignored. + </p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Low-Memory"></a>9.10.4. Low Memory Conditions</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Memory-Low-Memory-Heap"></a>9.10.4.1. Low on Heap Memory</h4></div></div></div><p> + When the broker runs low on heap memory performance will degrade because the JVM will trigger full garbage collection (GC) events in a struggle to free memory. + These full GC events are also called stop-the-world events as they completely halt the execution of the Java application. + Stop-the-world-events may take any where from a couple of milliseconds up to several minutes. + Should the heap memory demands rise even further the JVM will eventually throw an OutOfMemoryError which will cause the broker to shut down. + </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Memory-Low-Memory-Direct"></a>9.10.4.2. Low on Direct Memory</h4></div></div></div><p> + When the broker detects that it uses 40% of available direct memory it will start flowing incoming transient messages to disk and reading them back before delivery. + This will prevent the broker from running out of direct memory but may degrade performance by requiring disk I/O. + </p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Defaults"></a>9.10.5. Defaults</h3></div></div></div><p> + By default Qpid uses these settiongs: + </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"> + 0.5 GB heap memory + </li><li class="listitem"> + 1.5 GB direct memory + </li><li class="listitem"> + 5% of heap reserved for the JE cache. + </li><li class="listitem"> + Start flow-to-disk at 40% direct memory utilisation. + </li></ul></div><p> + As an example, this would accomodate a broker with 50 connections, each serving 5 sessions, and each session having 1000 messages of 1 kB on queues in the broker. + This means a total of 250 concurrent sessions and a total of 250000 messages without flowing messages to disk. + </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Tuning"></a>9.10.6. Memory Tuning the Broker</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5919"></a>9.10.6.1. Java Tuning</h4></div></div></div><p> + Most of these options are implementation specific. It is assumed you are using Oracle Java 1.7 and Qpid v6. + </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"> + Heap and direct memory can be configured through the <a class="link" href="Java-Broker-Appendix-Environment-Variables.html#Java-Broker-Appendix-Environment-Variables-Qpid-Java-Mem"><code class="literal">QPID_JAVA_MEM</code> environment variable</a>. + </li></ul></div><p> + </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5933"></a>9.10.6.2. Qpid Tuning</h4></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"> + The system property <code class="literal">qpid.broker.bdbTotalCacheSize</code> sets the total amount of heap memory (in bytes) allocated to BDB caches. + </li><li class="listitem"> + The system property <code class="literal">broker.flowToDiskThreshold</code> sets the threshold (in bytes) for flowing transient messages to disk. + Should the broker use more than direct memory it will flow incoming messages to disk. + Should utilisation fall beneath the threshold it will stop flowing messages to disk. + </li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5947"></a>9.10.6.3. Formulas</h4></div></div></div><p> + We developed a simple formula which estimates the <span class="emphasis"><em>minimum</em></span> memory usage of the broker under certain usage. + These are rough estimate so we strongly recommend testing your configuration extensively. + Also, if your machine has more memory available by all means use more memory as it can only improve the performance and stability of your broker. + However, remember that both heap and direct memory are served from your computer's physical memory so their sum should never exceed the physically available RAM (minus what other processes use). + </p><p> + <span class="mathphrase"> + memory<sub>heap</sub> = 15 MB + 15 kB * N<sub>sessions</sub> + 1 kB * N<sub>messages</sub> + 17 kB * N<sub>connections</sub> + </span> + <span class="mathphrase"> + memory<sub>direct</sub> = 2 MB + (200 B + averageSize<sub>msg</sub> *2)* N<sub>messages</sub> + 512 kB * N<sub>connections</sub> + </span> + Where <span class="mathphrase">N</span> denotes the total number of connections/sessions/messages on the broker. Furthermore, for direct memory only the messages that have not been flown to disk are relevant. + </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e5990"></a>9.10.6.4. Things to Consider</h4></div></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="d0e5993"></a>Performance</h5></div></div></div><p> + Choosing a smaller direct memory size will lower the threshold for flowing transient messages to disk when messages accumulate on a queue. + This can have impact on performance in the transient case where otherwise no disk I/O would be involved. + </p><p> + Having to little heap memory will result in poor performance due to frequent garbage collection events. See <a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory" title="9.10.4. Low Memory Conditions">Section 9.10.4, “Low Memory Conditions”</a> for more details. + </p></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="d0e6002"></a>OutOfMemoryError</h5></div></div></div><p> + Choosing too low heap memory can cause an OutOfMemoryError which will force the broker to shut down. + In this sense the available heap memory puts a hard limit on the number of messages you can have in the broker at the same time. + </p><p> + If the Java runs out of direct memory it also throws a OutOfMemoryError resulting the a broker shutdown. + Under normal circumstances this should not happen but needs to be considered when deviating from the default configuration, especially when changing the flowToDiskThreshold. + </p><p> + If you are sending very large messages you should accommodate for this by making sure you have enough direct memory. + </p></div></div></div></div><div class="navfooter"><hr /><table summary="Navigation footer" width="100%"><tr><td align="left" width="40%"><a accesskey="p" href="Java-Broker-Runtime-Connection-Limit.html">Prev</a> </td><td align="center" width="20%"><a accesskey="u" href="Java-Broker-Runtime.html">Up</a></td><td align="right" width="40%"> <a accesskey="n" href="Java-Broker-High-Availability.html">Next</a></td></tr><tr><td align="left" valign="top" width="40%">9.9. Connection Limits </td><td align="center" width="20%"><a accesskey="h" href="AMQP-Messaging-Broker-Java-Book.html">Home</a></td><td align="right" valign="top" width="40%"> Chapter 10. High Availability</td></tr></table></div></div> + + <hr/> + + <ul id="-apache-navigation"> + <li><a href="http://www.apache.org/";>Apache</a></li> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks!</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + <li><a href="http://www.apache.org/";><img id="-apache-feather" width="48" height="14" src="" alt="Apache"/></a></li> + </ul> + + <p id="-legal"> + Apache Qpid, Messaging built on AMQP; Copyright © 2015 + The Apache Software Foundation; Licensed under + the <a href="http://www.apache.org/licenses/LICENSE-2.0";>Apache + License, Version 2.0</a>; Apache Qpid, Qpid, Qpid Proton, + Proton, Apache, the Apache feather logo, and the Apache Qpid + project logo are trademarks of The Apache Software + Foundation; All other marks mentioned may be trademarks or + registered trademarks of their respective owners + </p> + </div> + </div> + </div> + </body> +</html>
Modified: qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Message-Compression.html URL: http://svn.apache.org/viewvc/qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Message-Compression.html?rev=1722689&r1=1722688&r2=1722689&view=diff ============================================================================== --- qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Message-Compression.html (original) +++ qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Message-Compression.html Sat Jan 2 23:59:48 2016 @@ -114,7 +114,7 @@ https://github.com/apache/qpid-proton/bl <ul id="-path-navigation"><li><a href="/index.html">Home</a></li><li><a href="/releases/index.html">Releases</a></li><li><a href="/releases/qpid-java-trunk/index.html">Qpid Java Trunk</a></li><li><a href="/releases/qpid-java-trunk/java-broker/book/index.html">AMQP Messaging Broker (Java)</a></li><li>9.8. Message Compression</li></ul> <div id="-middle-content"> - <div class="docbook"><div class="navheader"><table summary="Navigation header" width="100%"><tr><th align="center" colspan="3">9.8. Message Compression</th></tr><tr><td align="left" width="20%"><a accesskey="p" href="Java-Broker-Runtime-Background-Recovery.html">Prev</a> </td><th align="center" width="60%">Chapter 9. Runtime</th><td align="right" width="20%"> <a accesskey="n" href="Java-Broker-Runtime-Connection-Limit.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title"><a id="Java-Broker-Runtime-Message-Compression"></a>9.8. Message Compression</h2></div></div></div><p>The Java Broker supports<a class="footnote" href="#ftn.idm140601090063008" id="idm140601090063008"><sup class="footnote">[12]</sup></a> message compression. This feature works in co-operation with Qpid + <div class="docbook"><div class="navheader"><table summary="Navigation header" width="100%"><tr><th align="center" colspan="3">9.8. Message Compression</th></tr><tr><td align="left" width="20%"><a accesskey="p" href="Java-Broker-Runtime-Background-Recovery.html">Prev</a> </td><th align="center" width="60%">Chapter 9. Runtime</th><td align="right" width="20%"> <a accesskey="n" href="Java-Broker-Runtime-Connection-Limit.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Java-Broker-Runtime-Message-Compression"></a>9.8. Message Compression</h2></div></div></div><p>The Java Broker supports<a class="footnote" href="#ftn.d0e5714" id="d0e5714"><sup class="footnote">[13]</sup></a> message compression. This feature works in co-operation with Qpid Clients implementing the same feature.</p><p>Once the feature is enabled (using Broker context variable <span class="emphasis"><em>broker.messageCompressionEnabled</em></span>), the Broker will advertise support for the message compression feature to the client at connection time. This allows clients to opt to turn @@ -122,13 +122,13 @@ https://github.com/apache/qpid-proton/bl do not, it will internally, on-the-fly, decompress compressed messages when sending to clients without support and conversely, compress uncomressed messages when sending to clients who do.</p><p>The Broker has a threshold below which it will not consider compressing a message, this is controlled by Broker content variable - (<code class="literal">connection.messageCompressionThresholdSize</code>) and expresses a size in bytes.</p><p>This feature <span class="emphasis"><em>may</em></span> have a beneficial effect on performance by:</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>Reducing the number of bytes transmitted over the wire, both between Client and Broker, and + (<code class="literal">connection.messageCompressionThresholdSize</code>) and expresses a size in bytes.</p><p>This feature <span class="emphasis"><em>may</em></span> have a beneficial effect on performance by:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>Reducing the number of bytes transmitted over the wire, both between Client and Broker, and in the HA case, Broker to Broker, for replication purposes.</p></li><li class="listitem"><p>Reducing storage space when data is at rest within the Broker, both on disk and in memory.</p></li></ul></div><p>Of course, compression and decompression is computationally expensive. Turning on the feature may have a negative impact on CPU utilization on Broker and/or Client. Also for small messages payloads, message compression may increase the message size. It is recommended to test the feature - with representative data.</p><div class="footnotes"><br /><hr align="left" width="100" /><div class="footnote" id="ftn.idm140601090063008"><p><a class="para" href="#idm140601090063008"><sup class="para">[12] </sup></a>Message compression is not yet supported for the 1.0 - protocol.</p></div></div></div><div class="navfooter"><hr /><table summary="Navigation footer" width="100%"><tr><td align="left" width="40%"><a accesskey="p" href="Java-Broker-Runtime-Background-Recovery.html">Prev</a> </td><td align="center" width="20%"><a accesskey="u" href="Java-Broker-Runtime.html">Up</a></td><td align="right" width="40%"> <a accesskey="n" href="Java-Broker-Runtime-Connection-Limit.html">Next</a></td></tr><tr><td align="left" valign="top" width="40%">9.7. Background Recovery </td><td align="center" width="20%"><a accesskey="h" href="index.html">Home</a></td><td align="right" valign="top" width="40%"> 9.9. Connection Limits</td></tr></table></div></div> + with representative data.</p><div class="footnotes"><br /><hr style="width:100; text-align:left;margin-left: 0" /><div class="footnote" id="ftn.d0e5714"><p><a class="para" href="#d0e5714"><sup class="para">[13] </sup></a>Message compression is not yet supported for the 1.0 + protocol.</p></div></div></div><div class="navfooter"><hr /><table summary="Navigation footer" width="100%"><tr><td align="left" width="40%"><a accesskey="p" href="Java-Broker-Runtime-Background-Recovery.html">Prev</a> </td><td align="center" width="20%"><a accesskey="u" href="Java-Broker-Runtime.html">Up</a></td><td align="right" width="40%"> <a accesskey="n" href="Java-Broker-Runtime-Connection-Limit.html">Next</a></td></tr><tr><td align="left" valign="top" width="40%">9.7. Background Recovery </td><td align="center" width="20%"><a accesskey="h" href="AMQP-Messaging-Broker-Java-Book.html">Home</a></td><td align="right" valign="top" width="40%"> 9.9. Connection Limits</td></tr></table></div></div> <hr/> Modified: qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Producer-Transaction-Timeout.html URL: http://svn.apache.org/viewvc/qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Producer-Transaction-Timeout.html?rev=1722689&r1=1722688&r2=1722689&view=diff ============================================================================== --- qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Producer-Transaction-Timeout.html (original) +++ qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime-Producer-Transaction-Timeout.html Sat Jan 2 23:59:48 2016 @@ -114,9 +114,9 @@ https://github.com/apache/qpid-proton/bl <ul id="-path-navigation"><li><a href="/index.html">Home</a></li><li><a href="/releases/index.html">Releases</a></li><li><a href="/releases/qpid-java-trunk/index.html">Qpid Java Trunk</a></li><li><a href="/releases/qpid-java-trunk/java-broker/book/index.html">AMQP Messaging Broker (Java)</a></li><li>9.3. Producer Transaction Timeout</li></ul> <div id="-middle-content"> - <div class="docbook"><div class="navheader"><table summary="Navigation header" width="100%"><tr><th align="center" colspan="3">9.3. Producer Transaction Timeout</th></tr><tr><td align="left" width="20%"><a accesskey="p" href="Java-Broker-Runtime-Disk-Space-Management.html">Prev</a> </td><th align="center" width="60%">Chapter 9. Runtime</th><td align="right" width="20%"> <a accesskey="n" href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title"><a id="Java-Broker-Runtime-Producer-Transaction-Timeout"></a>9.3. Producer Transaction Timeout</h2></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Producer-Transaction-Timeout-GeneralInformation"></a>9.3.1. General Information</h3></div></div></div><p> The transaction timeout mechanism is used to control broker resour ces when clients + <div class="docbook"><div class="navheader"><table summary="Navigation header" width="100%"><tr><th align="center" colspan="3">9.3. Producer Transaction Timeout</th></tr><tr><td align="left" width="20%"><a accesskey="p" href="Java-Broker-Runtime-Disk-Space-Management.html">Prev</a> </td><th align="center" width="60%">Chapter 9. Runtime</th><td align="right" width="20%"> <a accesskey="n" href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Java-Broker-Runtime-Producer-Transaction-Timeout"></a>9.3. Producer Transaction Timeout</h2></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Producer-Transaction-Timeout-GeneralInformation"></a>9.3.1. General Information</h3></div></div></div><p> The transaction timeout mechanism is used to c ontrol broker resources when clients producing messages using transactional sessions hang or otherwise become unresponsive, or simply - begin a transaction and keep using it without ever calling <a class="ulink" href="http://docs.oracle.com/javaee/6/api/javax/jms/Session.html#commit"; target="_top">Session#commit()</a>.</p><p>Users can choose to configure an idleWarn or openWarn threshold, after which the identified + begin a transaction and keep using it without ever calling <a class="link" href="http://docs.oracle.com/javaee/6/api/javax/jms/Session.html#commit"; target="_top">Session#commit()</a>.</p><p>Users can choose to configure an idleWarn or openWarn threshold, after which the identified transaction should be logged as a WARN level alert as well as (more importantly) an idleClose or openClose threshold after which the transaction and the connection it applies to will be closed.</p><p>This feature is particularly useful in environments where the owner of the broker does not @@ -125,7 +125,7 @@ https://github.com/apache/qpid-proton/bl broker holds an open transaction on the persistent store. This can have undesirable consequences if the store does not time out or close long-running transactions, such as with BDB. This can can result in a rapid increase in disk usage size, bounded only by available space, due to growth of - the transaction log. </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Producer-Transaction-Timeout-Scope"></a>9.3.3. Scope</h3></div></div></div><p>Note that only <a class="ulink" href="http://docs.oracle.com/javaee/6/api/javax/jms/MessageProducer.html"; target="_top">MessageProducer</a> clients will be affected by a transaction timeout, since store + the transaction log. </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Producer-Transaction-Timeout-Scope"></a>9.3.3. Scope</h3></div></div></div><p>Note that only <a class="link" href="http://docs.oracle.com/javaee/6/api/javax/jms/MessageProducer.html"; target="_top">MessageProducer</a> clients will be affected by a transaction timeout, since store transaction lifespan on a consumer only spans the execution of the call to Session#commit() and there is no scope for a long-lived transaction to arise.</p><p>It is also important to note that the transaction timeout mechanism is purely a JMS transaction timeout, and unrelated to any other timeouts in the Qpid client library and will have @@ -133,7 +133,7 @@ https://github.com/apache/qpid-proton/bl gives a brief overview of what the Transaction Timeout feature can do.</p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Producer-Transaction-Timeout-Effect-Broker-Side"></a>9.3.4.1. Broker Logging and Connection Close</h4></div></div></div><p>When the openWarn or idleWarn specified threshold is exceeded, the broker will log a WARN level alert with details of the connection and channel on which the threshold has been exceeded, along with the age of the transaction.</p><p>When the openClose or idleClose specified threshold value is exceeded, the broker will - throw an exception back to the client connection via the <a class="ulink" href="http://docs.oracle.com/javaee/6/api/javax/jms/ExceptionListener.html"; target="_top">ExceptionListener</a>, log the + throw an exception back to the client connection via the <a class="link" href="http://docs.oracle.com/javaee/6/api/javax/jms/ExceptionListener.html"; target="_top">ExceptionListener</a>, log the action and then close the connection.</p><p>The example broker log output shown below is where the idleWarn threshold specified is lower than the idleClose threshold and the broker therefore logs the idle transaction 3 times before the close threshold is triggered and the connection closed out.</p><pre class="screen">CHN-1008 : Idle Transaction : 13,116 ms @@ -147,7 +147,7 @@ CHN-1007 : Open Transaction : 13,406 ms CHN-1007 : Open Transaction : 14,406 ms CHN-1003 : Close </pre></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Producer-Transaction-Timeout-Effect-Client-Side"></a>9.3.4.2. Client Side Effect</h4></div></div></div><p>After a Close threshold has been exceeded, the trigger client will receive this exception - on its <a class="ulink" href="http://docs.oracle.com/javaee/6/api/javax/jms/ExceptionListener.html"; target="_top">exception + on its <a class="link" href="http://docs.oracle.com/javaee/6/api/javax/jms/ExceptionListener.html"; target="_top">exception listener</a>, prior to being disconnected:</p><code class="computeroutput">org.apache.qpid.AMQConnectionClosedException: Error: Idle transaction timed out [error code 506: resource error]</code><p>Any later attempt to use the connection will result in this exception being thrown:</p><pre class="screen">Producer: Caught an Exception: javax.jms.IllegalStateException: Object org.apache.qpid.client.AMQSession_0_8@129b0e1 has been closed javax.jms.IllegalStateException: Object org.apache.qpid.client.AMQSession_0_8@129b0e1 has been closed @@ -170,7 +170,7 @@ CHN-1003 : Close Session and Connection. Clients that use the JMS API directly should be aware that sessions managing both consumers and producers, or multiple producers, will be affected by a single producer hanging or leaving a transaction idle or open, and closed, and must take appropriate - action to handle that scenario.</p></div></div></div><div class="navfooter"><hr /><table summary="Navigation footer" width="100%"><tr><td align="left" width="40%"><a accesskey="p" href="Java-Broker-Runtime-Disk-Space-Management.html">Prev</a> </td><td align="center" width="20%"><a accesskey="u" href="Java-Broker-Runtime.html">Up</a></td><td align="right" width="40%"> <a accesskey="n" href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html">Next</a></td></tr><tr><td align="left" valign="top" width="40%">9.2. Disk Space Management </td><td align="center" width="20%"><a accesskey="h" href="index.html">Home</a></td><td align="right" valign="top" width="40%"> 9.4. Handing Undeliverable Messages</td></tr></table></div></div> + action to handle that scenario.</p></div></div></div><div class="navfooter"><hr /><table summary="Navigation footer" width="100%"><tr><td align="left" width="40%"><a accesskey="p" href="Java-Broker-Runtime-Disk-Space-Management.html">Prev</a> </td><td align="center" width="20%"><a accesskey="u" href="Java-Broker-Runtime.html">Up</a></td><td align="right" width="40%"> <a accesskey="n" href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html">Next</a></td></tr><tr><td align="left" valign="top" width="40%">9.2. Disk Space Management </td><td align="center" width="20%"><a accesskey="h" href="AMQP-Messaging-Broker-Java-Book.html">Home</a></td><td align="right" valign="top" width="40%"> 9.4. Handing Undeliverable Messages</td></tr></table></div></div> <hr/> Modified: qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime.html URL: http://svn.apache.org/viewvc/qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime.html?rev=1722689&r1=1722688&r2=1722689&view=diff ============================================================================== --- qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime.html (original) +++ qpid/site/docs/releases/qpid-java-trunk/java-broker/book/Java-Broker-Runtime.html Sat Jan 2 23:59:48 2016 @@ -114,26 +114,101 @@ https://github.com/apache/qpid-proton/bl <ul id="-path-navigation"><li><a href="/index.html">Home</a></li><li><a href="/releases/index.html">Releases</a></li><li><a href="/releases/qpid-java-trunk/index.html">Qpid Java Trunk</a></li><li><a href="/releases/qpid-java-trunk/java-broker/book/index.html">AMQP Messaging Broker (Java)</a></li><li>Chapter 9. Runtime</li></ul> <div id="-middle-content"> - <div class="docbook"><div class="navheader"><table summary="Navigation header" width="100%"><tr><th align="center" colspan="3">Chapter 9. Runtime</th></tr><tr><td align="left" width="20%"><a accesskey="p" href="Java-Broker-Security-Configuration-Encryption.html">Prev</a> </td><th align="center" width="60%"> </th><td align="right" width="20%"> <a accesskey="n" href="Java-Broker-Runtime-Disk-Space-Management.html">Next</a></td></tr></table><hr /></div><div class="chapter"><div class="titlepage"><div><div><h1 class="title"><a id="Java-Broker-Runtime"></a>Chapter 9. Runtime</h1></div></div></div><div class="toc"><p><strong>Table of Contents</strong></p><dl class="toc"><dt><span class="section"><a href="Java-Broker-Runtime.html#Java-Broker-Runtime-Log-Files">9.1. Log Files</a></span></dt><dd><dl><dt><span class="section"><a href="Java-Broker-Runtime.html#Java-Broker-Runtime-Log-Files-Enable-Debug">9.1.1. Enabling Debug</a></span></dt></dl></dd ><dt><span class="section"><a >href="Java-Broker-Runtime-Disk-Space-Management.html">9.2. Disk Space >Management</a></span></dt><dd><dl><dt><span class="section"><a >href="Java-Broker-Runtime-Disk-Space-Management.html#Qpid-Producer-Flow-Control">9.2.1. > Producer Flow Control</a></span></dt></dl></dd><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html">9.3. Producer >Transaction Timeout</a></span></dt><dd><dl><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-GeneralInformation">9.3.1. > General Information</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-Purpose">9.3.2. > Purpose</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-Scope">9.3.3. > Scope</a></span></dt><d t><span class="section"><a href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-Effect">9.3.4. Effect</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-Configuration">9.3.5. Configuration</a></span></dt></dl></dd><dt><span class="section"><a href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html">9.4. Handing Undeliverable Messages</a></span></dt><dd><dl><dt><span class="section"><a href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html#Java-Broker-Runtime-Handling-Undeliverable-Messages-Introduction">9.4.1. Introduction</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html#Java-Broker-Runtime-Handling-Undeliverable-Messages-Maximum-Delivery-Count">9.4.2. Maximum Delivery Count</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Handling-Und eliverable-Messages.html#Java-Broker-Runtime-Handling-Undeliverable-Messages-Dead-Letter-Queues">9.4.3. Dead Letter Queues (DLQ)</a></span></dt></dl></dd><dt><span class="section"><a href="Java-Broker-Close-Connection-When-No-Route.html">9.5. Closing client connections on unroutable mandatory messages</a></span></dt><dd><dl><dt><span class="section"><a href="Java-Broker-Close-Connection-When-No-Route.html#Java-Broker-Close-Connection-When-No-Route-Summary">9.5.1. Summary</a></span></dt><dt><span class="section"><a href="Java-Broker-Close-Connection-When-No-Route.html#Java-Broker-Close-Connection-When-No-Route-Configuration">9.5.2. Configuring <span class="emphasis"><em>closeWhenNoRoute</em></span></a></span></dt></dl></dd><dt><span class="section"><a href="Java-Broker-Runtime-Flow-To-Disk.html">9.6. Flow to Disk</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Background-Recovery.html">9.7. Background Recovery</a></span></dt><dt><span class="section"><a href="Ja va-Broker-Runtime-Message-Compression.html">9.8. Message Compression</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Connection-Limit.html">9.9. Connection Limits</a></span></dt></dl></div><div class="section"><div class="titlepage"><div><div><h2 class="title"><a id="Java-Broker-Runtime-Log-Files"></a>9.1. Log Files</h2></div></div></div><p> The Broker uses the <a class="ulink" href="http://logging.apache.org/log4j/1.2/"; target="_top">Apache Log4J</a> - Logging Framework for all logging activity. </p><p> In the Broker's shipped configuration, all logging is directed to log file <code class="literal"><a class="link" href="Java-Broker-Appendix-Environment-Variables.html#Java-Broker-Appendix-Environment-Variables-Qpid-Work">${QPID_WORK}</a>/log/qpid.log</code>. The log file is not rotated and will be overwritten - when the Broker restarts. Logging levels are configured in such a way that the log will comprise - of:</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>Opertional Log Events. These report key events in the lifecycle of objects (Broker - start-up, Queue creation, Queue deletion etc) within the Broker. See <a class="xref" href="Java-Broker-Appendix-Operation-Logging.html" title="Appendix C. Operational Logging">Appendix C, <em>Operational Logging</em></a> for details of the formation of these - messages.</p></li><li class="listitem"><p>Queue Alert Events. These report when the queue thresholds have been breached. See <a class="xref" href="Java-Broker-Appendix-Queue-Alerts.html" title="Appendix D. Queue Alerts">Appendix D, <em>Queue Alerts</em></a> for details.</p></li><li class="listitem"><p>Any Error and Warning conditions.</p></li></ul></div><p>Logging can be reconfigured either by changing the logging configuration file <code class="literal"><a class="link" href="Java-Broker-Appendix-Environment-Variables.html#Java-Broker-Appendix-Environment-Variables-Qpid-Home">${QPID_HOME}</a>/etc/log4j.xml</code> or at runtime using the Logging Management MBean, - see <a class="xref" href="Java-Broker-Management-Channel-JMX.html#Java-Broker-Management-Channel-JMX-MBeans" title="6.4.5. The MBeans">Section 6.4.5, “The MBeans”</a> for - details.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Log-Files-Enable-Debug"></a>9.1.1. Enabling Debug</h3></div></div></div><p>It can be helpful to enable debug within the Broker in order to understand a problem more - clearly. If this is required, debug can be enabled at runtime (without restarting the Broker) - using the Logging Management MBean. The change can also be made by changing the log configuration - file and restarting the Broker. Whichever mechanism is chosen, change the appender associated - with <code class="literal">org.apache.qpid</code> from <code class="literal">WARN</code> to - <code class="literal">DEBUG</code>.</p><div class="example"><a id="idm140601089133056"></a><p class="title"><strong>Example 9.1. Changing the log4j.xml configuration file to enable debug</strong></p><div class="example-contents"><pre class="screen"> -... -<logger additivity="true" name="org.apache.qpid"> - <level value="debug"/> <!-- change the level value from warn to debug --> -</logger> -...</pre></div></div><br class="example-break" /><div class="important" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Important</h3><p>Running a production system at <code class="literal">DEBUG</code> level can have performance - implications by slowing the Broker down. It can also generate large log files. Take care to - revert the logging level back to <code class="literal">WARN</code> after the analysis is performed.</p></div></div></div></div><div class="navfooter"><hr /><table summary="Navigation footer" width="100%"><tr><td align="left" width="40%"><a accesskey="p" href="Java-Broker-Security-Configuration-Encryption.html">Prev</a> </td><td align="center" width="20%"> </td><td align="right" width="40%"> <a accesskey="n" href="Java-Broker-Runtime-Disk-Space-Management.html">Next</a></td></tr><tr><td align="left" valign="top" width="40%">8.4. Configuration Encryption </td><td align="center" width="20%"><a accesskey="h" href="index.html">Home</a></td><td align="right" valign="top" width="40%"> 9.2. Disk Space Management</td></tr></table></div></div> + <div class="docbook"><div class="navheader"><table summary="Navigation header" width="100%"><tr><th align="center" colspan="3">Chapter 9. Runtime</th></tr><tr><td align="left" width="20%"><a accesskey="p" href="Java-Broker-Security-Configuration-Encryption.html">Prev</a> </td><th align="center" width="60%"> </th><td align="right" width="20%"> <a accesskey="n" href="Java-Broker-Runtime-Disk-Space-Management.html">Next</a></td></tr></table><hr /></div><div class="chapter"><div class="titlepage"><div><div><h1 class="title"><a id="Java-Broker-Runtime"></a>Chapter 9. Runtime</h1></div></div></div><div class="toc"><p><strong>Table of Contents</strong></p><dl class="toc"><dt><span class="section"><a href="Java-Broker-Runtime.html#Java-Broker-Runtime-Logging">9.1. Logging</a></span></dt><dd><dl><dt><span class="section"><a href="Java-Broker-Runtime.html#Java-Broker-Runtime-Logging-Concepts">9.1.1. Concepts</a></span></dt><dt><span class="section" ><a >href="Java-Broker-Runtime.html#Java-Broker-Runtime-Logging-Default-Configuration">9.1.2. > Default Configuration</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime.html#Java-Broker-Runtime-Logging-Loggers">9.1.3. >Loggers</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime.html#Java-Broker-Runtime-Logging-InclusionRules">9.1.4. > Inclusion Rules</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime.html#Java-Broker-Runtime-Logging-Management">9.1.5. >Logging Management</a></span></dt></dl></dd><dt><span class="section"><a >href="Java-Broker-Runtime-Disk-Space-Management.html">9.2. Disk Space >Management</a></span></dt><dd><dl><dt><span class="section"><a >href="Java-Broker-Runtime-Disk-Space-Management.html#Qpid-Producer-Flow-Control">9.2.1. > Producer Flow Control</a></span></dt></dl></dd><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html">9.3. Producer >Transaction Timeout</a></span></dt><dd><dl><dt ><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-GeneralInformation">9.3.1. > General Information</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-Purpose">9.3.2. > Purpose</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-Scope">9.3.3. > Scope</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-Effect">9.3.4. > Effect</a></span></dt><dt><span class="section"><a >href="Java-Broker-Runtime-Producer-Transaction-Timeout.html#Java-Broker-Runtime-Producer-Transaction-Timeout-Configuration">9.3.5. > Configuration</a></span></dt></dl></dd><dt><span class="section"><a >href="Java-Broker-Runtime-Handling-Undeliverabl e-Messages.html">9.4. Handing Undeliverable Messages</a></span></dt><dd><dl><dt><span class="section"><a href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html#Java-Broker-Runtime-Handling-Undeliverable-Messages-Introduction">9.4.1. Introduction</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html#Java-Broker-Runtime-Handling-Undeliverable-Messages-Maximum-Delivery-Count">9.4.2. Maximum Delivery Count</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Handling-Undeliverable-Messages.html#Java-Broker-Runtime-Handling-Undeliverable-Messages-Dead-Letter-Queues">9.4.3. Dead Letter Queues (DLQ)</a></span></dt></dl></dd><dt><span class="section"><a href="Java-Broker-Runtime-Close-Connection-When-No-Route.html">9.5. Closing client connections on unroutable mandatory messages</a></span></dt><dd><dl><dt><span class="section"><a href="Java-Broker-Runtime-Close-Connection-When-No-Route.html#Java-Broker-Runtime-Clo se-Connection-When-No-Route-Summary">9.5.1. Summary</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Close-Connection-When-No-Route.html#Java-Broker-Runtime-Close-Connection-When-No-Route-Configuration">9.5.2. Configuring <span class="emphasis"><em>closeWhenNoRoute</em></span></a></span></dt></dl></dd><dt><span class="section"><a href="Java-Broker-Runtime-Flow-To-Disk.html">9.6. Flow to Disk</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Background-Recovery.html">9.7. Background Recovery</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Message-Compression.html">9.8. Message Compression</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Connection-Limit.html">9.9. Connection Limits</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Memory.html">9.10. Memory</a></span></dt><dd><dl><dt><span class="section"><a href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Introducti on">9.10.1. Introduction</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Types">9.10.2. Types of Memory</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Usage">9.10.3. Memory Usage in the Broker</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory">9.10.4. Low Memory Conditions</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Defaults">9.10.5. Defaults</a></span></dt><dt><span class="section"><a href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Tuning">9.10.6. Memory Tuning the Broker</a></span></dt></dl></dd></dl></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Java-Broker-Runtime-Logging"></a>9.1. Logging</h2></div></div></div><p>This section describes the flexib le logging capabilities of the Java Broker.</p><p> + </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>The Broker is capable of sending logging events to a variety of destinations including + plain files, remote syslog daemons, and an in-memory buffer (viewable from Management). + The system is also open for extension meaning it is possible to produce a plugin to log to + a bespoke destination.</p></li><li class="listitem"><p>Logging can be dynamically configured at runtime. For instance, it is possible to + temporarily increase the logging verbosity of the system whilst a problem is investigated + and then revert later, all without the need to restart the Broker.</p></li><li class="listitem"><p>Virtualhosts can be configured to generate their own separate log, and the Broker is + capable of generating a log either inclusive or exclusive of virtualhost events.</p></li><li class="listitem"><p>Logs are accessible over Management, removing the need for those operating the Broker + to have shell level access.</p></li></ul></div><p> + </p><p>In the remainder of this section you will first find a description of the concepts used in + the logging subsystem. Next, you find a description of the default configuration. The section + then concludes with a in-depth description of the loggers themselves and how they may be + configured.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Logging-Concepts"></a>9.1.1. Concepts</h3></div></div></div><p>The logging subsystem uses two concepts:</p><p> + </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>A <span class="emphasis"><em>Logger</em></span> is responsible for production of a log. The Broker + ships a variety of loggers, for instance, a file logger, which is capable of writing a + log file to the file system, a Syslog Logger capable of writing to a remote syslog + daemon and console logger capable of writing to stdout or stderr.</p><p>Loggers are attached at two points within the Broker Model; the Broker itself and + the virtualhosts. Loggers attached at the Broker can capture log events for the system + as a whole, or can exclude events related to virtualhosts.</p><p>Loggers attached to a virtualhost capture log events relating to that virtualhost + only.</p><p>The Broker and virtualhosts can have zero or more Loggers. If no loggers are + configured, no logging is generated at all.</p></li><li class="listitem"><p><span class="emphasis"><em>Inclusion rules</em></span> govern what appears within a log. Inclusion + rules are associated with Loggers. This means it is possible for different Loggers to + have different contents.</p><p>A Logger with no inclusion rules will produce an empty log.</p></li></ul></div><p> + </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Logging-Default-Configuration"></a>9.1.2. Default Configuration</h3></div></div></div><p>The default configuration is designed to be suitable for use without change in small + production environments. It has the following characteristics:</p><p> + </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>The Broker generates a single log file <code class="literal">qpid.log</code>. This logfile is + rolled automatically when the file reaches 100MB. A maximum history of one file is + retained. On restart the the log will be appended to.</p><p>The log contains: </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>All operational logging events. See <a class="xref" href="Java-Broker-Appendix-Operation-Logging.html" title="Appendix C. Operational Logging">Appendix C, <em>Operational Logging</em></a>.</p></li><li class="listitem"><p>Log events from Qpid itself deemed informational or + higher.</p></li><li class="listitem"><p>Log events from Qpid's dependencies (such as Derby or Jetty) that are + deemed warning or higher.</p></li></ul></div><p> + </p><p>The default location for the log file is + <code class="literal">${QPID_WORK}/log/qpid.log</code>.</p></li><li class="listitem"><p>The Broker also caches the last 4096 log events in a memory cache. By default, the + memory logger logs the same things the file logger does.</p></li></ul></div><p> + </p><p>The configuration can be customised at runtime using Management. This makes it possible to + investigate unusual conditions <span class="emphasis"><em>without</em></span> the need to restart the Broker. + For instance, you may alter the logging level so that a verbose log is produced whilst an + investigation is in progress and revert the setting later, all without the need to restart the + Broker.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Logging-Loggers"></a>9.1.3. Loggers</h3></div></div></div><p>Loggers are responsible for the writing of a log. The log includes log events that match a + Logger's inclusion rules.</p><p>Loggers are associated with either the Broker or a virtualhost. Virtualhost loggers write + only log events related to that virtualhost. Broker Loggers write log events from the Broker + as a whole. Optionally a Broker Logger can be configured to exclude log events coming from + virtualhosts. These abilities can be usefully exploited together in managed service scenarios + to produce separate logs for separate user groups.</p><p>Loggers can be added or removed at runtime, without restarting the Broker. However changes + to a Logger's configuration such as filenames and rolling options don't take effect until the + next restart. Changes to a Logger's inclusion rules take effect immediately.</p><p>All loggers allow the log event layout to be customised. Loggers understand <a class="link" href="http://logback.qos.ch/manual/layouts.html#ClassicPatternLayout"; target="_top"> Logback Classic + Pattern Layouts</a>. </p><p>The following sections describes each Logger implementation in detail.</p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Logging-Loggers-FileLogger"></a>9.1.3.1. FileLogger</h4></div></div></div><p>A <span class="emphasis"><em>FileLogger</em></span> - writes a log file to the filesystem. The name and + location of the log file, the rolling configuration, and compression options can be + configured.</p><p>The <span class="emphasis"><em>roll daily</em></span> option, if enabled, will cause the log file will be + rolled at midnight local time. The rolled over file will have a suffix in the form + <code class="literal">yyyy-mm-dd</code>. In roll daily mode, <span class="emphasis"><em>maximum number of rolled + files</em></span> controls the maximum number of <span class="emphasis"><em>days</em></span> to be retained. + Older files will be deleted.</p><p>The <span class="emphasis"><em>maximum file size</em></span> option limits the size of any one log file. + Once a log file reaches the given size, it will be rolled. The rolled over file will have + the numeric suffix, beginning at <code class="literal">1</code>. If the log file rolls again, first + the existing file with the suffix <code class="literal">.1</code> is renamed to <code class="literal">.2</code> + and so forth. If roll daily is not in use, <span class="emphasis"><em>maximum number of rolled + files</em></span> governs the number of rolled <span class="emphasis"><em>files</em></span> that will be + retained.</p><p><span class="emphasis"><em>Roll on restart</em></span> governs whether the log file is rolled when the + Broker is restarted. If not ticked, the Broker will append to the existing log file until it + needs to be rolled.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Logging-Loggers-ConsoleLogger"></a>9.1.3.2. ConsoleLogger</h4></div></div></div><p><span class="emphasis"><em>ConsoleLogger</em></span> - writes a log file standard out or standard + error.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Logging-Loggers-SyslogLogger"></a>9.1.3.3. SyslogLogger</h4></div></div></div><p><span class="emphasis"><em>SyslogLogger</em></span> - writes a log file to a syslog daemon using the + <code class="literal">USER</code> facility. The hostname and port number of the syslog daemon can be + configured.</p><p>Log entries can be prefixed with a string. This string defaults to include the word + <code class="literal">Qpid</code> and the name of the Broker or virtualhost. This serves to + distinguish the logging generated by this Qpid instance, from other Qpid instances, or other + applications using the <code class="literal">USER</code>.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Logging-Loggers-MemoryLogger"></a>9.1.3.4. MemoryLogger</h4></div></div></div><p><span class="emphasis"><em>MemoryLogger</em></span> - writes a log file to a circular in-memory buffer. By + default the circular buffer holds the last 4096 log events. The contents of the buffer can + be viewed via Management. See <a class="xref" href="Java-Broker-Runtime.html#Java-Broker-Runtime-Logging-Management-MemoryLogger" title="Figure 9.3. Viewing a memory logger">Figure 9.3, “Viewing a memory logger”</a></p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Logging-InclusionRules"></a>9.1.4. Inclusion Rules</h3></div></div></div><p>A <span class="emphasis"><em>Logger</em></span> has one or more <span class="emphasis"><em>inclusion rules</em></span>. These + govern what appears in the log. A Logger with no inclusion rules will log nothing.</p><p>Inclusion rules can be added, removed or changed at runtime. Changes take place + immediately.</p><p> + </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>The <span class="emphasis"><em>Name And Level</em></span> inclusion rule accepts log events that match + a given <span class="emphasis"><em>log event source name</em></span> and have a level that equals or + exceeds the specified value.</p><p>The log event source name refers to the fully qualified class name from which the + event originates. These names permit a trailing wild card <code class="literal">.*</code>. For + instance a source name of <code class="literal">org.apache.qpid.*</code> will match all events + from classes in the package <code class="literal">org.apache.qpid</code> and any sub packages + beneath.</p><p>The <span class="emphasis"><em>Level</em></span> governs the level of the events that will be included + in the log. It may take one of the following values: ERROR, WARN, INFO, DEBUG, TRACE + where ERROR is considered the highest and TRACE the lowest. In addition, there are two + special values: OFF and ALL, the former excludes all log events whereas the latter will + include everything. When considering whether a logging event should be included in the + log, the logging event must have a level that matches that of the inclusion rule or be + higher, otherwise the log event will not appear in the log.</p></li></ul></div><p> + </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Logging-Management"></a>9.1.5. Logging Management</h3></div></div></div><p>The logging subsystem can be completely managed from the Web Management Console or the + REST API. You can: </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>Add, remove, or change the configuration of Loggers.</p></li><li class="listitem"><p>Add, remove, or change the Inclusion Rules.</p></li><li class="listitem"><p>For FileLoggers, download the log file and rolled log files associated with + the Logger.</p></li><li class="listitem"><p>For MemoryLoggers, view the last <code class="literal">n</code> log + events</p></li></ul></div><p> + </p><p> The figure that follows shows a FileLogger. The attributes area shows the configuration + of the Logger. The inclusion rule table shows the rules that are associated with the Logger. + The area towards the bottom of the tab allows the log files to be downloaded to the browser. + </p><div class="figure"><a id="Java-Broker-Runtime-Logging-Management-FileLogger"></a><p class="title"><strong>Figure 9.1. Viewing a file logger</strong></p><div class="figure-contents"><div class="mediaobject"><table border="0" style="cellpadding: 0; cellspacing: 0;" summary="manufactured viewport for HTML img" width="900"><tr><td><img alt="Viewing a file logger" src="images/Management-Web-Logging-FileLogger.png" width="900" /></td></tr></table></div></div></div><p><br class="figure-break" /> + </p><p> The figure below shows the editing of the level of an inclusion rule. </p><div class="figure"><a id="Java-Broker-Runtime-Logging-Management-InclusionRule"></a><p class="title"><strong>Figure 9.2. Editing an inclusion rule</strong></p><div class="figure-contents"><div class="mediaobject"><table border="0" style="cellpadding: 0; cellspacing: 0;" summary="manufactured viewport for HTML img" width="900"><tr><td><img alt="Editing an inclusion rule" src="images/Management-Web-Logging-InclusionRule.png" width="900" /></td></tr></table></div></div></div><p><br class="figure-break" /> + </p><p> The figure below shows a Memory Logger. Note that the Memory Logger provides access to + the cached message via the viewer towards the bottom on the tab. </p><div class="figure"><a id="Java-Broker-Runtime-Logging-Management-MemoryLogger"></a><p class="title"><strong>Figure 9.3. Viewing a memory logger</strong></p><div class="figure-contents"><div class="mediaobject"><table border="0" style="cellpadding: 0; cellspacing: 0;" summary="manufactured viewport for HTML img" width="900"><tr><td><img alt="Viewing a memory logger" src="images/Management-Web-Logging-MemoryLogger.png" width="900" /></td></tr></table></div></div></div><p><br class="figure-break" /> + </p></div></div></div><div class="navfooter"><hr /><table summary="Navigation footer" width="100%"><tr><td align="left" width="40%"><a accesskey="p" href="Java-Broker-Security-Configuration-Encryption.html">Prev</a> </td><td align="center" width="20%"> </td><td align="right" width="40%"> <a accesskey="n" href="Java-Broker-Runtime-Disk-Space-Management.html">Next</a></td></tr><tr><td align="left" valign="top" width="40%">8.4. Configuration Encryption </td><td align="center" width="20%"><a accesskey="h" href="AMQP-Messaging-Broker-Java-Book.html">Home</a></td><td align="right" valign="top" width="40%"> 9.2. Disk Space Management</td></tr></table></div></div> <hr/> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
