http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/how_multisite_distribution_works.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/how_multisite_distribution_works.html.md.erb b/geode-docs/developing/events/how_multisite_distribution_works.html.md.erb deleted file mode 100644 index 7678e7a..0000000 --- a/geode-docs/developing/events/how_multisite_distribution_works.html.md.erb +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: Multi-Site (WAN) Event Distribution ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Geode distributes a subset of cache events between distributed systems, with a minimum impact on each system's performance. Events are distributed only for regions that you configure to use a gateway sender for distribution. - -## <a id="how_multisite_distribution_works__section_A16562611E094C88B12BC149D5EEEEBA" class="no-quick-link"></a>Queuing Events for Distribution - -In regions that are configured with one or more gateway senders (`gateway-sender-ids` attribute), events are automatically added to a gateway sender queue for distribution to other sites. Events that are placed in a gateway sender queue are distributed asynchronously to remote sites. For serial gateway queues, the ordering of events sent between sites can be preserved using the `order-policy` attribute. - -If a queue becomes too full, it is overflowed to disk to keep the member from running out of memory. You can optionally configure the queue to be persisted to disk (with the `enable-persistence` `gateway-sender` attribute). With persistence, if the member that manages the queue goes down, the member picks up where it left off after it restarts. - -## Operation Distribution from a Gateway Sender - -The multi-site installation is designed for minimal impact on distributed system performance, so only the farthest-reaching entry operations are distributed between sites. - -These operations are distributed: - -- entry create -- entry put -- entry distributed destroy, providing the operation is not an expiration action - -These operations are not distributed: - -- get -- invalidate -- local destroy -- expiration actions of any kind -- region operations - -## <a id="how_multisite_distribution_works__section_EE819CBF41274312BD5C3EA4A660475C" class="no-quick-link"></a>How a Gateway Sender Processes Its Queue - -Each primary gateway sender contains a processor thread that reads messages from the queue, batches them, and distributes the batches to a gateway receiver in a remote site. To process the queue, a gateway sender thread takes the following actions: - -1. Reads messages from the queue -2. Creates a batch of the messages -3. Synchronously distributes the batch to the other site and waits for a reply -4. Removes the batch from the queue after the other site has successfully replied - -Because the batch is not removed from the queue until after the other site has replied, the message cannot get lost. On the other hand, in this mode a message could be processed more than once. If a site goes offline in the middle of processing a batch of messages, then that same batch will be sent again once the site is back online. - -You can configure the batch size for messages as well as the batch time interval settings. A gateway sender processes a batch of messages from the queue when either the batch size or the time interval is reached. In an active network, it is likely that the batch size will be reached before the time interval. In an idle network, the time interval will most likely be reached before the batch size. This may result in some network latency that corresponds to the time interval. - -## <a id="how_multisite_distribution_works__section_EF240AB26CF242F99689222E9E1D2512" class="no-quick-link"></a>How a Gateway Sender Handles Batch Processing Failure - -Exceptions can occur at different points during batch processing: - -- The gateway receiver could fail with acknowledgment. If processing fails while the gateway receiver is processing a batch, the receiver replies with a failure acknowledgment that contains the exception, including the identity of the message that failed, and the ID of the last message that it successfully processed. The gateway sender then removes the successfully processed messages and the failed message from the queue and logs an exception with the failed message information. The sender then continues processing the messages remaining in the queue. -- The gateway receiver can fail without acknowledgment. If the gateway receiver does not acknowledge a sent batch, the gateway sender does not know which messages were successfully processed. In this case the gateway sender re-sends the entire batch. -- No gateway receivers may be available for processing. If a batch processing exception occurs because there are no remote gateway receivers available, then the batch remains in the queue. The gateway sender waits for a time, and then attempts to re-send the batch. The time period between attempts is five seconds. The existing server monitor continuously attempts to connect to the gateway receiver, so that a connection can be made and queue processing can continue. Messages build up in the queue and possibly overflow to disk while waiting for the connection. -
http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/implementing_cache_event_handlers.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/implementing_cache_event_handlers.html.md.erb b/geode-docs/developing/events/implementing_cache_event_handlers.html.md.erb deleted file mode 100644 index 0e639e7..0000000 --- a/geode-docs/developing/events/implementing_cache_event_handlers.html.md.erb +++ /dev/null @@ -1,153 +0,0 @@ ---- -title: Implementing Cache Event Handlers ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Depending on your installation and configuration, cache events can come from local operations, peers, servers, and remote sites. Event handlers register their interest in one or more events and are notified when the events occur. - -<a id="implementing_cache_event_handlers__section_9286E8C6B3C54089888E1680B4F43692"></a> -For each type of handler, Geode provides a convenience class with empty stubs for the interface callback methods. - -**Note:** -Write-behind cache listeners are created by extending the `AsyncEventListener` interface, and they are configured with an `AsyncEventQueue` that you assign to one or more regions. - -**Procedure** - -1. Decide which events your application needs to handle. For each region, decide which events you want to handle. For the cache, decide whether to handle transaction events. -2. For each event, decide which handlers to use. The `*Listener` and `*Adapter` classes in `org.apache.geode.cache.util` show the options. -3. Program each event handler: - - 1. Extend the handler's adapter class. - 2. If you want to declare the handler in the `cache.xml`, implement the `org.apache.geode.cache.Declarable` interface as well. - 3. Implement the handler's callback methods as needed by your application. - - **Note:** - Improperly programmed event handlers can block your distributed system. Cache events are synchronous. To modify your cache or perform distributed operations based on events, avoid blocking your system by following the guidelines in [How to Safely Modify the Cache from an Event Handler Callback](writing_callbacks_that_modify_the_cache.html#writing_callbacks_that_modify_the_cache). - - Example: - - ``` pre - package myPackage; - import org.apache.geode.cache.Declarable; - import org.apache.geode.cache.EntryEvent; - import org.apache.geode.cache.util.CacheListenerAdapter; - import java.util.Properties; - - public class MyCacheListener extends CacheListenerAdapter implements Declarable { - /** Processes an afterCreate event. - * @param event The afterCreate EntryEvent received - */ - public void afterCreate(EntryEvent event) { - String eKey = event.getKey(); - String eVal = event.getNewValue(); - ... do work with event info - } - ... process other event types - } - - ``` - -4. Install the event handlers, either through the API or the `cache.xml`. - - XML Region Event Handler Installation: - - ``` pre - <region name="trades"> - <region-attributes ... > - <!-- Cache listener --> - <cache-listener> - <class-name>myPackage.MyCacheListener</class-name> - <cache-listener> - </region-attributes> - </region> - ``` - - Java Region Event Handler Installation: - - ``` pre - tradesRegion = cache.createRegionFactory(RegionShortcut.PARTITION) - .addCacheListener(new MyCacheListener()) - .create("trades"); - ``` - - XML Transaction Writer and Listener Installation: - - ``` pre - <cache search-timeout="60"> - <cache-transaction-manager> - <transaction-listener> - <class-name>com.company.data.MyTransactionListener</class-name> - <parameter name="URL"> - <string>jdbc:cloudscape:rmi:MyData</string> - </parameter> - </transaction-listener> - <transaction-listener> - . . . - </transaction-listener> - <transaction-writer> - <class-name>com.company.data.MyTransactionWriter</class-name> - <parameter name="URL"> - <string>jdbc:cloudscape:rmi:MyData</string> - </parameter> - <parameter - ... - </parameter> - </transaction-writer> - </cache-transaction-manager> - . . . - </cache> - ``` - -The event handlers are initialized automatically during region creation when you start the member. - -## <a id="implementing_cache_event_handlers__section_C62E9535C43B4BC5A7AA7B8B4125D1EB" class="no-quick-link"></a>Installing Multiple Listeners on a Region - -XML: - -``` pre -<region name="exampleRegion"> - <region-attributes> - . . . - <cache-listener> - <class-name>myCacheListener1</class-name> - </cache-listener> - <cache-listener> - <class-name>myCacheListener2</class-name> - </cache-listener> - <cache-listener> - <class-name>myCacheListener3</class-name> - </cache-listener> - </region-attributes> -</region> -``` - -API: - -``` pre -CacheListener listener1 = new myCacheListener1(); -CacheListener listener2 = new myCacheListener2(); -CacheListener listener3 = new myCacheListener3(); - -Region nr = cache.createRegionFactory() - .initCacheListeners(new CacheListener[] - {listener1, listener2, listener3}) - .setScope(Scope.DISTRIBUTED_NO_ACK) - .create(name); - -``` http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/implementing_durable_client_server_messaging.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/implementing_durable_client_server_messaging.html.md.erb b/geode-docs/developing/events/implementing_durable_client_server_messaging.html.md.erb deleted file mode 100644 index 150a79d..0000000 --- a/geode-docs/developing/events/implementing_durable_client_server_messaging.html.md.erb +++ /dev/null @@ -1,199 +0,0 @@ ---- -title: Implementing Durable Client/Server Messaging ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="implementing_durable_client_server_messaging__section_7A0D0B7D1F2748C7BA7479E0FD5C6BFA"></a> -Use durable messaging for subscriptions that you need maintained for your clients even when your clients are down or disconnected. You can configure any of your event subscriptions as durable. Events for durable queries and subscriptions are saved in queue when the client is disconnected and played back when the client reconnects. Other queries and subscriptions are removed from the queue. - -Use durable messaging for client/server installations that use event subscriptions. - -These are the high-level tasks described in this topic: - -1. Configure your client as durable -2. Decide which subscriptions should be durable and configure accordingly -3. Program your client to manage durable messaging for disconnect, reconnect, and event handling - -## <a id="implementing_durable_client_server_messaging__section_643EB5FA6F09463C80646786394F1E02" class="no-quick-link"></a>Configure the Client as Durable - -Use one of the following methods: - -- `gemfire.properties` file: - - ``` pre - durable-client-id=31 - durable-client-timeout=200 - ``` - -- Java: - - ``` pre - Properties props = new Properties(); - props.setProperty("durable-client-id", "31"); - props.setProperty("durable-client-timeout", "" + 200); - CacheFactory cf = new CacheFactory(props); - ``` - -<a id="implementing_durable_client_server_messaging__section_B8E01FE4B5A347EB96085DA3194F6AE8"></a> - -The `durable-client-id` indicates that the client is durable and gives the server an identifier to correlate the client to its durable messages. For a non-durable client, this id is an empty string. The ID can be any number that is unique among the clients attached to servers in the same distributed system. - -The `durable-client-timeout` tells the server how long to wait for client reconnect. When this timeout is reached, the server stops storing to the client's message queue and discards any stored messages. The default is 300 seconds. This is a tuning parameter. If you change it, take into account the normal activity of your application, the average size of your messages, and the level of risk you can handle, both in lost messages and in the servers' capacity to store enqueued messages. Assuming that no messages are being removed from the queue, how long can the server run before the queue reaches the maximum capacity? How many durable clients can the server handle? To assist with tuning, use the Geode message queue statistics for durable clients through the disconnect and reconnect cycles. - -## <a id="implementing_durable_client_server_messaging__section_BB5DCCE0582E4FE8B62DE473512FC704" class="no-quick-link"></a>Configure Durable Subscriptions and Continuous Queries - -The register interest and query creation methods all have an optional boolean parameter for indicating durability. By default all are non-durable. - -``` pre -// Durable registration -// Define keySpecification, interestResultPolicy, durability -exampleRegion.registerInterest(keySpecification, interestResultPolicySpecification, true); - -// Durable CQ -// Define cqName, queryString, cqAttributes, durability -CqQuery myCq = queryService.newCq(cqName, queryString, cqAttributes, true); -``` - -Save only critical messages while the client is disconnected by only indicating durability for critical subscriptions and CQs. When the client is connected to its servers, it receives messages for all keys and queries reqistered. When the client is disconnected, non-durable interest registrations and CQs are discontinued but all messages already in the queue for them remain there. - -**Note:** -For a single durable client ID, you must maintain the same durability of your registrations and queries between client runs. - -## <a id="implementing_durable_client_server_messaging__section_0FBD23CC79784E588135FE93306EC0A4" class="no-quick-link"></a>Program the Client to Manage Durable Messaging - -Program your durable client to be durable-messaging aware when it disconnects, reconnects, and handles events from the server. - -1. Disconnect with a request to keep your queues active by using `Pool.close` or `ClientCache.close` with the boolean `keepalive` parameter. - - ``` pre - clientCache.close(true); - ``` - - To be retained during client down time, durable continuous queries (CQs) must be executing at the time of disconnect. - -2. Program your durable client's reconnection to: - - 1. If desired, detect whether the previously registered subscription queue is available upon durable client reconnection and the count of pending events in the queue. Based on the results, you can then decide whether to receive the remaining events or close the cache if the number is too large. - - For example, for a client with only the default pool created: - - ``` pre - int pendingEvents = cache.getDefaultPool().getPendingEventCount(); - - if (pendingEvents == -2) { // client connected for the first time ⦠// continue - } - else if (pendingEvents == -1) { // client reconnected but after the timeout period - ⦠// handle possible data loss - } - else { // pendingEvents >= 0 - ⦠// decide to invoke readyForEvents() or ClientCache::close(false)/pool.destroy() - } - ``` - - For a client with multiple pools: - - ``` pre - int pendingEvents = 0; - - int pendingEvents1 = PoolManager.find(âpool1â).getPendingEventCount(); - - pendingEvents += (pendingEvents1 > 0) ? pendingEvents1 : 0; - - int pendingEvents2 = PoolManager.find(âpool2â).getPendingEventCount(); - - pendingEvents += (pendingEvents2 > 0) ? pendingEvents2 : 0; - - // process individual pool counts separately. - ``` - - The `getPendingEventCount` API can return the following possible values: - - A value representing a count of events pending at the server. Note that this count is an approximate value based on the time the durable client pool connected or reconnected to the server. Any number of invocations will return the same value. - - A zero value if there are no events pending at server for this client pool - - A negative value indicates that no queue is available at the server for the client pool. - - -1 indicates that the client pool has reconnected to the server after its durable-client-timeout period has elapsed. The pool's subscription queue has been removed possibly causing data loss. - - A value of -2 indicates that this client pool has connected to server for the first time. - - 2. Connect, initialize the client cache, regions, and any cache listeners, and create and execute any durable continuous queries. - 3. Run all interest registration calls. - - **Note:** - Registering interest with `InterestResultPolicy.KEYS_VALUES` initializes the client cache with the *current* values of specified keys. If concurrency checking is enabled for the region, any earlier (older) region events that are replayed to the client are ignored and are not sent to configured listeners. If your client must process all replayed events for a region, register with `InterestResultPolicy.KEYS` or `InterestResultPolicy.NONE` when reconnecting. Or, disable concurrency checking for the region in the client cache. See [Consistency for Region Updates](../distributed_regions/region_entry_versions.html#topic_CF2798D3E12647F182C2CEC4A46E2045). - - 4. Call `ClientCache.readyForEvents` so the server will replay stored events. If the ready message is sent earlier, the client may lose events. - - ``` pre - ClientCache clientCache = ClientCacheFactory.create(); - // Here, create regions, listeners that are not defined in the cache.xml . . . - // Here, run all register interest calls before doing anything else - clientCache.readyForEvents(); - ``` - -3. When you program your durable client `CacheListener`: - 1. Implement the callback methods to behave properly when stored events are replayed. The durable clientâs `CacheListener` must be able to handle having events played after the fact. Generally listeners receive events very close to when they happen, but the durable client may receive events that occurred minutes before and are not relevant to current cache state. - 2. Consider whether to use the `CacheListener` callback method, `afterRegionLive`, which is provided specifically for the end of durable event replay. You can use it to perform application-specific operations before resuming normal event handling. If you do not wish to use this callback, and your listener is an instance of `CacheListener` (instead of a `CacheListenerAdapter`) implement `afterRegionLive` as an empty method. - -## Initial Operation - -The initial startup of a durable client is similar to the startup of any other client, except that it specifically calls the `ClientCache.readyForEvents` method when all regions and listeners on the client are ready to process messages from the server. - -## <a id="implementing_durable_client_server_messaging__section_9B9A9EE8C7FF47948C8108A0F7F4E32E" class="no-quick-link"></a>Disconnection - -While the client and servers are disconnected, their operation varies depending on the circumstances. - -- **Normal disconnect**. When a client closes its connection, the servers stop sending messages to the client and release its connection. If the client requests it, the servers maintain the queues and durable interest list information until the client reconnects or times out. The non-durable interest lists are discarded. The servers continue to queue up incoming messages for entries on the durable interest list. All messages that were in the queue when the client disconnected remain in the queue. If the client requests not to have its subscriptions maintained, or if there are no durable subscriptions, the servers unregister the client and do the same cleanup as for a non-durable client. -- **Abnormal disconnect**. If the client crashes or loses its connections to all servers, the servers automatically maintain its message queue and durable subscriptions until it reconnects or times out. -- **Client disconnected but operational**. If the client operates while it is disconnected, it gets what data it can from the local client cache. Since updates are not allowed, the data can become stale. An `UnconnectedException` occurs if an update is attempted. -- **Client stays disconnected past timeout period**. The servers track how long to keep a durable subscription queue alive based on the `durable-client-timeout` setting. If the client remains disconnected longer than the timeout, the servers unregister the client and do the same cleanup that is performed for a non-durable client. The servers also log an alert. When a timed-out client reconnects, the servers treat it as a new client making its initial connection. - -## <a id="implementing_durable_client_server_messaging__section_E3C42A6FDC884FC38ECC121955C06BDC" class="no-quick-link"></a>Reconnection - -During initialization, the client cache is not blocked from doing operations, so you might be receiving old stored events from the server at the same time that your client cache is being updated by much more current events. These are the things that can act on the cache concurrently: - -- Results returned by the server in response to the clientâs interest registrations. -- Client cache operations by the application. -- Callbacks triggered by replaying old events from the queue - -Geode handles the conflicts between the application and interest registrations so they do not create cache update conflicts. But you must program your event handlers so they don't conflict with current operations. This is true for all event handlers, but it is especially important for those used in durable clients. Your handlers may receive events well after the fact and you must ensure your programming takes that into account. - -This figure shows the three concurrent procedures during the initialization process. The application begins operations immediately on the client (step 1), while the clientâs cache ready message (also step 1) triggers a series of queue operations on the servers (starting with step 2 on the primary server). At the same time, the client registers interest (step 2 on the client) and receives a response from the server. Message B2 applies to an entry in Region A, so the cache listener handles B2âs event. Because B2 comes before the marker, the client does not apply the update to the cache. - -<img src="../../images/ClientServerAdvancedTopics-6.gif" alt="Durable client reconnection. " id="implementing_durable_client_server_messaging__image_068A59FA019E46FA9DE0BC7FA60AAADD" class="image" /> - -## <a id="implementing_durable_client_server_messaging__section_C848DF6D649F4DCAA2B895F5439BAA97" class="no-quick-link"></a>Durable Event Replay - -When a durable client reconnects before the timeout period, the servers replay the events that were stored while the client was gone and then resume normal event messaging to the client. To avoid overwriting current entries with old data, the stored events are not applied to the client cache. Stored events are distinguished from new normal events by a marker that is sent to the client once all old events are replayed. - -1. All servers with a queue for this client place a marker in their queue when the client reconnects. -2. The primary server sends the queued messages to the client up to the marker. -3. The client receives the messages but does not apply the usual automatic updates to its cache. If cache listeners are installed, they handle the events. -4. The client receives the marker message indicating that all past events have been played back. -5. The server sends the current list of live regions. -6. For every `CacheListener` in each live region on the client, the marker event triggers the `afterRegionLive` callback. After the callback, the client begins normal processing of events from the server and applies the updates to its cache. - -Even when a new client starts up for the first time, the client cache ready markers are inserted in the queues. If messages start coming into the new queues before the servers insert the marker, those messages are considered as having happened while the client was disconnected, and their events are replayed the same as in the reconnect case. - -## <a id="implementing_durable_client_server_messaging__section_E519D541E2844292ABD2E0BDF5FB5798" class="no-quick-link"></a>Application Operations During Interest Registration - -Application operations take precedence over interest registration responses. The client can perform operations while it is receiving its interest registration responses. When adding register interest responses to the client cache, the following rules are applied: - -- If the entry already exists in the cache with a valid value, it is not updated. -- If the entry is invalid, and the register interest response is valid, the valid value is put into the cache. -- If an entry is marked destroyed, it is not updated. Destroyed entries are removed from the system after the register interest response is completed. -- If the interest response does not contain any results, because all of those keys are absent from the serverâs cache, the clientâs cache can start out empty. If the queue contains old messages related to those keys, the events are still replayed in the clientâs cache. - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/implementing_write_behind_event_handler.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/implementing_write_behind_event_handler.html.md.erb b/geode-docs/developing/events/implementing_write_behind_event_handler.html.md.erb deleted file mode 100644 index 54cf174..0000000 --- a/geode-docs/developing/events/implementing_write_behind_event_handler.html.md.erb +++ /dev/null @@ -1,245 +0,0 @@ ---- -title: Implementing an AsyncEventListener for Write-Behind Cache Event Handling ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -An `AsyncEventListener` asynchronously processes batches of events after they have been applied to a region. You can use an `AsyncEventListener` implementation as a write-behind cache event handler to synchronize region updates with a database. - -## <a id="implementing_write_behind_cache_event_handling__section_35B3ADC77E1147468A568E49C8C308E1" class="no-quick-link"></a>How an AsyncEventListener Works - -An `AsyncEventListener` instance is serviced by its own dedicated thread in which a callback method is invoked. Events that update a region are placed in an internal `AsyncEventQueue`, and one or more threads dispatch batches of events at a time to the listener implementation. - -You can configure an `AsyncEventQueue` to be either serial or parallel. A serial queue is deployed to one Geode member, and it delivers all of a region's events, in order of occurrence, to a configured `AsyncEventListener` implementation. A parallel queue is deployed to multiple Geode members, and each instance of the queue delivers region events, possibly simultaneously, to a local `AsyncEventListener` implementation. - -While a parallel queue provides the best throughput for writing events, it provides less control for ordering those events. With a parallel queue, you cannot preserve event ordering for a region as a whole because multiple Geode servers queue and deliver the region's events at the same time. However, the ordering of events for a given partition (or for a given queue of a distributed region) can be preserved. - -For both serial and parallel queues, you can control the maximum amount of memory that each queue uses, as well as the batch size and frequency for processing batches in the queue. You can also configure queues to persist to disk (instead of simply overflowing to disk) so that write-behind caching can pick up where it left off when a member shuts down and is later restarted. - -Optionally, a queue can use multiple threads to dispatch queued events. When you configure multiple threads for a serial queue, the logical queue that is hosted on a Geode member is divided into multiple physical queues, each with a dedicated dispatcher thread. You can then configure whether the threads dispatch queued events by key, by thread, or in the same order in which events were added to the queue. When you configure multiple threads for a parallel queue, each queue hosted on a Geode member is processed by dispatcher threads; the total number of queues created depends on the number of members that host the region. - -A `GatewayEventFilter` can be placed on the `AsyncEventQueue` to control whether a particular event is sent to a selected `AsyncEventListener`. For example, events associated with sensitive data could be detected and not queued. For more detail, see the Javadocs for `GatewayEventFilter`. - -A `GatewayEventSubstitutionFilter` can specify whether the event is transmitted in its entirety or in an altered representation. For example, to reduce the size of the data being serialized, it might be a more efficient to represent a full object by only its key. For more detail, see the Javadocs for `GatewayEventSubstitutionFilter`. - -## Operation Distribution from an AsyncEventQueue - -An `AsyncEventQueue` distributes these operations: - -- Entry create -- Entry put -- Entry distributed destroy, providing the operation is not an expiration action -- Expiration destroy, if the `forward-expiration-destroy` attribute is set to `true`. By default, this attribute is `false`, but you can set it to `true` using `cache.xml` or `gfsh`. To set this attribute in the Java API, use `AsyncEventQueueFactory.setForwardExpirationDestroy()`. See the javadocs for details. - -These operations are not distributed: - -- Get -- Invalidate -- Local destroy -- Region operations -- Expiration actions -- Expiration destroy, if the `forward-expiration-destroy` attribute is set to `false`. The default value is `false`. - -## <a id="implementing_write_behind_cache_event_handling__section_6FDBAFCB9C194EB0AF0822A509F2F9F2" class="no-quick-link"></a>Guidelines for Using an AsyncEventListener - -Review the following guidelines before using an AsyncEventListener: - -- If you use an `AsyncEventListener` to implement a write-behind cache listener, your code should check for the possibility that an existing database connection may have been closed due to an earlier exception. For example, check for `Connection.isClosed()` in a catch block and re-create the connection as needed before performing further operations. -- Use a serial `AsyncEventQueue` if you need to preserve the order of region events within a thread when delivering events to your listener implementation. Use parallel queues when the order of events within a thread is not important, and when you require maximum throughput for processing events. In both cases, serial and parallel, the order of operations on a given key is preserved within the scope of the thread. -- You must install the `AsyncEventListener` implementation on a Geode member that hosts the region whose events you want to process. -- If you configure a parallel `AsyncEventQueue`, deploy the queue on each Geode member that hosts the region. -- You can install a listener on more than one member to provide high availability and guarantee delivery for events, in the event that a member with the active `AsyncEventListener` shuts down. At any given time only one member has an active listener for dispatching events. The listeners on other members remain on standby for redundancy. For best performance and most efficient use of memory, install only one standby listener (redundancy of at most one). -- Install no more than one standby listener (redundancy of at most one) for performance and memory reasons. -- To preserve pending events through member shutdowns, configure Geode to persist the internal queue of the `AsyncEventListener` to an available disk store. By default, any pending events that reside in the internal queue of an `AsyncEventListener` are lost if the active listener's member shuts down. -- To ensure high availability and reliable delivery of events, configure the event queue to be both persistent and redundant. - -## <a id="implementing_write_behind_cache_event_handling__section_FB3EB382E37945D9895E09B47A64D6B9" class="no-quick-link"></a>Implementing an AsyncEventListener - -To receive region events for processing, you create a class that implements the `AsyncEventListener` interface. The `processEvents` method in your listener receives a list of queued `AsyncEvent` objects in each batch. - -Each `AsyncEvent` object contains information about a region event, such as the name of the region where the event occurred, the type of region operation, and the affected key and value. - -The basic framework for implementing a write-behind event handler involves iterating through the batch of events and writing each event to a database. For example: - -``` pre -class MyAsyncEventListener implements AsyncEventListener { - - public boolean processEvents(List<AsyncEvent> events) { - - // Process each AsyncEvent - - for(AsyncEvent event: events) { - - // Write the event to a database - - } - } -} -``` - -## <a id="implementing_write_behind_cache_event_handling__section_AB80262CFB6D4867B52A5D6D880A5294" class="no-quick-link"></a>Processing AsyncEvents - -Use the [AsyncEventListener.processEvents](/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventListener.html) method to process AsyncEvents. This method is called asynchronously when events are queued to be processed. The size of the list reflects the number of batch events where batch size is defined in the AsyncEventQueueFactory. The `processEvents` method returns a boolean; true if the AsyncEvents are processed correctly, and false if any events fail processing. As long as `processEvents` returns false, Geode continues to re-try processing the events. - -You can use the `getDeserializedValue` method to obtain cache values for entries that have been updated or created. Since the `getDeserializedValue` method will return a null value for destroyed entries, you should use the `getKey` method to obtain references to cache objects that have been destroyed. Here's an example of processing AsyncEvents: - -``` pre -public boolean processEvents(@SuppressWarnings("rawtypes") List<AsyncEvent> list) - { - logger.log (Level.INFO, String.format("Size of List<GatewayEvent> = %s", list.size())); - List<JdbcBatch> newEntries = new ArrayList<JdbcBatch>(); - - List<JdbcBatch> updatedEntries = new ArrayList<JdbcBatch>(); - List<String> destroyedEntries = new ArrayList<String>(); - int possibleDuplicates = 0; - - for (@SuppressWarnings("rawtypes") AsyncEvent ge: list) - { - - if (ge.getPossibleDuplicate()) - possibleDuplicates++; - - if ( ge.getOperation().equals(Operation.UPDATE)) - { - updatedEntries.add((JdbcBatch) ge.getDeserializedValue()); - } - else if ( ge.getOperation().equals(Operation.CREATE)) - { - newEntries.add((JdbcBatch) ge.getDeserializedValue()); - } - else if ( ge.getOperation().equals(Operation.DESTROY)) - { - destroyedEntries.add(ge.getKey().toString()); - } - - } -``` - -## <a id="implementing_write_behind_cache_event_handling__section_9286E8C6B3C54089888E1680B4F43692" class="no-quick-link"></a>Configuring an AsyncEventListener - -To configure a write-behind cache listener, you first configure an asynchronous queue to dispatch the region events, and then create the queue with your listener implementation. You then assign the queue to a region in order to process that region's events. - -**Procedure** - -1. Configure a unique `AsyncEventQueue` with the name of your listener implementation. You can optionally configure the queue for parallel operation, persistence, batch size, and maximum memory size. See [WAN Configuration](../../reference/topics/elements_ref.html#topic_7B1CABCAD056499AA57AF3CFDBF8ABE3) for more information. - - **gfsh configuration** - - ``` pre - gfsh>create async-event-queue --id=sampleQueue --persistent --disk-store=exampleStore --listener=com.myCompany.MyAsyncEventListener --listener-param=url#jdbc:db2:SAMPLE,username#gfeadmin,password#admin1 - ``` - - The parameters for this command uses the following syntax: - - ``` pre - create async-event-queue --id=value --listener=value [--group=value] [--batch-size=value] - [--persistent(=value)?] [--disk-store=value] [--max-queue-memory=value] [--listener-param=value(,value)*] - ``` - - For more information, see [create async-event-queue](../../tools_modules/gfsh/command-pages/create.html#topic_ryz_pb1_dk). - - **cache.xml Configuration** - - ``` pre - <cache> - <async-event-queue id="sampleQueue" persistent="true" - disk-store-name="exampleStore" parallel="false"> - <async-event-listener> - <class-name>MyAsyncEventListener</class-name> - <parameter name="url"> - <string>jdbc:db2:SAMPLE</string> - </parameter> - <parameter name="username"> - <string>gfeadmin</string> - </parameter> - <parameter name="password"> - <string>admin1</string> - </parameter> - </async-event-listener> - </async-event-queue> - ... - </cache> - ``` - - **Java Configuration** - - ``` pre - Cache cache = new CacheFactory().create(); - AsyncEventQueueFactory factory = cache.createAsyncEventQueueFactory(); - factory.setPersistent(true); - factory.setDiskStoreName("exampleStore"); - factory.setParallel(false); - AsyncEventListener listener = new MyAsyncEventListener(); - AsyncEventQueue asyncQueue = factory.create("sampleQueue", listener); - ``` - -2. If you are using a parallel `AsyncEventQueue`, the gfsh example above requires no alteration, as gfsh applies to all members. If using cache.xml or the Java API to configure your `AsyncEventQueue`, repeat the above configuration in each Geode member that will host the region. Use the same ID and configuration settings for each queue configuration. - **Note:** - You can ensure other members use the sample configuration by using the cluster configuration service available in gfsh. See [Overview of the Cluster Configuration Service](../../configuring/cluster_config/gfsh_persist.html). - -3. On each Geode member that hosts the `AsyncEventQueue`, assign the queue to each region that you want to use with the `AsyncEventListener` implementation. - - **gfsh Configuration** - - ``` pre - gfsh>create region --name=Customer --async-event-queue-id=sampleQueue - ``` - - Note that you can specify multiple queues on the command line in a comma-delimited list. - - **cache.xml Configuration** - - ``` pre - <cache> - <region name="Customer"> - <region-attributes async-event-queue-ids="sampleQueue"> - </region-attributes> - </region> - ... - </cache> - ``` - - **Java Configuration** - - ``` pre - RegionFactory rf1 = cache.createRegionFactory(); - rf1.addAsyncEventQueue(sampleQueue); - Region customer = rf1.create("Customer"); - - // Assign the queue to multiple regions as needed - RegionFactory rf2 = cache.createRegionFactory(); - rf2.addAsyncEventQueue(sampleQueue); - Region order = rf2.create("Order"); - ``` - - Using the Java API, you can also add and remove queues to regions that have already been created: - - ``` pre - AttributesMutator mutator = order.getAttributesMutator(); - mutator.addAsyncEventQueueId("sampleQueue"); - ``` - - See the [Geode API documentation](/releases/latest/javadoc/org/apache/geode/cache/AttributesMutator.html) for more information. - -4. Optionally configure persistence and conflation for the queue. - **Note:** - You must configure your AsyncEventQueue to be persistent if you are using persistent data regions. Using a non-persistent queue with a persistent region is not supported. - -5. Optionally configure multiple dispatcher threads and the ordering policy for the queue using the instructions in [Configuring Dispatcher Threads and Order Policy for Event Distribution](configuring_gateway_concurrency_levels.html). - -The `AsyncEventListener` receives events from every region configured with the associated `AsyncEventQueue`. http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/limit_server_subscription_queue_size.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/limit_server_subscription_queue_size.html.md.erb b/geode-docs/developing/events/limit_server_subscription_queue_size.html.md.erb deleted file mode 100644 index a3d6682..0000000 --- a/geode-docs/developing/events/limit_server_subscription_queue_size.html.md.erb +++ /dev/null @@ -1,74 +0,0 @@ ---- -title: Limit the Server's Subscription Queue Memory Use ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="limit_server_subscription_queue_size__section_1791DFB89502480EB57F81D16AC0EBAC"></a> -These are options for limiting the amount of server memory the subscription queues consume. - -- Optional: Conflate the subscription queue messages. -- Optional: Increase the frequency of queue synchronization. This only applies to configurations where server redundancy is used for high availability. Increase the clientâs pool configuration, `subscription-ack-interval`. The client periodically sends a batch acknowledgment of messages to the server, rather than acknowledging each message individually. A lower setting speeds message delivery and generally reduces traffic between the server and client. A higher setting helps contain server queue size. Example: - - ``` pre - <!-- Set subscription ack interval to 3 seconds --> - <cache> - <pool ... subscription-enabled="true" - subscription-ack-interval="3000"> - ... - </pool> - ``` - - You might want to lower the interval if you have a very busy system and want to reduce the space required in the servers for the subscription queues. More frequent acknowledgments means fewer events held in the server queues awaiting acknowledgment. - -- Optional: Limit Queue Size. Cap the server queue size using overflow or blocking. These options help avoid out of memory errors on the server in the case of slow clients. A slow client slows the rate that the server can send messages, causing messages to back up in the queue, possibly leading to out of memory on the server. You can use one or the other of these options, but not both: - - Optional: Overflow to Disk. Configure subscription queue overflow by setting the serverâs `client-subscription` properties. With overflow, the most recently used (MRU) events are written out to disk, keeping the oldest events, the ones that are next in line to be sent to the client, available in memory. Example: - - ``` pre - <!-- Set overflow after 10K messages are enqueued --> - <cache-server port="40404"> - <client-subscription - eviction-policy="entry" - capacity="10000" - disk-store-name="svrOverflow"/> - </cache-server> - ``` - - - Optional: Block While Queue Full. Set the serverâs `maximum-message-count` to the maximum number of event messages allowed in any single subscription queue before incoming messages are blocked. You can only limit the message count, not the size allocated for messages. Examples: - - XML: - - ``` pre - <!-- Set the maximum message count to 50000 entries --> - <cache-server port="41414" maximum-message-count="50000" /> - ``` - - API: - - ``` pre - Cache cache = ...; - CacheServer cacheServer = cache.addCacheServer(); - cacheServer.setPort(41414); - cacheServer.setMaximumMessageCount(50000); - cacheServer.start(); - ``` - - **Note:** - With this setting, one slow client can slow the server and all of its other clients because this blocks the threads that write to the queues. All operations that add messages to the queue block until the queue size drops to an acceptable level. If the regions feeding these queues are partitioned or have `distributed-ack` or `global` scope, operations on them remain blocked until their event messages can be added to the queue. If you are using this option and see stalling on your server region operations, your queue capacity might be too low for your application behavior. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/list_of_event_handlers_and_events.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/list_of_event_handlers_and_events.html.md.erb b/geode-docs/developing/events/list_of_event_handlers_and_events.html.md.erb deleted file mode 100644 index cda18ee..0000000 --- a/geode-docs/developing/events/list_of_event_handlers_and_events.html.md.erb +++ /dev/null @@ -1,181 +0,0 @@ ---- -title: List of Event Handlers and Events ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Geode provides many types of events and event handlers to help you manage your different data and application needs. - -## <a id="event_handlers_and_events__section_E7B7502F673B43E794884D0F6BF537CF" class="no-quick-link"></a>Event Handlers - -Use either cache handlers or membership handlers in any single application. Do not use both. The event handlers in this table are cache handlers unless otherwise noted. - -<table> -<colgroup> -<col width="25%" /> -<col width="25%" /> -<col width="50%" /> -</colgroup> -<thead> -<tr class="header"> -<th>Handler API</th> -<th>Events received</th> -<th>Description</th> -</tr> -</thead> -<tbody> -<tr> -<td><code class="ph codeph">AsyncEventListener</code></td> -<td><code class="ph codeph">AsyncEvent</code></td> -<td><p>Tracks changes in a region for write-behind processing. Extends th <code class="ph codeph">CacheCallback</code> interface. You install a write-back cache listener to an <code class="ph codeph">AsyncEventQueue</code> instance. You can then add the <code class="ph codeph">AsyncEventQueue</code> instance to one or more regions for write-behind processing. See [Implementing an AsyncEventListener for Write-Behind Cache Event Handling](implementing_write_behind_event_handler.html#implementing_write_behind_cache_event_handling).</p></td> -</tr> -<tr> -<td><code class="ph codeph">CacheCallback</code></td> -<td> </td> -<td>Superinterface of all cache event listeners. Functions only to clean up resources that the callback allocated.</td> -</tr> -<tr> -<td><code class="ph codeph">CacheListener</code></td> -<td><code class="ph codeph">RegionEvent</code>, <code class="ph codeph">EntryEvent</code></td> -<td>Tracks changes to region and its data entries. Responds synchronously. Extends <code class="ph codeph">CacheCallback</code> interface. Installed in region. Receives only local cache events. Install one in every member where you want the events handled by this listener. In a partitioned region, the cache listener only fires in the primary data store. Listeners on secondaries are not fired.</td> -</tr> -<tr> -<td><code class="ph codeph">CacheWriter</code></td> -<td><code class="ph codeph">RegionEvent</code>, <code class="ph codeph">EntryEvent</code></td> -<td>Receives events for <em>pending</em> changes to the region and its data entries in this member or one of its peers. Has the ability to abort the operations in question. Extends <code class="ph codeph">CacheCallback</code> interface. Installed in region. Receives events from anywhere in the distributed region, so you can install one in one member for the entire distributed region. Receives events only in primary data store in partitioned regions, so install one in every data store.</td> -</tr> -<tr> -<td><code class="ph codeph">ClientMembershipListener</code> -<p>(org.apache.geode.management .membership.ClientMembershipListener)</p></td> -<td><code class="ph codeph">ClientMembershipEvent</code></td> -<td>One of the interfaces that replaces the deprecated Admin APIs. You can use the ClientMembershipListener to receive membership events only about clients. This listener's callback methods are invoked when this process detects connection changes to clients. Callback methods include <code class="ph codeph">memberCrashed</code>, <code class="ph codeph">memberJoined</code>, <code class="ph codeph">memberLeft</code> (graceful exit).</td> -</tr> -<tr> -<td><code class="ph codeph">CqListener</code></td> -<td><code class="ph codeph">CqEvent</code></td> -<td>Receives events from the server cache that satisfy a client-specified query. Extends <code class="ph codeph">CacheCallback</code> interface. Installed in the client inside a <code class="ph codeph">CqQuery</code>.</td> -</tr> -<tr> -<td><code class="ph codeph">GatewayConflictResolver</code></td> -<td><code class="ph codeph">TimestampedEntryEvent</code></td> -<td>Decides whether to apply a potentially conflicting event to a region that is distributed over a WAN configuration. This event handler is called only when the distributed system ID of an update event is different from the ID that last updated the region entry.</td> -</tr> -<tr> -<td><code class="ph codeph">MembershipListener</code> -<p>(org.apache.geode.management .membership.MembershipListener)</p></td> -<td><code class="ph codeph">MembershipEvent</code></td> -<td>Use this interface to receive membership events only about peers. This listener's callback methods are invoked when peer members join or leave the Geode distributed system. Callback methods include <code class="ph codeph">memberCrashed</code>, <code class="ph codeph">memberJoined</code>, and <code class="ph codeph">memberLeft</code> (graceful exit).</td> -</tr> -<tr> -<td><code class="ph codeph">RegionMembershipListener</code></td> -<td><code class="ph codeph">RegionEvent</code></td> -<td>Provides after-event notification when a region with the same name has been created in another member and when other members hosting the region join or leave the distributed system. Extends <code class="ph codeph">CacheCallback</code> and <code class="ph codeph">CacheListener</code>. Installed in region as a <code class="ph codeph">CacheListener</code>.</td> -</tr> -<tr> -<td><code class="ph codeph">TransactionListener</code></td> -<td><code class="ph codeph">TransactionEvent</code> with embedded list of <code class="ph codeph">EntryEvent</code></td> -<td>Tracks the outcome of transactions and changes to data entries in the transaction. -<div class="note note"> -**Note:** -<p>Multiple transactions on the same cache can cause concurrent invocation of <code class="ph codeph">TransactionListener</code> methods, so implement methods that do the appropriate synchronizing of the multiple threads for thread-safe operation.</p> -</div> -Extends <code class="ph codeph">CacheCallback</code> interface. Installed in cache using transaction manager. Works with region-level listeners if needed.</td> -</tr> -<tr> -<td><code class="ph codeph">TransactionWriter</code></td> -<td><code class="ph codeph">TransactionEvent</code> with embedded list of <code class="ph codeph">EntryEvent</code></td> -<td>Receives events for <em>pending</em> transaction commits. Has the ability to abort the transaction. Extends <code class="ph codeph">CacheCallback</code> interface. Installed in cache using transaction manager. At most one writer is called per transaction. Install a writer in every transaction host.</td> -</tr> -<tr> -<td><code class="ph codeph">UniversalMembershipListenerAdapter</code> -<p>(org.apache.geode .management.membership .UniversalMembershipListenerAdapter)</p></td> -<td><code class="ph codeph">MembershipEvent</code> and <code class="ph codeph">ClientMembershipEvent</code></td> -<td>One of the interfaces that replaces the deprecated Admin APIs. Provides a wrapper for MembershipListener and ClientMembershipListener callbacks for both clients and peers.</td> -</tr> -</tbody> -</table> - -## <a id="event_handlers_and_events__section_48C81FE4C1934DBBB287925A6F7A473D" class="no-quick-link"></a>Events - -The events in this table are cache events unless otherwise noted. - -<table> -<colgroup> -<col width="25%" /> -<col width="25%" /> -<col width="50%" /> -</colgroup> -<thead> -<tr> -<th>Event</th> -<th>Passed to handler ...</th> -<th>Description</th> -</tr> -</thead> -<tbody> -<tr> -<td><code class="ph codeph">AsyncEvent</code></td> -<td><code class="ph codeph">AsyncEventListener</code></td> -<td>Provides information about a single event in the cache for asynchronous, write-behind processing.</td> -</tr> -<tr> -<td><code class="ph codeph">CacheEvent</code></td> -<td> </td> -<td>Superinterface to <code class="ph codeph">RegionEvent</code> and <code class="ph codeph">EntryEvent</code>. This defines common event methods, and contains data needed to diagnose the circumstances of the event, including a description of the operation being performed, information about where the event originated, and any callback argument passed to the method that generated this event.</td> -</tr> -<tr> -<td><code class="ph codeph">ClientMembershipEvent</code></td> -<td><code class="ph codeph">ClientMembershipListener</code></td> -<td>An event delivered to a <code class="ph codeph">ClientMembershipListener</code> when this process detects connection changes to servers or clients.</td> -</tr> -<tr> -<td><code class="ph codeph">CqEvent</code></td> -<td><code class="ph codeph">CqListener</code></td> -<td>Provides information about a change to the results of a continuous query running on a server on behalf of a client. <code class="ph codeph">CqEvent</code>s are processed on the client.</td> -</tr> -<tr> -<td><code class="ph codeph">EntryEvent</code></td> -<td><code class="ph codeph">CacheListener</code>, <code class="ph codeph">CacheWriter</code>, <code class="ph codeph">TransactionListener</code> (inside the <code class="ph codeph">TransactionEvent</code>)</td> -<td>Extends <code class="ph codeph">CacheEvent</code> for entry events. Contains information about an event affecting a data entry in the cache. The information includes the key, the value before this event, and the value after this event. <code class="ph codeph">EntryEvent.getNewValue</code> returns the current value of the data entry. <code class="ph codeph">EntryEvent.getOldValue</code> returns the value before this event if it is available. For a partitioned region, returns the old value if the local cache holds the primary copy of the entry. <code class="ph codeph">EntryEvent</code> provides the Geode transaction ID if available. -<p>You can retrieve serialized values from <code class="ph codeph">EntryEvent</code> using the <code class="ph codeph">getSerialized</code>* methods. This is useful if you get values from one regionâs events just to put them into a separate cache region. There is no counterpart <code class="ph codeph">put</code> function as the put recognizes that the value is serialized and bypasses the serialization step.</p></td> -</tr> -<tr> -<td><code class="ph codeph">MembershipEvent</code> (membership event)</td> -<td><code class="ph codeph">MembershipListener</code></td> -<td><p>An event that describes the member that originated this event. Instances of this are delivered to a <code class="ph codeph">MembershipListener</code> when a member has joined or left the distributed system.</p></td> -</tr> -<tr> -<td><code class="ph codeph">RegionEvent</code></td> -<td><code class="ph codeph">CacheListener</code>, <code class="ph codeph">CacheWriter</code>, <code class="ph codeph">RegionMembershipListener</code></td> -<td>Extends <code class="ph codeph">CacheEvent</code> for region events. Provides information about operations that affect the whole region, such as reinitialization of the region after being destroyed.</td> -</tr> -<tr> -<td><code class="ph codeph">TimestampedEntryEvent</code></td> -<td><code class="ph codeph">GatewayConflictResolver</code></td> -<td>Extends <code class="ph codeph">EntryEvent</code> to include a timestamp and distributed system ID associated with the event. The conflict resolver can compare the timestamp and ID in the event with the values stored in the entry to decide whether the local system should apply the potentially conflicting event.</td> -</tr> -<tr> -<td><code class="ph codeph">TransactionEvent</code></td> -<td><code class="ph codeph">TransactionListener</code>, <code class="ph codeph">TransactionWriter</code></td> -<td>Describes the work done in a transaction. This event may be for a pending or committed transaction, or for the work abandoned by an explicit rollback or failed commit. The work is represented by an ordered list of <code class="ph codeph">EntryEvent</code> instances. The entry events are listed in the order in which the operations were performed in the transaction. -<p>As the transaction operations are performed, the entry events are conflated, with only the last event for each entry remaining in the list. So if entry A is modified, then entry B, then entry A, the list will contain the event for entry B followed by the second event for entry A.</p></td> -</tr> -</tbody> -</table> - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/resolving_multisite_conflicts.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/resolving_multisite_conflicts.html.md.erb b/geode-docs/developing/events/resolving_multisite_conflicts.html.md.erb deleted file mode 100644 index 259c319..0000000 --- a/geode-docs/developing/events/resolving_multisite_conflicts.html.md.erb +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: Resolving Conflicting Events ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -You can optionally create a `GatewayConflictResolver` cache plug-in to decide whether a potentially conflicting event that was delivered from another site should be applied to the local cache. - -By default, all regions perform consistency checks when a member applies an update received either from another cluster member or from a remote cluster over the WAN. The default consistency checking for WAN events is described in [How Consistency Is Achieved in WAN Deployments](../distributed_regions/how_region_versioning_works_wan.html#topic_fpy_z3h_j5). - -You can override the default consistency checking behavior by writing and configuring a custom `GatewayConflictResolver`. The `GatewayConflictResolver` implementation can use the timestamp and distributed system ID included in a WAN update event to determine whether or not to apply the update. For example, you may decide that updates from a particular cluster should always "win" a conflict when the timestamp difference between updates is less than some fixed period of time. - -## <a id="topic_E97BB68748F14987916CD1A50E4B4542__section_E20B4A8A98FD4EDAAA8C14B8059AA7F7" class="no-quick-link"></a>Implementing a GatewayConflictResolver - -**Note:** -A `GatewayConflictResolver` implementation is called only for update events that could cause a conflict in the region. This corresponds to update events that have a different distributed system ID than the distributed system that last updated the region entry. If the same distributed system ID makes consecutive updates to a region entry, no conflict is possible, and the `GatewayConflictResolver` is not called. - -**Procedure** - -1. Program the event handler: - 1. Create a class that implements the `GatewayConflictResolver` interface. - 2. If you want to declare the handler in `cache.xml`, implement the `org.apache.geode.cache.Declarable` interface as well. - 3. Implement the handler's `onEvent()` method to determine whether the WAN event should be allowed. `onEvent()` receives both a `TimestampedEntryEvent` and a `GatewayConflictHelper` instance. `TimestampedEntryEvent` has methods for obtaining the timestamp and distributed system ID of both the update event and the current region entry. Use methods in the `GatewayConflictHelper` to either disallow the update event (retaining the existing region entry value) or provide an alternate value. - - **Example:** - - ``` pre - public void onEvent(TimestampedEntryEvent event, GatewayConflictHelper helper) { - if (event.getOperation().isUpdate()) { - ShoppingCart oldCart = (ShoppingCart)event.getOldValue(); - ShoppingCart newCart = (ShoppingCart)event.getNewValue(); - oldCart.updateFromConflictingState(newCart); - helper.changeEventValue(oldCart); - } - } - ``` - - **Note:** - In order to maintain consistency in the region, your conflict resolver must always resolve two events in the same way regardless of which event it receives first. - -2. Install the conflict resolver for the cache, using either the `cache.xml` file or the Java API. - - **cache.xml** - - ``` pre - <cache> - ... - <gateway-conflict-resolver> - <class-name>myPackage.MyConflictResolver</class-name> - </gateway-conflict-resolver> - ... - </cache> - ``` - - **Java API** - - ``` pre - // Create or obtain the cache - Cache cache = new CacheFactory().create(); - - // Create and add a conflict resolver - cache.setGatewayConflictResolver(new MyConflictResolver); - ``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/tune_client_message_tracking_timeout.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/tune_client_message_tracking_timeout.html.md.erb b/geode-docs/developing/events/tune_client_message_tracking_timeout.html.md.erb deleted file mode 100644 index 4eaf852..0000000 --- a/geode-docs/developing/events/tune_client_message_tracking_timeout.html.md.erb +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: Tune the Client's Subscription Message Tracking Timeout ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="tune_client_message_tracking_timeout__section_C655A41D47694BDC9164E5D83C23FA7C"></a> -If the client pool's `subscription-message-tracking-timeout` is set too low, your client will discard tracking records for live threads, increasing the likelihood of processing duplicate events from those threads. - -This setting is especially important in systems where it is vital to avoid or greatly minimize duplicate events. If you detect that duplicate messages are being processed by your clients, increasing the timeout may help. Setting `subscription-message-tracking-timeout` may not completely eliminate duplicate entries, but careful configuration can help minimize occurrences. - -Duplicates are monitored by keeping track of message sequence IDs from the source thread where the operation originated. For a long-running system, you would not want to track this information for very long periods or the information may be kept long enough for a thread ID to be recycled. If this happens, messages from a new thread may be discarded mistakenly as duplicates of messages from an old thread with the same ID. In addition, maintaining this tracking information for old threads uses memory that might be freed up for other things. - -To minimize duplicates and reduce the size of the message tracking list, set your client `subscription-message-tracking-timeout` higher than double the sum of these times: - -- The longest time your originating threads might wait between operations -- For redundant servers add: - - The serverâs `message-sync-interval` - - Total time required for failover (usually 7-10 seconds, including the time to detect failure) - -You risk losing live thread tracking records if you set the value lower than this. This could result in your client processing duplicate event messages into its cache for the associated threads. It is worth working to set the `subscription-message-tracking-timeout` as low as you reasonably can. - -``` pre -<!-- Set the tracking timeout to 70 seconds --> -<pool name="client" subscription-enabled="true" subscription-message-tracking-timeout="70000"> - ... -</pool> -``` http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/tune_client_server_event_messaging.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/tune_client_server_event_messaging.html.md.erb b/geode-docs/developing/events/tune_client_server_event_messaging.html.md.erb deleted file mode 100644 index db43ed5..0000000 --- a/geode-docs/developing/events/tune_client_server_event_messaging.html.md.erb +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Tuning Client/Server Event Messaging ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="client_server_event_messaging__section_0894E034A285456EA01D5903248F9B3B"></a> -The server uses an asynchronous messaging queue to send events to its clients. Every event in the queue originates in an operation performed by a thread in a client, a server, or an application in the serverâs or some other distributed system. The event message has a unique identifier composed of the originating threadâs ID combined with its memberâs distributed system member ID, and the sequential ID of the operation. So the event messages originating in any single thread can be grouped and ordered by time from lowest sequence ID to highest. Servers and clients track the highest sequential ID for each member thread ID. - -A single client thread receives and processes messages from the server, tracking received messages to make sure it does not process duplicate sends. It does this using the process IDs from originating threads. - -<img src="../../images_svg/tune_cs_event_messaging.svg" id="client_server_event_messaging__image_656BDF5E745F4C6D92C844C423102948" class="image" /> - -The clientâs message tracking list holds the highest sequence ID of any message received for each originating thread. The list can become quite large in systems where there are many different threads coming and going and doing work on the cache. After a thread dies, its tracking entry is not needed. To avoid maintaining tracking information for threads that have died, the client expires entries that have had no activity for more than the `subscription-message-tracking-timeout`. - -- **[Conflate the Server Subscription Queue](../../developing/events/conflate_server_subscription_queue.html)** - -- **[Limit the Server's Subscription Queue Memory Use](../../developing/events/limit_server_subscription_queue_size.html)** - -- **[Tune the Client's Subscription Message Tracking Timeout](../../developing/events/tune_client_message_tracking_timeout.html)** - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/writing_callbacks_that_modify_the_cache.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/writing_callbacks_that_modify_the_cache.html.md.erb b/geode-docs/developing/events/writing_callbacks_that_modify_the_cache.html.md.erb deleted file mode 100644 index 7b201bc..0000000 --- a/geode-docs/developing/events/writing_callbacks_that_modify_the_cache.html.md.erb +++ /dev/null @@ -1,65 +0,0 @@ ---- -title: How to Safely Modify the Cache from an Event Handler Callback ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Event handlers are synchronous. If you need to change the cache or perform any other distributed operation from event handler callbacks, be careful to avoid activities that might block and affect your overall system performance. - -## <a id="writing_callbacks_that_modify_the_cache__section_98E49363C91945DEB0A3B2FD9A209969" class="no-quick-link"></a>Operations to Avoid in Event Handlers - -Do not perform distributed operations of any kind directly from your event handler. Geode is a highly distributed system and many operations that may seem local invoke distributed operations. - -These are common distributed operations that can get you into trouble: - -- Calling `Region` methods, on the event's region or any other region. -- Using the Geode `DistributedLockService`. -- Modifying region attributes. -- Executing a function through the Geode `FunctionService`. - -To be on the safe side, do not make any calls to the Geode API directly from your event handler. Make all Geode API calls from within a separate thread or executor. - -## <a id="writing_callbacks_that_modify_the_cache__section_78648D4177E14EA695F0B059E336137C" class="no-quick-link"></a>How to Perform Distributed Operations Based on Events - -If you need to use the Geode API from your handlers, make your work asynchronous to the event handler. You can spawn a separate thread or use a solution like the `java.util.concurrent.Executor` interface. - -This example shows a serial executor where the callback creates a `Runnable` that can be pulled off a queue and run by another object. This preserves the ordering of events. - -``` pre -public void afterCreate(EntryEvent event) { - final Region otherRegion = cache.getRegion("/otherRegion"); - final Object key = event.getKey(); - final Object val = event.getNewValue(); - - serialExecutor.execute(new Runnable() { - public void run() { - try { - otherRegion.create(key, val); - } - catch (org.apache.geode.cache.RegionDestroyedException e) { - ... - } - catch (org.apache.geode.cache.EntryExistsException e) { - ... - } - } - }); - } -``` - -For additional information on the `Executor`, see the `SerialExecutor` example on the Oracle Java web site. http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/eviction/chapter_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/eviction/chapter_overview.html.md.erb b/geode-docs/developing/eviction/chapter_overview.html.md.erb deleted file mode 100644 index c5d1417..0000000 --- a/geode-docs/developing/eviction/chapter_overview.html.md.erb +++ /dev/null @@ -1,34 +0,0 @@ ---- -title: Eviction ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Use eviction to control data region size. - -<a id="eviction__section_C3409270DD794822B15E819E2276B21A"></a> - -- **[How Eviction Works](../../developing/eviction/how_eviction_works.html)** - - Eviction settings cause Apache Geode to work to keep a region's resource use under a specified level by removing least recently used (LRU) entries to make way for new entries. - -- **[Configure Data Eviction](../../developing/eviction/configuring_data_eviction.html)** - - Use eviction controllers to configure the eviction-attributes region attribute settings to keep your region within a specified limit. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/eviction/configuring_data_eviction.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/eviction/configuring_data_eviction.html.md.erb b/geode-docs/developing/eviction/configuring_data_eviction.html.md.erb deleted file mode 100644 index 6c22284..0000000 --- a/geode-docs/developing/eviction/configuring_data_eviction.html.md.erb +++ /dev/null @@ -1,88 +0,0 @@ ---- -title: Configure Data Eviction ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Use eviction controllers to configure the eviction-attributes region attribute settings to keep your region within a specified limit. - -<a id="configuring_data_eviction__section_8515EC9635C342C0916EE9E6120E2AC9"></a> -Eviction controllers monitor region and memory use and, when the limit is reached, remove older entries to make way for new data. For heap percentage, the controller used is the Geode resource manager, configured in conjunction with the JVM's garbage collector for optimum performance. - -Configure data eviction as follows. You do not need to perform these steps in the sequence shown. - -1. Decide whether to evict based on: - - Entry count (useful if your entry sizes are relatively uniform). - - Total bytes used. In partitioned regions, this is set using `local-max-memory`. In non-partitioned, it is set in `eviction-attributes`. - - Percentage of application heap used. This uses the Geode resource manager. When the manager determines that eviction is required, the manager orders the eviction controller to start evicting from all regions where the eviction algorithm is set to `lru-heap-percentage`. Eviction continues until the manager calls a halt. Geode evicts the least recently used entry hosted by the member for the region. See [Managing Heap and Off-heap Memory](../../managing/heap_use/heap_management.html#resource_manager). - -2. Decide what action to take when the limit is reached: - - Locally destroy the entry. - - Overflow the entry data to disk. See [Persistence and Overflow](../storing_data_on_disk/chapter_overview.html). - -3. Decide the maximum amount of data to allow in the member for the eviction measurement indicated. This is the maximum for all storage for the region in the member. For partitioned regions, this is the total for all buckets stored in the member for the region - including any secondary buckets used for redundancy. -4. Decide whether to program a custom sizer for your region. If you are able to provide such a class, it might be faster than the standard sizing done by Geode. Your custom class must follow the guidelines for defining custom classes and, additionally, must implement `org.apache.geode.cache.util.ObjectSizer`. See [Requirements for Using Custom Classes in Data Caching](../../basic_config/data_entries_custom_classes/using_custom_classes.html). - -**Note:** -You can also configure Regions using the gfsh command-line interface, however, you cannot configure `eviction-attributes` using gfsh. See [Region Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_EF03119A40EE492984F3B6248596E1DD) and [Disk Store Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_1ACC91B493EE446E89EC7DBFBBAE00EA). - -Examples: - -``` pre -// Create an LRU memory eviction controller with max bytes of 1000 MB -// Use a custom class for measuring the size of each object in the region -<region-attributes refid="REPLICATE"> - <eviction-attributes> - <lru-memory-size maximum="1000" action="overflow-to-disk"> - <class-name>com.myLib.MySizer</class-name> - <parameter name="name"> - <string>Super Sizer</string> - </parameter> - </lru-memory-size> - </eviction-attributes> - </region-attributes> -``` - -``` pre -// Create a memory eviction controller on a partitioned region with max bytes of 512 MB -<region name="demoPR"> - <region-attributes refid="PARTITION"> - <partition-attributes local-max-memory="512" total-num-buckets="13"/> - <eviction-attributes> - <lru-memory-size action="local-destroy"/> - <class-name>org.apache.geode.cache.util.ObjectSizer - </class-name> - </eviction-attributes> - </region-attributes> -</region> - -``` - -``` pre -// Configure a partitioned region for heap LRU eviction. The resource manager controls the limits. -<region-attributes refid="PARTITION_HEAP_LRU"> -</region-attributes> -``` - -``` pre -Region currRegion = cache.createRegionFactory() - .setEvictionAttributes(EvictionAttributes.createLRUHeapAttributes(EvictionAction.LOCAL_DESTROY)) - .create("root"); -``` - -
