http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/delta_propagation/delta_propagation_properties.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/delta_propagation/delta_propagation_properties.html.md.erb b/geode-docs/developing/delta_propagation/delta_propagation_properties.html.md.erb deleted file mode 100644 index 0b5b40a..0000000 --- a/geode-docs/developing/delta_propagation/delta_propagation_properties.html.md.erb +++ /dev/null @@ -1,96 +0,0 @@ ---- -title: Delta Propagation Properties ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -This topic describes the properties that can be used to configure delta propagation. - -Delta propagation properties can be configured through the API and through the `gemfire.properties` and `cache.xml` files. - -## <a id="delta_propagation_properties__section_561D6DA876E24469B7536E98AB12F676" class="no-quick-link"></a>delta-propagation - -A `gemfire.properties` boolean that enables or disables delta propagation. When false, full entry values are sent for every update. The default setting is true, which enables delta propagation. - -Disable delta propagation as follows: - -- `gemfire.properties`: - - ``` pre - delta-propagation=false - ``` - -- API: - - ``` pre - Properties props = new Properties(); - props.setProperty("delta-propagation", false); - this.cache = new ClientCacheFactory(props).create(); - ``` - -## <a id="delta_propagation_properties__section_7D4590512D1548FD94F81C8726A2CA44" class="no-quick-link"></a>cloning-enabled - -A region attributes boolean that affects how `fromDelta` applies deltas to the local cache. When true, the updates are applied to a clone of the value and then the clone is saved to the cache. When false, the value is modified in place in the cache. The default value is false. - -Exceptions to this behavior: - -- If the `Cache` attribute `copy-on-read` is true, cloning is enabled, regardless of what this attribute is set to. -- If the `Region` attribute `off-heap` is true, cloning is enabled, regardless of what this attribute is set to. - -Cloning can be expensive, but it ensures that the new object is fully initialized with the delta before any application code sees it. - -When cloning is enabled, by default Geode does a deep copy of the object, using serialization. You may be able to improve performance by implementing `java.lang.Cloneable` and then implementing the `clone` method, making a deep copy of anything to which a delta may be applied. The goal is to reduce significantly the overhead of copying the object while still retaining the isolation needed for your deltas. - -Without cloning: - -- It is possible for application code to read the entry value as it is being modified, possibly seeing the value in an intermediate, inconsistent state, with just part of the delta applied. You may choose to resolve this issue by having your application code synchronize on reads and writes. -- Geode loses any reference to the old value because the old value is transformed in place into the new value. Because of this, your `CacheListener` sees the same new value returned for `EntryEvent.getOldValue` and `EntryEvent.getNewValue` . -- Exceptions thrown from `fromDelta` may leave your cache in an inconsistent state. Without cloning, any interruption of the delta application could leave you with some of the fields in your cached object changed and others unchanged. If you do not use cloning, keep this in mind when you program your error handling in your `fromDelta` implementation. - -With cloning: - -- The `fromDelta` method generates more garbage in memory. -- Performance is reduced. - -Enable cloning as follows: - -- `cache.xml`: - - ``` pre - <region name="region_with_cloning"> - <region-attributes refid="REPLICATE" cloning-enabled="true"> - </region-attributes> - </region> - ``` - -- API: - - ``` pre - RegionFactory rf = cache.createRegionFactory(REPLICATE); - rf.setCloningEnabled(true); - custRegion = rf.create("customer"); - ``` - -- gfsh: - - ``` pre - gfsh>create region --name="region_with_cloning" --type=REPLICATE - --enable-cloning=true - ``` - -
http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/delta_propagation/errors_in_delta_propagation.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/delta_propagation/errors_in_delta_propagation.html.md.erb b/geode-docs/developing/delta_propagation/errors_in_delta_propagation.html.md.erb deleted file mode 100644 index 6f0cdf4..0000000 --- a/geode-docs/developing/delta_propagation/errors_in_delta_propagation.html.md.erb +++ /dev/null @@ -1,35 +0,0 @@ ---- -title: Errors In Delta Propagation ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -This topic lists the errors that can occur when using delta propagation. - -<a id="errors_in_delta_propagation__section_877AC61D691C44078A782683F90D169B"></a> -Errors in delta propagation fall into two categories based on how they are handled by the system: - -- Problems applying the delta that can be remedied by requesting the full value in place of the delta. Your `put` operation does not see errors or exceptions related to this type of delta propagation failure. The system automatically does a full value distribution from the sender to the receiver where the problem occurs. This type of error includes: - - Unavailable entry value in the receiving cache, either because the entry is missing or its value is null. In both cases, there is nothing to apply the delta to and the full value must be sent. This is most likely to occur if you destroy or invalidate your entries locally, either through application calls or through configured actions like eviction or entry expiration. - - `InvalidDeltaException` thrown by `fromDelta` method, programmed by you. This exception enables you to avoid applying deltas that would violate data consistency checks or other application requirements. - - Any error applying the delta in a client in server-to-client propagation. The client logs a warning in addition to retrieving the full value from the server. -- Problems creating or distributing the delta that cannot be fixed by distributing the full value. In these cases, your `put` operation fails with an exception. This type of error includes: - - Error or exception in `hasDelta` or `toDelta`. - - Error or exception in a server or peer receiver that fall outside of the situations described above in the first category. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/delta_propagation/how_delta_propagation_works.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/delta_propagation/how_delta_propagation_works.html.md.erb b/geode-docs/developing/delta_propagation/how_delta_propagation_works.html.md.erb deleted file mode 100644 index 6437781..0000000 --- a/geode-docs/developing/delta_propagation/how_delta_propagation_works.html.md.erb +++ /dev/null @@ -1,69 +0,0 @@ ---- -title: How Delta Propagation Works ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Delta propagation reduces the amount of data you send over the network. You do this by only sending the change, or delta, information about an object, instead of sending the entire changed object. If you do not use cloning when applying the deltas, you can also expect to generate less garbage in your receiving JVMs. - -<a id="how_delta_propagation_works__section_78D584B3FFD04D1D9BA83203FF2B55A9"></a> -In most distributed data management systems, the data stored in the system tends to be created once and then updated frequently. These updates are sent to other members for event propagation, redundancy management, and cache consistency in general. Tracking only the changes in an updated object and sending only the deltas mean lower network transmission costs and lower object serialization/deserialization costs. Performance improvements can be significant, especially when changes to an object are small relative to its overall size. - -<a id="how_delta_propagation_works__section_ABE3589920D6477BBB2223A583AF169A"></a> - -Geode propagates object deltas using methods that you program. The methods are in the `Delta` interface, which you implement in your cached objects' classes. If any of your classes are plain old Java objects, you need to wrap them for this implementation. - -This figure shows delta propagation for a change to an entry with key, k, and value object, v. - -<img src="../../images/DeltaPropagation-1.gif" id="how_delta_propagation_works__image_06A25185C31548669423DDA3136B1851" class="image" /> - -1. **`get` operation**. The `get` works as usual: the cache returns the full entry object from the local cache or, if it isn't available there, from a remote cache or from a loader. -2. **update methods**. You need to add code to the object's update methods so that they save delta information for object updates, in addition to the work they were already doing. -3. **`put` operation**. The `put` works as usual in the local cache, using the full value, then calls `hasDelta` to see if there are deltas and `toDelta` to serialize the information. Distribution is the same as for full values, according to member and region configuration. -4. **receipt of delta at remote member**. `fromDelta` extracts the delta information that was serialized by `toDelta` and applies it to the object in the local cache. The delta is applied directly to the existing value or to a clone, depending on how you configure it for the region. -5. **additional distributions**. As with full distributions, receiving members forward the delta according to their configurations and connections to other members. For example, if VM1 is a client and VM2 is a server, VM2 forwards the delta to its peers and its other clients as needed. Receiving members do not recreate the delta; `toDelta` is only called in the originating member. - -## <a id="how_delta_propagation_works__section_25EC5BE960F0402DAEDBE0A5A6589ACA" class="no-quick-link"></a>General Characteristics of Delta Propagation - -To use the delta propagation feature, all updates on a key in a region must have value types that implement the `Delta` interface. You cannot mix object types for an entry key where some of the types implement delta and some do not. This is because, when a type implementing the delta interface is received for an update, the existing value for the key is cast to a `Delta` type to apply the received delta. If the existing type does not also implement the `Delta` interface, the operation throws a `ClassCastException`. - -**Note:** Only the object itself being placed in the cache can implement the `Delta` interface and propagate changes. Any sub-objects of the cache object do not propagate their changes. - -Sometimes `fromDelta` cannot be invoked because there is no object to apply the delta to in the receiving cache. When this happens, the system automatically does a full value distribution to the receiver. These are the possible scenarios: -1. If the system can determine beforehand that the receiver does not have a local copy, it sends the initial message with the full value. This is possible when regions are configured with no local data storage, such as with the region shortcut settings `PARTITION_PROXY` and `REPLICATE_PROXY`. These configurations are used to accomplish things like provide data update information to listeners and to pass updates forward to clients. -2. In less obvious cases, such as when an entry has been locally deleted, first the delta is sent, then the receiver requests a full value and that is sent. Whenever the full value is received, any further distributions to the receiver's peers or clients uses the full value. - -Geode also does not propagate deltas for: - -- Transactional commit -- The `putAll` operation -- JVMs running Geode versions that do not support delta propagation (6.0 and earlier) - -## <a id="how_delta_propagation_works__section_F4A102A74530429F87BEA53C90D5CCFB" class="no-quick-link"></a>Supported Topologies and Limitations - -The following topologies support delta propagation (with some limitations): - -- **Peer-to-peer**. Geode system members distribute and receive entry changes using delta propagation, with these requirements and caveats: - - Regions must be partitioned or have their scope set to `distributed-ack` or `global`. The region shortcut settings for distributed regions use `distributed-ack` `scope`. Delta propagation does not work for regions with `distributed-no-ack` `scope` because the receiver could not recover if an exception occurred while applying the delta. - - For partitioned regions, if a receiving peer does not hold the primary or a secondary copy of the entry, but still requires a value, the system automatically sends the full value. - - To receive deltas, a region must be non-empty. The system automatically sends the full value to empty regions. Empty regions can send deltas. -- **Client/server**. Geode clients can always send deltas to the servers, and servers can usually sent deltas to clients. These configurations require the servers to send full values to the clients, instead of deltas: - - When the client's `gemfire.properties` setting `conflate-events` is set to true, the servers send full values for all regions. - - When the server region attribute `enable-subscription-conflation` is set to true and the client `gemfire.properties` setting `conflate-events` is set to `server`, the servers send full values for the region. - - When the client region is configured with the `PROXY` client region shortcut setting (empty client region), servers send full values. - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/delta_propagation/implementing_delta_propagation.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/delta_propagation/implementing_delta_propagation.html.md.erb b/geode-docs/developing/delta_propagation/implementing_delta_propagation.html.md.erb deleted file mode 100644 index 4b6ae99..0000000 --- a/geode-docs/developing/delta_propagation/implementing_delta_propagation.html.md.erb +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: Implementing Delta Propagation ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -By default, delta propagation is enabled in your distributed system. When enabled, delta propagation is used for objects that implement `org.apache.geode.Delta`. You program the methods to store and extract delta information for your entries and to apply received delta information. - -<a id="implementing_delta_propagation__section_877AC61D691C44078A782683F90D169B"></a> -Use the following procedure to implement delta propagation in your distributed system. - -1. Study your object types and expected application behavior to determine which regions can benefit from using delta propagation. Delta propagation does not improve performance for all data and data modification scenarios. See [When to Avoid Delta Propagation](when_to_use_delta_prop.html#when_to_use_delta_prop). -2. For each region where you are using delta propagation, choose whether to enable cloning using the delta propagation property `cloning-enabled`. Cloning is disabled by default. See [Delta Propagation Properties](delta_propagation_properties.html#delta_propagation_properties). -3. If you do not enable cloning, review all associated listener code for dependencies on `EntryEvent.getOldValue`. Without cloning, Geode modifies the entry in place and so loses its reference to the old value. For delta events, the `EntryEvent` methods `getOldValue` and `getNewValue` both return the new value. -4. For every class where you want delta propagation, implement `org.apache.geode.Delta` and update your methods to support delta propagation. Exactly how you do this depends on your application and object needs, but these steps describe the basic approach: - 1. If the class is a plain old Java object (POJO), wrap it for this implementation and update your code to work with the wrapper class. - 2. Define as transient any extra object fields that you use to manage delta state. This can help performance when the full object is distributed. Whenever standard Java serialization is used, the transient keyword indicates to Java to not serialize the field. - 3. Study the object contents to decide how to handle delta changes. Delta propagation has the same issues of distributed concurrency control as the distribution of full objects, but on a more detailed level. Some parts of your objects may be able to change independent of one another while others may always need to change together. Send deltas large enough to keep your data logically consistent. If, for example, field A and field B depend on each other, then your delta distributions should either update both fields or neither. As with regular updates, the fewer producers you have on a data region, the lower your likelihood of concurrency issues. - 4. In the application code that puts entries, put the fully populated object into the local cache. Even though you are planning to send only deltas, errors on the receiving end could cause Geode to request the full object, so you must provide it to the originating put method. Do this even in empty producers, with regions configured for no local data storage. This usually means doing a get on the entry unless you are sure it does not already exist anywhere in the distributed region. - 5. Change each field's update method to record information about the update. The information must be sufficient for `toDelta` to encode the delta and any additional required delta information when it is invoked. - 6. Write `hasDelta` to report on whether a delta is available. - 7. Write `toDelta` to create a byte stream with the changes to the object and any other information `fromDelta` will need to apply the changes. Before returning from `toDelta`, reset your delta state to indicate that there are no delta changes waiting to be sent. - 8. Write `fromDelta` to decode the byte stream that `toDelta` creates and update the object. - 9. Make sure you provide adequate synchronization to your object to maintain a consistent object state. If you do not use cloning, you will probably need to synchronize on reads and writes to avoid reading partially written updates from the cache.This synchronization might involve `toDelta`, `fromDelta`, `toData`, `fromData`, and other methods that access or update the object. Additionally, your implementation should take into account the possibility of concurrent invocations of `fromDelta` and one or more of the object's update methods. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/delta_propagation/when_to_use_delta_prop.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/delta_propagation/when_to_use_delta_prop.html.md.erb b/geode-docs/developing/delta_propagation/when_to_use_delta_prop.html.md.erb deleted file mode 100644 index 0c53082..0000000 --- a/geode-docs/developing/delta_propagation/when_to_use_delta_prop.html.md.erb +++ /dev/null @@ -1,34 +0,0 @@ ---- -title: When to Avoid Delta Propagation ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Generally, the larger your objects and the smaller the deltas, the greater the benefits of using delta propagation. Partitioned regions with higher redundancy levels generally benefit more from delta propagation. However, in some application scenarios, delta propagation does not show any significant benefits. On occasion it results in performance degradation. - -<a id="when_to_use_delta_prop__section_83BA84BB08194FC58F2BCE149AA0F0EC"></a> -By default, delta propagation is enabled in your distributed system. - -These are the main factors that can reduce the performance benefits of using delta propagation: - -- The added costs of deserializing your objects to apply deltas. Applying a delta requires the entry value to be deserialized. Once this is done, the object is stored back in the cache in deserialized form. This aspect of delta propagation only negatively impacts your system if your objects are not already being deserialized for other reasons, such as for indexing and querying or for listener operations. Once stored in deserialized form, there are reserialization costs for operations that send the object outside of the member, like distribution from a gateway sender, values sent in response to `netSearch` or client requests, and storage to disk. The more operations that require reserialization, the higher the overhead of deserializing the object. As with all serialization efforts, you can improve performance in serialization and deserialization by providing custom implementations of `DataSerializable` for your objects. -- Cloning when applying the delta. Using cloning can affect performance and generates extra garbage. Not using cloning is risky however, as you are modifying cached values in place. Without cloning, make sure you synchronize your entry access to keep your cache from becoming inconsistent. -- Problems applying the delta that cause the system to go back to the originator for the full entry value. When this happens, the overall operation costs more than sending the full entry value in the first place. This can be additionally aggravated if your delta is sent to a number of recipients, all or most of them request a full value, and the full value send requires the object to be serialized. -- Disk I/O costs associated with overflow regions. If you use eviction with overflow to disk, on-disk values must be brought into memory in order to apply the delta. This is much more costly than just removing the reference to the disk copy, as you would do with a full value distribution into the cache. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/chapter_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/chapter_overview.html.md.erb b/geode-docs/developing/distributed_regions/chapter_overview.html.md.erb deleted file mode 100644 index d24de37..0000000 --- a/geode-docs/developing/distributed_regions/chapter_overview.html.md.erb +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Distributed and Replicated Regions ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -In addition to basic region management, distributed and replicated regions include options for things like push and pull distribution models, global locking, and region entry versions to ensure consistency across Geode members. - -- **[How Distribution Works](../../developing/distributed_regions/how_distribution_works.html)** - - To use distributed and replicated regions, you should understand how they work and your options for managing them. - -- **[Options for Region Distribution](../../developing/distributed_regions/choosing_level_of_dist.html)** - - You can use distribution with and without acknowledgment, or global locking for your region distribution. Regions that are configured for distribution with acknowledgment can also be configured to resolve concurrent updates consistently across all Geode members that host the region. - -- **[How Replication and Preloading Work](../../developing/distributed_regions/how_replication_works.html)** - - To work with replicated and preloaded regions, you should understand how their data is initialized and maintained in the cache. - -- **[Configure Distributed, Replicated, and Preloaded Regions](../../developing/distributed_regions/managing_distributed_regions.html)** - - Plan the configuration and ongoing management of your distributed, replicated, and preloaded regions, and configure the regions. - -- **[Locking in Global Regions](../../developing/distributed_regions/locking_in_global_regions.html)** - - In global regions, the system locks entries and the region during updates. You can also explicitly lock the region and its entries as needed by your application. Locking includes system settings that help you optimize performance and locking behavior between your members. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/choosing_level_of_dist.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/choosing_level_of_dist.html.md.erb b/geode-docs/developing/distributed_regions/choosing_level_of_dist.html.md.erb deleted file mode 100644 index 3d48ab4..0000000 --- a/geode-docs/developing/distributed_regions/choosing_level_of_dist.html.md.erb +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: Options for Region Distribution ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -You can use distribution with and without acknowledgment, or global locking for your region distribution. Regions that are configured for distribution with acknowledgment can also be configured to resolve concurrent updates consistently across all Geode members that host the region. - -<a id="choosing_level_of_dist__section_F2528B151DD54CEFA05C4BA655BCF016"></a> -Each distributed region must have the same scope and concurrency checking setting throughout the distributed system. - -Distributed scope is provided at three levels: - -- **distributed-no-ack**. Distribution operations return without waiting for a response from other caches. This scope provides the best performance and uses the least amount of overhead, but it is also most prone to having inconsistencies caused by network problems. For example, a temporary disruption of the network transport layer could cause a failure in distributing updates to a cache on a remote machine, while the local cache continues being updated. -- **distributed-ack**. Distribution waits for acknowledgment from other caches before continuing. This is slower than `distributed-no-ack`, but covers simple communication problems such as temporary network disruptions. - - In systems where there are many `distributed-no-ack` operations, it is possible for `distributed-ack` operations to take a long time to complete. The distributed system has a configurable time to wait for acknowledgment to any `distributed-ack` message before sending alerts to the logs about a possible problem with the unresponsive member. No matter how long the wait, the sender keeps waiting in order to honor the distributed-ack region setting. The `gemfire.properties` attribute governing this is `ack-wait-threshold`. - -- **global**. Entries and regions are automatically locked across the distributed system during distribution operations. All load, create, put, invalidate, and destroy operations on the region and its entries are performed with a distributed lock. The global scope enforces strict consistency across the distributed system, but it is the slowest mechanism for achieving consistency. In addition to the implicit locking performed by distribution operations, regions with global scope and their contents can be explicitly locked through the application APIs. This allows applications to perform atomic, multi-step operations on regions and region entries. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/how_distribution_works.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/how_distribution_works.html.md.erb b/geode-docs/developing/distributed_regions/how_distribution_works.html.md.erb deleted file mode 100644 index 857a194..0000000 --- a/geode-docs/developing/distributed_regions/how_distribution_works.html.md.erb +++ /dev/null @@ -1,48 +0,0 @@ ---- -title: How Distribution Works ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -To use distributed and replicated regions, you should understand how they work and your options for managing them. - -<a id="how_distribution_works__section_2F892A4987C547E68CA78067133C2C2C"></a> -**Note:** -The management of replicated and distributed regions supplements the general information for managing data regions provided in [Basic Configuration and Programming](../../basic_config/book_intro.html). See also `org.apache.geode.cache.PartitionAttributes`. - -A distributed region automatically sends entry value updates to remote caches and receives updates from them. - -- Distributed entry updates come from the `Region` `put` and `create` operations (the creation of an entry with a non-null value is seen as an update by remote caches that already have the entry key). Entry updates are distributed selectively - only to caches where the entry key is already defined. This provides a pull model of distribution, compared to the push model that you get with replication. -- Distribution alone does not cause new entries to be copied from remote caches. -- A distributed region shares cache loader and cache writer application event handler plug-ins across the distributed system. - -In a distributed region, new and updated entry values are automatically distributed to remote caches that already have the entries defined. - -**Step 1:** The application updates or creates the entry. At this point, the entry in the M1 cache may not yet exist. - -<img src="../../images_svg/distributed_how_1.svg" id="how_distribution_works__image_40EFE6E95E6945A1B08A68508ECBCC60" class="image" /> - -**Step 2:** The new value is automatically distributed to caches holding the entry. - -<img src="../../images_svg/distributed_how_2.svg" id="how_distribution_works__image_AF8A3ADEB5D94E20B101FDA92BF6D002" class="image" /> - -**Step 3:** The entry's value is the same throughout the distributed system. - -<img src="../../images_svg/distributed_how_3.svg" id="how_distribution_works__image_5B1F06B54C9047E28A8C8673D1D5BD27" class="image" /> - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/how_region_versioning_works.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/how_region_versioning_works.html.md.erb b/geode-docs/developing/distributed_regions/how_region_versioning_works.html.md.erb deleted file mode 100644 index c8b98f7..0000000 --- a/geode-docs/developing/distributed_regions/how_region_versioning_works.html.md.erb +++ /dev/null @@ -1,127 +0,0 @@ ---- -title: Consistency Checking by Region Type ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="topic_7A4B6C6169BD4B1ABD356294F744D236"></a> - -Geode performs different consistency checks depending on the type of region you have configured. - -## <a id="topic_7A4B6C6169BD4B1ABD356294F744D236__section_B090F5FB87D84104A7BE4BCEA6BAE6B7" class="no-quick-link"></a>Partitioned Region Consistency - -For a partitioned region, Geode maintains consistency by routing all updates on a given key to the Geode member that holds the primary copy of that key. That member holds a lock on the key while distributing updates to other members that host a copy of the key. Because all updates to a partitioned region are serialized on the primary Geode member, all members apply the updates in the same order and consistency is maintained at all times. See [Understanding Partitioning](../partitioned_regions/how_partitioning_works.html). - -## <a id="topic_7A4B6C6169BD4B1ABD356294F744D236__section_72DFB366C8F14ADBAF2A136669ECAB1E" class="no-quick-link"></a>Replicated Region Consistency - -For a replicated region, any member that hosts the region can update a key and distribute that update to other members without locking the key. It is possible that two members can update the same key at the same time (a concurrent update). It is also possible that, due to network latency, an update in one member is distributed to other members at a later time, after those members have already applied more recent updates to the key (an out-of-order update). By default, Geode members perform conflict checking before applying region updates in order to detect and consistently resolve concurrent and out-of-order updates. Conflict checking ensures that region data eventually becomes consistent on all members that host the region. The conflict checking behavior for replicated regions is summarized as follows: - -- If two members update the same key at the same time, conflict checking ensures that all members eventually apply the same value, which is the value of one of the two concurrent updates. -- If a member receives an out-of-order update (an update that is received after one or more recent updates were applied), conflict checking ensures that the out-of-order update is discarded and not applied to the cache. - -[How Consistency Checking Works for Replicated Regions](#topic_C5B74CCDD909403C815639339AA03758) and [How Destroy and Clear Operations Are Resolved](#topic_321B05044B6641FCAEFABBF5066BD399) provide more details about how Geode performs conflict checking when applying an update. - -## <a id="topic_7A4B6C6169BD4B1ABD356294F744D236__section_313045F430EE459CB411CAAE7B00F3D8" class="no-quick-link"></a>Non-Replicated Regions and Client Cache Consistency - -When a member receives an update for an entry in a non-replicated region and applies an update, it performs conflict checking in the same way as for a replicated region. However, if the member initiates an operation on an entry that is not present in the region, it first passes that operation to a member that hosts a replicate. The member that hosts the replica generates and provides the version information necessary for subsequent conflict checking. See [How Consistency Checking Works for Replicated Regions](#topic_C5B74CCDD909403C815639339AA03758). - -Client caches also perform consistency checking in the same way when they receive an update for a region entry. However, all region operations that originate in the client cache are first passed onto an available Geode server, which generates the version information necessary for subsequent conflict checking. - -## <a id="topic_B64891585E7F4358A633C792F10FA23E" class="no-quick-link"></a>Configuring Consistency Checking - -Geode enables consistency checking by default. You cannot disable consistency checking for persistent regions. For all other regions, you can explicitly enable or disable consistency checking by setting the `concurrency-checks-enabled` region attribute in `cache.xml` to "true" or "false." - -All Geode members that host a region must use the same `concurrency-checks-enabled` setting for that region. - -A client cache can disable consistency checking for a region even if server caches enable consistency checking for the same region. This configuration ensures that the client sees all events for the region, but it does not prevent the client cache region from becoming out-of-sync with the server cache. - -See [<region-attributes>](../../reference/topics/cache_xml.html#region-attributes). - -**Note:** -Regions that do not enable consistency checking remain subject to race conditions. Concurrent updates may result in one or more members having different values for the same key. Network latency can result in older updates being applied to a key after more recent updates have occurred. - -## <a id="topic_0BDACA590B2C4974AC9C450397FE70B2" class="no-quick-link"></a>Overhead for Consistency Checks - -Consistency checking requires additional overhead for storing and distributing version and timestamp information, as well as for maintaining destroyed entries for a period of time to meet consistency requirements. - -To provide consistency checking, each region entry uses an additional 16 bytes. When an entry is deleted, a tombstone entry of approximately 13 bytes is created and maintained until the tombstone expires or is garbage-collected in the member. (When an entry is destroyed, the member temporarily retains the entry with its current version stamp to detect possible conflicts with operations that have occurred. The retained entry is referred to as a tombstone.) See [How Destroy and Clear Operations Are Resolved](#topic_321B05044B6641FCAEFABBF5066BD399). - -If you cannot support the additional overhead in your deployment, you can disable consistency checks by setting `concurrency-checks-enabled` to "false" for each region. See [Consistency for Region Updates](region_entry_versions.html#topic_CF2798D3E12647F182C2CEC4A46E2045). - -## <a id="topic_C5B74CCDD909403C815639339AA03758" class="no-quick-link"></a>How Consistency Checking Works for Replicated Regions - -Each region stores version and timestamp information for use in conflict detection. Geode members use the recorded information to detect and resolve conflicts consistently before applying a distributed update. - -<a id="topic_C5B74CCDD909403C815639339AA03758__section_763B071061C94D1E82E8883325294547"></a> -By default, each entry in a region stores the ID of the Geode member that last updated the entry, as well as a version stamp for the entry that is incremented each time an update occurs. The version information is stored in each local entry, and the version stamp is distributed to other Geode members when the local entry is updated. - -A Geode member or client that receives an update message first compares the update version stamp with the version stamp recorded in its local cache. If the update version stamp is larger, it represents a newer version of the entry, so the receiving member applies the update locally and updates the version information. A smaller update version stamp indicates an out-of-order update, which is discarded. - -An identical version stamp indicates that multiple Geode members updated the same entry at the same time. To resolve a concurrent update, a Geode member always applies (or keeps) the region entry that has the highest membership ID; the region entry having the lower membership ID is discarded. - -**Note:** -When a Geode member discards an update message (either for an out-of-order update or when resolving a concurrent update), it does not pass the discarded event to an event listener for the region. You can track the number of discarded updates for each member using the `conflatedEvents` statistic. See [Geode Statistics List](../../reference/statistics/statistics_list.html#statistics_list). Some members may discard an update while other members apply the update, depending on the order in which each member receives the update. For this reason, the `conflatedEvents` statistic differs for each Geode member. The example below describes this behavior in more detail. - -The following example shows how a concurrent update is handled in a distributed system of three Geode members. Assume that Members A, B, and C have membership IDs of 1, 2, and 3, respectively. Each member currently stores an entry, X, in their caches at version C2 (the entry was last updated by member C): - -**Step 1:** An application updates entry X on Geode member A at the same time another application updates entry X on member C. Each member increments the version stamp for the entry and records the version stamp with their member ID in their local caches. In this case the entry was originally at version C2, so each member updates the version to 3 (A3 and C3, respectively) in their local caches. - -<img src="../../images_svg/region_entry_versions_1.svg" id="topic_C5B74CCDD909403C815639339AA03758__image_nt5_ptw_4r" class="image" /> - -**Step 2:** Member A distributes its update message to members B and C. - -Member B compares the update version stamp (3) to its recorded version stamp (2) and applies the update to its local cache as version A3. In this member, the update is applied for the time being, and passed on to configured event listeners. - -Member C compares the update version stamp (3) to its recorded version stamp (3) and identifies a concurrent update. To resolve the conflict, member C next compares the membership ID of the update to the membership ID stored in its local cache. Because the distributed system ID the update (A3) is lower than the ID stored in the cache (C3), member C discards the update (and increments the `conflatedEvents` statistic). - -<img src="../../images_svg/region_entry_versions_2.svg" id="topic_C5B74CCDD909403C815639339AA03758__image_ocs_35b_pr" class="image" /> - -**Step 3:** Member C distributes the update message to members A and B. - -Members A and B compare the update version stamp (3) to their recorded version stamps (3) and identify the concurrent update. To resolve the conflict, both members compare the membership ID of the update with the membership ID stored in their local caches. Because the distributed system ID of A in the cache value is less than the ID of C in the update, both members record the update C3 in their local caches, overwriting the previous value. - -At this point, all members that host the region have achieved a consistent state for the concurrent updates on members A and C. - -<img src="../../images_svg/region_entry_versions_3.svg" id="topic_C5B74CCDD909403C815639339AA03758__image_gsv_k5b_pr" class="image" /> - -## <a id="topic_321B05044B6641FCAEFABBF5066BD399" class="no-quick-link"></a>How Destroy and Clear Operations Are Resolved - -When consistency checking is enabled for a region, a Geode member does not immediately remove an entry from the region when an application destroys the entry. Instead, the member retains the entry with its current version stamp for a period of time in order to detect possible conflicts with operations that have occurred. The retained entry is referred to as a *tombstone*. Geode retains tombstones for partitioned regions and non-replicated regions as well as for replicated regions, in order to provide consistency. - -A tombstone in a client cache or a non-replicated region expires after 8 minutes, at which point the tombstone is immediately removed from the cache. - -A tombstone for a replicated or partitioned region expires after 10 minutes. Expired tombstones are eligible for garbage collection by the Geode member. Garbage collection is automatically triggered after 100,000 tombstones of any type have timed out in the local Geode member. You can optionally set the `gemfire.tombstone-gc-threshold` property to a value smaller than 100000 to perform garbage collection more frequently. - -**Note:** -To avoid out-of-memory errors, a Geode member also initiates garbage collection for tombstones when the amount of free memory drops below 30 percent of total memory. - -You can monitor the total number of tombstones in a cache using the `tombstoneCount` statistic in `CachePerfStats`. The `tombstoneGCCount` statistic records the total number of tombstone garbage collection cycles that a member has performed. `replicatedTombstonesSize` and `nonReplicatedTombstonesSize` show the approximate number of bytes that are currently consumed by tombstones in replicated or partitioned regions, and in non-replicated regions, respectively. See [Geode Statistics List](../../reference/statistics/statistics_list.html#statistics_list). - -## <a id="topic_321B05044B6641FCAEFABBF5066BD399__section_4D0140E96A3141EB8D983D0A43464097" class="no-quick-link"></a>About Region.clear() Operations - -Region entry version stamps and tombstones ensure consistency only when individual entries are destroyed. A `Region.clear()` operation, however, operates on all entries in a region at once. To provide consistency for `Region.clear()` operations, Geode obtains a distributed read/write lock for the region, which blocks all concurrent updates to the region. Any updates that were initiated before the clear operation are allowed to complete before the region is cleared. - -## <a id="topic_32ACFA5542C74F3583ECD30467F352B0" class="no-quick-link"></a>Transactions with Consistent Regions - -A transaction that modifies a region having consistency checking enabled generates all necessary version information for region updates when the transaction commits. - -If a transaction modifies a normal, preloaded or empty region, the transaction is first delegated to a Geode member that holds a replicate for the region. This behavior is similar to the transactional behavior for partitioned regions, where the partitioned region transaction is forwarded to a member that hosts the primary for the partitioned region update. - -The limitation for transactions on normal, preloaded or or empty regions is that, when consistency checking is enabled, a transaction cannot perform a `localDestroy` or `localInvalidate` operation against the region. Geode throws an `UnsupportedOperationInTransactionException` exception in such cases. An application should use a `Destroy` or `Invalidate` operation in place of a `localDestroy` or `localInvalidate` when consistency checks are enabled. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/how_region_versioning_works_wan.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/how_region_versioning_works_wan.html.md.erb b/geode-docs/developing/distributed_regions/how_region_versioning_works_wan.html.md.erb deleted file mode 100644 index 275d496..0000000 --- a/geode-docs/developing/distributed_regions/how_region_versioning_works_wan.html.md.erb +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: How Consistency Is Achieved in WAN Deployments ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -When two or more Geode systems are configured to distribute events over a WAN, each system performs local consistency checking before it distributes an event to a configured gateway sender. Discarded events are not distributed across the WAN. - -Regions can also be configured to distribute updates to other Geode clusters over a WAN. With a distributed WAN configuration, multiple gateway senders asynchronously queue and send region updates to another Geode cluster. It is possible for multiple sites to send updates to the same region entry at the same time. It is also possible that, due to a slow WAN connection, a cluster might receive region updates after a considerable delay, and after it has applied more recent updates to a region. To ensure that WAN-replicated regions eventually reach a consistent state, Geode first ensures that each cluster performs consistency checking to regions before queuing updates to a gateway sender for WAN distribution. In order words, region conflicts are first detected and resolved in the local cluster, using the techniques described in the previous sections. - -When a Geode cluster in a WAN configuration receives a distributed update, conflict checking is performed to ensure that all sites apply updates in the same way. This ensures that regions eventually reach a consistent state across all Geode clusters. The default conflict checking behavior for WAN-replicated regions is summarized as follows: - -- If an update is received from the same Geode cluster that last updated the region entry, then there is no conflict and the update is applied. -- If an update is received from a different Geode cluster than the one that last updated the region entry, then a potential conflict exists. A cluster applies the update only when the update has a timestamp that is later than the timestamp currently recorded in the cache. - -**Note:** -If you use the default conflict checking feature for WAN deployments, you must ensure that all Geode members in all clusters synchronize their system clocks. For example, use a common NTP server for all Geode members that participate in a WAN deployment. - -As an alternative to the default conflict checking behavior for WAN deployments, you can develop and deploy a custom conflict resolver for handling region events that are distributed over a WAN. Using a custom resolver enables you to handle conflicts using criteria other than, or in addition to, timestamp information. For example, you might always prioritize updates that originate from a particular site, given that the timestamp value is within a certain range. - -When a gateway sender distributes an event to another Geode site, it adds the distributed system ID of the local cluster, as well as a timestamp for the event. In a default configuration, the cluster that receives the event examines the timestamp to determine whether or not the event should be applied. If the timestamp of the update is earlier than the local timestamp, the cluster discards the event. If the timestamp is the same as the local timestamp, then the entry having the highest distributed system ID is applied (or kept). - -You can override the default consistency checking for WAN events by installing a conflict resolver plug-in for the region. If a conflict resolver is installed, then any event that can potentially cause a conflict (any event that originated from a different distributed system ID than the ID that last modified the entry) is delivered to the conflict resolver. The resolver plug-in then makes the sole determination for which update to apply or keep. - -See "Implementing a GatewayConflictResolver" under [Resolving Conflicting Events](../events/resolving_multisite_conflicts.html#topic_E97BB68748F14987916CD1A50E4B4542) to configure a custom resolver. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/how_replication_works.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/how_replication_works.html.md.erb b/geode-docs/developing/distributed_regions/how_replication_works.html.md.erb deleted file mode 100644 index 6d2ccc5..0000000 --- a/geode-docs/developing/distributed_regions/how_replication_works.html.md.erb +++ /dev/null @@ -1,51 +0,0 @@ ---- -title: How Replication and Preloading Work ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -To work with replicated and preloaded regions, you should understand how their data is initialized and maintained in the cache. - -<a id="how_replication_works__section_C75BB463A0584491ABD982A55E5A050F"></a> -Replicated and preloaded regions are configured by using one of the `REPLICATE` region shortcut settings, or by setting the region attribute `data-policy` to `replicate`, `persistent-replicate`, or `preloaded`. - -## <a id="how_replication_works__section_B4E76BBCC6104A27BC0A8ECA6B9CDF91" class="no-quick-link"></a>Initialization of Replicated and Preloaded Regions - -At region creation, the system initializes the preloaded or replicated region with the most complete and up-to-date data set it can find. The system uses these data sources to initialize the new region, following this order of preference: - -1. Another replicated region that is already defined in the distributed system. -2. For persistent replicate only. Disk files, followed by a union of all copies of the region in the distributed cache. -3. For preloaded region only. Another preloaded region that is already defined in the distributed system. -4. The union of all copies of the region in the distributed cache. - -<img src="../../images_svg/distributed_replica_preload.svg" id="how_replication_works__image_5F50EBA30CE3408091F07A198F821741" class="image" /> - -While a region is being initialized from a replicated or preloaded region, if the source region crashes, the initialization starts over. - -If a union of regions is used for initialization, as in the figure, and one of the individual source regions goes away during the initialization (due to cache closure, member crash, or region destruction), the new region may contain a partial data set from the crashed source region. When this happens, there is no warning logged or exception thrown. The new region still has a complete set of the remaining members' regions. - -## <a id="how_replication_works__section_6BE7555A711E4CA490B02E58B5DDE396" class="no-quick-link"></a>Behavior of Replicated and Preloaded Regions After Initialization - -Once initialized, the preloaded region operates like the region with a `normal` `data-policy`, receiving distributions only for entries it has defined in the local cache. - -<img src="../../images_svg/distributed_preload.svg" id="how_replication_works__image_994CA599B1004D3F95E1BB7C4FAC2AEF" class="image" /> - -If the region is configured as a replicated region, it receives all new creations in the distributed region from the other members. This is the push distribution model. Unlike the preloaded region, the replicated region has a contract that states it will hold all entries that are present anywhere in the distributed region. - -<img src="../../images_svg/distributed_replica.svg" id="how_replication_works__image_2E7F3EB6213A47FEA3ABE32FD2CB1503" class="image" /> - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/locking_in_global_regions.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/locking_in_global_regions.html.md.erb b/geode-docs/developing/distributed_regions/locking_in_global_regions.html.md.erb deleted file mode 100644 index 2dc0c8a..0000000 --- a/geode-docs/developing/distributed_regions/locking_in_global_regions.html.md.erb +++ /dev/null @@ -1,109 +0,0 @@ ---- -title: Locking in Global Regions ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -In global regions, the system locks entries and the region during updates. You can also explicitly lock the region and its entries as needed by your application. Locking includes system settings that help you optimize performance and locking behavior between your members. - -<a id="locking_in_global_regions__section_065B3A57CCCA4F17821D170A312B6675"></a> -In regions with global scope, locking helps ensure cache consistency. - -Locking of regions and entries is done in two ways: - -1. **Implicit**. Geode automatically locks global regions and their data entries during most operations. Region invalidation and destruction do not acquire locks. -2. **Explicit**. You can use the API to explicitly lock the region and its entries. Do this to guarantee atomicity in tasks with multi-step distributed operations. The `Region` methods `org.apache.geode.cache.Region.getDistributedLock` and `org.apache.geode.cache.Region.getRegionDistributedLock` return instances of `java.util.concurrent.locks.Lock` for a region and a specified key. - - **Note:** - You must use the `Region` API to lock regions and region entries. Do not use the `DistributedLockService` in the `org.apache.geode.distributed` package. That service is available only for locking in arbitrary distributed applications. It is not compatible with the `Region` locking methods. - -## <a id="locking_in_global_regions__section_5B47F9C5C27A4B789A3498AC553BB1FB" class="no-quick-link"></a>Lock Timeouts - -Getting a lock on a region or entry is a two-step process of getting a lock instance for the entity and then using the instance to set the lock. Once you have the lock, you hold it for your operations, then release it for someone else to use. You can set limits on the time spent waiting to get a lock and the time spent holding it. Both implicit and explicit locking operations are affected by the timeouts: - -- The lock timeout limits the wait to get a lock. The cache attribute `lock-timeout` governs implicit lock requests. For explicit locking, specify the wait time through your calls to the instance of `java.util.concurrent.locks.Lock` returned from the `Region` API. You can wait a specific amount of time, return immediately either with or without the lock, or wait indefinitely. - - ``` pre - <cache lock-timeout="60"> - </cache> - ``` - - gfsh: - - ``` pre - gfsh>alter runtime --lock-timeout=60 - ``` - -- The lock lease limits how long a lock can be held before it is automatically released. A timed lock allows the application to recover when a member fails to release an obtained lock within the lease time. For all locking, this timeout is set with the cache attribute `lock-lease`. - - ``` pre - <cache lock-lease="120"> </cache> - ``` - - gfsh: - - ``` pre - gfsh>alter runtime --lock-lease=120 - ``` - -## <a id="locking_in_global_regions__section_031727F04D114B42944872360A386907" class="no-quick-link"></a>Optimize Locking Performance - -For each global region, one of the members with the region defined will be assigned the job of lock grantor. The lock grantor runs the lock service that receives lock requests from system members, queues them as needed, and grants them in the order received. - -The lock grantor is at a slight advantage over other members as it is the only one that does not have to send a message to request a lock. The grantorâs requests cost the least for the same reason. Thus, you can optimize locking in a region by assigning lock grantor status to the member that acquires the most locks. This may be the member that performs the most puts and thus requires the most implicit locks or this may be the member that performs many explicit locks. - -The lock grantor is assigned as follows: - -- Any member with the region defined that requests lock grantor status is assigned it. Thus at any time, the most recent member to make the request is the lock grantor. -- If no member requests lock grantor status for a region, or if the current lock grantor goes away, the system assigns a lock grantor from the members that have the region defined in their caches. - -You can request lock grantor status: - -1. At region creation through the `is-lock-grantor` attribute. You can retrieve this attribute through the region method, `getAttributes`, to see whether you requested to be lock grantor for the region. - **Note:** - The `is-lock-grantor` attribute does not change after region creation. - -2. After region creation through the region `becomeLockGrantor` method. Changing lock grantors should be done with care, however, as doing so takes cycles from other operations. In particular, be careful to avoid creating a situation where you have members vying for lock grantor status. - -## <a id="locking_in_global_regions__section_34661E38DFF9420B89C1A2B25F232D53" class="no-quick-link"></a>Examples - -These two examples show entry locking and unlocking. Note how the entryâs `Lock` object is obtained and then its lock method invoked to actually set the lock. The example program stores the entry lock information in a hash table for future reference. - -``` pre -/* Lock a data entry */ -HashMap lockedItemsMap = new HashMap(); -... - String entryKey = ... - if (!lockedItemsMap.containsKey(entryKey)) - { - Lock lock = this.currRegion.getDistributedLock(entryKey); - lock.lock(); - lockedItemsMap.put(name, lock); - } - ... -``` - -``` pre -/* Unlock a data entry */ - String entryKey = ... - if (lockedItemsMap.containsKey(entryKey)) - { - Lock lock = (Lock) lockedItemsMap.remove(name); - lock.unlock(); - } -``` http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/managing_distributed_regions.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/managing_distributed_regions.html.md.erb b/geode-docs/developing/distributed_regions/managing_distributed_regions.html.md.erb deleted file mode 100644 index 7d4cf37..0000000 --- a/geode-docs/developing/distributed_regions/managing_distributed_regions.html.md.erb +++ /dev/null @@ -1,64 +0,0 @@ ---- -title: Configure Distributed, Replicated, and Preloaded Regions ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Plan the configuration and ongoing management of your distributed, replicated, and preloaded regions, and configure the regions. - -<a id="configure_distributed_region__section_11E9E1B3EB5845D9A4FB226A992B8D0D"></a> -Before you begin, understand [Basic Configuration and Programming](../../basic_config/book_intro.html). - -1. Choose the region shortcut setting that most closely matches your region configuration. See **`org.apache.geode.cache.RegionShortcut`** or [Region Shortcuts](../../reference/topics/chapter_overview_regionshortcuts.html#concept_ymp_rkz_4dffhdfhk). To create a replicated region, use one of the `REPLICATE` shortcut settings. To create a preloaded region, set your region `data-policy` to `preloaded`. This `cache.xml` declaration creates a replicated region: - - ``` pre - <region-attributes refid="REPLICATE"> - </region-attributes> - ``` - - You can also use gfsh to configure a region. For example: - - ``` pre - gfsh>create region --name=regionA --type=REPLICATE - ``` - - See [Region Types](../region_options/region_types.html#region_types). - -2. Choose the level of distribution for your region. The region shortcuts in `RegionShortcut` for distributed regions use `distributed-ack` scope. If you need a different scope, set the `region-attributes` `scope` to `distributed-no-ack` or `global`. - - Example: - - ``` pre - <region-attributes refid="REPLICATE" scope="distributed-no-ack"> - </region-attributes> - ``` - -3. If you are using the `distributed-ack` scope, optionally enable concurrency checks for the region. - - Example: - - ``` pre - <region-attributes refid="REPLICATE" scope="distributed-ack" concurrency-checks-enabled="true"> - </region-attributes> - ``` - -4. If you are using `global` scope, program any explicit locking you need in addition to the automated locking provided by Geode. - -## <a id="configure_distributed_region__section_6F53FB58B8A84D0F8086AFDB08A649F9" class="no-quick-link"></a>Local Destroy and Invalidate in the Replicated Region - -Of all the operations that affect the local cache only, only local region destroy is allowed in a replicated region. Other operations are not configurable or throw exceptions. For example, you cannot use local destroy as the expiration action on a replicated region. This is because local operations like entry invalidation and destruction remove data from the local cache only. A replicated region would no longer be complete if data were removed locally but left intact. http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/distributed_regions/region_entry_versions.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/distributed_regions/region_entry_versions.html.md.erb b/geode-docs/developing/distributed_regions/region_entry_versions.html.md.erb deleted file mode 100644 index 9aa8dde..0000000 --- a/geode-docs/developing/distributed_regions/region_entry_versions.html.md.erb +++ /dev/null @@ -1,51 +0,0 @@ ---- -title: Consistency for Region Updates ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -<a id="topic_CF2798D3E12647F182C2CEC4A46E2045"></a> - - -Geode ensures that all copies of a region eventually reach a consistent state on all members and clients that host the region, including Geode members that distribute region events. - -- **[Consistency Checking by Region Type](../../developing/distributed_regions/how_region_versioning_works.html#topic_7A4B6C6169BD4B1ABD356294F744D236)** - - Geode performs different consistency checks depending on the type of region you have configured. - -- **[Configuring Consistency Checking](../../developing/distributed_regions/how_region_versioning_works.html#topic_B64891585E7F4358A633C792F10FA23E)** - - Geode enables consistency checking by default. You cannot disable consistency checking for persistent regions. For all other regions, you can explicitly enable or disable consistency checking by setting the `concurrency-checks-enabled` region attribute in `cache.xml` to "true" or "false." - -- **[Overhead for Consistency Checks](../../developing/distributed_regions/how_region_versioning_works.html#topic_0BDACA590B2C4974AC9C450397FE70B2)** - - Consistency checking requires additional overhead for storing and distributing version and timestamp information, as well as for maintaining destroyed entries for a period of time to meet consistency requirements. - -- **[How Consistency Checking Works for Replicated Regions](../../developing/distributed_regions/how_region_versioning_works.html#topic_C5B74CCDD909403C815639339AA03758)** - - Each region stores version and timestamp information for use in conflict detection. Geode members use the recorded information to detect and resolve conflicts consistently before applying a distributed update. - -- **[How Destroy and Clear Operations Are Resolved](../../developing/distributed_regions/how_region_versioning_works.html#topic_321B05044B6641FCAEFABBF5066BD399)** - - When consistency checking is enabled for a region, a Geode member does not immediately remove an entry from the region when an application destroys the entry. Instead, the member retains the entry with its current version stamp for a period of time in order to detect possible conflicts with operations that have occurred. The retained entry is referred to as a *tombstone*. Geode retains tombstones for partitioned regions and non-replicated regions as well as for replicated regions, in order to provide consistency. - -- **[Transactions with Consistent Regions](../../developing/distributed_regions/how_region_versioning_works.html#topic_32ACFA5542C74F3583ECD30467F352B0)** - - A transaction that modifies a region having consistency checking enabled generates all necessary version information for region updates when the transaction commits. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/cache_event_handler_examples.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/cache_event_handler_examples.html.md.erb b/geode-docs/developing/events/cache_event_handler_examples.html.md.erb deleted file mode 100644 index 85ccf46..0000000 --- a/geode-docs/developing/events/cache_event_handler_examples.html.md.erb +++ /dev/null @@ -1,155 +0,0 @@ ---- -title: Cache Event Handler Examples ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Some examples of cache event handlers. - -## <a id="cache_event_handler_examples__section_F2790678E9DE4A81B73A4B6346CB210B" class="no-quick-link"></a>Declaring and Loading an Event Handler with Parameters - -This declares an event handler for a region in the `cache.xml`. The handler is a cache listener designed to communicate changes to a DB2 database. The declaration includes the listenerâs parameters, which are the database path, username, and password. - -``` pre -<region name="exampleRegion"> - <region-attributes> - . . . - <cache-listener> - <class-name>JDBCListener</class-name> - <parameter name="url"> - <string>jdbc:db2:SAMPLE</string> - </parameter> - <parameter name="username"> - <string>gfeadmin</string> - </parameter> - <parameter name="password"> - <string>admin1</string> - </parameter> - </cache-listener> - </region-attributes> - </region> -``` - -This code listing shows part of the implementation of the `JDBCListener` declared in the `cache.xml`. This listener implements the `Declarable` interface. When an entry is created in the cache, this listenerâs `afterCreate` callback method is triggered to update the database. Here the listenerâs properties, provided in the `cache.xml`, are passed into the `Declarable.init` method and used to create a database connection. - -``` pre -. . . -public class JDBCListener -extends CacheListenerAdapter -implements Declarable { - public void afterCreate(EntryEvent e) { - . . . - // Initialize the database driver and connection using input parameters - Driver driver = (Driver) Class.forName(DRIVER_NAME).newInstance(); - Connection connection = - DriverManager.getConnection(_url, _username, _password); - System.out.println(_connection); - . . . - } - . . . - public void init(Properties props) { - this._url = props.getProperty("url"); - this._username = props.getProperty("username"); - this._password = props.getProperty("password"); - } -} -``` - -## <a id="cache_event_handler_examples__section_2B4275C1AE744794AAD22530E5ECA8CC" class="no-quick-link"></a>Installing an Event Handler Through the API - -This listing defines a cache listener using the `RegionFactory` method `addCacheListener`. - -``` pre -Region newReg = cache.createRegionFactory() - .addCacheListener(new SimpleCacheListener()) - .create(name); - -``` - -You can create a cache writer similarly, using the `RegionFactory` method `setCacheWriter`, like this: - -``` pre -Region newReg = cache.createRegionFactory() - .setCacheWriter(new SimpleCacheWriter()) - .create(name); - -``` - -## <a id="cache_event_handler_examples__section_C62E9535C43B4BC5A7AA7B8B4125D1EB" class="no-quick-link"></a>Installing Multiple Listeners on a Region - -XML: - -``` pre -<region name="exampleRegion"> - <region-attributes> - . . . - <cache-listener> - <class-name>myCacheListener1</class-name> - </cache-listener> - <cache-listener> - <class-name>myCacheListener2</class-name> - </cache-listener> - <cache-listener> - <class-name>myCacheListener3</class-name> - </cache-listener> - </region-attributes> -</region> -``` - -API: - -``` pre -CacheListener listener1 = new myCacheListener1(); -CacheListener listener2 = new myCacheListener2(); -CacheListener listener3 = new myCacheListener3(); - -Region nr = cache.createRegionFactory() - .initCacheListeners(new CacheListener[] - {listener1, listener2, listener3}) - .setScope(Scope.DISTRIBUTED_NO_ACK) - .create(name); -``` - -## <a id="cache_event_handler_examples__section_3AF3D7C9927F491F8BACDB72834E42AA" class="no-quick-link"></a>Installing a Write-Behind Cache Listener - -``` pre -//AsyncEventQueue with listener that performs WBCL work -<cache> - <async-event-queue id="sampleQueue" persistent="true" - disk-store-name="exampleStore" parallel="false"> - <async-event-listener> - <class-name>MyAsyncListener</class-name> - <parameter name="url"> - <string>jdbc:db2:SAMPLE</string> - </parameter> - <parameter name="username"> - <string>gfeadmin</string> - </parameter> - <parameter name="password"> - <string>admin1</string> - </parameter> - </async-event-listener> - </async-event-queue> - -// Add the AsyncEventQueue to region(s) that use the WBCL - <region name="data"> - <region-attributes async-event-queue-ids="sampleQueue"> - </region-attributes> - </region> -</cache> -``` http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/84cfbdfc/geode-docs/developing/events/chapter_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/developing/events/chapter_overview.html.md.erb b/geode-docs/developing/events/chapter_overview.html.md.erb deleted file mode 100644 index f5b46f4..0000000 --- a/geode-docs/developing/events/chapter_overview.html.md.erb +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Events and Event Handling ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one or more -contributor license agreements. See the NOTICE file distributed with -this work for additional information regarding copyright ownership. -The ASF licenses this file to You under the Apache License, Version 2.0 -(the "License"); you may not use this file except in compliance with -the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> - -Geode provides versatile and reliable event distribution and handling for your cached data and system member events. - -- **[How Events Work](../../developing/events/how_events_work.html)** - - Members in your Geode distributed system receive cache updates from other members through cache events. The other members can be peers to the member, clients or servers or other distributed systems. - -- **[Implementing Geode Event Handlers](../../developing/events/event_handler_overview.html)** - - You can specify event handlers for region and region entry operations and for administrative events. - -- **[Configuring Peer-to-Peer Event Messaging](../../developing/events/configure_p2p_event_messaging.html)** - - You can receive events from distributed system peers for any region that is not a local region. Local regions receive only local cache events. - -- **[Configuring Client/Server Event Messaging](../../developing/events/configure_client_server_event_messaging.html)** - - You can receive events from your servers for server-side cache events and query result changes. - -- **[Configuring Multi-Site (WAN) Event Queues](../../developing/events/configure_multisite_event_messaging.html)** - - In a multi-site (WAN) installation, Geode uses gateway sender queues to distribute events for regions that are configured with a gateway sender. AsyncEventListeners also use an asynchronous event queue to distribute events for configured regions. This section describes additional options for configuring the event queues that are used by gateway senders or AsyncEventListener implementations. - -
