http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/auto_serialization.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/auto_serialization.html.md.erb b/developing/data_serialization/auto_serialization.html.md.erb deleted file mode 100644 index 7e3dfa2..0000000 --- a/developing/data_serialization/auto_serialization.html.md.erb +++ /dev/null @@ -1,124 +0,0 @@ ---- -title: Using Automatic Reflection-Based PDX Serialization ---- - -You can configure your cache to automatically serialize and deserialize domain objects without having to add any extra code to them. - -<a id="auto_serialization__section_E2B7719D3C1545808CC21E0FDBD2D610"></a> -You can automatically serialize and deserialize domain objects without coding a `PdxSerializer` class. You do this by registering your domain objects with a custom `PdxSerializer` called `ReflectionBasedAutoSerializer` that uses Java reflection to infer which fields to serialize. - -You can also extend the ReflectionBasedAutoSerializer to customize its behavior. For example, you could add optimized serialization support for BigInteger and BigDecimal types. See [Extending the ReflectionBasedAutoSerializer](extending_the_autoserializer.html#concept_9E020566EE794A81A48A90BA798EC279) for details. - -**Note:** -Your custom PDX autoserializable classes cannot use the `com.gemstone` package. If they do, the classes will be ignored by the PDX auto serializer. - -<a id="auto_serialization__section_C69046B44729454F8CD464B0289EFDD8"></a> - -**Prerequisites** - -- Understand generally how to configure the Geode cache. -- Understand how PDX serialization works and how to configure your application to use `PdxSerializer`. - -<a id="auto_serialization__section_43F6E45FF69E470897FD9D002FBE896D"><strong>Procedure</strong></a> - -In your application where you manage data from the cache, provide the following configuration and code as appropriate: - -1. In the domain classes that you wish to autoserialize, make sure each class has a zero-arg constructor. For example: - - ``` pre - public PortfolioPdx(){} - ``` - -2. Using one of the following methods, set the PDX serializer to `ReflectionBasedAutoSerializer`. - 1. In gfsh, execute the following command prior to starting up any members that host data: - - ``` pre - gfsh>configure pdx --auto-serializable-classes=com\.company\.domain\..* - ``` - - By using gfsh, this configuration can propagated across the cluster through the [Cluster Configuration Service](../../configuring/cluster_config/gfsh_persist.html). - - 2. Alternately, in `cache.xml`: - - ``` pre - <!-- Cache configuration configuring auto serialization behavior --> - <cache> - <pdx> - <pdx-serializer> - <class-name> - org.apache.geode.pdx.ReflectionBasedAutoSerializer - </class-name> - <parameter name="classes"> - <string>com.company.domain.DomainObject</string> - </parameter> - </pdx-serializer> - </pdx> - ... - </cache> - ``` - - The parameter, `classes`, takes a comma-separated list of class patterns to define the domain classes to serialize. If your domain object is an aggregation of other domain classes, you need to register the domain object and each of those domain classes explicitly for the domain object to be serialized completely. - - 3. Using the Java API: - - ``` pre - Cache c = new CacheFactory() - .setPdxSerializer(new ReflectionBasedAutoSerializer("com.company.domain.DomainObject")) - .create(); - ``` - -3. Customize the behavior of the `ReflectionBasedAutoSerializer` using one of the following mechanisms: - - By using a class pattern string to specify the classes to auto-serialize and customize how the classes are serialized. Class pattern strings can be specified in the API by passing strings to the `ReflectionBasedAutoSerializer` constructor or by specifying them in cache.xml. See [Customizing Serialization with Class Pattern Strings](autoserialization_with_class_pattern_strings.html#concept_9B67BBE94B414B7EA63BD7E8D61D0312) for details. - - By creating a subclass of `ReflectionBasedAutoSerializer` and overriding specific methods. See [Extending the ReflectionBasedAutoSerializer](extending_the_autoserializer.html#concept_9E020566EE794A81A48A90BA798EC279) for details. - -4. If desired, configure the `ReflectionBasedAutoSerializer` to check the portability of the objects it is passed before it tries to autoserialize them. When this flag is set to true, the `ReflectionBasedAutoSerializer` will throw a `NonPortableClassException` error when trying to autoserialize a non-portable object. To set this, use the following configuration: - - In gfsh, use the following command: - - ``` pre - gfsh>configure pdx --portable-auto-serializable-classes=com\.company\.domain\..* - ``` - - By using gfsh, this configuration can propagated across the cluster through the [Cluster Configuration Service](../../configuring/cluster_config/gfsh_persist.html). - - In cache.xml: - - ``` pre - <!-- Cache configuration configuring auto serialization behavior --> - <cache> - <pdx> - <pdx-serializer> - <class-name> - org.apache.geode.pdx.ReflectionBasedAutoSerializer - </class-name> - <parameter name="classes"> - <string>com.company.domain.DomainObject</string> - </parameter> - <parameter name="check-portability"> - <string>true</string> - </parameter> - </pdx-serializer> - </pdx> - ... - </cache> - ``` - - Using the Java API: - - ``` pre - Cache c = new CacheFactory() - .setPdxSerializer(new ReflectionBasedAutoSerializer(true,"com.company.domain.DomainObject")) - .create(); - ``` - -For each domain class you provide, all fields are considered for serialization except those defined as `static` or `transient` and those you explicitly exclude using the class pattern strings. - -**Note:** -The `ReflectionBasedAutoSerializer` traverses the given domain object's class hierarchy to retrieve all fields to be considered for serialization. So if `DomainObjectB` inherits from `DomainObjectA`, you only need to register `DomainObjectB` to have all of `DomainObjectB` serialized. - -- **[Customizing Serialization with Class Pattern Strings](../../developing/data_serialization/autoserialization_with_class_pattern_strings.html)** - - Use class pattern strings to name the classes that you want to serialize using Geode's reflection-based autoserializer and to specify object identity fields and to specify fields to exclude from serialization. - -- **[Extending the ReflectionBasedAutoSerializer](../../developing/data_serialization/extending_the_autoserializer.html)** - - You can extend the `ReflectionBasedAutoSerializer` to handle serialization in a customized manner. This section provides an overview of the available method-based customization options and an example of extending the serializer to support BigDecimal and BigInteger types. - -
http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/autoserialization_with_class_pattern_strings.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/autoserialization_with_class_pattern_strings.html.md.erb b/developing/data_serialization/autoserialization_with_class_pattern_strings.html.md.erb deleted file mode 100644 index ba18558..0000000 --- a/developing/data_serialization/autoserialization_with_class_pattern_strings.html.md.erb +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: Customizing Serialization with Class Pattern Strings ---- - -Use class pattern strings to name the classes that you want to serialize using Geode's reflection-based autoserializer and to specify object identity fields and to specify fields to exclude from serialization. - -The class pattern strings used to configured the `ReflectionBasedAutoSerializer` are standard regular expressions. For example, this expression would select all classes defined in the `com.company.domain` package and its subpackages: - -``` pre -com\.company\.domain\..* -``` - -You can augment the pattern strings with a special notation to define fields to exclude from serialization and to define fields to mark as PDX identity fields. The full syntax of the pattern string is: - -``` pre -<class pattern> [# (identity|exclude) = <field pattern>]... [, <class pattern>...] -``` - -The following example pattern string sets these PDX serialization criteria: - -- Classes with names matching the pattern `com.company.DomainObject.*` are serialized. In those classes, fields beginning with `id` are marked as identity fields and fields named `creationDate` are not serialized. -- The class `com.company.special.Patient` is serialized. In the class, the field, `ssn` is marked as an identity field - -``` pre -com.company.DomainObject.*#identity=id.*#exclude=creationDate, -com.company.special.Patient#identity=ssn -``` - -**Note:** -There is no association between the `identity` and `exclude` options, so the pattern above could also be expressed as: - -``` pre -com.company.DomainObject.*#identity=id.*, com.company.DomainObject.*#exclude=creationDate, -com.company.special.Patient#identity=ssn - -``` - -**Note:** -The order of the patterns is not relevant. All defined class patterns are used when determining whether a field should be considered as an identity field or should be excluded. - -Examples: - -- This XML uses the example pattern shown above: - - ``` pre - <parameter name="classes"> - <string>com.company.DomainObject.*#identity=id.*#exclude=creationDate, - com.company.special.Patient#identity=ssn</string> - </parameter> - ``` - -- This application code sets the same pattern: - - ``` pre - classPatterns.add("com.company.DomainObject.*#identity=id.*#exclude=creationDate, - com.company.special.Patient#identity=ssn"); - ``` - -- This application code has the same effect: - - ``` pre - Cache c = new CacheFactory().set("cache-xml-file", cacheXmlFileName) - .setPdxSerializer(new ReflectionBasedAutoSerializer("com.foo.DomainObject*#identity=id.*", - "com.company.DomainObject.*#exclude=creationDate","com.company.special.Patient#identity=ssn")) - .create(); - ``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/chapter_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/chapter_overview.html.md.erb b/developing/data_serialization/chapter_overview.html.md.erb deleted file mode 100644 index 5ef0877..0000000 --- a/developing/data_serialization/chapter_overview.html.md.erb +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: Data Serialization ---- - -Data that you manage in Geode must be serialized and deserialized for storage and transmittal between processes. You can choose among several options for data serialization. - -- **[Overview of Data Serialization](../../developing/data_serialization/data_serialization_options.html)** - - Geode offers serialization options other than Java serialization that give you higher performance and greater flexibility for data storage, transfers, and language types. - -- **[Geode PDX Serialization](../../developing/data_serialization/gemfire_pdx_serialization.html)** - - Geode's Portable Data eXchange (PDX) is a cross-language data format that can reduce the cost of distributing and serializing your objects. PDX stores data in named fields that you can access individually, to avoid the cost of deserializing the entire data object. PDX also allows you to mix versions of objects where you have added or removed fields. - -- **[Geode Data Serialization (DataSerializable and DataSerializer)](../../developing/data_serialization/gemfire_data_serialization.html)** - - Geode's `DataSerializable` interface gives you quick serialization of your objects. - -- **[Standard Java Serialization](../../developing/data_serialization/java_serialization.html)** - - You can use standard Java serialization for data you only distribute between Java applications. If you distribute your data between non-Java clients and Java servers, you need to do additional programming to get the data between the various class formats. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/data_serialization_options.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/data_serialization_options.html.md.erb b/developing/data_serialization/data_serialization_options.html.md.erb deleted file mode 100644 index 7402ee7..0000000 --- a/developing/data_serialization/data_serialization_options.html.md.erb +++ /dev/null @@ -1,51 +0,0 @@ ---- -title: Overview of Data Serialization ---- - -Geode offers serialization options other than Java serialization that give you higher performance and greater flexibility for data storage, transfers, and language types. - -<a id="data_serialization_options__section_B1BDB0E7F6814DFD8BACD8D8C5CAA81B"></a> -All data that Geode moves out of the local cache must be serializable. However, you do not necessarily need to implement `java.io.Serializable` since other serialization options are available in Geode. Region data that must be serializable falls under the following categories: - -- Partitioned regions -- Distributed regions -- Regions that are persisted or overflowed to disk -- Server or client regions in a client/server installation -- Regions configured with a gateway sender for distributing events in a multi-site installation -- Regions that receive events from remote caches -- Regions that provide function arguments and results - -**Note:** -If you are storing objects with the [HTTP Session Management Modules](../../tools_modules/http_session_mgmt/chapter_overview.html), these objects must be serializable since they are serialized before being stored in the region. - -To minimize the cost of serialization and deserialization, Geode avoids changing the data format whenever possible. This means your data might be stored in the cache in serialized or deserialized form, depending on how you use it. For example, if a server acts only as a storage location for data distribution between clients, it makes sense to leave the data in serialized form, ready to be transmitted to clients that request it. Partitioned region data is always initially stored in serialized form. - -## <a id="data_serialization_options__section_691C2CF5A4E24D599070A7AADEDF2BEC" class="no-quick-link"></a>Data Serialization Options - -<a id="data_serialization_options__section_44CC2DEEDA0F41D49D416ABA921A6436"></a> - -With Geode, you have the option to serialize your domain objects automatically or to implement serialization using one of Geode's interfaces. Enabling automatic serialization means that domain objects are serialized and deserialized without your having to make any code changes to those objects. This automatic serialization is performed by registering your domain objects with a custom `PdxSerializer` called the `ReflectionBasedAutoSerializer`, which uses Java reflection to infer which fields to serialize. - -If autoserialization does not meet your needs, you can serialize your objects by implementing one of the Geode interfaces, `PdxSerializable` or `DataSerializable`. You can use these interfaces to replace any standard Java data serialization for better performance. If you cannot or do not want to modify your domain classes, each interface has an alternate serializer class, `PdxSerializer` and `DataSerializer`. To use these, you create your custom serializer class and then associate it with your domain class in the Geode cache configuration. - -Geode Data serialization is about 25% faster than PDX serialization, however using PDX serialization will help you to avoid the even larger costs of performing deserialization. - -<a id="data_serialization_options__section_993B4A298874459BB4A8A0A9811854D9"></a><a id="data_serialization_options__table_ccf00c9f-9b98-47f7-ab30-3d23ecaff0a1"></a> - -| Capability | Geode Data Serializable | Geode PDX Serializable | -|----------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------|-----------------------------------------------------| -| Implements Java Serializable. | X |  | -| Handles multiple versions of application domain objects, providing the versions differ by the addition or subtraction of fields. |  | X | -| Provides single field access of serialized data, without full deserialization - supported also for OQL querying. |  | X | -| Automatically ported to other languages by Geode |  | X | -| Works with .NET clients. | X | X | -| Works with C++ clients. | X | X | -| Works with Geode delta propagation. | X | X (See note below.) | - -<span class="tablecap">**Table 1.** Serialization Options: Comparison of Features</span> - -**Note:** By default, you can use Geode delta propagation with PDX serialization. However, delta propagation will not work if you have set the Geode property `read-serialized` to "true". In terms of deserialization, to apply a change delta propagation requires a domain class instance and the `fromDelta `method. If you have set `read-serialized` to true, then you will receive a `PdxInstance` instead of a domain class instance and `PdxInstance` does not have the `fromDelta` method required for delta propagation. - -## <a id="data_serialization_options__section_D90C2C09B95C40B6803CF202CF8008BF" class="no-quick-link"></a>Differences between Geode Serialization (PDX or Data Serializable) and Java Serialization - -Geode serialization (either PDX Serialization or Data Serialization) does not support circular object graphs whereas Java serialization does. In Geode serialization, if the same object is referenced more than once in an object graph, the object is serialized for each reference, and deserialization produces multiple copies of the object. By contrast in this situation, Java serialization serializes the object once and when deserializing the object, it produces one instance of the object with multiple references. http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/extending_the_autoserializer.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/extending_the_autoserializer.html.md.erb b/developing/data_serialization/extending_the_autoserializer.html.md.erb deleted file mode 100644 index 35cbe66..0000000 --- a/developing/data_serialization/extending_the_autoserializer.html.md.erb +++ /dev/null @@ -1,106 +0,0 @@ ---- -title: Extending the ReflectionBasedAutoSerializer ---- - -You can extend the `ReflectionBasedAutoSerializer` to handle serialization in a customized manner. This section provides an overview of the available method-based customization options and an example of extending the serializer to support BigDecimal and BigInteger types. - -## <a id="concept_9E020566EE794A81A48A90BA798EC279__section_378C0C68A1B342DD9D754DDBDC4874B3" class="no-quick-link"></a>Reasons to Extend the ReflectionBasedAutoSerializer - -One of the main use cases for extending the `ReflectionBasedAutoSerializer` is that you want it to handle an object that would currently need to be handled by standard Java serialization. There are several issues with having to use standard Java serialization that can be addressed by extending the PDX `ReflectionBasedAutoSerializer`. - -- Each time we transition from a Geode serialized object to an object that will be Java I/O serialized, extra data must get serialized. This can cause a great deal of serialization overhead. This is why it is worth extending the `ReflectionBasedAutoSerializer` to handle any classes that normally would have to be Java I/O serialized. -- Expanding the number of classes that can use the `ReflectionBasedAutoSerializer` is beneficial when you encounter object graphs. After we use Java I/O serialization on an object, any objects under that object in the object graph will also have to be Java I/O serialized. This includes objects that normally would have been serialized using PDX or `DataSerializable`. -- If standard Java I/O serialization is done on an object and you have enabled check-portability, then an exception will be thrown. Even if you are not concerned with the object's portability, you can use this flag to find out what classes would use standard Java serialization (by getting an exception on them) and then enhancing your auto serializer to handle them. - -## <a id="concept_9E020566EE794A81A48A90BA798EC279__section_A739691C60FB4EB291289AADCD66C675" class="no-quick-link"></a>Overriding ReflectionBasedAutoSerializer Behavior - -You can customize the specific behaviors in `ReflectionBasedAutoSerializer` by overriding the following methods: - -- **`isClassAutoSerialized`** customizes which classes to autoserialize. -- **`isFieldIncluded`** specifies which fields of a class to autoserialize. -- **`getFieldName`** defines the specific field names that will be generated during autoserialization. -- **`isIdentifyField`** controls which field is marked as the identity field. Identity fields are used when a PdxInstance computes its hash code to determine whether it is equal to another object. -- **`getFieldType`** determines the field type that will be used when autoserializing the given field. -- **`transformFieldValue`** controls whether specific field values of a PDX object can be transformed during serialization. -- **`writeTransform`** controls what field value is written during auto serialization. -- **`readTransform`** controls what field value is read during auto deserialization. - -These methods are only called the first time the `ReflectionBasedAutoSerializer` sees a new class. The results will be remembered and used the next time the same class is seen. - -For details on these methods and their default behaviors, see the JavaDocs on [ReflectionBasedAutoSerializer](/releases/latest/javadoc/org/apache/geode/pdx/ReflectionBasedAutoSerializer.html) for details. - -## <a id="concept_9E020566EE794A81A48A90BA798EC279__section_7C4CC39FD82A48A9B5F8376522078192" class="no-quick-link"></a>Example of Optimizing Autoserialization of BigInteger and BigDecimal Types - -This section provides an example of extending the `ReflectionBasedAutoSerializer` to optimize the automatic serialization of BigInteger and BigDecimal types. - -The following code sample illustrates a subclass of the `ReflectionBasedAutoSerializer` that optimizes BigInteger and BigDecimal autoserialization: - -``` pre -public static class BigAutoSerializer extends ReflectionBasedAutoSerializer { - public BigAutoSerializer(Boolean checkPortability, string⦠patterns) { - super(checkPortability, patterns); -} - -@Override -public FieldType get FieldType(Field f, Class<?> clazz) { - if (f.getType().equals(BigInteger.class)) { - return FieldType.BYTE_ARRAY; - } else if (f.getType().equals(BigDecimal.class)) { - return FieldType.STRING; - } else { - return super.getFieldType(f, clazz); - } - } -@Override - public boolean transformFieldValue(Field f, Class<?> clazz) { - if (f.getType().equals(BigInteger.class)) { - return true; - } else if (f.getType().equals(BigDecimal.class)) { - return true; - } else { - return super.transformFieldValue(f, clazz); - } - } - -@Override - public Object writeTransform(Field f, Class<?> clazz, Object originalValue) { - if (f.getType().equals(BigInteger.class)) { - byte[] result = null; - if (originalValue != null) { - BigInteger bi = (BigInteger)originalValue; - result = bi.toByteArray(); - } - return result; - } else if (f.getType().equals(BigDecimal.class)) { - Object result = null; - if (originalValue != null) { - BigDecimal bd = (BigDecimal)originalValue; - result = bd.toString(); - } - return result; - } else { - return super.writeTransform(f, clazz, originalValue); - } - } - -@Override - public Object readTransform(Field f, Class<?> clazz, Object serializedValue) { - if (f.getType().equals(BigInteger.class)) { - BigInteger result = null; - if (serializedValue != null) { - result = new BigInteger((byte[])serializedValue); - } - return result; - } else if (f.getType().equals(BigDecimal.class)) { - BigDecimal result = null; - if (serializedValue != null) { - result = new BigDecimal((String)serializedValue); - } - return result; - } else { - return super.readTransform(f, clazz, serializedValue); - } - } - - } -``` http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/gemfire_data_serialization.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/gemfire_data_serialization.html.md.erb b/developing/data_serialization/gemfire_data_serialization.html.md.erb deleted file mode 100644 index 58aeb7e..0000000 --- a/developing/data_serialization/gemfire_data_serialization.html.md.erb +++ /dev/null @@ -1,35 +0,0 @@ ---- -title: Geode Data Serialization (DataSerializable and DataSerializer) ---- - -Geode's `DataSerializable` interface gives you quick serialization of your objects. - -## <a id="gemfire_data_serialization__section_0C84D6BF5E9748CB865E6BB944A077DE" class="no-quick-link"></a>Data Serialization with the DataSerializable Interface - -Geode's `DataSerializable` interface gives you faster and more compact data serialization than the standard Java serialization or Geode PDX serialization. However, while Geode `DataSerializable` interface is generally more performant than Geode's `PdxSerializable`, it requires full deserialization on the server and then reserialization to send the data back to the client. - -You can further speed serialization by registering the instantiator for your `DataSerializable` class through `Instantiator`, eliminating the need for reflection to find the right serializer. You can provide your own serialization through the API. - -The recommended way to register your custom `Instantiator` is by specifying it in the `serialization-registration` element of cache.xml. - -For more information, see the online Java documentation for `DataSerializable` and `DataSerializer`. - -**Example cache.xml:** - -The following provides an example of how to register an instantiator using cache.xml. - -``` pre -<serialization-registration> -<instantiator id="30"> - <class-name>com.package.MyClass</class-name> -</instantiator> -</serialization-registration> -``` - -In addition to speeding standard object serialization, you can use the `DataSerializable` interface to serialize any custom objects you store in the cache. - -## <a id="gemfire_data_serialization__section_B21408E7090C41B08BF300146F87648B" class="no-quick-link"></a>Serializing Your Domain Object with DataSerializer - -You can also use `DataSerializer` to serialize domain objects. It serializes data in the same way as `DataSerializable` but allows you to serialize classes without modifying the domain class code. - -See the JavaDocs on [DataSerializable](/releases/latest/javadoc/org/apache/geode/DataSerializable.html) and [DataSerializer](/releases/latest/javadoc/org/apache/geode/DataSerializer.html) for more information. http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/gemfire_pdx_serialization.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/gemfire_pdx_serialization.html.md.erb b/developing/data_serialization/gemfire_pdx_serialization.html.md.erb deleted file mode 100644 index 1053618..0000000 --- a/developing/data_serialization/gemfire_pdx_serialization.html.md.erb +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: Geode PDX Serialization ---- - -Geode's Portable Data eXchange (PDX) is a cross-language data format that can reduce the cost of distributing and serializing your objects. PDX stores data in named fields that you can access individually, to avoid the cost of deserializing the entire data object. PDX also allows you to mix versions of objects where you have added or removed fields. - -- **[Geode PDX Serialization Features](../../developing/data_serialization/PDX_Serialization_Features.html)** - - Geode PDX serialization offers several advantages in terms of functionality. - -- **[High Level Steps for Using PDX Serialization](../../developing/data_serialization/use_pdx_high_level_steps.html)** - - To use PDX serialization, you can configure and use Geode's reflection-based autoserializer, or you can program the serialization of your objects by using the PDX interfaces and classes. - -- **[Using Automatic Reflection-Based PDX Serialization](../../developing/data_serialization/auto_serialization.html)** - - You can configure your cache to automatically serialize and deserialize domain objects without having to add any extra code to them. - -- **[Serializing Your Domain Object with a PdxSerializer](../../developing/data_serialization/use_pdx_serializer.html)** - - For a domain object that you cannot or do not want to modify, use the `PdxSerializer` class to serialize and deserialize the object's fields. You use one `PdxSerializer` implementation for the entire cache, programming it for all of the domain objects that you handle in this way. - -- **[Implementing PdxSerializable in Your Domain Object](../../developing/data_serialization/use_pdx_serializable.html)** - - For a domain object with source that you can modify, implement the `PdxSerializable` interface in the object and use its methods to serialize and deserialize the object's fields. - -- **[Programming Your Application to Use PdxInstances](../../developing/data_serialization/program_application_for_pdx.html)** - - A `PdxInstance` is a light-weight wrapper around PDX serialized bytes. It provides applications with run-time access to fields of a PDX serialized object. - -- **[Adding JSON Documents to the Geode Cache](../../developing/data_serialization/jsonformatter_pdxinstances.html)** - - The `JSONFormatter` API allows you to put JSON formatted documents into regions and retrieve them later by storing the documents internally as PdxInstances. - -- **[Using PdxInstanceFactory to Create PdxInstances](../../developing/data_serialization/using_PdxInstanceFactory.html)** - - You can use the `PdxInstanceFactory` interface to create a `PdxInstance` from raw data when the domain class is not available on the server. - -- **[Persisting PDX Metadata to Disk](../../developing/data_serialization/persist_pdx_metadata_to_disk.html)** - - Geode allows you to persist PDX metadata to disk and specify the disk store to use. - -- **[Using PDX Objects as Region Entry Keys](../../developing/data_serialization/using_pdx_region_entry_keys.html)** - - Using PDX objects as region entry keys is highly discouraged. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/java_serialization.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/java_serialization.html.md.erb b/developing/data_serialization/java_serialization.html.md.erb deleted file mode 100644 index 14b9ea3..0000000 --- a/developing/data_serialization/java_serialization.html.md.erb +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Standard Java Serialization ---- - -You can use standard Java serialization for data you only distribute between Java applications. If you distribute your data between non-Java clients and Java servers, you need to do additional programming to get the data between the various class formats. - -<a id="java_serialization__section_AD2524E4E3C145D4A2CEB379DD8E9389"></a> -Standard Java types are serializable by definition. For your domain classes, implement `java.io.Serializable`, then make sure to mark your transient and static variables as needed for your objects. For information, see the online documentation for `java.io.Serializable` for your Java version. - -Mixing `DataSerializable` with `Serializable` or `PdxSerializable` use on the same data can result in increased memory use and lower throughput than using just `Serializable` on the entire data, especially if the `Serializable` entries are in collections. The bigger the data collection, the lower the throughput as the metadata for the collection entries is not shared when using `DataSerializable`. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/jsonformatter_pdxinstances.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/jsonformatter_pdxinstances.html.md.erb b/developing/data_serialization/jsonformatter_pdxinstances.html.md.erb deleted file mode 100644 index 59ceb3c..0000000 --- a/developing/data_serialization/jsonformatter_pdxinstances.html.md.erb +++ /dev/null @@ -1,29 +0,0 @@ ---- -title: Adding JSON Documents to the Geode Cache ---- - -The `JSONFormatter` API allows you to put JSON formatted documents into regions and retrieve them later by storing the documents internally as PdxInstances. - -Geode now supports the use of JSON formatted documents natively. When you add a JSON document to a Geode cache, you call the JSONFormatter APIs to transform them into the PDX format (as a `PdxInstance`), which enables Geode to understand the JSON document at a field level. - -In terms of querying and indexing, because the documents are stored internally as PDX, applications can index on any field contained inside the JSON document including any nested field (within JSON objects or JSON arrays.) Any queries run on these stored documents will return PdxInstances as results. To update a JSON document stored in Geode , you can execute a function on the PdxInstance. - -You can then use the `JSONFormatter` to convert the PdxInstance results back into the JSON document. - -`JSONFormatter` uses a streaming parser ([Jackson](http://wiki.fasterxml.com/JacksonHome), JSON processor) to turn JSON documents into the optimized PDX format. To use the JSONFormatter, make sure that `$GEMFIRE/lib/server-dependencies.jar` is available in your application's CLASSPATH. - -The `JSONFormatter` class has four static methods that are used to convert JSON document into PdxInstances and then to convert those PdxInstances back into JSON document. - -You need to call the following methods before putting any JSON document into the Geode region: - -- `fromJSON`. Creates a PdxInstance from a JSON byte array. Returns the PdxInstance. -- `fromJSON`. Creates a PdxInstance from a JSON string. Returns the PdxInstance. - -After putting the JSON document into a region as a PdxInstance, you can execute standard Geode queries and create indexes on the JSON document in the same manner you would query or index any other Geode PdxInstance. - -After executing a Geode query or calling `region.get`, you can use the following methods to convert a PdxInstance back into the JSON format: - -- `toJSON`. Reads a PdxInstance and returns a JSON string. -- `toJSONByteArray`. Reads a PdxInstance and returns a JSON byte array. - -For more information on using the JSONFormatter, see the Java API documentation for `org.apache.geode.pdx.JSONFormatter`. http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/persist_pdx_metadata_to_disk.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/persist_pdx_metadata_to_disk.html.md.erb b/developing/data_serialization/persist_pdx_metadata_to_disk.html.md.erb deleted file mode 100644 index 37b11fc..0000000 --- a/developing/data_serialization/persist_pdx_metadata_to_disk.html.md.erb +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: Persisting PDX Metadata to Disk ---- - -Geode allows you to persist PDX metadata to disk and specify the disk store to use. - -<a id="persist_pdx_metadata_to_disk__section_7F357A8E56B54BFB9A5778C0F89E034E"></a> -**Prerequisites** - -- Understand generally how to configure the Geode cache. See [Basic Configuration and Programming](../../basic_config/book_intro.html). -- Understand how Geode disk stores work. See [Disk Storage](../../managing/disk_storage/chapter_overview.html). - -**Procedure** - -1. Set the `<pdx>` attribute `persistent` to true in your cache configuration. This is required for caches that use PDX with persistent regions and with regions that use a gateway sender to distribute events across a WAN.. Otherwise, it is optional. -2. (Optional) If you want to use a disk store that is not the Geode default disk store, set the `<pdx>` attribute `disk-store-name` to the name of your non-default disk store. - **Note:** - If you are using PDX serialized objects as region entry keys and you are using persistent regions, then you must configure your PDX disk store to be a different one than the disk store used by the persistent regions. - -3. (Optional) If you later want to rename the PDX types that are persisted to disk, you can do so on your offline disk-stores by executing the `pdx rename` command. See [pdx rename](../../tools_modules/gfsh/command-pages/pdx.html). - -**Example cache.xml:** - -This example `cache.xml` enables PDX persistence and sets a non-default disk store in a server cache configuration: - -``` pre - <pdx read-serialized="true" - persistent="true" disk-store-name="SerializationDiskStore"> - <pdx-serializer> - <class-name>pdxSerialization.defaultSerializer</class-name> - </pdx-serializer> - </pdx> - <region ... -``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/program_application_for_pdx.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/program_application_for_pdx.html.md.erb b/developing/data_serialization/program_application_for_pdx.html.md.erb deleted file mode 100644 index 9bfbe12..0000000 --- a/developing/data_serialization/program_application_for_pdx.html.md.erb +++ /dev/null @@ -1,90 +0,0 @@ ---- -title: Programming Your Application to Use PdxInstances ---- - -A `PdxInstance` is a light-weight wrapper around PDX serialized bytes. It provides applications with run-time access to fields of a PDX serialized object. - -<a id="program_application_for_pdx__section_7F357A8E56B54BFB9A5778C0F89E034E"></a> -You can configure your cache to return a `PdxInstance` when a PDX serialized object is deserialized instead of deserializing the object to a domain class. You can then program your application code that reads your entries to handle `PdxInstance`s fetched from the cache. - -**Note:** -This applies only to entry retrieval that you explicitly code using methods like `EntryEvent.getNewValue` and `Region.get`, as you do inside functions or in cache listener code. This does not apply to querying because the query engine retrieves the entries and handles object access for you. - -If you configure your cache to allow PDX serialized reads, a fetch from the cache returns the data in the form it is found. If the object is not serialized, the fetch returns the domain object. If the object is serialized, the fetch returns the `PdxInstance` for the object. - -**Note:** -If you are using `PdxInstance`s, you cannot use delta propagation to apply changes to PDX serialized objects. - -For example, in client/server applications that are programmed and configured to handle all data activity from the client, PDX serialized reads done on the server side will always return a `PdxInstance`. This is because all of data is serialized for transfer from the client, and you are not performing any server-side activities that would deserialize the objects in the server cache. - -In mixed situations, such as where a server cache is populated from client operations and also from data loads done on the server side, fetches done on the server can return a mix of `PdxInstance`s and domain objects. - -When fetching data in a cache with PDX serialized reads enabled, the safest approach is to code to handle both types, receiving an `Object` from the fetch operation, checking the type and casting as appropriate. However, if you know that the class is not available in the JVM, then you can avoid performing the type check. - -`PdxInstance` overrides any custom implementation you might have coded for your object's `equals` and `hashcode` methods. Make sure you have marked at least one identity field when writing PDX serialized objects. If you do not set as least one identity field, then the PdxInstance`equals` and `hashCode` methods will use all PDX fields to compare objects and consequently, will not perform as well. - -<a id="program_application_for_pdx__section_03F8918B4CAD49CCA1823FAAE25D53A8"></a> - -**Prerequisites** - -- Understand generally how to configure the Geode cache. See [Basic Configuration and Programming](../../basic_config/book_intro.html#basic_config_management). - -<a id="program_application_for_pdx__section_B3C7C7629DFD4483B32B27F84D64DFCF"></a> - -**Procedure** - -In your application where you fetch data from the cache, provide the following configuration and code as appropriate: - -1. In the cache.xml file of the member where entry fetches are run, set the `<pdx>` `read-serialized` attribute to true. - Data is not necessarily accessed on the member that you have coded for it. For example, if a client application runs a function on a server, the actual data access is done on the server, so you set `read-serialized` to true on the server. - - For example: - - ``` pre - // Cache configuration setting PDX read behavior - <cache> - <pdx read-serialized="true" /> - ... - </cache> - - ``` - -2. Write the application code that fetches data from the cache to handle a `PdxInstance`. If you are sure you will only retrieve `PdxInstance`s from the cache, you can code only for that. In many cases, a `PdxInstance` or a domain object may be returned from your cache entry retrieval operation, so you should check the object type and handle each possible type. - - For example: - - ``` pre - // put/get code with serialized read behavior - // put is done as normal - myRegion.put(myKey, myPdxSerializableObject); - - // get checks Object type and handles each appropriately - Object myObject = myRegion.get(myKey); - if (myObject instanceof PdxInstance) { - // get returned PdxInstance instead of domain object - PdxInstance myPdxInstance = (PdxInstance)myObject; - - // PdxInstance.getField deserializes the field, but not the object - String fieldValue = myPdxInstance.getField("stringFieldName"); - - // Update a field and put it back into the cache - // without deserializing the entire object - WritablePdxInstance myWritablePdxI = myPdxInstance.createWriter(); - myWritablePdxI.setField("fieldName", fieldValue); - region.put(key, myWritablePdxI); - - // Deserialize the entire object if needed, from the PdxInstance - DomainClass myPdxObject = (DomainClass)myPdxInstance.getObject(); - } - else if (myObject instanceof DomainClass) { - // get returned instance of domain object - // code to handle domain object instance - ... - } - ... - ``` - - **Note:** - Due to a limitation with PDX, if your PDX-enabled cache contains TreeSet domain objects, you should implement a Comparator that can handle both your domain objects and PdxInstance objects. You will also need to make the domain classes available on the server. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/use_pdx_high_level_steps.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/use_pdx_high_level_steps.html.md.erb b/developing/data_serialization/use_pdx_high_level_steps.html.md.erb deleted file mode 100644 index f47e81f..0000000 --- a/developing/data_serialization/use_pdx_high_level_steps.html.md.erb +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: High Level Steps for Using PDX Serialization ---- - -To use PDX serialization, you can configure and use Geode's reflection-based autoserializer, or you can program the serialization of your objects by using the PDX interfaces and classes. - -<a id="concept_A7C8890826394B4293C036DD739835BD__section_7F357A8E56B54BFB9A5778C0F89E034E"></a> -Optionally, program your application code to deserialize individual fields out of PDX representations of your serialized objects. You may also need to persist your PDX metadata to disk for recovery on startup. - -**Procedure** - -1. Use one of these serialization options for each object type that you want to serialize using PDX serialization: - - [Using Automatic Reflection-Based PDX Serialization](auto_serialization.html) - - [Serializing Your Domain Object with a PdxSerializer](use_pdx_serializer.html) - - [Implementing PdxSerializable in Your Domain Object](use_pdx_serializable.html) - -2. To ensure that your servers do not need to load the application classes, set the `pdx` `read-serialized` attribute to true. In gfsh, execute the following command before starting up your servers: - - ``` pre - gfsh>configure pdx --read-serialized=true - ``` - - By using gfsh, this configuration can propagated across the cluster through the [Cluster Configuration Service](../../configuring/cluster_config/gfsh_persist.html). Alternately, you would need to configure `pdx read-serialized` in each server's `cache.xml` file. - -3. If you are storing any Geode data on disk, then you must configure PDX serialization to use persistence. See [Persisting PDX Metadata to Disk](persist_pdx_metadata_to_disk.html) for more information. -4. (Optional) Wherever you run explicit application code to retrieve and manage your cached entries, you may want to manage your data objects without using full deserialization. To do this, see [Programming Your Application to Use PdxInstances](program_application_for_pdx.html). - -## PDX and Multi-Site (WAN) Deployments - -For multisite (WAN) installations only-- if you will use PDX serialization in any of your WAN-enabled regions, for each distributed system, you must choose a unique integer between 0 (zero) and 255 and set the `distributed-system-id` in every member's `gemfire.properties` file. See [Configuring a Multi-site (WAN) System](../../topologies_and_comm/multi_site_configuration/setting_up_a_multisite_system.html). - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/use_pdx_serializable.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/use_pdx_serializable.html.md.erb b/developing/data_serialization/use_pdx_serializable.html.md.erb deleted file mode 100644 index 948112b..0000000 --- a/developing/data_serialization/use_pdx_serializable.html.md.erb +++ /dev/null @@ -1,98 +0,0 @@ ---- -title: Implementing PdxSerializable in Your Domain Object ---- - -For a domain object with source that you can modify, implement the `PdxSerializable` interface in the object and use its methods to serialize and deserialize the object's fields. - -<a id="use_pdx_serializable__section_7F357A8E56B54BFB9A5778C0F89E034E"></a> -**Procedure** - -1. In your domain class, implement `PdxSerializable`, importing the required `org.apache.geode.pdx` classes. - - For example: - - ``` pre - import org.apache.geode.pdx.PdxReader; - import org.apache.geode.pdx.PdxSerializable; - import org.apache.geode.pdx.PdxWriter; - - public class PortfolioPdx implements PdxSerializable { - ... - ``` - -2. If your domain class does not have a zero-arg constructor, create one for it. - - For example: - - ``` pre - public PortfolioPdx(){} - ``` - -3. Program `PdxSerializable.toData.` - 1. Write each standard Java data field of your domain class using the `PdxWriter` write methods. Geode automatically provides `PdxWriter` to the `toData` method for `PdxSerializable` objects. - 2. Call the `PdxWriter` `markIdentifyField` method for each field you want to have Geode use to identify your object. Put this after the field's write method. Geode uses this information to compare objects for operations like distinct queries. If you do not set as least one identity field, then the `equals` and `hashCode` methods will use all PDX fields to compare objects and consequently, will not perform as well. It is important that the fields used by your `equals` and `hashCode` implementations are the same fields that you mark as identity fields. - 3. For a particular version of your class, you need to consistently write the same named field each time. The field names or number of fields must not change from one instance to another for the same class version. - 4. For best performance, do fixed width fields first and then variable length fields. - - Example `toData` code: - - ``` pre - // PortfolioPdx fields - private int id; - private String pkid; - private Map<String, PositionPdx> positions; - private String type; - private String status; - private String[] names; - private byte[] newVal; - private Date creationDate; - ... - - public void toData(PdxWriter writer) - { - writer.writeInt("id", id) - // The markIdentifyField call for a field must - // come after the field's write method - .markIdentityField("id") - .writeDate("creationDate", creationDate) //fixed length field - .writeString("pkid", pkid) - .writeObject("positions", positions) - .writeString("type", type) - .writeString("status", status) - .writeStringArray("names", names) - .writeByteArray("newVal", newVal) - } - ``` - -4. Program `PdxSerializable.fromData` to read your data fields from the serialized form into the object's fields using the `PdxReader` read methods. - - Provide the same names that you did in `toData` and call the read operations in the same order as you called the write operations in your `toData` implementation. - - Geode automatically provides `PdxReader` to the `fromData` method for `PdxSerializable` objects. - - Example `fromData` code: - - ``` pre - - public void fromData(PdxReader reader) - { - id = reader.readInt("id"); - creationDate = reader.readDate("creationDate"); - pkid = reader.readString("pkid"); - position1 = (PositionPdx)reader.readObject("position1"); - position2 = (PositionPdx)reader.readObject("position2"); - positions = (Map<String, PositionPdx>)reader.readObject("positions"); - type = reader.readString("type"); - status = reader.readString("status"); - names = reader.readStringArray("names"); - newVal = reader.readByteArray("newVal"); - arrayNull = reader.readByteArray("arrayNull"); - arrayZeroSize = reader.readByteArray("arrayZeroSize"); - } - ``` - -**What to do next** - -- As needed, configure and program your Geode applications to use `PdxInstance` for selective object deserialization. See [Programming Your Application to Use PdxInstances](program_application_for_pdx.html). - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/use_pdx_serializer.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/use_pdx_serializer.html.md.erb b/developing/data_serialization/use_pdx_serializer.html.md.erb deleted file mode 100644 index 7ef401e..0000000 --- a/developing/data_serialization/use_pdx_serializer.html.md.erb +++ /dev/null @@ -1,128 +0,0 @@ ---- -title: Serializing Your Domain Object with a PdxSerializer ---- - -For a domain object that you cannot or do not want to modify, use the `PdxSerializer` class to serialize and deserialize the object's fields. You use one `PdxSerializer` implementation for the entire cache, programming it for all of the domain objects that you handle in this way. - -With `PdxSerializer`, you leave your domain object as-is and handle the serialization and deserialization in the separate serializer. You register the serializer in your cache PDX configuration. Program the serializer to handle all of the domain objects you need. - -If you write your own `PdxSerializer` and you also use the `ReflectionBasedAutoSerializer`, then the `PdxSerializer` needs to own the `ReflectionBasedAutoSerializer` and delegate to it. A Cache can only have a single `PdxSerializer` instance. - -**Note:** -The `PdxSerializer` `toData` and `fromData` methods differ from those for `PdxSerializable`. They have different parameters and results. - -**Procedure** - -1. In the domain classes that you wish to PDX serialize, make sure each class has a zero-arg constructor. For example: - - ``` pre - public PortfolioPdx(){} - ``` - -2. If you have not already implemented `PdxSerializer` for some other domain object, perform these steps: - 1. Create a new class as your cache-wide serializer and make it implement `PdxSerializer`. If you want to declare your new class in the `cache.xml` file, have it also implement `Declarable`. - - Example: - - ``` pre - import org.apache.geode.cache.Declarable; - import org.apache.geode.pdx.PdxReader; - import org.apache.geode.pdx.PdxSerializer; - import org.apache.geode.pdx.PdxWriter; - - public class ExamplePdxSerializer implements PdxSerializer, Declarable { - ... - ``` - - 2. In your cache pdx configuration, register the serializer class in the cache's `<pdx>` `<pdx-serializer>` `<class-name>` attribute. - - Example: - - ``` pre - // Configuration setting PDX serializer for the cache - <cache> - <pdx> - <pdx-serializer> - <class-name>com.company.ExamplePdxSerializer</class-name> - </pdx-serializer> - </pdx> - ... - </cache> - ``` - - Or use the `CacheFactory.setPdxSerializer ` API. - - ``` pre - Cache c = new CacheFactory - .setPdxSerializer(new ExamplePdxSerializer()) - .create(); - ``` - - **Note:** - You cannot specify a custom `pdx-serializer` class using gfsh, however the `configure pdx` command automatically configures the <span class="keyword apiname">org.apache.geode.pdx.ReflectionBasedAutoSerializer</span> class. See [configure pdx](../../tools_modules/gfsh/command-pages/configure.html#topic_jdkdiqbgphqh). - -3. Program `PdxSerializer.toData` to recognize, cast, and handle your domain object: - - 1. Write each standard Java data field of your domain class using the `PdxWriter` write methods. - 2. Call the `PdxWriter` `markIdentityField` method for each field you want to have Geode use to identify your object. Put this after the field's write method. Geode uses this information to compare objects for operations like distinct queries. If you do not set as least one identity field, then the `equals` and `hashCode` methods will use all PDX fields to compare objects and consequently, will not perform as well. It is important that the fields used by your `equals` and `hashCode` implementations are the same fields that you mark as identity fields. - 3. For a particular version of your class, you need to consistently write the same named field each time. The field names or number of fields must not change from one instance to another for the same class version. - 4. For best performance, do fixed width fields first and then variable length fields. - 5. If desired, you can check the portability of the object before serializing it by adding the `checkPortability` parameter when using the` PdxWriter` `writeObject`, `writeObjectArray`, and `writeField` methods. - - Example `toData` code: - - ``` pre - public boolean toData(Object o, PdxWriter writer) - { - if(!(o instanceof PortfolioPdx)) { - return false; - } - - PortfolioPdx instance = (PortfolioPdx) o; - writer.writeInt("id", instance.id) - //identity field - .markIdentityField("id") - .writeDate("creationDate", instance.creationDate) - .writeString("pkid", instance.pkid) - .writeObject("positions", instance.positions) - .writeString("type", instance.type) - .writeString("status", instance.status) - .writeStringArray("names", instance.names) - .writeByteArray("newVal", instance.newVal) - - return true; - } - ``` - - 1. Program `PdxSerializer.fromData` to create an instance of your class, read your data fields from the serialized form into the object's fields using the `PdxReader` read methods, and return the created object. - - Provide the same names that you did in `toData` and call the read operations in the same order as you called the write operations in your `toData` implementation. - - Geode provides the domain class type and `PdxReader` to the `fromData` method. - - Example `fromData` code: - - ``` pre - public Object fromData(Class<?> clazz, PdxReader reader) - { - if(!clazz.equals(PortfolioPdx.class)) { - return null; - } - - PortfolioPdx instance = new PortfolioPdx(); - instance.id = reader.readInt("id"); - instance.creationDate = reader.readDate("creationDate"); - instance.pkid = reader.readString("pkid"); - instance.positions = (Map<String, PositionPdx>)reader.readObject("positions"); - instance.type = reader.readString("type"); - instance.status = reader.readString("status"); - instance.names = reader.readStringArray("names"); - instance.newVal = reader.readByteArray("newVal"); - - return instance; - } - ``` - -4. If desired, you can also enable extra validation in your use of `PdxWriter`. You can set this by setting the system property `gemfire.validatePdxWriters` to **true**. Note that you should only set this option if you are debugging new code as this option can decrease system performance. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/using_PdxInstanceFactory.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/using_PdxInstanceFactory.html.md.erb b/developing/data_serialization/using_PdxInstanceFactory.html.md.erb deleted file mode 100644 index 7b9501d..0000000 --- a/developing/data_serialization/using_PdxInstanceFactory.html.md.erb +++ /dev/null @@ -1,34 +0,0 @@ ---- -title: Using PdxInstanceFactory to Create PdxInstances ---- - -You can use the `PdxInstanceFactory` interface to create a `PdxInstance` from raw data when the domain class is not available on the server. - -This can be particularly useful when you need an instance of a domain class for plug in code such as a function or a loader. If you have the raw data for the domain object (the class name and each field's type and data), then you can explicitly create a `PdxInstance`. The `PdxInstanceFactory` is very similar to the `PdxWriter` except that after writing each field, you need to call the create method which returns the created PdxInstance. - -To create a factory call `RegionService.createPdxInstanceFactory`. A factory can only create a single instance. To create multiple instances create multiple factories or use `PdxInstance.createWriter()` to create subsequent instances. Using `PdxInstance.createWriter()` is usually faster. - -When you create a PdxInstance, set as least one identity field using the `markIndentityField` method. If you do not mark an identity field, the PdxInstance`equals` and `hashCode` methods will use all PDX fields to compare objects and consequently, will not perform as well. It is important that the fields used by your `equals` and `hashCode` implementations are the same fields that you mark as identity fields. - -The following is a code example of using `PdxInstanceFactory`: - -``` pre -PdxInstance pi = cache.createPdxInstanceFactory("com.company.DomainObject") - .writeInt("id", 37) - .markIdentityField("id") - .writeString("name", "Mike Smith") - .writeObject("favoriteDay", cache.createPdxEnum("com.company.Day", "FRIDAY", 5)) - .create(); -``` - -For more information, see `PdxInstanceFactory` in the Java API documentation. - -## <a id="concept_FFECBE8249D848E9A2CF7FD02514EC68__section_F4EC56197730427084FBF040820A6149" class="no-quick-link"></a>Enum Objects as PdxInstances - -You can now work with enum objects as PdxInstances. When you fetch an enum object from the cache, you can now deserialize it as a `PdxInstance`. To check whether a `PdxInstance` is an enum, use the `PdxInstance.isEnum` method. An enum `PdxInstance` will have one field named "name" whose value is a String that corresponds to the enum constant name. - -An enum `PdxInstance` is not writable; if you call `createWriter` it will throw an exception. - -The `RegionService` has a method that allows you to create a `PdxInstance` that represents an enum. See `RegionService.createPdxEnum` in the Java API documentation. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/data_serialization/using_pdx_region_entry_keys.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/data_serialization/using_pdx_region_entry_keys.html.md.erb b/developing/data_serialization/using_pdx_region_entry_keys.html.md.erb deleted file mode 100644 index 91c91c4..0000000 --- a/developing/data_serialization/using_pdx_region_entry_keys.html.md.erb +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: Using PDX Objects as Region Entry Keys ---- - -Using PDX objects as region entry keys is highly discouraged. - -<a id="concept_E5B95958B8D04B2285CB5E4DC6FABC78__section_900BD620D716407AA78BE0E38C2C396D"></a> -The best practice for creating region entry keys is to use a simple key; for example, use a String or Integer. If the key must be a domain class, then you should use a non-PDX-serialized class. - -If you must use PDX serialized objects as region entry keys, ensure that you do not set `read-serialized` to `true`. This configuration setting will cause problems in partitioned regions because partitioned regions require the hash code of the key to be the same on all JVMs in the distributed system. When the key is a `PdxInstance` object, its hash code will likely not be the same as the hash code of the domain object. - -If you are using PDX serialized objects as region entry keys and you are using persistent regions, then you must configure your PDX disk store to be a different one than the disk store used by the persistent regions. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/delta_propagation/chapter_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/delta_propagation/chapter_overview.html.md.erb b/developing/delta_propagation/chapter_overview.html.md.erb deleted file mode 100644 index 1beb847..0000000 --- a/developing/delta_propagation/chapter_overview.html.md.erb +++ /dev/null @@ -1,31 +0,0 @@ ---- -title: Delta Propagation ---- - -Delta propagation allows you to reduce the amount of data you send over the network by including only changes to objects rather than the entire object. - -- **[How Delta Propagation Works](how_delta_propagation_works.html)** - - Delta propagation reduces the amount of data you send over the network. You do this by only sending the change, or delta, information about an object, instead of sending the entire changed object. If you do not use cloning when applying the deltas, you can also expect to generate less garbage in your receiving JVMs. - -- **[When to Avoid Delta Propagation](when_to_use_delta_prop.html)** - - Generally, the larger your objects and the smaller the deltas, the greater the benefits of using delta propagation. Partitioned regions with higher redundancy levels generally benefit more from delta propagation. However, in some application scenarios, delta propagation does not show any significant benefits. On occasion it results in performance degradation. - -- **[Delta Propagation Properties](delta_propagation_properties.html)** - - This topic describes the properties that can be used to configure delta propagation. - -- **[Implementing Delta Propagation](implementing_delta_propagation.html)** - - By default, delta propagation is enabled in your distributed system. When enabled, delta propagation is used for objects that implement `org.apache.geode.Delta`. You program the methods to store and extract delta information for your entries and to apply received delta information. - -- **[Errors In Delta Propagation](errors_in_delta_propagation.html)** - - This topic lists the errors that can occur when using delta propagation. - -- **[Delta Propagation Example](delta_propagation_example.html)** - - This topic provides an example of delta propagation. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/delta_propagation/delta_propagation_example.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/delta_propagation/delta_propagation_example.html.md.erb b/developing/delta_propagation/delta_propagation_example.html.md.erb deleted file mode 100644 index 075b32e..0000000 --- a/developing/delta_propagation/delta_propagation_example.html.md.erb +++ /dev/null @@ -1,113 +0,0 @@ ---- -title: Delta Propagation Example ---- - -This topic provides an example of delta propagation. - -<a id="delta_propagation_example__section_F6700EB7F305462D9F0DEE3080497A14"></a> -In this example, the feeder client is connected to the first server, and the receiver client is connected to the second. The servers are peers to each other. - -<img src="../../images/DeltaPropagation-3.gif" id="delta_propagation_example__image_E20DAE5925C44EBDA03D1AC8325D1122" class="image" /> - -The example demonstrates the following operations: - -1. In the Feeder client, the application updates the entry object and puts the entry. In response to the `put`, Geode calls `hasDelta`, which returns true, so Geode calls `toDelta` and forwards the extracted delta to the server. If `hasDelta` returned false, Geode would distribute the full entry value. -2. In Server1, Geode applies the delta to the cache, distributes the received delta to the server's peers, and forwards it to any other clients with interest in the entry (there are no other clients to Server1 in this example) -3. In Server2, Geode applies the delta to the cache and forwards it to its interested clients, which in this case is just the Receiver client. - -<a id="delta_propagation_example__section_185444FC51FB467587A62DFEC07C9C7D"></a> - -This example shows the basic approach to programming a `Delta` implementation. - -``` pre -package delta; - -import org.apache.geode.Delta; -import org.apache.geode.InvalidDeltaException; -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.io.Serializable; - -/** - * Sample implementation of Delta - * - * @author GemStone Systems, Inc. - * @since 6.1 - */ -public class SimpleDelta implements Delta, Serializable { - -// Original object fields - private int intVal; - private double doubleVal; - - // Added for delta - one boolean per field to track changed status - private transient boolean intFldChd = false; - private transient boolean dblFldChd = false; - - public SimpleDelta(){} - - public SimpleDelta(int i, double d){ - this.intVal = i; - this.doubleVal = d; - } - - public boolean hasDelta() { - return this.intFldChd || this.dblFldChd; - } - - public void toDelta(DataOutput out) throws IOException { - System.out.println("Extracting delta from " + this.toString()); - // Write information on what has changed to the - // data stream, so fromDelta knows what it's getting - out.writeBoolean(intFldChd); - if (intFldChd) { - // Write just the changes into the data stream - out.writeInt(this.intVal); - // Once the delta information is written, reset the delta status field - this.intFldChd = false; - System.out.println(" Extracted delta from field 'intVal' = " - + this.intVal); - } - out.writeBoolean(dblFldChd); - if (dblFldChd) { - out.writeDouble(this.doubleVal); - this.dblFldChd = false; - System.out.println(" Extracted delta from field 'doubleVal' = " - + this.doubleVal); - } - } - - public void fromDelta(DataInput in) throws IOException, InvalidDeltaException { - System.out.println("Applying delta to " + this.toString()); - // For each field, read whether there is a change - if (in.readBoolean()) { - // Read the change and apply it to the object - this.intVal = in.readInt(); - System.out.println(" Applied delta to field 'intVal' = " - + this.intVal); - } - if (in.readBoolean()) { - this.doubleVal = in.readDouble(); - System.out.println(" Applied delta to field 'doubleVal' = " - + this.doubleVal); - } - } - // In the setter methods, add setting of delta-related - // fields indicating what has changed - public void setIntVal(int anIntVal) { - this.intFldChd = true; - this.intVal = anIntVal; - } - - public void setDoubleVal(double aDoubleVal) { - this.dblFldChd = true; - this.doubleVal = aDoubleVal; - } - - public String toString() { - return "SimpleDelta [ hasDelta = " + hasDelta() + ", intVal = " + - this.intVal + ", doubleVal = {" + this.doubleVal + "} ]"; - } -} -``` http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/delta_propagation/delta_propagation_properties.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/delta_propagation/delta_propagation_properties.html.md.erb b/developing/delta_propagation/delta_propagation_properties.html.md.erb deleted file mode 100644 index c95fab4..0000000 --- a/developing/delta_propagation/delta_propagation_properties.html.md.erb +++ /dev/null @@ -1,79 +0,0 @@ ---- -title: Delta Propagation Properties ---- - -This topic describes the properties that can be used to configure delta propagation. - -Delta propagation properties can be configured through the API and through the `gemfire.properties` and `cache.xml` files. - -## <a id="delta_propagation_properties__section_561D6DA876E24469B7536E98AB12F676" class="no-quick-link"></a>delta-propagation - -A `gemfire.properties` boolean that enables or disables delta propagation. When false, full entry values are sent for every update. The default setting is true, which enables delta propagation. - -Disable delta propagation as follows: - -- `gemfire.properties`: - - ``` pre - delta-propagation=false - ``` - -- API: - - ``` pre - Properties props = new Properties(); - props.setProperty("delta-propagation", false); - this.cache = new ClientCacheFactory(props).create(); - ``` - -## <a id="delta_propagation_properties__section_7D4590512D1548FD94F81C8726A2CA44" class="no-quick-link"></a>cloning-enabled - -A region attributes boolean that affects how `fromDelta` applies deltas to the local cache. When true, the updates are applied to a clone of the value and then the clone is saved to the cache. When false, the value is modified in place in the cache. The default value is false. - -Exceptions to this behavior: - -- If the `Cache` attribute `copy-on-read` is true, cloning is enabled, regardless of what this attribute is set to. -- If the `Region` attribute `off-heap` is true, cloning is enabled, regardless of what this attribute is set to. - -Cloning can be expensive, but it ensures that the new object is fully initialized with the delta before any application code sees it. - -When cloning is enabled, by default Geode does a deep copy of the object, using serialization. You may be able to improve performance by implementing `java.lang.Cloneable` and then implementing the `clone` method, making a deep copy of anything to which a delta may be applied. The goal is to reduce significantly the overhead of copying the object while still retaining the isolation needed for your deltas. - -Without cloning: - -- It is possible for application code to read the entry value as it is being modified, possibly seeing the value in an intermediate, inconsistent state, with just part of the delta applied. You may choose to resolve this issue by having your application code synchronize on reads and writes. -- Geode loses any reference to the old value because the old value is transformed in place into the new value. Because of this, your `CacheListener` sees the same new value returned for `EntryEvent.getOldValue` and `EntryEvent.getNewValue` . -- Exceptions thrown from `fromDelta` may leave your cache in an inconsistent state. Without cloning, any interruption of the delta application could leave you with some of the fields in your cached object changed and others unchanged. If you do not use cloning, keep this in mind when you program your error handling in your `fromDelta` implementation. - -With cloning: - -- The `fromDelta` method generates more garbage in memory. -- Performance is reduced. - -Enable cloning as follows: - -- `cache.xml`: - - ``` pre - <region name="region_with_cloning"> - <region-attributes refid="REPLICATE" cloning-enabled="true"> - </region-attributes> - </region> - ``` - -- API: - - ``` pre - RegionFactory rf = cache.createRegionFactory(REPLICATE); - rf.setCloningEnabled(true); - custRegion = rf.create("customer"); - ``` - -- gfsh: - - ``` pre - gfsh>create region --name="region_with_cloning" --type=REPLICATE - --enable-cloning=true - ``` - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/delta_propagation/errors_in_delta_propagation.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/delta_propagation/errors_in_delta_propagation.html.md.erb b/developing/delta_propagation/errors_in_delta_propagation.html.md.erb deleted file mode 100644 index f6441e8..0000000 --- a/developing/delta_propagation/errors_in_delta_propagation.html.md.erb +++ /dev/null @@ -1,18 +0,0 @@ ---- -title: Errors In Delta Propagation ---- - -This topic lists the errors that can occur when using delta propagation. - -<a id="errors_in_delta_propagation__section_877AC61D691C44078A782683F90D169B"></a> -Errors in delta propagation fall into two categories based on how they are handled by the system: - -- Problems applying the delta that can be remedied by requesting the full value in place of the delta. Your `put` operation does not see errors or exceptions related to this type of delta propagation failure. The system automatically does a full value distribution from the sender to the receiver where the problem occurs. This type of error includes: - - Unavailable entry value in the receiving cache, either because the entry is missing or its value is null. In both cases, there is nothing to apply the delta to and the full value must be sent. This is most likely to occur if you destroy or invalidate your entries locally, either through application calls or through configured actions like eviction or entry expiration. - - `InvalidDeltaException` thrown by `fromDelta` method, programmed by you. This exception enables you to avoid applying deltas that would violate data consistency checks or other application requirements. - - Any error applying the delta in a client in server-to-client propagation. The client logs a warning in addition to retrieving the full value from the server. -- Problems creating or distributing the delta that cannot be fixed by distributing the full value. In these cases, your `put` operation fails with an exception. This type of error includes: - - Error or exception in `hasDelta` or `toDelta`. - - Error or exception in a server or peer receiver that fall outside of the situations described above in the first category. - - http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/ccc2fbda/developing/delta_propagation/how_delta_propagation_works.html.md.erb ---------------------------------------------------------------------- diff --git a/developing/delta_propagation/how_delta_propagation_works.html.md.erb b/developing/delta_propagation/how_delta_propagation_works.html.md.erb deleted file mode 100644 index a37bd8a..0000000 --- a/developing/delta_propagation/how_delta_propagation_works.html.md.erb +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: How Delta Propagation Works ---- - -Delta propagation reduces the amount of data you send over the network. You do this by only sending the change, or delta, information about an object, instead of sending the entire changed object. If you do not use cloning when applying the deltas, you can also expect to generate less garbage in your receiving JVMs. - -<a id="how_delta_propagation_works__section_78D584B3FFD04D1D9BA83203FF2B55A9"></a> -In most distributed data management systems, the data stored in the system tends to be created once and then updated frequently. These updates are sent to other members for event propagation, redundancy management, and cache consistency in general. Tracking only the changes in an updated object and sending only the deltas mean lower network transmission costs and lower object serialization/deserialization costs. Performance improvements can be significant, especially when changes to an object are small relative to its overall size. - -<a id="how_delta_propagation_works__section_ABE3589920D6477BBB2223A583AF169A"></a> - -Geode propagates object deltas using methods that you program. The methods are in the `Delta` interface, which you implement in your cached objects' classes. If any of your classes are plain old Java objects, you need to wrap them for this implementation. - -This figure shows delta propagation for a change to an entry with key, k, and value object, v. - -<img src="../../images/DeltaPropagation-1.gif" id="how_delta_propagation_works__image_06A25185C31548669423DDA3136B1851" class="image" /> - -1. **`get` operation**. The `get` works as usual: the cache returns the full entry object from the local cache or, if it isn't available there, from a remote cache or from a loader. -2. **update methods**. You need to add code to the object's update methods so that they save delta information for object updates, in addition to the work they were already doing. -3. **`put` operation**. The `put` works as usual in the local cache, using the full value, then calls `hasDelta` to see if there are deltas and `toDelta` to serialize the information. Distribution is the same as for full values, according to member and region configuration. -4. **receipt of delta at remote member**. `fromDelta` extracts the delta information that was serialized by `toDelta` and applies it to the object in the local cache. The delta is applied directly to the existing value or to a clone, depending on how you configure it for the region. -5. **additional distributions**. As with full distributions, receiving members forward the delta according to their configurations and connections to other members. For example, if VM1 is a client and VM2 is a server, VM2 forwards the delta to its peers and its other clients as needed. Receiving members do not recreate the delta; `toDelta` is only called in the originating member. - -## <a id="how_delta_propagation_works__section_25EC5BE960F0402DAEDBE0A5A6589ACA" class="no-quick-link"></a>General Characteristics of Delta Propagation - -To use the delta propagation feature, all updates on a key in a region must have value types that implement the `Delta` interface. You cannot mix object types for an entry key where some of the types implement delta and some do not. This is because, when a type implementing the delta interface is received for an update, the existing value for the key is cast to a `Delta` type to apply the received delta. If the existing type does not also implement the `Delta` interface, the operation throws a `ClassCastException`. - -**Note:** Only the object itself being placed in the cache can implement the `Delta` interface and propagate changes. Any sub-objects of the cache object do not propagate their changes. - -Sometimes `fromDelta` cannot be invoked because there is no object to apply the delta to in the receiving cache. When this happens, the system automatically does a full value distribution to the receiver. These are the possible scenarios: -1. If the system can determine beforehand that the receiver does not have a local copy, it sends the initial message with the full value. This is possible when regions are configured with no local data storage, such as with the region shortcut settings `PARTITION_PROXY` and `REPLICATE_PROXY`. These configurations are used to accomplish things like provide data update information to listeners and to pass updates forward to clients. -2. In less obvious cases, such as when an entry has been locally deleted, first the delta is sent, then the receiver requests a full value and that is sent. Whenever the full value is received, any further distributions to the receiver's peers or clients uses the full value. - -Geode also does not propagate deltas for: - -- Transactional commit -- The `putAll` operation -- JVMs running Geode versions that do not support delta propagation (6.0 and earlier) - -## <a id="how_delta_propagation_works__section_F4A102A74530429F87BEA53C90D5CCFB" class="no-quick-link"></a>Supported Topologies and Limitations - -The following topologies support delta propagation (with some limitations): - -- **Peer-to-peer**. Geode system members distribute and receive entry changes using delta propagation, with these requirements and caveats: - - Regions must be partitioned or have their scope set to `distributed-ack` or `global`. The region shortcut settings for distributed regions use `distributed-ack` `scope`. Delta propagation does not work for regions with `distributed-no-ack` `scope` because the receiver could not recover if an exception occurred while applying the delta. - - For partitioned regions, if a receiving peer does not hold the primary or a secondary copy of the entry, but still requires a value, the system automatically sends the full value. - - To receive deltas, a region must be non-empty. The system automatically sends the full value to empty regions. Empty regions can send deltas. -- **Client/server**. Geode clients can always send deltas to the servers, and servers can usually sent deltas to clients. These configurations require the servers to send full values to the clients, instead of deltas: - - When the client's `gemfire.properties` setting `conflate-events` is set to true, the servers send full values for all regions. - - When the server region attribute `enable-subscription-conflation` is set to true and the client `gemfire.properties` setting `conflate-events` is set to `server`, the servers send full values for the region. - - When the client region is configured with the `PROXY` client region shortcut setting (empty client region), servers send full values. -
