QI-318 Documentation!
Project: http://git-wip-us.apache.org/repos/asf/zest-qi4j/repo Commit: http://git-wip-us.apache.org/repos/asf/zest-qi4j/commit/f875d7cf Tree: http://git-wip-us.apache.org/repos/asf/zest-qi4j/tree/f875d7cf Diff: http://git-wip-us.apache.org/repos/asf/zest-qi4j/diff/f875d7cf Branch: refs/heads/master Commit: f875d7cf02bcf2e0c12b1ca1ad902965b145bacf Parents: bb5fcda Author: Paul Merlin <[email protected]> Authored: Mon Feb 11 20:05:53 2013 +0100 Committer: Paul Merlin <[email protected]> Committed: Mon Feb 11 20:05:53 2013 +0100 ---------------------------------------------------------------------- core/api/src/docs/valuecomposite.txt | 121 +++++++- .../qi4j/api/value/DocumentationSupport.java | 298 +++++++++++++++++++ core/spi/src/docs/metrics.txt | 2 +- core/spi/src/docs/spi.txt | 7 +- core/spi/src/docs/valueserialization.txt | 52 ++++ .../src/docs/vs-jackson.txt | 24 ++ .../src/docs/vs-orgjson.txt | 24 ++ .../src/docs/vs-stax.txt | 24 ++ manual/src/docs/userguide/extensions.txt | 13 + 9 files changed, 562 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/core/api/src/docs/valuecomposite.txt ---------------------------------------------------------------------- diff --git a/core/api/src/docs/valuecomposite.txt b/core/api/src/docs/valuecomposite.txt index 7e003f1..7536401 100644 --- a/core/api/src/docs/valuecomposite.txt +++ b/core/api/src/docs/valuecomposite.txt @@ -1,5 +1,6 @@ ////////////////////// * Copyright (c) 2007-2012, Niclas Hedhman. All Rights Reserved. + * Copyright (c) 2013, Paul Merlin. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -28,4 +29,122 @@ The characteristics of a ValueComposite compared to other Composite meta types a * It is Immutable. * Its equals/hashCode works on the values of the ValueComposite. - * Can be used as Property types, but will not be indexed and searchable. \ No newline at end of file + * Can be used as Property types. + +== Value Serialization == +Value objects can be serialized and deserialized using the ValueSerialization API which is a Service API implemented +by SPI and extensions. + +TIP: +ValueSerialization extends ValueSerializer, ValueDeserializer+. See the <<javadocs>> for interfaces detail. + +The ValueSerialization mechanism apply to the following object types : + + * ValueComposite, + * EntityReference, + * Iterable, + * Map, + * Plain Value. + +Nested Plain Values, EntityReferences, Iterables, Maps, ValueComposites are supported. +EntityComposites and EntityReferences are serialized as their identity string. + +Plain Values can be one of : + + * String, + * Boolean or boolean, + * Integer or int, + * Long or long, + * Short or short, + * Byte or byte, + * Float or float, + * Double or double, + * BigInteger, + * BigDecimal, + * Date, + * DateTime (JodaTime), + * LocalDateTime (JodaTime), + * LocalDate (JodaTime). + +Values of unknown types and all arrays are considered as +java.io.Serializable+ and by so are (de)serialized to (from) +base64 encoded bytes using pure Java serialization. If it happens that the value is not Serializable or the input to +deserialize is invalid, a +ValueSerializationException+ is thrown. + +Methods of +ValueSerializer+ allow to specify if the serialized state should contain extra type information about the +serialized value. Having type information in the serialized payload allows to keep actual ValueComposite types and by so +circumvent +AmbiguousTypeException+ when deserializing. + +Core Runtime provides a default ValueSerialization system based on the +https://github.com/douglascrockford/JSON-java[org.json] Java library producing and consuming JSON. + +Let's see how it works in practice. + +[snippet,java] +---- +source=core/api/src/test/java/org/qi4j/api/value/DocumentationSupport.java +tag=default +---- + +Reading this first example step by step we ; + + . declare a ValueComposite, + . assemble it, + . create a new Value instance, + . use the +ValueComposite#toString()+ method to get a JSON representation of the Value, + . and finally, use the +Module#newValueFromSerializedState()+ method to create a new Value instance from the JSON + state. + ++ValueComposite#toString()+ method leverage Value Serialization and by so provide JSON based representation. The Module +API allows to create new Value instances from serialized state. + +On top of that, Application assemblies can register different implementation of ValueSerialization as Services to +support more formats, see the <<extensions>> section. Note that the default behaviour described above is overriden if a +ValueSerialization Service is visible. + +Let's see how to use the ValueSerialization Services. + +[snippet,java] +---- +source=core/api/src/test/java/org/qi4j/api/value/DocumentationSupport.java +tag=service +---- + +In this second example, we ; + + . declare a ValueComposite, + . assemble it, + . assemble a ValueSerialization Service backed by the +org.json+ package, + . get the +ValueSerializer+ and +ValueDeserializer+ Services injected, + . create a new Value instance, + . use the +ValueSerializer#serialize()+ method to get a JSON representation of the Value, + . and finally, use the +ValueDeserializer#eserialize()+ method to create a new Value instance from the JSON state. + +Many applications need to stream data. The ValueSerialization API support such use cases in two ways. + +The first one use classic streams. + +[snippet,java] +---- +source=core/api/src/test/java/org/qi4j/api/value/DocumentationSupport.java +tag=stream +---- + + . get a handle on a source of values and an +OutputStream+, + . serialize data into the +OutputStream+, + . get a handle on an +InputStream+, + . deserialize data from the +InputStream+. + +The second one use the <<core-io>>: + +[snippet,java] +---- +source=core/api/src/test/java/org/qi4j/api/value/DocumentationSupport.java +tag=io +---- + + . get a handle on a source of values and a +Writer+, + . prepare the serialization +Function+, + . serialize a collection of values, one serialized value per line, + . get a handle on a serialized values +Reader+ and create a new empty +List+ of values, + . prepare the deserialization +Function+, + . deserialize a collection of values from read lines. + http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/core/api/src/test/java/org/qi4j/api/value/DocumentationSupport.java ---------------------------------------------------------------------- diff --git a/core/api/src/test/java/org/qi4j/api/value/DocumentationSupport.java b/core/api/src/test/java/org/qi4j/api/value/DocumentationSupport.java new file mode 100644 index 0000000..32595b2 --- /dev/null +++ b/core/api/src/test/java/org/qi4j/api/value/DocumentationSupport.java @@ -0,0 +1,298 @@ +/* + * Copyright (c) 2013, Paul Merlin. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * + * You may obtain a copy of the License at + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.qi4j.api.value; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.Reader; +import java.io.StringReader; +import java.io.StringWriter; +import java.io.Writer; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import org.junit.Before; +import org.junit.Test; +import org.qi4j.api.injection.scope.Service; +import org.qi4j.api.property.Property; +import org.qi4j.api.structure.Application; +import org.qi4j.api.structure.Module; +import org.qi4j.api.type.CollectionType; +import org.qi4j.bootstrap.ApplicationAssembler; +import org.qi4j.bootstrap.ApplicationAssembly; +import org.qi4j.bootstrap.ApplicationAssemblyFactory; +import org.qi4j.bootstrap.Assembler; +import org.qi4j.bootstrap.AssemblyException; +import org.qi4j.bootstrap.Energy4Java; +import org.qi4j.bootstrap.ModuleAssembly; +import org.qi4j.functional.Function; +import org.qi4j.io.Inputs; +import org.qi4j.io.Outputs; +import org.qi4j.io.Transforms; +import org.qi4j.test.AbstractQi4jTest; +import org.qi4j.valueserialization.orgjson.OrgJsonValueSerializationAssembler; + +import static org.hamcrest.CoreMatchers.equalTo; +import static org.junit.Assert.assertThat; + +/** + * Snippets: + * - default : default ValueSerialization + * - service : assembled service ValueSerialization + * - lookup : ValueSerialization values module finder + */ +public class DocumentationSupport + extends AbstractQi4jTest +{ + + @Before + public void injectToThis() + { + module.injectTo( this ); + } + + // START SNIPPET: default + // START SNIPPET: service + public interface SomeValue // (1) + { + + Property<String> foo(); + } + + @Override + public void assemble( ModuleAssembly module ) + throws AssemblyException + { + module.values( SomeValue.class ); // (2) + // END SNIPPET: default + new OrgJsonValueSerializationAssembler().assemble( module ); // (3) + // START SNIPPET: default + } + // END SNIPPET: default + // END SNIPPET: service + + @Test + // START SNIPPET: default + public void defaultValueSerialization() + { + SomeValue someValue = someNewValueInstance( module ); // (3) + String json = someValue.toString(); // (4) + SomeValue someNewValue = module.newValueFromSerializedState( SomeValue.class, json ); // (5) + // END SNIPPET: default + + assertThat( json, equalTo( "{\"foo\":\"bar\"}" ) ); + assertThat( someNewValue, equalTo( someValue ) ); + + // START SNIPPET: default + } + // END SNIPPET: default + // START SNIPPET: service + @Service + private ValueSerializer valueSerializer; // (4) + @Service + private ValueDeserializer valueDeserializer; // (4) + + // END SNIPPET: service + @Test + // START SNIPPET: service + public void assembledDefaultServiceSerialization() + { + SomeValue someValue = someNewValueInstance( module ); // (5) + String json = valueSerializer.serialize( someValue ); // (6) + SomeValue someNewValue = valueDeserializer.deserialize( SomeValue.class, json ); // (7) + // END SNIPPET: service + + assertThat( json, equalTo( "{\"foo\":\"bar\"}" ) ); + assertThat( someNewValue, equalTo( someValue ) ); + + // START SNIPPET: service + } + // END SNIPPET: service + + static enum AcmeValue + { + + foo, bar + } + + @Test + // START SNIPPET: stream + public void assembledServiceStreamingSerialization() + { + // END SNIPPET: stream + + List<AcmeValue> dataSource = Arrays.asList( AcmeValue.values() ); + ByteArrayOutputStream targetStream = new ByteArrayOutputStream(); + + // START SNIPPET: stream + // (1) + Iterable<AcmeValue> data = dataSource; // Eg. Entities converted to Values + OutputStream output = targetStream; // Eg. streaming JSON over HTTP + + // (2) + valueSerializer.serialize( data, output ); + // END SNIPPET: stream + + byte[] serialized = targetStream.toByteArray(); + ByteArrayInputStream sourceStream = new ByteArrayInputStream( serialized ); + + // START SNIPPET: stream + // (3) + InputStream input = sourceStream; // Eg. reading incoming JSON + + // (4) + List<AcmeValue> values = valueDeserializer.deserialize( CollectionType.listOf( AcmeValue.class ), input ); + // END SNIPPET: stream + + assertThat( values, equalTo( dataSource ) ); + + // START SNIPPET: stream + } + // END SNIPPET: stream + + @Test + // START SNIPPET: io + public void assembledServiceIOSerialization() + throws IOException + { + // END SNIPPET: io + + List<AcmeValue> dataSource = Arrays.asList( AcmeValue.values() ); + StringWriter outputWriter = new StringWriter(); + + // START SNIPPET: io + // (1) + Iterable<AcmeValue> queryResult = dataSource; // Eg. Entities converted to Values + Writer writer = outputWriter; // Eg. to pipe data to another process or to a file + + // (2) + Function<AcmeValue, String> serialize = valueSerializer.serialize(); + + // (3) + Inputs.iterable( queryResult ).transferTo( Transforms.map( serialize, Outputs.text( writer ) ) ); + // END SNIPPET: io + + String string = writer.toString(); + StringReader inputReader = new StringReader( string ); + + // START SNIPPET: io + // (4) + Reader reader = inputReader; + List<AcmeValue> values = new ArrayList<AcmeValue>(); + + // (5) + Function<String, AcmeValue> deserialize = valueDeserializer.deserialize( AcmeValue.class ); + + // Deserialization of a collection of AcmeValue from a String. + // One serialized AcmeValue per line. + // (6) + Inputs.text( reader ).transferTo( Transforms.map( deserialize, Outputs.collection( values ) ) ); + // END SNIPPET: io + + assertThat( dataSource, equalTo( values ) ); + + // START SNIPPET: io + } + // END SNIPPET: io + + @Test + // TODO Move to SPI ! + // TODO Include in each ValueSerialization extensions documentation + public void assembledWithValuesModuleSerialization() + throws Exception + { + Application app = new Energy4Java().newApplication( new ApplicationAssembler() + { + @Override + public ApplicationAssembly assemble( ApplicationAssemblyFactory applicationFactory ) + throws AssemblyException + { + Assembler[][][] pancakes = new Assembler[][][] + { + { + { + new Assembler() + { + @Override + public void assemble( ModuleAssembly valuesModule ) + throws AssemblyException + { + valuesModule.layer().setName( "SINGLE-Layer" ); + valuesModule.setName( "VALUES-Module" ); + + valuesModule.values( SomeValue.class ); + } + } + }, + { + new Assembler() + { + @Override + public void assemble( ModuleAssembly servicesModule ) + throws AssemblyException + { + servicesModule.setName( "SERVICES-Module" ); + + Function<Application, Module> valuesModuleFinder = new Function<Application, Module>() + { + @Override + public Module map( Application app ) + { + return app.findModule( "SINGLE-Layer", "VALUES-Module" ); + } + }; + new OrgJsonValueSerializationAssembler(). + withValuesModuleFinder( valuesModuleFinder ). + assemble( servicesModule ); + } + } + } + } + }; + return applicationFactory.newApplicationAssembly( pancakes ); + } + } ); + app.activate(); + try + { + Module valuesModule = app.findModule( "SINGLE-Layer", "VALUES-Module" ); + SomeValue someValue = someNewValueInstance( valuesModule ); + + Module servicesModule = app.findModule( "SINGLE-Layer", "SERVICES-Module" ); + ValueSerialization valueSerialization = servicesModule.findService( ValueSerialization.class ).get(); + + String json = valueSerialization.serialize( someValue ); + assertThat( json, equalTo( "{\"foo\":\"bar\"}" ) ); + + SomeValue someNewValue = valueSerialization.deserialize( SomeValue.class, json ); + assertThat( someNewValue, equalTo( someValue ) ); + } + finally + { + app.passivate(); + } + } + + private SomeValue someNewValueInstance( Module module ) + { + ValueBuilder<SomeValue> builder = module.newValueBuilder( SomeValue.class ); + builder.prototype().foo().set( "bar" ); + return builder.newInstance(); + } +} http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/core/spi/src/docs/metrics.txt ---------------------------------------------------------------------- diff --git a/core/spi/src/docs/metrics.txt b/core/spi/src/docs/metrics.txt index 97ad11f..678db90 100644 --- a/core/spi/src/docs/metrics.txt +++ b/core/spi/src/docs/metrics.txt @@ -14,7 +14,7 @@ ////////////////////// [[core-spi-metrics,Metrics SPI]] -= Qi4j Metrics SPI = += Metrics SPI = It is very easy to create an extension for the Metrics SPI, simply by implementing the MetricsProvider. If only a subset of the factories/types are supported, there is a convenience adapter call MetricsProviderAdapter in the Metrics SPI package. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/core/spi/src/docs/spi.txt ---------------------------------------------------------------------- diff --git a/core/spi/src/docs/spi.txt b/core/spi/src/docs/spi.txt index 8dfb96e..df223b2 100644 --- a/core/spi/src/docs/spi.txt +++ b/core/spi/src/docs/spi.txt @@ -27,8 +27,9 @@ applications during the bootstrap phase. include::../../build/docs/buildinfo/artifact.txt[] -There are currently 4 Core SPI extensions; +There are currently 5 Core SPI extensions; + * ValueSerialization SPI * EntityStore SPI * Cache SPI * Indexing/Query SPI @@ -41,6 +42,10 @@ can be solved in a support manner, or that we need to extend the Core API to sup :leveloffset: {level3} +include::valueserialization.txt[] + +:leveloffset: {level3} + include::entitystore.txt[] :leveloffset: {level3} http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/core/spi/src/docs/valueserialization.txt ---------------------------------------------------------------------- diff --git a/core/spi/src/docs/valueserialization.txt b/core/spi/src/docs/valueserialization.txt new file mode 100644 index 0000000..9029cbc --- /dev/null +++ b/core/spi/src/docs/valueserialization.txt @@ -0,0 +1,52 @@ +[[core-spi-valueserialization,ValueSerialization SPI]] += ValueSerialization SPI = + +Simply implement ValueSerialization to create an extension for the ValueSerialization SPI. +The Core SPI module provides adapters to create pull-parsing capable ValueSerializers and pull-parsing and tree-parsing +capable ValueDeserializers. + +The behaviour described here apply to all ValueSerialization services implemented using the Core SPI adapters. Note that +nothing stops you from implementing an extension for the ValueSerialization SPI without relying on theses adapters. + +Theses adapters are tailored for serialization mechanisms that support the following two structures that can be nested: + + * an collection of name/value pairs. In various languages, this is realized as an object, record, struct, + dictionary, hash table, keyed list, or associative array, + * an ordered list of values. In most languages, this is realized as an array, vector, list, or sequence ; + +in other words, a JSON-like structure. + +Special attention is taken when dealing with Maps. They are serialized as an ordered list of collections of +name/value pairs to keep the Map order for least surprise. That way, even when the underlying serialization mechanism +do not keep the collection of name/value pairs order we can rely on it being kept. + +Here is a sample Map with two entries in JSON notation to make things clear: + +[source,javascript] +---- +[ + { "key": "foo", "value": "bar" }, + { "key": "cathedral", "value": "bazar" } +] +---- + +Among Plain Values (see the <<core-api-value,ValueSerialization API>> section) some are considered primitives to +underlying serialization mechanisms and by so handed/come without conversion to/from implementations. + +Primitive values can be one of: + + * String, + * Boolean or boolean, + * Integer or int, + * Long or long, + * Short or short, + * Byte or byte, + * Float or float, + * Double or double. + +Serialization is always done in a streaming manner using a pull-parsing based approach. + +Deserialization is done in a streaming manner using a pull-parsing based approach except when encountering a +ValueComposite. ValueComposite types are deserialized using a tree-parsing based approach. + +All this means that you can serialize and deserialize large collections of values without filling the heap. http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/extensions/valueserialization-jackson/src/docs/vs-jackson.txt ---------------------------------------------------------------------- diff --git a/extensions/valueserialization-jackson/src/docs/vs-jackson.txt b/extensions/valueserialization-jackson/src/docs/vs-jackson.txt new file mode 100644 index 0000000..3071215 --- /dev/null +++ b/extensions/valueserialization-jackson/src/docs/vs-jackson.txt @@ -0,0 +1,24 @@ +[[extension-vs-jackson, Jackson ValueSerialization]] += Jackson ValueSerialization = + +[devstatus] +-------------- +source=extensions/valueserialization-jackson/dev-status.xml +-------------- + +ValueSerialization Service backed by http://wiki.fasterxml.com/JacksonHome[Jackson]. + +include::../../build/docs/buildinfo/artifact.txt[] + +== Assembly == + +Assembly is done as follows: + +[snippet,java] +---- +source=extensions/valueserialization-jackson/src/test/java/org/qi4j/valueserialization/jackson/JacksonPlainValueSerializationTest.java +tag=assembly +---- + +See the ValueSerialization <<core-api-value,API>> and <<core-spi-valueserialization,SPI>> documentation for details and +usage. http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/extensions/valueserialization-orgjson/src/docs/vs-orgjson.txt ---------------------------------------------------------------------- diff --git a/extensions/valueserialization-orgjson/src/docs/vs-orgjson.txt b/extensions/valueserialization-orgjson/src/docs/vs-orgjson.txt new file mode 100644 index 0000000..087a006 --- /dev/null +++ b/extensions/valueserialization-orgjson/src/docs/vs-orgjson.txt @@ -0,0 +1,24 @@ +[[extension-vs-orgjson, org.json ValueSerialization]] += org.json ValueSerialization = + +[devstatus] +-------------- +source=extensions/valueserialization-orgjson/dev-status.xml +-------------- + +ValueSerialization Service backed by https://github.com/douglascrockford/JSON-java[org.json]. + +include::../../build/docs/buildinfo/artifact.txt[] + +== Assembly == + +Assembly is done as follows: + +[snippet,java] +---- +source=extensions/valueserialization-orgjson/src/test/java/org/qi4j/valueserialization/orgjson/OrgJsonPlainValueSerializationTest.java +tag=assembly +---- + +See the ValueSerialization <<core-api-value,API>> and <<core-spi-valueserialization,SPI>> documentation for details and +usage. http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/extensions/valueserialization-stax/src/docs/vs-stax.txt ---------------------------------------------------------------------- diff --git a/extensions/valueserialization-stax/src/docs/vs-stax.txt b/extensions/valueserialization-stax/src/docs/vs-stax.txt new file mode 100644 index 0000000..abe9ecf --- /dev/null +++ b/extensions/valueserialization-stax/src/docs/vs-stax.txt @@ -0,0 +1,24 @@ +[[extension-vs-stax, StAX ValueSerialization]] += StAX ValueSerialization = + +[devstatus] +-------------- +source=extensions/valueserialization-stax/dev-status.xml +-------------- + +ValueSerialization Service backed by http://jcp.org/en/jsr/detail?id=173[StAX]. + +include::../../build/docs/buildinfo/artifact.txt[] + +== Assembly == + +Assembly is done as follows: + +[snippet,java] +---- +source=extensions/valueserialization-stax/src/test/java/org/qi4j/valueserialization/stax/StaxPlainValueSerializationTest.java +tag=assembly +---- + +See the ValueSerialization <<core-api-value,API>> and <<core-spi-valueserialization,SPI>> documentation for details and +usage. http://git-wip-us.apache.org/repos/asf/zest-qi4j/blob/f875d7cf/manual/src/docs/userguide/extensions.txt ---------------------------------------------------------------------- diff --git a/manual/src/docs/userguide/extensions.txt b/manual/src/docs/userguide/extensions.txt index 56b958f..e2de272 100644 --- a/manual/src/docs/userguide/extensions.txt +++ b/manual/src/docs/userguide/extensions.txt @@ -9,6 +9,7 @@ We try to keep the Qi4j Core Runtime as lean as possible, and a lot of the power Extension SPI, which defines clear ways to extend the platform. There are currently the following Extensions types, each with possibly more than one implementation; + * Value Serialization * Entity Stores * Index / Query Engines * Entity Caches @@ -25,6 +26,18 @@ for our users. :leveloffset: 2 +include::../../../../extensions/valueserialization-orgjson/src/docs/vs-orgjson.txt[] + +:leveloffset: 2 + +include::../../../../extensions/valueserialization-jackson/src/docs/vs-jackson.txt[] + +:leveloffset: 2 + +include::../../../../extensions/valueserialization-stax/src/docs/vs-stax.txt[] + +:leveloffset: 2 + include::../../../../extensions/cache-ehcache/src/docs/ehcache.txt[] :leveloffset: 2
