Hi there Dan,
You are correct, the thought is there to add a flag to the registry to
indicate that a definition is custom and thus should not conflict with
the existing ids. Even if they types were to be stored with the current
Pdx type definitions, upon loading/registration of the custom type
definitions, any conflict will be reported and the custom set will not
be registered until all issues were addressed.
I also had the opinion of the "if they can provide me a typeId, then
surely they can provide me with a fully populated JSON document".
Referencing the example document from the wiki, an user can be created
with just a first and surname. It is not required to provide
currentAddress, previousAddresses, dob,etc... Whilst one could force the
client to provide all fields in the JSON document, it is not always
possible nor feasible to do so. In the POJO world we have a structured
data definition and the generation of a type definition is simple. This
done because from a serialization perspective we always make sure that
all fields are serialized. BUT if we were to change the serialization,
i.e not serialize a field because it is null, the type definition
behavior would be exactly the same as JSON. Only, in this case, because
we changed the type definition for the 'com.demo.User' object (at
runtime) the deserialization step for previous versions would fail.
I believe that if we were to be able to describe WHAT the structure of a
JSON document should be and define the type according to that
definition, we could improve performance (as we don't have to determine
type definitions for every JSON document), be more flexible in consuming
JSON documents that are only partially populated and lastly not
potentially cause a vast amount of JSON-based type definitions to be
generated.
In addition to just the JSON benefits, having a formal way of describing
the type definitions will allow us to better maintain the current
registered type definitions. In addition to this, it would allow
customers/clients to create type definitions, by hand, if they were to
have lost their type registry.
As final thought, the addition of the external type registration
process is not meant replace the current behavior. But rather enhance
its capabilities. If no external types will have been defined OR the
client does not provide a '@typeId' tag, the current JSON type
definition behavior will stay the same.
--Udo
On 12/21/16 18:20, Dan Smith wrote:
I'm assuming the type ids here are a different set than the type ids used
with regular PDX serialization so they won't conflict if the pdx registry
assigns 1 to some class and a user puts @typeId: 1 in their json?
I'm concerned that this won't really address the type explosion issue.
Users that are able to go to the effort of adding these typeIds to all of
their json are probably users that can produce consistently formatted json
in the first place. Users that have inconsistently formatted json are
probably not going to want or be able to add these type ids.
It might be better for us to pursue a way to store arbitrary documents that
are self describing. Our current approach for json documents is assuming
that the documents are all consistently formatted. We are infer a schema
for their documents store the field names in the type registry and the
field values in the serialized data. If we give people the option to store
and query self describing values, then users with inconsistent json could
just use that option and pay the extra storage cost.
-Dan
On Tue, Dec 20, 2016 at 4:53 PM, Udo Kohlmeyer <ukohlme...@gmail.com> wrote:
Hey there,
I've just completed a new proposal on the wiki for a new mechanism that
could be used to define a type definition for an object.
https://cwiki.apache.org/confluence/display/GEODE/Custom+
External+Type+Definition+Proposal+for+JSON
Primarily the new type definition proposal will hopefully help with the
"structuring" of JSON document definitions in a manner that will allow
users to submit JSON documents for data types without the need to provide
every field of the whole domain object type.
Please review and comment as required.
--Udo
