Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Jacob Barrett]

Correct, that is what this ticket is asking for.

> On Jan 26, 2018, at 10:09 AM, Anilkumar Gingade  wrote:
> 
> Jake, the requirement seems like users doesn't have to set the
> read-serialized flag; system needs to take care of that based on
> availability of domain-class in classpath.
> 
> -Anil.
> 
> 
> 
>> On Thu, Jan 25, 2018 at 2:01 PM, Jacob Barrett  wrote:
>> 
>> It is no different really than how the Java client works. If on a Java
>> client I put an object who's class implements PdxSerializable the object is
>> serialized as PDX. If form the Java client I call a function with that
>> object as an argument then the server will try to deserialize the PDX to
>> the named domain class. If the domain class has not been deployed in the
>> class path then a ClassNotFoundException is throw. So if from the .NET,
>> C++, whatever language, I have a domain class that implements some language
>> specific version of PdxSerializable (IPdxSerializable in .NET) the object
>> is sent as PDX. Since I haven't created a corresponding Java domain object
>> and deployed it then I will get a ClassNotFoundException. Yes, I could
>> write a Java domain object and deploy it, but I shouldn't have to. The
>> crappy work around we have for this is the pdx read serializable flag and
>> with it set to true my non-Java client can run server side functions and
>> queries without also having to write and deploy matching Java domain
>> objects. The downside is that it global and changes all the behavior of PDX
>> serialized objects.
>> 
>> 
>> On Thu, Jan 25, 2018 at 1:43 PM Galen O'Sullivan 
>> wrote:
>> 
>>> Hi Jake,
>>> 
>>> I'm not really familiar with how the .NET client works - from what you're
>>> saying, it sounds like there is a way to serialize PDX objects without
>>> having access to a domain class? Can the .NET client tag PDX-serialized
>>> objects with the name of the domain class that they represent, so that
>> the
>>> Java classes will get the objects they expect?
>>> 
>>> Thanks,
>>> Galen
>>> 
>>> On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett 
>>> wrote:
>>> 
 Anil, this wouldn't fix a function execution where a parameter sent to
>>> the
 function is a PDX serialized object. The function would be invoked on
>> the
 server with the current global setting and file to deserialize the PDX
>> to
 domain object. This is why the ask is for it to fall back to
>> PdxInstance
>>> if
 the domain object is not deployed.
 
 -Jake
 
 On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade >> 
 wrote:
 
> Internally, there is an option to override read-serialized flag (to
 true);
> the query engine and other components uses this to keep the data in
> serialized form and work with PdxInstance...
> 
> public static void setPdxReadSerialized(Cache cache, boolean
> readSerialized);
> 
> We had discussed, making this as a public api...so that any thread
>> that
 can
> work on PdxInstance can take advantage of it...
> 
> -Anil.
> 
> 
> On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett 
> wrote:
> 
>> Bruce, the flag only applies to values serialized with PDX,
>> DataSerializable objects are not effected by this property.
>> 
>> I think there is some real value here as a stop gap until we have a
> better
>> solution in Geode 2 where the user can have a per request context
>>> that
>> specifies what return type they would like. Consider the user that
>>> has
 an
>> existing application that uses domain objects but wants to deploy
 another
>> application that doesn't to the same Geode cluster. The only option
>>> is
 to
>> either have all PDX deserialize to domain objects or all returned
>> as
>> PdxInstance. One of the two applications will not work without
>> modification. Changing the behavior described by Addison splits the
>> difference. If the application is, like it is by default, configure
>>> to
>> deserialize PDX to the domain object but the domain object is not
> deployed
>> it will now give back the PDX instance rather than failing.
>> 
>> An explicit use case is user that has both a Java and .NET
>>> application.
> The
>> .NET application does not have any Java domain objects to deploy to
>>> the
>> server but does want to query or run server side functions. The
>> Java
>> application has deployed the domain objects to the server and
 distributed
>> functions are written expecting those domain objects on the server.
>>> The
>> user would have to create Java domain objects for the .NET
>>> application
 or
>> modify their Java application to expect PdxInstance.
>> 
>> 
>> -Jake
>> 
>> 
>> On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt <
 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Anilkumar Gingade]

Jake, the requirement seems like users doesn't have to set the
read-serialized flag; system needs to take care of that based on
availability of domain-class in classpath.

-Anil.



On Thu, Jan 25, 2018 at 2:01 PM, Jacob Barrett  wrote:

> It is no different really than how the Java client works. If on a Java
> client I put an object who's class implements PdxSerializable the object is
> serialized as PDX. If form the Java client I call a function with that
> object as an argument then the server will try to deserialize the PDX to
> the named domain class. If the domain class has not been deployed in the
> class path then a ClassNotFoundException is throw. So if from the .NET,
> C++, whatever language, I have a domain class that implements some language
> specific version of PdxSerializable (IPdxSerializable in .NET) the object
> is sent as PDX. Since I haven't created a corresponding Java domain object
> and deployed it then I will get a ClassNotFoundException. Yes, I could
> write a Java domain object and deploy it, but I shouldn't have to. The
> crappy work around we have for this is the pdx read serializable flag and
> with it set to true my non-Java client can run server side functions and
> queries without also having to write and deploy matching Java domain
> objects. The downside is that it global and changes all the behavior of PDX
> serialized objects.
>
>
> On Thu, Jan 25, 2018 at 1:43 PM Galen O'Sullivan 
> wrote:
>
> > Hi Jake,
> >
> > I'm not really familiar with how the .NET client works - from what you're
> > saying, it sounds like there is a way to serialize PDX objects without
> > having access to a domain class? Can the .NET client tag PDX-serialized
> > objects with the name of the domain class that they represent, so that
> the
> > Java classes will get the objects they expect?
> >
> > Thanks,
> > Galen
> >
> > On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett 
> > wrote:
> >
> > > Anil, this wouldn't fix a function execution where a parameter sent to
> > the
> > > function is a PDX serialized object. The function would be invoked on
> the
> > > server with the current global setting and file to deserialize the PDX
> to
> > > domain object. This is why the ask is for it to fall back to
> PdxInstance
> > if
> > > the domain object is not deployed.
> > >
> > > -Jake
> > >
> > > On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade  >
> > > wrote:
> > >
> > > > Internally, there is an option to override read-serialized flag (to
> > > true);
> > > > the query engine and other components uses this to keep the data in
> > > > serialized form and work with PdxInstance...
> > > >
> > > > public static void setPdxReadSerialized(Cache cache, boolean
> > > > readSerialized);
> > > >
> > > > We had discussed, making this as a public api...so that any thread
> that
> > > can
> > > > work on PdxInstance can take advantage of it...
> > > >
> > > > -Anil.
> > > >
> > > >
> > > > On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett 
> > > > wrote:
> > > >
> > > > > Bruce, the flag only applies to values serialized with PDX,
> > > > > DataSerializable objects are not effected by this property.
> > > > >
> > > > > I think there is some real value here as a stop gap until we have a
> > > > better
> > > > > solution in Geode 2 where the user can have a per request context
> > that
> > > > > specifies what return type they would like. Consider the user that
> > has
> > > an
> > > > > existing application that uses domain objects but wants to deploy
> > > another
> > > > > application that doesn't to the same Geode cluster. The only option
> > is
> > > to
> > > > > either have all PDX deserialize to domain objects or all returned
> as
> > > > > PdxInstance. One of the two applications will not work without
> > > > > modification. Changing the behavior described by Addison splits the
> > > > > difference. If the application is, like it is by default, configure
> > to
> > > > > deserialize PDX to the domain object but the domain object is not
> > > > deployed
> > > > > it will now give back the PDX instance rather than failing.
> > > > >
> > > > > An explicit use case is user that has both a Java and .NET
> > application.
> > > > The
> > > > > .NET application does not have any Java domain objects to deploy to
> > the
> > > > > server but does want to query or run server side functions. The
> Java
> > > > > application has deployed the domain objects to the server and
> > > distributed
> > > > > functions are written expecting those domain objects on the server.
> > The
> > > > > user would have to create Java domain objects for the .NET
> > application
> > > or
> > > > > modify their Java application to expect PdxInstance.
> > > > >
> > > > >
> > > > > -Jake
> > > > >
> > > > >
> > > > > On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt <
> > > bschucha...@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > > > 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Jacob Barrett]

It is no different really than how the Java client works. If on a Java
client I put an object who's class implements PdxSerializable the object is
serialized as PDX. If form the Java client I call a function with that
object as an argument then the server will try to deserialize the PDX to
the named domain class. If the domain class has not been deployed in the
class path then a ClassNotFoundException is throw. So if from the .NET,
C++, whatever language, I have a domain class that implements some language
specific version of PdxSerializable (IPdxSerializable in .NET) the object
is sent as PDX. Since I haven't created a corresponding Java domain object
and deployed it then I will get a ClassNotFoundException. Yes, I could
write a Java domain object and deploy it, but I shouldn't have to. The
crappy work around we have for this is the pdx read serializable flag and
with it set to true my non-Java client can run server side functions and
queries without also having to write and deploy matching Java domain
objects. The downside is that it global and changes all the behavior of PDX
serialized objects.


On Thu, Jan 25, 2018 at 1:43 PM Galen O'Sullivan 
wrote:

> Hi Jake,
>
> I'm not really familiar with how the .NET client works - from what you're
> saying, it sounds like there is a way to serialize PDX objects without
> having access to a domain class? Can the .NET client tag PDX-serialized
> objects with the name of the domain class that they represent, so that the
> Java classes will get the objects they expect?
>
> Thanks,
> Galen
>
> On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett 
> wrote:
>
> > Anil, this wouldn't fix a function execution where a parameter sent to
> the
> > function is a PDX serialized object. The function would be invoked on the
> > server with the current global setting and file to deserialize the PDX to
> > domain object. This is why the ask is for it to fall back to PdxInstance
> if
> > the domain object is not deployed.
> >
> > -Jake
> >
> > On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade 
> > wrote:
> >
> > > Internally, there is an option to override read-serialized flag (to
> > true);
> > > the query engine and other components uses this to keep the data in
> > > serialized form and work with PdxInstance...
> > >
> > > public static void setPdxReadSerialized(Cache cache, boolean
> > > readSerialized);
> > >
> > > We had discussed, making this as a public api...so that any thread that
> > can
> > > work on PdxInstance can take advantage of it...
> > >
> > > -Anil.
> > >
> > >
> > > On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett 
> > > wrote:
> > >
> > > > Bruce, the flag only applies to values serialized with PDX,
> > > > DataSerializable objects are not effected by this property.
> > > >
> > > > I think there is some real value here as a stop gap until we have a
> > > better
> > > > solution in Geode 2 where the user can have a per request context
> that
> > > > specifies what return type they would like. Consider the user that
> has
> > an
> > > > existing application that uses domain objects but wants to deploy
> > another
> > > > application that doesn't to the same Geode cluster. The only option
> is
> > to
> > > > either have all PDX deserialize to domain objects or all returned as
> > > > PdxInstance. One of the two applications will not work without
> > > > modification. Changing the behavior described by Addison splits the
> > > > difference. If the application is, like it is by default, configure
> to
> > > > deserialize PDX to the domain object but the domain object is not
> > > deployed
> > > > it will now give back the PDX instance rather than failing.
> > > >
> > > > An explicit use case is user that has both a Java and .NET
> application.
> > > The
> > > > .NET application does not have any Java domain objects to deploy to
> the
> > > > server but does want to query or run server side functions. The Java
> > > > application has deployed the domain objects to the server and
> > distributed
> > > > functions are written expecting those domain objects on the server.
> The
> > > > user would have to create Java domain objects for the .NET
> application
> > or
> > > > modify their Java application to expect PdxInstance.
> > > >
> > > >
> > > > -Jake
> > > >
> > > >
> > > > On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt <
> > bschucha...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > I've found the current read-serialized property to be pretty
> useless.
> > > > >
> > > > > Having said that, what if the value isn't actually in serialized
> form
> > > in
> > > > > the local cache?  Is Geode supposed to serialize it & return it?
> > What
> > > > > if it isn't PDX-serialized?  Do we return a byte array?
> > > > >
> > > > >
> > > > > On 1/24/18 12:21 PM, Dan Smith wrote:
> > > > > > Is this really just a workaround for the fact that the
> > > read-serialized

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Galen O'Sullivan]

Hi Jake,

I'm not really familiar with how the .NET client works - from what you're
saying, it sounds like there is a way to serialize PDX objects without
having access to a domain class? Can the .NET client tag PDX-serialized
objects with the name of the domain class that they represent, so that the
Java classes will get the objects they expect?

Thanks,
Galen

On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett  wrote:

> Anil, this wouldn't fix a function execution where a parameter sent to the
> function is a PDX serialized object. The function would be invoked on the
> server with the current global setting and file to deserialize the PDX to
> domain object. This is why the ask is for it to fall back to PdxInstance if
> the domain object is not deployed.
>
> -Jake
>
> On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade 
> wrote:
>
> > Internally, there is an option to override read-serialized flag (to
> true);
> > the query engine and other components uses this to keep the data in
> > serialized form and work with PdxInstance...
> >
> > public static void setPdxReadSerialized(Cache cache, boolean
> > readSerialized);
> >
> > We had discussed, making this as a public api...so that any thread that
> can
> > work on PdxInstance can take advantage of it...
> >
> > -Anil.
> >
> >
> > On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett 
> > wrote:
> >
> > > Bruce, the flag only applies to values serialized with PDX,
> > > DataSerializable objects are not effected by this property.
> > >
> > > I think there is some real value here as a stop gap until we have a
> > better
> > > solution in Geode 2 where the user can have a per request context that
> > > specifies what return type they would like. Consider the user that has
> an
> > > existing application that uses domain objects but wants to deploy
> another
> > > application that doesn't to the same Geode cluster. The only option is
> to
> > > either have all PDX deserialize to domain objects or all returned as
> > > PdxInstance. One of the two applications will not work without
> > > modification. Changing the behavior described by Addison splits the
> > > difference. If the application is, like it is by default, configure to
> > > deserialize PDX to the domain object but the domain object is not
> > deployed
> > > it will now give back the PDX instance rather than failing.
> > >
> > > An explicit use case is user that has both a Java and .NET application.
> > The
> > > .NET application does not have any Java domain objects to deploy to the
> > > server but does want to query or run server side functions. The Java
> > > application has deployed the domain objects to the server and
> distributed
> > > functions are written expecting those domain objects on the server. The
> > > user would have to create Java domain objects for the .NET application
> or
> > > modify their Java application to expect PdxInstance.
> > >
> > >
> > > -Jake
> > >
> > >
> > > On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt <
> bschucha...@apache.org
> > >
> > > wrote:
> > >
> > > > +1
> > > >
> > > > I've found the current read-serialized property to be pretty useless.
> > > >
> > > > Having said that, what if the value isn't actually in serialized form
> > in
> > > > the local cache?  Is Geode supposed to serialize it & return it?
> What
> > > > if it isn't PDX-serialized?  Do we return a byte array?
> > > >
> > > >
> > > > On 1/24/18 12:21 PM, Dan Smith wrote:
> > > > > Is this really just a workaround for the fact that the
> > read-serialized
> > > > flag
> > > > > applies to the whole cache? I can see that if you have mix of
> regions
> > > > with
> > > > > and without domain classes on the server you might want this
> feature.
> > > Can
> > > > > you provide some more background on your use case?
> > > > >
> > > > > IMO we should get rid of read-serialized in favor of APIs that let
> > the
> > > > user
> > > > > decide whether they get a domain class or a PdxInstance.
> > > > >
> > > > > -Dan
> > > > >
> > > > > On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan <
> > > gosulli...@pivotal.io
> > > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Addison,
> > > > >>
> > > > >> What kind of setup do you have that is causing you to have PDX
> > > > serialized
> > > > >> objects that cannot be deserialized? Do you have classes that are
> > > > present
> > > > >> on some servers but not others?
> > > > >>
> > > > >> This change would break the contract of region.get() , which is
> that
> > > it
> > > > >> returns a value of a type that can be put into the region.
> > > > >>
> > > > >> Returning something that isn't what the user is expecting to be in
> > the
> > > > >> region would require users to check for PdxInstances every time
> they
> > > > get a
> > > > >> value -- even though the type annotations say that you can't get a
> > > > >> PdxInstance back (except for Region ).
> > > > >>
> > > > >> I think it would be 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[John Blum]

*One more thing...*

Regarding arguments passed to Functions, which can equally apply to return
types as well, particularly in SDG, since Functions can be methods on a
POJO, having strongly typed return values, for example ...

@GemfireFunction(..)
Customer someFunction(CustomerIdentifier id, PdxInstance someObject, Object
someValue, ...) { .. }

I had to handle this very problem, that is, when PDX is enabled and the
Function either takes a PdxInstance or perhaps a Java type (e.g. Customer)
and PDX bytes are sent; what then; how to handle?

Another scenario that needs special consideration is, if the Function takes
a generic argument, like Object, do you hand the user a PdxInstance (if PDX
is in play), or does the framework try to convert the type (i.e.
PdxInstance.getObject()) if possible, meaning the type is possibly on the
CLASSPATH if the Function argument is strongly typed?

Anyway, awhile back, I added some additional logic in SDG to more
intelligently handle such scenarios [1].  While it isn't perfect, it is far
more robust than what existed previously (thus, "building up").

The point is, the user did not need to do anything (i.e. extra
configuration) in order for SDG to do the right thing for the user,
covering 80% of the most common scenarios.

-j


[1]
https://github.com/spring-projects/spring-data-geode/blob/2.0.3.RELEASE/src/main/java/org/springframework/data/gemfire/function/PdxFunctionArgumentResolver.java





On Thu, Jan 25, 2018 at 1:16 PM, John Blum  wrote:

> To Swap's point about making ReflectionBasedAutoSerializer the default
> (which I think is a good idea), comparably, in SDG, I have made SDG's
> o.s.d.g.mapping.MappingPdxSerializer [1] the default when PDX is enabled
> (i.e. @EnablePdx).  No need to explicitly declare/register it.  SDG's
> MappingPdxSerializer leverage's *Spring Data's* powerful mapping
> infrastructure to convert Java objects to PDX and back.  This is even more
> robust than Apache Geode's own ReflectionBasedAutoSerializer, but clearly
> *Spring* specific.
>
> More details available here [2] (which as it turns out, needs to be
> updated based on the latest developments, namely DATAGEODE-72 [3]).
>
> -John
>
>
> [1] https://docs.spring.io/spring-data/geode/docs/current/api/org/
> springframework/data/gemfire/mapping/MappingPdxSerializer.html
> [2] https://docs.spring.io/spring-data/geode/docs/current/reference/html/#
> bootstrap-annotation-config-pdx
> [3] https://jira.spring.io/browse/DATAGEODE-72
>
>
> On Thu, Jan 25, 2018 at 12:44 PM, Jacob Barrett 
> wrote:
>
>>
>>
>> > On Jan 25, 2018, at 12:06 PM, Anilkumar Gingade 
>> wrote:
>> >
>> > Jake,
>> >
>> > Interesting...Its the argument passed to function that is in
>> deserialized
>> > form, right?
>>
>> Yes, if pdx read serialized is false (default) the argument is
>> deserialized to domain object or fails if no domain object found. If read
>> serialized is true then all arguments that are PDX serialized and passed in
>> as PdxInstance. Sett my this flag breaks all previous deployed functions
>> that expected domain objects in the argument.
>>
>> -Jake
>>
>>
>> >
>> > -Anil.
>> >
>> >
>> >
>> >
>> >
>> >
>> >> On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett 
>> wrote:
>> >>
>> >> Anil, this wouldn't fix a function execution where a parameter sent to
>> the
>> >> function is a PDX serialized object. The function would be invoked on
>> the
>> >> server with the current global setting and file to deserialize the PDX
>> to
>> >> domain object. This is why the ask is for it to fall back to
>> PdxInstance if
>> >> the domain object is not deployed.
>> >>
>> >> -Jake
>> >>
>> >> On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade > >
>> >> wrote:
>> >>
>> >>> Internally, there is an option to override read-serialized flag (to
>> >> true);
>> >>> the query engine and other components uses this to keep the data in
>> >>> serialized form and work with PdxInstance...
>> >>>
>> >>> public static void setPdxReadSerialized(Cache cache, boolean
>> >>> readSerialized);
>> >>>
>> >>> We had discussed, making this as a public api...so that any thread
>> that
>> >> can
>> >>> work on PdxInstance can take advantage of it...
>> >>>
>> >>> -Anil.
>> >>>
>> >>>
>> >>> On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett 
>> >>> wrote:
>> >>>
>>  Bruce, the flag only applies to values serialized with PDX,
>>  DataSerializable objects are not effected by this property.
>> 
>>  I think there is some real value here as a stop gap until we have a
>> >>> better
>>  solution in Geode 2 where the user can have a per request context
>> that
>>  specifies what return type they would like. Consider the user that
>> has
>> >> an
>>  existing application that uses domain objects but wants to deploy
>> >> another
>>  application that doesn't to the same Geode cluster. The only option
>> is
>> >> to
>> 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[John Blum]

To Swap's point about making ReflectionBasedAutoSerializer the default
(which I think is a good idea), comparably, in SDG, I have made SDG's
o.s.d.g.mapping.MappingPdxSerializer [1] the default when PDX is enabled
(i.e. @EnablePdx).  No need to explicitly declare/register it.  SDG's
MappingPdxSerializer leverage's *Spring Data's* powerful mapping
infrastructure to convert Java objects to PDX and back.  This is even more
robust than Apache Geode's own ReflectionBasedAutoSerializer, but clearly
*Spring* specific.

More details available here [2] (which as it turns out, needs to be updated
based on the latest developments, namely DATAGEODE-72 [3]).

-John


[1]
https://docs.spring.io/spring-data/geode/docs/current/api/org/springframework/data/gemfire/mapping/MappingPdxSerializer.html
[2]
https://docs.spring.io/spring-data/geode/docs/current/reference/html/#bootstrap-annotation-config-pdx
[3] https://jira.spring.io/browse/DATAGEODE-72


On Thu, Jan 25, 2018 at 12:44 PM, Jacob Barrett  wrote:

>
>
> > On Jan 25, 2018, at 12:06 PM, Anilkumar Gingade 
> wrote:
> >
> > Jake,
> >
> > Interesting...Its the argument passed to function that is in deserialized
> > form, right?
>
> Yes, if pdx read serialized is false (default) the argument is
> deserialized to domain object or fails if no domain object found. If read
> serialized is true then all arguments that are PDX serialized and passed in
> as PdxInstance. Sett my this flag breaks all previous deployed functions
> that expected domain objects in the argument.
>
> -Jake
>
>
> >
> > -Anil.
> >
> >
> >
> >
> >
> >
> >> On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett 
> wrote:
> >>
> >> Anil, this wouldn't fix a function execution where a parameter sent to
> the
> >> function is a PDX serialized object. The function would be invoked on
> the
> >> server with the current global setting and file to deserialize the PDX
> to
> >> domain object. This is why the ask is for it to fall back to
> PdxInstance if
> >> the domain object is not deployed.
> >>
> >> -Jake
> >>
> >> On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade 
> >> wrote:
> >>
> >>> Internally, there is an option to override read-serialized flag (to
> >> true);
> >>> the query engine and other components uses this to keep the data in
> >>> serialized form and work with PdxInstance...
> >>>
> >>> public static void setPdxReadSerialized(Cache cache, boolean
> >>> readSerialized);
> >>>
> >>> We had discussed, making this as a public api...so that any thread that
> >> can
> >>> work on PdxInstance can take advantage of it...
> >>>
> >>> -Anil.
> >>>
> >>>
> >>> On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett 
> >>> wrote:
> >>>
>  Bruce, the flag only applies to values serialized with PDX,
>  DataSerializable objects are not effected by this property.
> 
>  I think there is some real value here as a stop gap until we have a
> >>> better
>  solution in Geode 2 where the user can have a per request context that
>  specifies what return type they would like. Consider the user that has
> >> an
>  existing application that uses domain objects but wants to deploy
> >> another
>  application that doesn't to the same Geode cluster. The only option is
> >> to
>  either have all PDX deserialize to domain objects or all returned as
>  PdxInstance. One of the two applications will not work without
>  modification. Changing the behavior described by Addison splits the
>  difference. If the application is, like it is by default, configure to
>  deserialize PDX to the domain object but the domain object is not
> >>> deployed
>  it will now give back the PDX instance rather than failing.
> 
>  An explicit use case is user that has both a Java and .NET
> application.
> >>> The
>  .NET application does not have any Java domain objects to deploy to
> the
>  server but does want to query or run server side functions. The Java
>  application has deployed the domain objects to the server and
> >> distributed
>  functions are written expecting those domain objects on the server.
> The
>  user would have to create Java domain objects for the .NET application
> >> or
>  modify their Java application to expect PdxInstance.
> 
> 
>  -Jake
> 
> 
>  On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt <
> >> bschucha...@apache.org
> 
>  wrote:
> 
> > +1
> >
> > I've found the current read-serialized property to be pretty useless.
> >
> > Having said that, what if the value isn't actually in serialized form
> >>> in
> > the local cache?  Is Geode supposed to serialize it & return it?
> >> What
> > if it isn't PDX-serialized?  Do we return a byte array?
> >
> >
> >> On 1/24/18 12:21 PM, Dan Smith wrote:
> >> Is this really just a workaround for the fact that the
> >>> 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Jacob Barrett]

> On Jan 25, 2018, at 12:06 PM, Anilkumar Gingade  wrote:
> 
> Jake,
> 
> Interesting...Its the argument passed to function that is in deserialized
> form, right?

Yes, if pdx read serialized is false (default) the argument is deserialized to 
domain object or fails if no domain object found. If read serialized is true 
then all arguments that are PDX serialized and passed in as PdxInstance. Sett 
my this flag breaks all previous deployed functions that expected domain 
objects in the argument.

-Jake


> 
> -Anil.
> 
> 
> 
> 
> 
> 
>> On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett  wrote:
>> 
>> Anil, this wouldn't fix a function execution where a parameter sent to the
>> function is a PDX serialized object. The function would be invoked on the
>> server with the current global setting and file to deserialize the PDX to
>> domain object. This is why the ask is for it to fall back to PdxInstance if
>> the domain object is not deployed.
>> 
>> -Jake
>> 
>> On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade 
>> wrote:
>> 
>>> Internally, there is an option to override read-serialized flag (to
>> true);
>>> the query engine and other components uses this to keep the data in
>>> serialized form and work with PdxInstance...
>>> 
>>> public static void setPdxReadSerialized(Cache cache, boolean
>>> readSerialized);
>>> 
>>> We had discussed, making this as a public api...so that any thread that
>> can
>>> work on PdxInstance can take advantage of it...
>>> 
>>> -Anil.
>>> 
>>> 
>>> On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett 
>>> wrote:
>>> 
 Bruce, the flag only applies to values serialized with PDX,
 DataSerializable objects are not effected by this property.
 
 I think there is some real value here as a stop gap until we have a
>>> better
 solution in Geode 2 where the user can have a per request context that
 specifies what return type they would like. Consider the user that has
>> an
 existing application that uses domain objects but wants to deploy
>> another
 application that doesn't to the same Geode cluster. The only option is
>> to
 either have all PDX deserialize to domain objects or all returned as
 PdxInstance. One of the two applications will not work without
 modification. Changing the behavior described by Addison splits the
 difference. If the application is, like it is by default, configure to
 deserialize PDX to the domain object but the domain object is not
>>> deployed
 it will now give back the PDX instance rather than failing.
 
 An explicit use case is user that has both a Java and .NET application.
>>> The
 .NET application does not have any Java domain objects to deploy to the
 server but does want to query or run server side functions. The Java
 application has deployed the domain objects to the server and
>> distributed
 functions are written expecting those domain objects on the server. The
 user would have to create Java domain objects for the .NET application
>> or
 modify their Java application to expect PdxInstance.
 
 
 -Jake
 
 
 On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt <
>> bschucha...@apache.org
 
 wrote:
 
> +1
> 
> I've found the current read-serialized property to be pretty useless.
> 
> Having said that, what if the value isn't actually in serialized form
>>> in
> the local cache?  Is Geode supposed to serialize it & return it?
>> What
> if it isn't PDX-serialized?  Do we return a byte array?
> 
> 
>> On 1/24/18 12:21 PM, Dan Smith wrote:
>> Is this really just a workaround for the fact that the
>>> read-serialized
> flag
>> applies to the whole cache? I can see that if you have mix of
>> regions
> with
>> and without domain classes on the server you might want this
>> feature.
 Can
>> you provide some more background on your use case?
>> 
>> IMO we should get rid of read-serialized in favor of APIs that let
>>> the
> user
>> decide whether they get a domain class or a PdxInstance.
>> 
>> -Dan
>> 
>> On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan <
 gosulli...@pivotal.io
>> 
>> wrote:
>> 
>>> Hi Addison,
>>> 
>>> What kind of setup do you have that is causing you to have PDX
> serialized
>>> objects that cannot be deserialized? Do you have classes that are
> present
>>> on some servers but not others?
>>> 
>>> This change would break the contract of region.get() , which is
>> that
 it
>>> returns a value of a type that can be put into the region.
>>> 
>>> Returning something that isn't what the user is expecting to be in
>>> the
>>> region would require users to check for PdxInstances every time
>> they
> get a
>>> value -- even though the type annotations say that you can't 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Swapnil Bawaskar]

+1 to making PDX being more usable by adding an API rather than the current
flag.

I also think we should make the client-side usage of PDX simpler by always
using ReflectionBasedAutoSerializer when no serialization mechanism has
been specified see GEODE-2722
.

On Thu, Jan 25, 2018 at 11:07 AM John Blum  wrote:

> I have always thought/wondered, why not just store the data in serialized
> form always.  There are several reasons to do so...
>
> 1. Whenever data is transferred between client & server, between peers,
> over the WAN, overflowed to disk or persisted to disk, it must be
> serialized.
> 2. Naturally it follows that if the data is always stored in serialized
> form, it cuts down on de/serialization overhead.
> 3. Additionally, there is no need for or reduces the flags and other
> configuration settings to configure serialization making it simpler to
> understand, simpler to use.
> 4. When using PDX, Apache Geode is immediately interoperable between
> multiple language clients, primarily Java and .NET/C++, but even other
> language clients, e.g. JavaScript, Ruby, etc, where JSON is serialized to
> PDX.
> 5. PDX is queryable without deserialization. This is HUGE and maybe the
> most important reason!
>
>
>
> The last 2 points suggest that the default serialization format should be
> PDX, and truthfully, I am not really opposed to that.  Although, there are
> some problems with this.
>
> A. PDX does not handle cyclic dependencies unlike Java Serialization.
> However, Java Serialization has massive overhead and is not interoperable
> with native language and other language clients (e.g. JavaScript).
>
> B. PDX does not handle Deltas unlike DataSerialization.  However, even when
> using Deltas with DataSerialization, you must deserialize the data to apply
> the delta.  Quite frankly and ironically, PDX seems better suited to
> handling Deltas than DataSerialization, and without deserializing.
>
> So, I would double down on PDX and forget DataSerialization and Java
> Serialization.  And by "forget", I mean that Apache Geode never "stores"
> DataSerialized or Java Serialized bytes; only PDX!
>
> Therefore, solve the cyclic dependency problem and introduce proper Delta
> handling without deserialization.  Then, optimize it!  Make PDX the best
> serialization option for Java, and specifically for Apache Geode.  With 1
> serialization format to worry about there is less to maintain, less data to
> convert if the user needs to switch.  Flexibility is not always a good
> thing.  It is easier to build up than to build down if you know what I
> mean.
>
> I have made PDX a first class citizen in *Spring Data for Apache Geode*, in
> multiple key functional areas of the framework (e.g. Repositories) and dead
> simple to use/enable (i.e. @EnablePdx).
>
>
>
> Regarding .NET/C++.  Truthfully, I don't really buy the argument that
> .NET/C++ users shouldn't have to write Java types.  If the data is always
> kept serialized, then technically they shouldn't have to, but they are
> already writing their Functions in Java.  Besides, it is not like every
> type needs a Java type, only types that need to be deserialized, if at all.
>
> If the application consists of both Java and .NET/C++ clients, and the Java
> devs want to work with high-level Java types, then they don't really have a
> choice.  However, we can keep the de/serialization overhead at the point of
> access (e.g. in the Function, executed on a particular node, at the time of
> access), to a minimum.
>
> A simple API like...
>
> JavaType object = pdxInstance.getObject(Class type);
>
> ... would do the trick.
>
> The type argument does not need to be the original type that the PDX type
> meta-data was created from, either.  It could be a "projection".  The only
> concern Apache Geode has is mapping PDX fields to an instance of
> "JavaType", where PDX fields are mapped to writable "JavaType" properties
> (perhaps using Reflection here).
>
> If the JavaType does not contain a property matching a PDX field, no big
> deal.  This is the basis for our versioned type handling anyhow
> (adding/removing a field/property).  However, the inverse is a bit more an
> interesting problem, the JavaType has a field/property that is not
> currently stored in PDX.  Perhaps throw an error, or provide a default
> value, or whatever.  That could be configurable.
>
> Maybe, just maybe, a user has the ability to provide their own Converter,
> with it's own custom behavior...
>
> interface Converter {
>
>   T convert(PdxInstance pdxInstance);
>
> }
>
> class JavaTypeConverter extends Converter {
>
>   JavaType convert(PdxInstance pdxInstance) { ... }
>
> }
>
> Then...
>
> Converter javaTypeConverter = new JavaTypeConverter();
> ...
> JavaType object = pdxInstance.getObject(javaTypeConverter);
>
>
> *One final thought...*
>
> Ultimately, I'd like to see Apache Geode introduce a common
> framework/interface for 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Anilkumar Gingade]

+1 storing data in binary form (long time ask).




On Thu, Jan 25, 2018 at 11:07 AM, John Blum  wrote:

> I have always thought/wondered, why not just store the data in serialized
> form always.  There are several reasons to do so...
>
> 1. Whenever data is transferred between client & server, between peers,
> over the WAN, overflowed to disk or persisted to disk, it must be
> serialized.
> 2. Naturally it follows that if the data is always stored in serialized
> form, it cuts down on de/serialization overhead.
> 3. Additionally, there is no need for or reduces the flags and other
> configuration settings to configure serialization making it simpler to
> understand, simpler to use.
> 4. When using PDX, Apache Geode is immediately interoperable between
> multiple language clients, primarily Java and .NET/C++, but even other
> language clients, e.g. JavaScript, Ruby, etc, where JSON is serialized to
> PDX.
> 5. PDX is queryable without deserialization. This is HUGE and maybe the
> most important reason!
>
>
>
> The last 2 points suggest that the default serialization format should be
> PDX, and truthfully, I am not really opposed to that.  Although, there are
> some problems with this.
>
> A. PDX does not handle cyclic dependencies unlike Java Serialization.
> However, Java Serialization has massive overhead and is not interoperable
> with native language and other language clients (e.g. JavaScript).
>
> B. PDX does not handle Deltas unlike DataSerialization.  However, even when
> using Deltas with DataSerialization, you must deserialize the data to apply
> the delta.  Quite frankly and ironically, PDX seems better suited to
> handling Deltas than DataSerialization, and without deserializing.
>
> So, I would double down on PDX and forget DataSerialization and Java
> Serialization.  And by "forget", I mean that Apache Geode never "stores"
> DataSerialized or Java Serialized bytes; only PDX!
>
> Therefore, solve the cyclic dependency problem and introduce proper Delta
> handling without deserialization.  Then, optimize it!  Make PDX the best
> serialization option for Java, and specifically for Apache Geode.  With 1
> serialization format to worry about there is less to maintain, less data to
> convert if the user needs to switch.  Flexibility is not always a good
> thing.  It is easier to build up than to build down if you know what I
> mean.
>
> I have made PDX a first class citizen in *Spring Data for Apache Geode*, in
> multiple key functional areas of the framework (e.g. Repositories) and dead
> simple to use/enable (i.e. @EnablePdx).
>
>
>
> Regarding .NET/C++.  Truthfully, I don't really buy the argument that
> .NET/C++ users shouldn't have to write Java types.  If the data is always
> kept serialized, then technically they shouldn't have to, but they are
> already writing their Functions in Java.  Besides, it is not like every
> type needs a Java type, only types that need to be deserialized, if at all.
>
> If the application consists of both Java and .NET/C++ clients, and the Java
> devs want to work with high-level Java types, then they don't really have a
> choice.  However, we can keep the de/serialization overhead at the point of
> access (e.g. in the Function, executed on a particular node, at the time of
> access), to a minimum.
>
> A simple API like...
>
> JavaType object = pdxInstance.getObject(Class type);
>
> ... would do the trick.
>
> The type argument does not need to be the original type that the PDX type
> meta-data was created from, either.  It could be a "projection".  The only
> concern Apache Geode has is mapping PDX fields to an instance of
> "JavaType", where PDX fields are mapped to writable "JavaType" properties
> (perhaps using Reflection here).
>
> If the JavaType does not contain a property matching a PDX field, no big
> deal.  This is the basis for our versioned type handling anyhow
> (adding/removing a field/property).  However, the inverse is a bit more an
> interesting problem, the JavaType has a field/property that is not
> currently stored in PDX.  Perhaps throw an error, or provide a default
> value, or whatever.  That could be configurable.
>
> Maybe, just maybe, a user has the ability to provide their own Converter,
> with it's own custom behavior...
>
> interface Converter {
>
>   T convert(PdxInstance pdxInstance);
>
> }
>
> class JavaTypeConverter extends Converter {
>
>   JavaType convert(PdxInstance pdxInstance) { ... }
>
> }
>
> Then...
>
> Converter javaTypeConverter = new JavaTypeConverter();
> ...
> JavaType object = pdxInstance.getObject(javaTypeConverter);
>
>
> *One final thought...*
>
> Ultimately, I'd like to see Apache Geode introduce a common
> framework/interface for serialization, so that different serialization
> strategies, or "providers", could be introduced and used by our users based
> on their preferences and/or application's needs.
>
> Keep in mind, the users data might not just live in Apache Geode, which is
> 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[John Blum]

I have always thought/wondered, why not just store the data in serialized
form always.  There are several reasons to do so...

1. Whenever data is transferred between client & server, between peers,
over the WAN, overflowed to disk or persisted to disk, it must be
serialized.
2. Naturally it follows that if the data is always stored in serialized
form, it cuts down on de/serialization overhead.
3. Additionally, there is no need for or reduces the flags and other
configuration settings to configure serialization making it simpler to
understand, simpler to use.
4. When using PDX, Apache Geode is immediately interoperable between
multiple language clients, primarily Java and .NET/C++, but even other
language clients, e.g. JavaScript, Ruby, etc, where JSON is serialized to
PDX.
5. PDX is queryable without deserialization. This is HUGE and maybe the
most important reason!



The last 2 points suggest that the default serialization format should be
PDX, and truthfully, I am not really opposed to that.  Although, there are
some problems with this.

A. PDX does not handle cyclic dependencies unlike Java Serialization.
However, Java Serialization has massive overhead and is not interoperable
with native language and other language clients (e.g. JavaScript).

B. PDX does not handle Deltas unlike DataSerialization.  However, even when
using Deltas with DataSerialization, you must deserialize the data to apply
the delta.  Quite frankly and ironically, PDX seems better suited to
handling Deltas than DataSerialization, and without deserializing.

So, I would double down on PDX and forget DataSerialization and Java
Serialization.  And by "forget", I mean that Apache Geode never "stores"
DataSerialized or Java Serialized bytes; only PDX!

Therefore, solve the cyclic dependency problem and introduce proper Delta
handling without deserialization.  Then, optimize it!  Make PDX the best
serialization option for Java, and specifically for Apache Geode.  With 1
serialization format to worry about there is less to maintain, less data to
convert if the user needs to switch.  Flexibility is not always a good
thing.  It is easier to build up than to build down if you know what I mean.

I have made PDX a first class citizen in *Spring Data for Apache Geode*, in
multiple key functional areas of the framework (e.g. Repositories) and dead
simple to use/enable (i.e. @EnablePdx).



Regarding .NET/C++.  Truthfully, I don't really buy the argument that
.NET/C++ users shouldn't have to write Java types.  If the data is always
kept serialized, then technically they shouldn't have to, but they are
already writing their Functions in Java.  Besides, it is not like every
type needs a Java type, only types that need to be deserialized, if at all.

If the application consists of both Java and .NET/C++ clients, and the Java
devs want to work with high-level Java types, then they don't really have a
choice.  However, we can keep the de/serialization overhead at the point of
access (e.g. in the Function, executed on a particular node, at the time of
access), to a minimum.

A simple API like...

JavaType object = pdxInstance.getObject(Class type);

... would do the trick.

The type argument does not need to be the original type that the PDX type
meta-data was created from, either.  It could be a "projection".  The only
concern Apache Geode has is mapping PDX fields to an instance of
"JavaType", where PDX fields are mapped to writable "JavaType" properties
(perhaps using Reflection here).

If the JavaType does not contain a property matching a PDX field, no big
deal.  This is the basis for our versioned type handling anyhow
(adding/removing a field/property).  However, the inverse is a bit more an
interesting problem, the JavaType has a field/property that is not
currently stored in PDX.  Perhaps throw an error, or provide a default
value, or whatever.  That could be configurable.

Maybe, just maybe, a user has the ability to provide their own Converter,
with it's own custom behavior...

interface Converter {

  T convert(PdxInstance pdxInstance);

}

class JavaTypeConverter extends Converter {

  JavaType convert(PdxInstance pdxInstance) { ... }

}

Then...

Converter javaTypeConverter = new JavaTypeConverter();
...
JavaType object = pdxInstance.getObject(javaTypeConverter);


*One final thought...*

Ultimately, I'd like to see Apache Geode introduce a common
framework/interface for serialization, so that different serialization
strategies, or "providers", could be introduced and used by our users based
on their preferences and/or application's needs.

Keep in mind, the users data might not just live in Apache Geode, which is
particularly true in an increasingly Microservices world.  Other
technologies (e.g. Messaging Buses/Queues) are not going to know PDX.  PDX
would be the default, enabled serialization strategy/provider for Apache
Geode, provided by Apache Geode OOTB. This maybe 1 reason to still support
Java Serialization, given it is a 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Jacob Barrett]

Anil, this wouldn't fix a function execution where a parameter sent to the
function is a PDX serialized object. The function would be invoked on the
server with the current global setting and file to deserialize the PDX to
domain object. This is why the ask is for it to fall back to PdxInstance if
the domain object is not deployed.

-Jake

On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade 
wrote:

> Internally, there is an option to override read-serialized flag (to true);
> the query engine and other components uses this to keep the data in
> serialized form and work with PdxInstance...
>
> public static void setPdxReadSerialized(Cache cache, boolean
> readSerialized);
>
> We had discussed, making this as a public api...so that any thread that can
> work on PdxInstance can take advantage of it...
>
> -Anil.
>
>
> On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett 
> wrote:
>
> > Bruce, the flag only applies to values serialized with PDX,
> > DataSerializable objects are not effected by this property.
> >
> > I think there is some real value here as a stop gap until we have a
> better
> > solution in Geode 2 where the user can have a per request context that
> > specifies what return type they would like. Consider the user that has an
> > existing application that uses domain objects but wants to deploy another
> > application that doesn't to the same Geode cluster. The only option is to
> > either have all PDX deserialize to domain objects or all returned as
> > PdxInstance. One of the two applications will not work without
> > modification. Changing the behavior described by Addison splits the
> > difference. If the application is, like it is by default, configure to
> > deserialize PDX to the domain object but the domain object is not
> deployed
> > it will now give back the PDX instance rather than failing.
> >
> > An explicit use case is user that has both a Java and .NET application.
> The
> > .NET application does not have any Java domain objects to deploy to the
> > server but does want to query or run server side functions. The Java
> > application has deployed the domain objects to the server and distributed
> > functions are written expecting those domain objects on the server. The
> > user would have to create Java domain objects for the .NET application or
> > modify their Java application to expect PdxInstance.
> >
> >
> > -Jake
> >
> >
> > On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt  >
> > wrote:
> >
> > > +1
> > >
> > > I've found the current read-serialized property to be pretty useless.
> > >
> > > Having said that, what if the value isn't actually in serialized form
> in
> > > the local cache?  Is Geode supposed to serialize it & return it?  What
> > > if it isn't PDX-serialized?  Do we return a byte array?
> > >
> > >
> > > On 1/24/18 12:21 PM, Dan Smith wrote:
> > > > Is this really just a workaround for the fact that the
> read-serialized
> > > flag
> > > > applies to the whole cache? I can see that if you have mix of regions
> > > with
> > > > and without domain classes on the server you might want this feature.
> > Can
> > > > you provide some more background on your use case?
> > > >
> > > > IMO we should get rid of read-serialized in favor of APIs that let
> the
> > > user
> > > > decide whether they get a domain class or a PdxInstance.
> > > >
> > > > -Dan
> > > >
> > > > On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan <
> > gosulli...@pivotal.io
> > > >
> > > > wrote:
> > > >
> > > >> Hi Addison,
> > > >>
> > > >> What kind of setup do you have that is causing you to have PDX
> > > serialized
> > > >> objects that cannot be deserialized? Do you have classes that are
> > > present
> > > >> on some servers but not others?
> > > >>
> > > >> This change would break the contract of region.get() , which is that
> > it
> > > >> returns a value of a type that can be put into the region.
> > > >>
> > > >> Returning something that isn't what the user is expecting to be in
> the
> > > >> region would require users to check for PdxInstances every time they
> > > get a
> > > >> value -- even though the type annotations say that you can't get a
> > > >> PdxInstance back (except for Region ).
> > > >>
> > > >> I think it would be better to have a second API that allows users to
> > get
> > > >> and put PdxInstance objects in regions. That way, if they want to
> > handle
> > > >> the exceptional case where they have a serialized object that can't
> be
> > > >> deserialized, they can catch the ClassNotFound exception and get the
> > > >> underlying PdxInstance.
> > > >>
> > > >> I do think that the possibility of a ClassNotFoundException should
> be
> > > >> documented in the Region API.
> > > >>
> > > >> Cheers,
> > > >> Galen
> > > >>
> > > >> On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy 
> > > wrote:
> > > >>
> > > >>> Hi Geode Devs,
> > > >>>
> > > >>> I'm proposing the following change 

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Anilkumar Gingade]

Internally, there is an option to override read-serialized flag (to true);
the query engine and other components uses this to keep the data in
serialized form and work with PdxInstance...

public static void setPdxReadSerialized(Cache cache, boolean
readSerialized);

We had discussed, making this as a public api...so that any thread that can
work on PdxInstance can take advantage of it...

-Anil.


On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett  wrote:

> Bruce, the flag only applies to values serialized with PDX,
> DataSerializable objects are not effected by this property.
>
> I think there is some real value here as a stop gap until we have a better
> solution in Geode 2 where the user can have a per request context that
> specifies what return type they would like. Consider the user that has an
> existing application that uses domain objects but wants to deploy another
> application that doesn't to the same Geode cluster. The only option is to
> either have all PDX deserialize to domain objects or all returned as
> PdxInstance. One of the two applications will not work without
> modification. Changing the behavior described by Addison splits the
> difference. If the application is, like it is by default, configure to
> deserialize PDX to the domain object but the domain object is not deployed
> it will now give back the PDX instance rather than failing.
>
> An explicit use case is user that has both a Java and .NET application. The
> .NET application does not have any Java domain objects to deploy to the
> server but does want to query or run server side functions. The Java
> application has deployed the domain objects to the server and distributed
> functions are written expecting those domain objects on the server. The
> user would have to create Java domain objects for the .NET application or
> modify their Java application to expect PdxInstance.
>
>
> -Jake
>
>
> On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt 
> wrote:
>
> > +1
> >
> > I've found the current read-serialized property to be pretty useless.
> >
> > Having said that, what if the value isn't actually in serialized form in
> > the local cache?  Is Geode supposed to serialize it & return it?  What
> > if it isn't PDX-serialized?  Do we return a byte array?
> >
> >
> > On 1/24/18 12:21 PM, Dan Smith wrote:
> > > Is this really just a workaround for the fact that the read-serialized
> > flag
> > > applies to the whole cache? I can see that if you have mix of regions
> > with
> > > and without domain classes on the server you might want this feature.
> Can
> > > you provide some more background on your use case?
> > >
> > > IMO we should get rid of read-serialized in favor of APIs that let the
> > user
> > > decide whether they get a domain class or a PdxInstance.
> > >
> > > -Dan
> > >
> > > On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan <
> gosulli...@pivotal.io
> > >
> > > wrote:
> > >
> > >> Hi Addison,
> > >>
> > >> What kind of setup do you have that is causing you to have PDX
> > serialized
> > >> objects that cannot be deserialized? Do you have classes that are
> > present
> > >> on some servers but not others?
> > >>
> > >> This change would break the contract of region.get() , which is that
> it
> > >> returns a value of a type that can be put into the region.
> > >>
> > >> Returning something that isn't what the user is expecting to be in the
> > >> region would require users to check for PdxInstances every time they
> > get a
> > >> value -- even though the type annotations say that you can't get a
> > >> PdxInstance back (except for Region ).
> > >>
> > >> I think it would be better to have a second API that allows users to
> get
> > >> and put PdxInstance objects in regions. That way, if they want to
> handle
> > >> the exceptional case where they have a serialized object that can't be
> > >> deserialized, they can catch the ClassNotFound exception and get the
> > >> underlying PdxInstance.
> > >>
> > >> I do think that the possibility of a ClassNotFoundException should be
> > >> documented in the Region API.
> > >>
> > >> Cheers,
> > >> Galen
> > >>
> > >> On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy 
> > wrote:
> > >>
> > >>> Hi Geode Devs,
> > >>>
> > >>> I'm proposing the following change to how we handle deserialization
> > when
> > >>> Domain Objects can't be found and pdx-serialize=false.
> > >>>
> > >>> https://issues.apache.org/jira/browse/GEODE-4367
> > >>>
> > >>> Looking forward to the discussion.
> > >>>
> > >>> \ah
> > >>>
> >
> >
>

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Jacob Barrett]

Bruce, the flag only applies to values serialized with PDX,
DataSerializable objects are not effected by this property.

I think there is some real value here as a stop gap until we have a better
solution in Geode 2 where the user can have a per request context that
specifies what return type they would like. Consider the user that has an
existing application that uses domain objects but wants to deploy another
application that doesn't to the same Geode cluster. The only option is to
either have all PDX deserialize to domain objects or all returned as
PdxInstance. One of the two applications will not work without
modification. Changing the behavior described by Addison splits the
difference. If the application is, like it is by default, configure to
deserialize PDX to the domain object but the domain object is not deployed
it will now give back the PDX instance rather than failing.

An explicit use case is user that has both a Java and .NET application. The
.NET application does not have any Java domain objects to deploy to the
server but does want to query or run server side functions. The Java
application has deployed the domain objects to the server and distributed
functions are written expecting those domain objects on the server. The
user would have to create Java domain objects for the .NET application or
modify their Java application to expect PdxInstance.


-Jake


On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt 
wrote:

> +1
>
> I've found the current read-serialized property to be pretty useless.
>
> Having said that, what if the value isn't actually in serialized form in
> the local cache?  Is Geode supposed to serialize it & return it?  What
> if it isn't PDX-serialized?  Do we return a byte array?
>
>
> On 1/24/18 12:21 PM, Dan Smith wrote:
> > Is this really just a workaround for the fact that the read-serialized
> flag
> > applies to the whole cache? I can see that if you have mix of regions
> with
> > and without domain classes on the server you might want this feature. Can
> > you provide some more background on your use case?
> >
> > IMO we should get rid of read-serialized in favor of APIs that let the
> user
> > decide whether they get a domain class or a PdxInstance.
> >
> > -Dan
> >
> > On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan  >
> > wrote:
> >
> >> Hi Addison,
> >>
> >> What kind of setup do you have that is causing you to have PDX
> serialized
> >> objects that cannot be deserialized? Do you have classes that are
> present
> >> on some servers but not others?
> >>
> >> This change would break the contract of region.get() , which is that it
> >> returns a value of a type that can be put into the region.
> >>
> >> Returning something that isn't what the user is expecting to be in the
> >> region would require users to check for PdxInstances every time they
> get a
> >> value -- even though the type annotations say that you can't get a
> >> PdxInstance back (except for Region ).
> >>
> >> I think it would be better to have a second API that allows users to get
> >> and put PdxInstance objects in regions. That way, if they want to handle
> >> the exceptional case where they have a serialized object that can't be
> >> deserialized, they can catch the ClassNotFound exception and get the
> >> underlying PdxInstance.
> >>
> >> I do think that the possibility of a ClassNotFoundException should be
> >> documented in the Region API.
> >>
> >> Cheers,
> >> Galen
> >>
> >> On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy 
> wrote:
> >>
> >>> Hi Geode Devs,
> >>>
> >>> I'm proposing the following change to how we handle deserialization
> when
> >>> Domain Objects can't be found and pdx-serialize=false.
> >>>
> >>> https://issues.apache.org/jira/browse/GEODE-4367
> >>>
> >>> Looking forward to the discussion.
> >>>
> >>> \ah
> >>>
>
>

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Bruce Schuchardt]

+1

I've found the current read-serialized property to be pretty useless.

Having said that, what if the value isn't actually in serialized form in 
the local cache?  Is Geode supposed to serialize it & return it?  What 
if it isn't PDX-serialized?  Do we return a byte array?



On 1/24/18 12:21 PM, Dan Smith wrote:

Is this really just a workaround for the fact that the read-serialized flag
applies to the whole cache? I can see that if you have mix of regions with
and without domain classes on the server you might want this feature. Can
you provide some more background on your use case?

IMO we should get rid of read-serialized in favor of APIs that let the user
decide whether they get a domain class or a PdxInstance.

-Dan

On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan 
wrote:


Hi Addison,

What kind of setup do you have that is causing you to have PDX serialized
objects that cannot be deserialized? Do you have classes that are present
on some servers but not others?

This change would break the contract of region.get() , which is that it
returns a value of a type that can be put into the region.

Returning something that isn't what the user is expecting to be in the
region would require users to check for PdxInstances every time they get a
value -- even though the type annotations say that you can't get a
PdxInstance back (except for Region ).

I think it would be better to have a second API that allows users to get
and put PdxInstance objects in regions. That way, if they want to handle
the exceptional case where they have a serialized object that can't be
deserialized, they can catch the ClassNotFound exception and get the
underlying PdxInstance.

I do think that the possibility of a ClassNotFoundException should be
documented in the Region API.

Cheers,
Galen

On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy  wrote:


Hi Geode Devs,

I'm proposing the following change to how we handle deserialization when
Domain Objects can't be found and pdx-serialize=false.

https://issues.apache.org/jira/browse/GEODE-4367

Looking forward to the discussion.

\ah

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Michael Stolz]

I should have said Geode 2.0 rather than GemFire 10...
Sorry. My GemFire roots are showing ;(

--
Mike Stolz
Principal Engineer, GemFire Product Lead
Mobile: +1-631-835-4771
Download the new GemFire book here.


On Wed, Jan 24, 2018 at 2:40 PM, Michael Stolz  wrote:

> I agree that we should have api level choice for whether to read
> serialized or not.
> We can deprecate read-serialized and get rid of it in GemFire 10.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771 <(631)%20835-4771>
> Download the new GemFire book here.
> 
>
> On Wed, Jan 24, 2018 at 12:21 PM, Dan Smith  wrote:
>
>> Is this really just a workaround for the fact that the read-serialized
>> flag
>> applies to the whole cache? I can see that if you have mix of regions with
>> and without domain classes on the server you might want this feature. Can
>> you provide some more background on your use case?
>>
>> IMO we should get rid of read-serialized in favor of APIs that let the
>> user
>> decide whether they get a domain class or a PdxInstance.
>>
>> -Dan
>>
>> On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan 
>> wrote:
>>
>> > Hi Addison,
>> >
>> > What kind of setup do you have that is causing you to have PDX
>> serialized
>> > objects that cannot be deserialized? Do you have classes that are
>> present
>> > on some servers but not others?
>> >
>> > This change would break the contract of region.get() , which is that it
>> > returns a value of a type that can be put into the region.
>> >
>> > Returning something that isn't what the user is expecting to be in the
>> > region would require users to check for PdxInstances every time they
>> get a
>> > value -- even though the type annotations say that you can't get a
>> > PdxInstance back (except for Region ).
>> >
>> > I think it would be better to have a second API that allows users to get
>> > and put PdxInstance objects in regions. That way, if they want to handle
>> > the exceptional case where they have a serialized object that can't be
>> > deserialized, they can catch the ClassNotFound exception and get the
>> > underlying PdxInstance.
>> >
>> > I do think that the possibility of a ClassNotFoundException should be
>> > documented in the Region API.
>> >
>> > Cheers,
>> > Galen
>> >
>> > On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy 
>> wrote:
>> >
>> > > Hi Geode Devs,
>> > >
>> > > I'm proposing the following change to how we handle deserialization
>> when
>> > > Domain Objects can't be found and pdx-serialize=false.
>> > >
>> > > https://issues.apache.org/jira/browse/GEODE-4367
>> > >
>> > > Looking forward to the discussion.
>> > >
>> > > \ah
>> > >
>> >
>>
>
>

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Michael Stolz]

I agree that we should have api level choice for whether to read serialized
or not.
We can deprecate read-serialized and get rid of it in GemFire 10.

--
Mike Stolz
Principal Engineer, GemFire Product Lead
Mobile: +1-631-835-4771
Download the new GemFire book here.


On Wed, Jan 24, 2018 at 12:21 PM, Dan Smith  wrote:

> Is this really just a workaround for the fact that the read-serialized flag
> applies to the whole cache? I can see that if you have mix of regions with
> and without domain classes on the server you might want this feature. Can
> you provide some more background on your use case?
>
> IMO we should get rid of read-serialized in favor of APIs that let the user
> decide whether they get a domain class or a PdxInstance.
>
> -Dan
>
> On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan 
> wrote:
>
> > Hi Addison,
> >
> > What kind of setup do you have that is causing you to have PDX serialized
> > objects that cannot be deserialized? Do you have classes that are present
> > on some servers but not others?
> >
> > This change would break the contract of region.get() , which is that it
> > returns a value of a type that can be put into the region.
> >
> > Returning something that isn't what the user is expecting to be in the
> > region would require users to check for PdxInstances every time they get
> a
> > value -- even though the type annotations say that you can't get a
> > PdxInstance back (except for Region ).
> >
> > I think it would be better to have a second API that allows users to get
> > and put PdxInstance objects in regions. That way, if they want to handle
> > the exceptional case where they have a serialized object that can't be
> > deserialized, they can catch the ClassNotFound exception and get the
> > underlying PdxInstance.
> >
> > I do think that the possibility of a ClassNotFoundException should be
> > documented in the Region API.
> >
> > Cheers,
> > Galen
> >
> > On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy 
> wrote:
> >
> > > Hi Geode Devs,
> > >
> > > I'm proposing the following change to how we handle deserialization
> when
> > > Domain Objects can't be found and pdx-serialize=false.
> > >
> > > https://issues.apache.org/jira/browse/GEODE-4367
> > >
> > > Looking forward to the discussion.
> > >
> > > \ah
> > >
> >
>

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Galen O'Sullivan]

Hi Addison,

What kind of setup do you have that is causing you to have PDX serialized
objects that cannot be deserialized? Do you have classes that are present
on some servers but not others?

This change would break the contract of region.get() , which is that it
returns a value of a type that can be put into the region.

Returning something that isn't what the user is expecting to be in the
region would require users to check for PdxInstances every time they get a
value -- even though the type annotations say that you can't get a
PdxInstance back (except for Region ).

I think it would be better to have a second API that allows users to get
and put PdxInstance objects in regions. That way, if they want to handle
the exceptional case where they have a serialized object that can't be
deserialized, they can catch the ClassNotFound exception and get the
underlying PdxInstance.

I do think that the possibility of a ClassNotFoundException should be
documented in the Region API.

Cheers,
Galen

On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy  wrote:

> Hi Geode Devs,
>
> I'm proposing the following change to how we handle deserialization when
> Domain Objects can't be found and pdx-serialize=false.
>
> https://issues.apache.org/jira/browse/GEODE-4367
>
> Looking forward to the discussion.
>
> \ah
>

Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Addison Huddy]

Hi Geode Devs,

I'm proposing the following change to how we handle deserialization when
Domain Objects can't be found and pdx-serialize=false.

https://issues.apache.org/jira/browse/GEODE-4367

Looking forward to the discussion.

\ah