Re: REST: support getting objects from cache.
Thanks, Alexey! I'll update the docs. -P On Fri, Mar 2, 2018 at 1:42 AM, Alexey Kuznetsov wrote: > I merged IGNITE-7803 into master. > Created ticket for documentation: https://issues. > apache.org/jira/browse/IGNITE-7867 > > Prachi, could you update documentation for REST? > > On Thu, Mar 1, 2018 at 2:41 PM, Alexey Kuznetsov > wrote: > >> Vladimir, >> >> I finished with this feature: https://issues.apache >> .org/jira/browse/IGNITE-7803 >> >> Could you please review my changes in branch ignite-7803? >> >> DIFF: https://github.com/apache/ignite/commit/da03195ef5c39deabfc4 >> 961b2a0336fdbc98aa31 >> >> -- >> Alexey Kuznetsov >> > > > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com >
Re: REST: support getting objects from cache.
I merged IGNITE-7803 into master. Created ticket for documentation: https://issues.apache.org/jira/browse/IGNITE-7867 Prachi, could you update documentation for REST? On Thu, Mar 1, 2018 at 2:41 PM, Alexey Kuznetsov wrote: > Vladimir, > > I finished with this feature: https://issues. > apache.org/jira/browse/IGNITE-7803 > > Could you please review my changes in branch ignite-7803? > > DIFF: https://github.com/apache/ignite/commit/ > da03195ef5c39deabfc4961b2a0336fdbc98aa31 > > -- > Alexey Kuznetsov > -- Alexey Kuznetsov GridGain Systems www.gridgain.com
Re: REST: support getting objects from cache.
Vladimir, I finished with this feature: https://issues.apache.org/jira/browse/IGNITE-7803 Could you please review my changes in branch ignite-7803? DIFF: https://github.com/apache/ignite/commit/da03195ef5c39deabfc4961b2a0336fdbc98aa31 -- Alexey Kuznetsov
Re: REST: support getting objects from cache.
> private int orgId;
> public int getOrganizationId() {...}
What an insane naming convention, who would do that? :)
Anyway, JSON field names should match binary metadata.
We have SQL via REST, so users would expect to use the same field names in
binary objects and SQL queries.
On Tue, Feb 27, 2018 at 12:41 AM, Denis Magda wrote:
> Hi guys,
>
> 1. Alex, the point that Pavel tried to convey is that a user who will
> get/put data using REST doesn't care if the data will be deserialized on
> the server side. It just needs to work. So, the same user shouldn't know
> about the keepBinary parameter, just turn it on for REST protocol by
> default.
>
> 3. Vote for skipping this use case and throw a meaningful exception if it
> occurs so that the user can let us know if there is a demand.
>
>
>
> On Mon, Feb 26, 2018 at 7:12 AM, Alexey Kuznetsov > wrote:
>
>> Vova, Pavel, Sergey,
>>
>> Thanks for you comments.
>>
>> 1) There one corner case with POJO objects in cache.
>> Binary marshaler convert by fields, not by getters. So if you have
>>
>> class Person {
>> private int orgId;
>> ...
>>
>> public int getOrganizationId() {...}
>> }
>>
>> POJO will be transformed to {"organizationId": 1} JSON
>> binary will be transformed to {"orgId": 1} JSON
>>
>> So in this case [keepBinary=true|false] can make sense for end user.
>>
>> We can have keepBinary=true by default, and left an option to user if he
>> needs to transform POJO to JSON for some reason.
>> But I agree that double transformation is just a waste of CPU and memory
>> :)
>>
>> 3) How about to throw exception by default and output "$circular_ref" in
>> case of some option, for example, "allowCircularRefs=true".
>>
>>
>> On Mon, Feb 26, 2018 at 6:39 PM, Sergey Kozlov
>> wrote:
>>
>> > Vova, Alexey
>> >
>> > 3) The exception is not enough for that case. We should return a proper
>> > error message in the json reply.
>> >
>> > On Mon, Feb 26, 2018 at 2:20 PM, Vladimir Ozerov
>> > wrote:
>> >
>> > > My 50 cents:
>> > > 1) Agree with Pavel, not need to deserialize at all
>> > > 2) May be you see everything in the upper case because this is how
>> CREATE
>> > > TABLE works - every object name (table name, column name, etc.) are
>> > > converted to upper case by default. This is expected behavior of every
>> > SQL
>> > > engine. If you want to preserve cases please consider using qoutes
>> around
>> > > column names (e.g. CREATE TABLE "myTable"("myColumn" INT PRIMARY KEY).
>> > > 3) I think we should throw an exception in this case - objects with
>> > > recursive dependencies cannot be expressed in text form (whether this
>> is
>> > > XML, JSON or anything else).
>> > >
>> > > On Mon, Feb 26, 2018 at 10:00 AM, Pavel Tupitsyn <
>> ptupit...@apache.org>
>> > > wrote:
>> > >
>> > > > Hi Alexey,
>> > > >
>> > > > 1) IMO for this task you should ALWAYS work in binary mode.
>> > > > What is the use case for deserializing with a real class and then
>> > > > serializing to JSON?
>> > > > Looks like a waste of resources to me.
>> > > >
>> > > > 2) This should not be the case, please re-check your code.
>> > > > Binary meta preserves original case (stores field names as is), just
>> > > > checked this with 2.4 build.
>> > > >
>> > > > 3) JSON serializers typically handle this by adding special fields
>> ($id
>> > > and
>> > > > $ref in Json.NET, for example).
>> > > > But I believe this is a rare use case and can be skipped in initial
>> > > > implementation.
>> > > >
>> > > > Thanks,
>> > > > Pavel
>> > > >
>> > > > On Mon, Feb 26, 2018 at 7:38 AM, Alexey Kuznetsov <
>> > akuznet...@apache.org
>> > > >
>> > > > wrote:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > I'm working on IGNITE-7803 REST: Add support to get values
>> inserted
>> > > via
>> > > > > API or SQL[1]
>> > > > >
>> > > > > And found following issues:
>> > > > >
>> > > > > 1. First, if server node that will handle REST request does not
>> have
>> > > > class
>> > > > > of object that we want to get from cache I need to set
>> > > cache.keepBinary()
>> > > > > in order to avoid object deserialization and work directly with
>> > binary
>> > > > > metadata directly.
>> > > > >
>> > > > > But in some cases node could have class in classpath and user may
>> > need
>> > > to
>> > > > > use that class.
>> > > > >
>> > > > > How about to add option "keepBinary=true" and let user handle
>> this
>> > by
>> > > > > himself?
>> > > > >
>> > > > > 2. Second, in binary metadata all names stored in upper case. So,
>> > > binary
>> > > > > object converted to JSON will be like: {"ID": 1, "NAME": "Alex",
>> > > > "SALARY":
>> > > > > 300}
>> > > > >
>> > > > > It is OK?
>> > > > >
>> > > > > 3. Should we handle circular references in binary objects? If yes,
>> > then
>> > > > > how?
>> > > > >
>> > > > >
>> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7803
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Alexey Kuznetsov
>> > > > >
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Sergey Kozlov
Re: REST: support getting objects from cache.
Hi guys,
1. Alex, the point that Pavel tried to convey is that a user who will
get/put data using REST doesn't care if the data will be deserialized on
the server side. It just needs to work. So, the same user shouldn't know
about the keepBinary parameter, just turn it on for REST protocol by
default.
3. Vote for skipping this use case and throw a meaningful exception if it
occurs so that the user can let us know if there is a demand.
On Mon, Feb 26, 2018 at 7:12 AM, Alexey Kuznetsov
wrote:
> Vova, Pavel, Sergey,
>
> Thanks for you comments.
>
> 1) There one corner case with POJO objects in cache.
> Binary marshaler convert by fields, not by getters. So if you have
>
> class Person {
> private int orgId;
> ...
>
> public int getOrganizationId() {...}
> }
>
> POJO will be transformed to {"organizationId": 1} JSON
> binary will be transformed to {"orgId": 1} JSON
>
> So in this case [keepBinary=true|false] can make sense for end user.
>
> We can have keepBinary=true by default, and left an option to user if he
> needs to transform POJO to JSON for some reason.
> But I agree that double transformation is just a waste of CPU and memory :)
>
> 3) How about to throw exception by default and output "$circular_ref" in
> case of some option, for example, "allowCircularRefs=true".
>
>
> On Mon, Feb 26, 2018 at 6:39 PM, Sergey Kozlov
> wrote:
>
> > Vova, Alexey
> >
> > 3) The exception is not enough for that case. We should return a proper
> > error message in the json reply.
> >
> > On Mon, Feb 26, 2018 at 2:20 PM, Vladimir Ozerov
> > wrote:
> >
> > > My 50 cents:
> > > 1) Agree with Pavel, not need to deserialize at all
> > > 2) May be you see everything in the upper case because this is how
> CREATE
> > > TABLE works - every object name (table name, column name, etc.) are
> > > converted to upper case by default. This is expected behavior of every
> > SQL
> > > engine. If you want to preserve cases please consider using qoutes
> around
> > > column names (e.g. CREATE TABLE "myTable"("myColumn" INT PRIMARY KEY).
> > > 3) I think we should throw an exception in this case - objects with
> > > recursive dependencies cannot be expressed in text form (whether this
> is
> > > XML, JSON or anything else).
> > >
> > > On Mon, Feb 26, 2018 at 10:00 AM, Pavel Tupitsyn >
> > > wrote:
> > >
> > > > Hi Alexey,
> > > >
> > > > 1) IMO for this task you should ALWAYS work in binary mode.
> > > > What is the use case for deserializing with a real class and then
> > > > serializing to JSON?
> > > > Looks like a waste of resources to me.
> > > >
> > > > 2) This should not be the case, please re-check your code.
> > > > Binary meta preserves original case (stores field names as is), just
> > > > checked this with 2.4 build.
> > > >
> > > > 3) JSON serializers typically handle this by adding special fields
> ($id
> > > and
> > > > $ref in Json.NET, for example).
> > > > But I believe this is a rare use case and can be skipped in initial
> > > > implementation.
> > > >
> > > > Thanks,
> > > > Pavel
> > > >
> > > > On Mon, Feb 26, 2018 at 7:38 AM, Alexey Kuznetsov <
> > akuznet...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I'm working on IGNITE-7803 REST: Add support to get values
> inserted
> > > via
> > > > > API or SQL[1]
> > > > >
> > > > > And found following issues:
> > > > >
> > > > > 1. First, if server node that will handle REST request does not
> have
> > > > class
> > > > > of object that we want to get from cache I need to set
> > > cache.keepBinary()
> > > > > in order to avoid object deserialization and work directly with
> > binary
> > > > > metadata directly.
> > > > >
> > > > > But in some cases node could have class in classpath and user may
> > need
> > > to
> > > > > use that class.
> > > > >
> > > > > How about to add option "keepBinary=true" and let user handle this
> > by
> > > > > himself?
> > > > >
> > > > > 2. Second, in binary metadata all names stored in upper case. So,
> > > binary
> > > > > object converted to JSON will be like: {"ID": 1, "NAME": "Alex",
> > > > "SALARY":
> > > > > 300}
> > > > >
> > > > > It is OK?
> > > > >
> > > > > 3. Should we handle circular references in binary objects? If yes,
> > then
> > > > > how?
> > > > >
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7803
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Alexey Kuznetsov
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Sergey Kozlov
> > GridGain Systems
> > www.gridgain.com
> >
>
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com
>
Re: REST: support getting objects from cache.
Vova, Pavel, Sergey,
Thanks for you comments.
1) There one corner case with POJO objects in cache.
Binary marshaler convert by fields, not by getters. So if you have
class Person {
private int orgId;
...
public int getOrganizationId() {...}
}
POJO will be transformed to {"organizationId": 1} JSON
binary will be transformed to {"orgId": 1} JSON
So in this case [keepBinary=true|false] can make sense for end user.
We can have keepBinary=true by default, and left an option to user if he
needs to transform POJO to JSON for some reason.
But I agree that double transformation is just a waste of CPU and memory :)
3) How about to throw exception by default and output "$circular_ref" in
case of some option, for example, "allowCircularRefs=true".
On Mon, Feb 26, 2018 at 6:39 PM, Sergey Kozlov wrote:
> Vova, Alexey
>
> 3) The exception is not enough for that case. We should return a proper
> error message in the json reply.
>
> On Mon, Feb 26, 2018 at 2:20 PM, Vladimir Ozerov
> wrote:
>
> > My 50 cents:
> > 1) Agree with Pavel, not need to deserialize at all
> > 2) May be you see everything in the upper case because this is how CREATE
> > TABLE works - every object name (table name, column name, etc.) are
> > converted to upper case by default. This is expected behavior of every
> SQL
> > engine. If you want to preserve cases please consider using qoutes around
> > column names (e.g. CREATE TABLE "myTable"("myColumn" INT PRIMARY KEY).
> > 3) I think we should throw an exception in this case - objects with
> > recursive dependencies cannot be expressed in text form (whether this is
> > XML, JSON or anything else).
> >
> > On Mon, Feb 26, 2018 at 10:00 AM, Pavel Tupitsyn
> > wrote:
> >
> > > Hi Alexey,
> > >
> > > 1) IMO for this task you should ALWAYS work in binary mode.
> > > What is the use case for deserializing with a real class and then
> > > serializing to JSON?
> > > Looks like a waste of resources to me.
> > >
> > > 2) This should not be the case, please re-check your code.
> > > Binary meta preserves original case (stores field names as is), just
> > > checked this with 2.4 build.
> > >
> > > 3) JSON serializers typically handle this by adding special fields ($id
> > and
> > > $ref in Json.NET, for example).
> > > But I believe this is a rare use case and can be skipped in initial
> > > implementation.
> > >
> > > Thanks,
> > > Pavel
> > >
> > > On Mon, Feb 26, 2018 at 7:38 AM, Alexey Kuznetsov <
> akuznet...@apache.org
> > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I'm working on IGNITE-7803 REST: Add support to get values inserted
> > via
> > > > API or SQL[1]
> > > >
> > > > And found following issues:
> > > >
> > > > 1. First, if server node that will handle REST request does not have
> > > class
> > > > of object that we want to get from cache I need to set
> > cache.keepBinary()
> > > > in order to avoid object deserialization and work directly with
> binary
> > > > metadata directly.
> > > >
> > > > But in some cases node could have class in classpath and user may
> need
> > to
> > > > use that class.
> > > >
> > > > How about to add option "keepBinary=true" and let user handle this
> by
> > > > himself?
> > > >
> > > > 2. Second, in binary metadata all names stored in upper case. So,
> > binary
> > > > object converted to JSON will be like: {"ID": 1, "NAME": "Alex",
> > > "SALARY":
> > > > 300}
> > > >
> > > > It is OK?
> > > >
> > > > 3. Should we handle circular references in binary objects? If yes,
> then
> > > > how?
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-7803
> > > >
> > > >
> > > >
> > > > --
> > > > Alexey Kuznetsov
> > > >
> > >
> >
>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>
--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com
Re: REST: support getting objects from cache.
Vova, Alexey
3) The exception is not enough for that case. We should return a proper
error message in the json reply.
On Mon, Feb 26, 2018 at 2:20 PM, Vladimir Ozerov
wrote:
> My 50 cents:
> 1) Agree with Pavel, not need to deserialize at all
> 2) May be you see everything in the upper case because this is how CREATE
> TABLE works - every object name (table name, column name, etc.) are
> converted to upper case by default. This is expected behavior of every SQL
> engine. If you want to preserve cases please consider using qoutes around
> column names (e.g. CREATE TABLE "myTable"("myColumn" INT PRIMARY KEY).
> 3) I think we should throw an exception in this case - objects with
> recursive dependencies cannot be expressed in text form (whether this is
> XML, JSON or anything else).
>
> On Mon, Feb 26, 2018 at 10:00 AM, Pavel Tupitsyn
> wrote:
>
> > Hi Alexey,
> >
> > 1) IMO for this task you should ALWAYS work in binary mode.
> > What is the use case for deserializing with a real class and then
> > serializing to JSON?
> > Looks like a waste of resources to me.
> >
> > 2) This should not be the case, please re-check your code.
> > Binary meta preserves original case (stores field names as is), just
> > checked this with 2.4 build.
> >
> > 3) JSON serializers typically handle this by adding special fields ($id
> and
> > $ref in Json.NET, for example).
> > But I believe this is a rare use case and can be skipped in initial
> > implementation.
> >
> > Thanks,
> > Pavel
> >
> > On Mon, Feb 26, 2018 at 7:38 AM, Alexey Kuznetsov >
> > wrote:
> >
> > > Hi,
> > >
> > > I'm working on IGNITE-7803 REST: Add support to get values inserted
> via
> > > API or SQL[1]
> > >
> > > And found following issues:
> > >
> > > 1. First, if server node that will handle REST request does not have
> > class
> > > of object that we want to get from cache I need to set
> cache.keepBinary()
> > > in order to avoid object deserialization and work directly with binary
> > > metadata directly.
> > >
> > > But in some cases node could have class in classpath and user may need
> to
> > > use that class.
> > >
> > > How about to add option "keepBinary=true" and let user handle this by
> > > himself?
> > >
> > > 2. Second, in binary metadata all names stored in upper case. So,
> binary
> > > object converted to JSON will be like: {"ID": 1, "NAME": "Alex",
> > "SALARY":
> > > 300}
> > >
> > > It is OK?
> > >
> > > 3. Should we handle circular references in binary objects? If yes, then
> > > how?
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-7803
> > >
> > >
> > >
> > > --
> > > Alexey Kuznetsov
> > >
> >
>
--
Sergey Kozlov
GridGain Systems
www.gridgain.com
Re: REST: support getting objects from cache.
My 50 cents:
1) Agree with Pavel, not need to deserialize at all
2) May be you see everything in the upper case because this is how CREATE
TABLE works - every object name (table name, column name, etc.) are
converted to upper case by default. This is expected behavior of every SQL
engine. If you want to preserve cases please consider using qoutes around
column names (e.g. CREATE TABLE "myTable"("myColumn" INT PRIMARY KEY).
3) I think we should throw an exception in this case - objects with
recursive dependencies cannot be expressed in text form (whether this is
XML, JSON or anything else).
On Mon, Feb 26, 2018 at 10:00 AM, Pavel Tupitsyn
wrote:
> Hi Alexey,
>
> 1) IMO for this task you should ALWAYS work in binary mode.
> What is the use case for deserializing with a real class and then
> serializing to JSON?
> Looks like a waste of resources to me.
>
> 2) This should not be the case, please re-check your code.
> Binary meta preserves original case (stores field names as is), just
> checked this with 2.4 build.
>
> 3) JSON serializers typically handle this by adding special fields ($id and
> $ref in Json.NET, for example).
> But I believe this is a rare use case and can be skipped in initial
> implementation.
>
> Thanks,
> Pavel
>
> On Mon, Feb 26, 2018 at 7:38 AM, Alexey Kuznetsov
> wrote:
>
> > Hi,
> >
> > I'm working on IGNITE-7803 REST: Add support to get values inserted via
> > API or SQL[1]
> >
> > And found following issues:
> >
> > 1. First, if server node that will handle REST request does not have
> class
> > of object that we want to get from cache I need to set cache.keepBinary()
> > in order to avoid object deserialization and work directly with binary
> > metadata directly.
> >
> > But in some cases node could have class in classpath and user may need to
> > use that class.
> >
> > How about to add option "keepBinary=true" and let user handle this by
> > himself?
> >
> > 2. Second, in binary metadata all names stored in upper case. So, binary
> > object converted to JSON will be like: {"ID": 1, "NAME": "Alex",
> "SALARY":
> > 300}
> >
> > It is OK?
> >
> > 3. Should we handle circular references in binary objects? If yes, then
> > how?
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-7803
> >
> >
> >
> > --
> > Alexey Kuznetsov
> >
>
Re: REST: support getting objects from cache.
Hi Alexey,
1) IMO for this task you should ALWAYS work in binary mode.
What is the use case for deserializing with a real class and then
serializing to JSON?
Looks like a waste of resources to me.
2) This should not be the case, please re-check your code.
Binary meta preserves original case (stores field names as is), just
checked this with 2.4 build.
3) JSON serializers typically handle this by adding special fields ($id and
$ref in Json.NET, for example).
But I believe this is a rare use case and can be skipped in initial
implementation.
Thanks,
Pavel
On Mon, Feb 26, 2018 at 7:38 AM, Alexey Kuznetsov
wrote:
> Hi,
>
> I'm working on IGNITE-7803 REST: Add support to get values inserted via
> API or SQL[1]
>
> And found following issues:
>
> 1. First, if server node that will handle REST request does not have class
> of object that we want to get from cache I need to set cache.keepBinary()
> in order to avoid object deserialization and work directly with binary
> metadata directly.
>
> But in some cases node could have class in classpath and user may need to
> use that class.
>
> How about to add option "keepBinary=true" and let user handle this by
> himself?
>
> 2. Second, in binary metadata all names stored in upper case. So, binary
> object converted to JSON will be like: {"ID": 1, "NAME": "Alex", "SALARY":
> 300}
>
> It is OK?
>
> 3. Should we handle circular references in binary objects? If yes, then
> how?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7803
>
>
>
> --
> Alexey Kuznetsov
>
REST: support getting objects from cache.
Hi,
I'm working on IGNITE-7803 REST: Add support to get values inserted via
API or SQL[1]
And found following issues:
1. First, if server node that will handle REST request does not have class
of object that we want to get from cache I need to set cache.keepBinary()
in order to avoid object deserialization and work directly with binary
metadata directly.
But in some cases node could have class in classpath and user may need to
use that class.
How about to add option "keepBinary=true" and let user handle this by
himself?
2. Second, in binary metadata all names stored in upper case. So, binary
object converted to JSON will be like: {"ID": 1, "NAME": "Alex", "SALARY":
300}
It is OK?
3. Should we handle circular references in binary objects? If yes, then how?
[1] https://issues.apache.org/jira/browse/IGNITE-7803
--
Alexey Kuznetsov
