Thanks for the suggestion. A quick summary. there are several ways being mentioned to achieve it.
1. Use PDX ReflectionBasedAutoSerializer. write a server-side function to get only the fields I need. Good: Always consistent. Only 1 region to manage. Bad: Need a jar containing the Serializer + server function deployed to geode server. 2. 2 region, 1 for data, 1 for property co-locating. Good: Simple to do Bad. Not always consistent, though eventual consistency. 3. Projection using OQL Good: Always consistent. Only 1 region to manage. And the domain object still need to deploy to geode server? Bad: Not sure the speed of OQL compare to region.get() From: Udo Kohlmeyer [mailto:[email protected]] Sent: Wednesday, August 30, 2017 12:41 PM To: [email protected] Subject: Re: Get part of the object Hi there Nan, Given that you would have 2 separate regions, you should not ever run into the situation where properties and data are significantly "out-of-sync". If you were to use the same WAN Sender for both regions, the data and properties should be sent in the same batch or subsequent batches. This approach is not "atomic" and yes, you could end up in a situation where there might be small point in time where they are "out-of-sync". IF there is an absolute requirement for atomicity AND smaller region.get() operations, there are other approaches that you could use. Maybe insert the big object (properties + data) into an "ingestion" region. Then have a CacheListener split the big object into data and properties regions. WAN Replication can then be setup on the "ingestion" region, thus data + properties move atomically. Of course, the remote cluster will have to then take the WAN replicated "big Object" and split it into the querying regions. Once again, many more moving parts.... and possibly overkill if you don't have the requirement of atomic and eventual consistency (within 1 or batches) is good enough. --Udo On 8/30/17 10:04, Xu, Nan wrote: collocating is a good suggestion, so assume I have data region and property region collocated. We need the data distributed to multiple datacenter and to save latency and bandwidth, we are going to use conflation. What I am afraid is data get conflated and old data not send out and properties get send out. So they are not exactly 1:1 match then. Suggestion? Would be eventual consistent?(I think so) Thanks, Nan From: Udo Kohlmeyer [mailto:[email protected]] Sent: Wednesday, August 30, 2017 11:01 AM To: [email protected]<mailto:[email protected]> Subject: Re: Get part of the object Hi the Nan, From experience I can tell you it is better to safe the byte[] to a separate (colocated region). You can then have a data and properties region, which use the same key to store the data in both regions. The only thing you would have to manage yourself would be the CRUD operations for each. i.e If you insert properties you'll have to insert the data into the data region. If you remove/update properties you'll potentially have to do the same to the data region. Geode does not provide the mechanism to manage data splits. --Udo On 8/30/17 08:20, Lyndon Adams wrote: Yep thanks Luke. On 30 Aug 2017, at 16:12, Xu, Nan <[email protected]<mailto:[email protected]>> wrote: “Use selective gettter methods on the wrapper object” How? Region.get() already grab the full object from network. Or something I am missing? From: Lyndon Adams [mailto:[email protected]] Sent: Wednesday, August 30, 2017 10:05 AM To: [email protected]<mailto:[email protected]> Subject: Re: Get part of the object Use selective gettter methods on the wrapper object to achieve this requirement. On 30 Aug 2017, at 16:03, Xu, Nan <[email protected]<mailto:[email protected]>> wrote: Yes, but because data is big, >100K, and other is small, to save the bandwidth, I don’t want to get the full object back but only the other part. From: Dharam Thacker [mailto:[email protected]] Sent: Wednesday, August 30, 2017 10:01 AM To: [email protected]<mailto:[email protected]> Subject: Re: Get part of the object Region.get() is going to return you domain object. So let's say yoy have domain class with { byte[] data Map<String,Object> other } You can say, Region.get(data).getOther() Thanks, Dharam On Aug 30, 2017 8:18 PM, "Lyndon Adams" <[email protected]<mailto:[email protected]>> wrote: Yes On 30 Aug 2017, at 15:47, Xu, Nan <[email protected]<mailto:[email protected]>> wrote: Not sure I get it. you mean save a list, and list comprise of 2 objects -- data and properties? From: Lyndon Adams [mailto:[email protected]] Sent: Wednesday, August 30, 2017 9:45 AM To: [email protected]<mailto:[email protected]> Subject: Re: Get part of the object Make the value a collection wrapper. On 30 Aug 2017, at 15:39, Xu, Nan <[email protected]<mailto:[email protected]>> wrote: Thanks, wondering if geode can save 2 objects for a single key? So I can put data and other property separately. Or every key is for 1 object? Never use the PDX and server side function, is that mean those need to be deployed to the server? or there is a way to programming register the function to the server? Thanks, Nan From: Luke Shannon [mailto:[email protected]] Sent: Tuesday, August 29, 2017 8:02 PM To: [email protected]<mailto:[email protected]> Subject: Re: Get part of the object +1 on Mike's suggestion for the ReflectionBasedAutoSerializer To get name and author you could use OQL: https://gemfire.docs.pivotal.io/geode/getting_started/querying_quick_reference.html#reference_D5CE64F5FD6F4A808AEFB748C867189E<https://urldefense.proofpoint.com/v2/url?u=https-3A__gemfire.docs.pivotal.io_geode_getting-5Fstarted_querying-5Fquick-5Freference.html-23reference-5FD5CE64F5FD6F4A808AEFB748C867189E&d=DwMFaQ&c=SFszdw3oxIkTvaP4xmzq_apLU3uL-3SxdAPNkldf__Q&r=HB5LZowSGF4DiMmOUsCX6Q&m=8beRJvUxxhwMi53HGv0iYc9bpWOpIjRLWs6L_12UPVQ&s=XWQnjvDN7lTRBBHYWEMKtnyPfsLxW_Pq0wL1ZSMK1No&e=> On Tue, Aug 29, 2017 at 5:45 PM, Michael Stolz <[email protected]<mailto:[email protected]>> wrote: You should probably use the PDX ReflectionBasedAutoSerializer. Then you can write a server-side function that you call to get just the fields you want and return them. -- Mike Stolz Principal Engineer, GemFire Product Manager Mobile: +1-631-835-4771<tel:%28631%29%20835-4771> On Tue, Aug 29, 2017 at 4:49 PM, Xu, Nan <[email protected]<mailto:[email protected]>> wrote: Hi, I have an object comprise of MyObject{ byte[] data; String name; String author; …. Other properties; } Or I can use MyObject{ Byte[] data; Other allOther; } When I get by key, I only want to get all the properties except the data itself, because data could be large, is that has to be done with query? Not sure how the performance of query compare to region.get(key). Another question is to serialize the data. Do I need to upload the jar containing MyObject to geode server? Thanks, Nan ________________________________ This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. -- Luke Shannon | Platform Engineering | Pivotal ------------------------------------------------------------------------- Mobile:416-571-9495<tel:%28416%29%20571-9495> Join the Toronto Pivotal Usergroup: http://www.meetup.com/Toronto-Pivotal-User-Group/<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.meetup.com_Toronto-2DPivotal-2DUser-2DGroup_&d=DwMFaQ&c=SFszdw3oxIkTvaP4xmzq_apLU3uL-3SxdAPNkldf__Q&r=HB5LZowSGF4DiMmOUsCX6Q&m=8beRJvUxxhwMi53HGv0iYc9bpWOpIjRLWs6L_12UPVQ&s=6_3MUQQcXn4OGBkgQjyTWec_NaJU4gvp3tHRQfoV-38&e=> ________________________________ This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. ________________________________ This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. ________________________________ This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. ________________________________ This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. ________________________________ This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. ---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.
