Re: SnapshotVersion

[Markus Reich]

Perfect!!! Thx Meex

Andrus Adamchik  schrieb am Do., 9. März 2017 um
07:20 Uhr:

> You can store those values in each DataObject instance variables and map a
> PostLoad callback on the entity (Modeler > Select ObjEntity in question >
> Select "Callbacks" tab). In the callback method you can reset the values.
> More info on callbacks:
>
> http://cayenne.apache.org/docs/4.0/cayenne-guide/lifecycle-events.html
>
> Andrus
>
>
> > On Mar 8, 2017, at 10:31 PM, Markus Reich 
> wrote:
> >
> > Hi Andrus,
> >
> > I have some getters in DataObjects that use some DB properties for
> > calculation, e.g. I read other data from DB and to kind of lookups, so I
> > got a much better performance when I introtuced a caching of this
> > calculated values. I just use a concurrent hash map to realize the
> caching.
> > Now I need a trigger for invalidation of the cached values, when the
> > DataObject itself get's invalidated?
> >
> > regards
> > Meex
> >
> > Andrus Adamchik  schrieb am Mi., 8. März 2017 um
> > 20:24 Uhr:
> >
> >> Hi Markus,
> >>
> >> Snapshot version reflects the version of a cached snapshot that was used
> >> to sync the current state of the object (this is probably obvious). The
> >> behavior observed by you may actually be a bug (and we may need to
> >> investigate it).
> >>
> >> Generally I would recommend against relying too much on snapshot version
> >> API. I always considered it an implementation detail (though of course
> it
> >> is publicly accessible... oh well) that may easily fall victim of a
> future
> >> DataObject redesign (e.g. see the parallel 'Fetching lots of objects'
> >> thread).
> >>
> >> Can you explain how your cache is intended to work? Perhaps there is a
> >> callback/listener solution to it?
> >>
> >> Andrus
> >>
> >>> On Mar 6, 2017, at 5:04 PM, Markus Reich 
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I always thought that when the content of an object changes (by commit)
> >> the
> >>> version gets incremented. I analyzed this a bit further and found sthg.
> >>> which is very clear for me :-/
> >>>
> >>> When I have two DataObjects with the same key/id and I change a value
> in
> >>> one Object, I got a higher version after committing the changes. The
> >> other
> >>> DataObjects also "gets" the new value, but the snapshot version stays
> the
> >>> same?
> >>>
> >>> The background is that I try to build a cache on computed values of
> >>> DataObjects, so I need an indicator for invalidating my cache, I
> thought
> >>> snapshot version would be perfect :-) but so it seems that this won't
> >> work
> >>> for my idea :-/
> >>>
> >>> best regards
> >>> Meex
> >>
> >>
>
>

Re: ObjectCache

[Musall, Maik]

And unless I'm explictly creating a query cache somewhere in my application,
I won't have one?

> Am 08.03.2017 um 22:05 schrieb John Huss :
> 
> Unless you are using a query cache then running an ObjectSelect will always
> give you fresh data from the DB.
> On Wed, Mar 8, 2017 at 3:01 PM Musall, Maik  wrote:
> 
>> Can someone confirm this? Will I always get fresh data from DB with an
>> explicit query, or am I at risk being returned stale data from a
>> context-local cache that doesn't even see changes that haven been recorded
>> in the shared snapshot cache in the meantime, let alone in the database?
>> 
>> I'm still a bit puzzled and out of documentation about the question of how
>> to control data freshness. Cayenne seems to be a bit different than EOF in
>> this regard, with it's multi-level caching.
>> 
>> Maik
>> 
>> 
>>> Am 24.02.2017 um 23:24 schrieb Markus Reich >> :
>>> 
>>> I think if you start a "standard" query it goes always against the DB,
>> this
>>> would be the query cache but this is not implicit done, you have to use
>>> groups for that.
>>> Object cache as I understand works behind the scenes, when you access
>>> properties of you persistent entity with getters, then values where read
>>> through the GraphManager from Cache.
>>> Only OID Queries and RelationalQueries can get their data from cache.
>>> Standard SelectQuery always runs a query on DB. But maybe I'm wrong :-/
>>> 
>>> Musall, Maik  schrieb am Fr., 24. Feb. 2017 um
>>> 23:14 Uhr:
>>> 
 Hi Lon,
 
 so with a context-local cache, you would still execute a regular query,
 but that query would not actually hit the database but the cache would
 return the result instead? Is it like a result set per query SQL string
 which is cached? I don't really understand how those local caches are
>> keyed.
 
 If there is any Cayenne documentation explaining this which I missed,
>> I'd
 be happt to get a pointer to that.
 
 Maik
 
> Am 24.02.2017 um 18:25 schrieb Lon Varscsak :
> 
> I built something similar in EOF to local cache, so I think I can
>> answer
 at
> least part of the question.
> 
> It’s not uncommon for me to have a complex set of queries to do
>> something
> like compute pricing on an order.  Rather than having to maintain many
 tiny
> caches or ivars with query results, all of my code executes queries to
 get
> the data set it needs whenever the pricing calculation is called.  It
 makes
> the code simpler, but gives you the performance of having stored the
> results yourself in an ivar (or dictionary).
> 
> Add in EHCache and now you can control how long those objects live
 (rather
> than the life of an ivar). If they get removed from cache due to cache
> control settings, the next time through the code, it will refetch them
 but
> I don’t have to be aware of this from a code perspective.
> 
> Where I’m still hung up, is that it’s common for me to have many parts
>> of
> my code that are not aware of each other that display/operate on the
>> same
> data, and I’m unsure how to update the cache (since there are multiple
> local caches) in between those two components after commit (since
>> change
> propagation is turned off by default and even when on is in a separate
> thread so I can’t assume the objects will be refreshed when I need).
> 
> -Lon
> 
> On Fri, Feb 24, 2017 at 3:00 AM, Musall, Maik 
 wrote:
> 
>> Hi all,
>> 
>> I'd like to extend this question a bit. I just read the entire
 performance
>> tuning chapter again [1], and I'm a bit puzzled especially about the
>> ObjectContext's local caches, which Andrus also recommended to use in
 the
>> "A way to refreshObject()" thread:
>> 
>>> So instead of micro-optimizations with shared snapshot cache, you
 should
>> use queries with cache groups and query cache. They get refetched
>> when a
>> cache group is flushed in response to an object commit. So technically
 you
>> are not reusing the snapshot cache, but it really doesn't matter. The
>> benefit you get in code simplicity, consistency, and often
>> performance,
>> always outweighs that. Besides this approach is cluster-friendly.
>> 
>> Coming from EOF I grew the habit of frequently creating short-lived
>> ObjectContexts, so basically every page load uses it's own new
>> ObjectContext to get fresh data from the shared global cache, because
 other
>> users may have altered data between page loads. While working on a
 page, I
>> have references to the objects that I need there anyway, so I don't
>> understand at all what that context-local cache mechanism should do
>> for
 me,
>> unless 

Re: ObjectCache

[John Huss]

Unless you are using a query cache then running an ObjectSelect will always
give you fresh data from the DB.
On Wed, Mar 8, 2017 at 3:01 PM Musall, Maik  wrote:

> Can someone confirm this? Will I always get fresh data from DB with an
> explicit query, or am I at risk being returned stale data from a
> context-local cache that doesn't even see changes that haven been recorded
> in the shared snapshot cache in the meantime, let alone in the database?
>
> I'm still a bit puzzled and out of documentation about the question of how
> to control data freshness. Cayenne seems to be a bit different than EOF in
> this regard, with it's multi-level caching.
>
> Maik
>
>
> > Am 24.02.2017 um 23:24 schrieb Markus Reich  >:
> >
> > I think if you start a "standard" query it goes always against the DB,
> this
> > would be the query cache but this is not implicit done, you have to use
> > groups for that.
> > Object cache as I understand works behind the scenes, when you access
> > properties of you persistent entity with getters, then values where read
> > through the GraphManager from Cache.
> > Only OID Queries and RelationalQueries can get their data from cache.
> > Standard SelectQuery always runs a query on DB. But maybe I'm wrong :-/
> >
> > Musall, Maik  schrieb am Fr., 24. Feb. 2017 um
> > 23:14 Uhr:
> >
> >> Hi Lon,
> >>
> >> so with a context-local cache, you would still execute a regular query,
> >> but that query would not actually hit the database but the cache would
> >> return the result instead? Is it like a result set per query SQL string
> >> which is cached? I don't really understand how those local caches are
> keyed.
> >>
> >> If there is any Cayenne documentation explaining this which I missed,
> I'd
> >> be happt to get a pointer to that.
> >>
> >> Maik
> >>
> >>> Am 24.02.2017 um 18:25 schrieb Lon Varscsak :
> >>>
> >>> I built something similar in EOF to local cache, so I think I can
> answer
> >> at
> >>> least part of the question.
> >>>
> >>> It’s not uncommon for me to have a complex set of queries to do
> something
> >>> like compute pricing on an order.  Rather than having to maintain many
> >> tiny
> >>> caches or ivars with query results, all of my code executes queries to
> >> get
> >>> the data set it needs whenever the pricing calculation is called.  It
> >> makes
> >>> the code simpler, but gives you the performance of having stored the
> >>> results yourself in an ivar (or dictionary).
> >>>
> >>> Add in EHCache and now you can control how long those objects live
> >> (rather
> >>> than the life of an ivar). If they get removed from cache due to cache
> >>> control settings, the next time through the code, it will refetch them
> >> but
> >>> I don’t have to be aware of this from a code perspective.
> >>>
> >>> Where I’m still hung up, is that it’s common for me to have many parts
> of
> >>> my code that are not aware of each other that display/operate on the
> same
> >>> data, and I’m unsure how to update the cache (since there are multiple
> >>> local caches) in between those two components after commit (since
> change
> >>> propagation is turned off by default and even when on is in a separate
> >>> thread so I can’t assume the objects will be refreshed when I need).
> >>>
> >>> -Lon
> >>>
> >>> On Fri, Feb 24, 2017 at 3:00 AM, Musall, Maik 
> >> wrote:
> >>>
>  Hi all,
> 
>  I'd like to extend this question a bit. I just read the entire
> >> performance
>  tuning chapter again [1], and I'm a bit puzzled especially about the
>  ObjectContext's local caches, which Andrus also recommended to use in
> >> the
>  "A way to refreshObject()" thread:
> 
> > So instead of micro-optimizations with shared snapshot cache, you
> >> should
>  use queries with cache groups and query cache. They get refetched
> when a
>  cache group is flushed in response to an object commit. So technically
> >> you
>  are not reusing the snapshot cache, but it really doesn't matter. The
>  benefit you get in code simplicity, consistency, and often
> performance,
>  always outweighs that. Besides this approach is cluster-friendly.
> 
>  Coming from EOF I grew the habit of frequently creating short-lived
>  ObjectContexts, so basically every page load uses it's own new
>  ObjectContext to get fresh data from the shared global cache, because
> >> other
>  users may have altered data between page loads. While working on a
> >> page, I
>  have references to the objects that I need there anyway, so I don't
>  understand at all what that context-local cache mechanism should do
> for
> >> me,
>  unless that "create new contexts all the time" strategy is just plain
> >> wrong
>  for Cayenne-based applications?
> 
>  And even if I would use a longer-lived ObjectContext for something,
> >> where
>  would 

Re: ObjectCache

[Musall, Maik]

Can someone confirm this? Will I always get fresh data from DB with an explicit 
query, or am I at risk being returned stale data from a context-local cache 
that doesn't even see changes that haven been recorded in the shared snapshot 
cache in the meantime, let alone in the database?

I'm still a bit puzzled and out of documentation about the question of how to 
control data freshness. Cayenne seems to be a bit different than EOF in this 
regard, with it's multi-level caching.

Maik


> Am 24.02.2017 um 23:24 schrieb Markus Reich :
> 
> I think if you start a "standard" query it goes always against the DB, this
> would be the query cache but this is not implicit done, you have to use
> groups for that.
> Object cache as I understand works behind the scenes, when you access
> properties of you persistent entity with getters, then values where read
> through the GraphManager from Cache.
> Only OID Queries and RelationalQueries can get their data from cache.
> Standard SelectQuery always runs a query on DB. But maybe I'm wrong :-/
> 
> Musall, Maik  schrieb am Fr., 24. Feb. 2017 um
> 23:14 Uhr:
> 
>> Hi Lon,
>> 
>> so with a context-local cache, you would still execute a regular query,
>> but that query would not actually hit the database but the cache would
>> return the result instead? Is it like a result set per query SQL string
>> which is cached? I don't really understand how those local caches are keyed.
>> 
>> If there is any Cayenne documentation explaining this which I missed, I'd
>> be happt to get a pointer to that.
>> 
>> Maik
>> 
>>> Am 24.02.2017 um 18:25 schrieb Lon Varscsak :
>>> 
>>> I built something similar in EOF to local cache, so I think I can answer
>> at
>>> least part of the question.
>>> 
>>> It’s not uncommon for me to have a complex set of queries to do something
>>> like compute pricing on an order.  Rather than having to maintain many
>> tiny
>>> caches or ivars with query results, all of my code executes queries to
>> get
>>> the data set it needs whenever the pricing calculation is called.  It
>> makes
>>> the code simpler, but gives you the performance of having stored the
>>> results yourself in an ivar (or dictionary).
>>> 
>>> Add in EHCache and now you can control how long those objects live
>> (rather
>>> than the life of an ivar). If they get removed from cache due to cache
>>> control settings, the next time through the code, it will refetch them
>> but
>>> I don’t have to be aware of this from a code perspective.
>>> 
>>> Where I’m still hung up, is that it’s common for me to have many parts of
>>> my code that are not aware of each other that display/operate on the same
>>> data, and I’m unsure how to update the cache (since there are multiple
>>> local caches) in between those two components after commit (since change
>>> propagation is turned off by default and even when on is in a separate
>>> thread so I can’t assume the objects will be refreshed when I need).
>>> 
>>> -Lon
>>> 
>>> On Fri, Feb 24, 2017 at 3:00 AM, Musall, Maik 
>> wrote:
>>> 
 Hi all,
 
 I'd like to extend this question a bit. I just read the entire
>> performance
 tuning chapter again [1], and I'm a bit puzzled especially about the
 ObjectContext's local caches, which Andrus also recommended to use in
>> the
 "A way to refreshObject()" thread:
 
> So instead of micro-optimizations with shared snapshot cache, you
>> should
 use queries with cache groups and query cache. They get refetched when a
 cache group is flushed in response to an object commit. So technically
>> you
 are not reusing the snapshot cache, but it really doesn't matter. The
 benefit you get in code simplicity, consistency, and often performance,
 always outweighs that. Besides this approach is cluster-friendly.
 
 Coming from EOF I grew the habit of frequently creating short-lived
 ObjectContexts, so basically every page load uses it's own new
 ObjectContext to get fresh data from the shared global cache, because
>> other
 users may have altered data between page loads. While working on a
>> page, I
 have references to the objects that I need there anyway, so I don't
 understand at all what that context-local cache mechanism should do for
>> me,
 unless that "create new contexts all the time" strategy is just plain
>> wrong
 for Cayenne-based applications?
 
 And even if I would use a longer-lived ObjectContext for something,
>> where
 would that local cache get in effect? Either I have references to
>> objects
 anyway, or I would need to refetch them, which would go through to the
 database, right? In what case would I get a benefit from a local cache?
>> I
 didn't find answers to this in the Cayenne Guide.
 
 Thanks
 Maik
 
 [1] https://cayenne.apache.org/docs/4.0/cayenne-guide/
 

Re: SnapshotVersion

[Markus Reich]

Hi Andrus,

I have some getters in DataObjects that use some DB properties for
calculation, e.g. I read other data from DB and to kind of lookups, so I
got a much better performance when I introtuced a caching of this
calculated values. I just use a concurrent hash map to realize the caching.
Now I need a trigger for invalidation of the cached values, when the
 DataObject itself get's invalidated?

regards
Meex

Andrus Adamchik  schrieb am Mi., 8. März 2017 um
20:24 Uhr:

> Hi Markus,
>
> Snapshot version reflects the version of a cached snapshot that was used
> to sync the current state of the object (this is probably obvious). The
> behavior observed by you may actually be a bug (and we may need to
> investigate it).
>
> Generally I would recommend against relying too much on snapshot version
> API. I always considered it an implementation detail (though of course it
> is publicly accessible... oh well) that may easily fall victim of a future
> DataObject redesign (e.g. see the parallel 'Fetching lots of objects'
> thread).
>
> Can you explain how your cache is intended to work? Perhaps there is a
> callback/listener solution to it?
>
> Andrus
>
> > On Mar 6, 2017, at 5:04 PM, Markus Reich 
> wrote:
> >
> > Hi,
> >
> > I always thought that when the content of an object changes (by commit)
> the
> > version gets incremented. I analyzed this a bit further and found sthg.
> > which is very clear for me :-/
> >
> > When I have two DataObjects with the same key/id and I change a value in
> > one Object, I got a higher version after committing the changes. The
> other
> > DataObjects also "gets" the new value, but the snapshot version stays the
> > same?
> >
> > The background is that I try to build a cache on computed values of
> > DataObjects, so I need an indicator for invalidating my cache, I thought
> > snapshot version would be perfect :-) but so it seems that this won't
> work
> > for my idea :-/
> >
> > best regards
> > Meex
>
>

Re: SnapshotVersion

[Andrus Adamchik]

Hi Markus,

Snapshot version reflects the version of a cached snapshot that was used to 
sync the current state of the object (this is probably obvious). The behavior 
observed by you may actually be a bug (and we may need to investigate it). 

Generally I would recommend against relying too much on snapshot version API. I 
always considered it an implementation detail (though of course it is publicly 
accessible... oh well) that may easily fall victim of a future DataObject 
redesign (e.g. see the parallel 'Fetching lots of objects' thread). 

Can you explain how your cache is intended to work? Perhaps there is a 
callback/listener solution to it?

Andrus

> On Mar 6, 2017, at 5:04 PM, Markus Reich  wrote:
> 
> Hi,
> 
> I always thought that when the content of an object changes (by commit) the
> version gets incremented. I analyzed this a bit further and found sthg.
> which is very clear for me :-/
> 
> When I have two DataObjects with the same key/id and I change a value in
> one Object, I got a higher version after committing the changes. The other
> DataObjects also "gets" the new value, but the snapshot version stays the
> same?
> 
> The background is that I try to build a cache on computed values of
> DataObjects, so I need an indicator for invalidating my cache, I thought
> snapshot version would be perfect :-) but so it seems that this won't work
> for my idea :-/
> 
> best regards
> Meex

Re: Fetching lots of objects

[Andrus Adamchik]

Hi Maik,

> On Mar 8, 2017, at 7:47 PM, Musall, Maik  wrote:
> 
> Well, if I need them all in the same context to work with after this, I would 
> then need to localObject() them and be back at locking, this time against the 
> graph manager. Dang.

Yes. Unfortunately.

> It would be nice if Cayenne would internally parallelize things like 
> ObjectResolver.objectsFromDataRows() and use lock-free strategies to deal 
> with the caching.

This is probably the last (and consequently the worst) place in Cayenne where 
locking still occurs. After I encountered this problem in a high-concurrency 
system, I've done some analysis of it (see [1] and also [2]), and this has been 
my "Cayenne 5.0" plan for a long time. With 4.0 making such progress as it does 
now, we may actually start contemplating it again.

Andrus


[1] 
https://lists.apache.org/thread.html/b3a990f94a8db3818c7f12eb433a8fef89d5e0afee653def11da1aa9@1382717376@%3Cdev.cayenne.apache.org%3E
[2] 
https://lists.apache.org/thread.html/bfcf79ffa521e402d080e3aafc5f0444fa0ab7d09045ec3092aee6c2@1382706785@%3Cdev.cayenne.apache.org%3E

Re: DataView

[Andrus Adamchik]

Hi Adrian,

You haven't missed anything, other than the new Cayenne APIs :) DataViews 
development was frozen back then, and to the best of my knowledge no one did 
any work on that.

Andrus

> On Mar 8, 2017, at 9:33 PM, Adrian Wiesmann  wrote:
> 
> Hi list
> 
> Back in the times there was that thing called DataView. DataViews take care 
> how a UI should work with data from Cayenne. See 
> https://cayenne.apache.org/docs/1.2/dataviews.html for a quick reminder.
> 
> Since things slowed down around v1.2 of Cayenne, I started to fork DataViews 
> and integrated them into what I called Gozer. While Gozer could render PDF 
> and CSV, it first of all was meant as the renderer for my Wicket based OSS 
> app.
> 
> I haven't worked with Cayenne in a while and am currently thinking about 
> dusting off Gozer (DataView) and bringing it up to date.
> 
> Just a quick check so that I do not re-invent the wheel: Is there anything I 
> missed in the last few years regarding the DataViews or are there similar 
> projects nowadays which I should look at first?
> 
> Regards,
> Adrian
> 

DataView

[Adrian Wiesmann]

Hi list

Back in the times there was that thing called DataView. DataViews take 
care how a UI should work with data from Cayenne. See 
https://cayenne.apache.org/docs/1.2/dataviews.html for a quick reminder.


Since things slowed down around v1.2 of Cayenne, I started to fork 
DataViews and integrated them into what I called Gozer. While Gozer 
could render PDF and CSV, it first of all was meant as the renderer for 
my Wicket based OSS app.


I haven't worked with Cayenne in a while and am currently thinking about 
dusting off Gozer (DataView) and bringing it up to date.


Just a quick check so that I do not re-invent the wheel: Is there 
anything I missed in the last few years regarding the DataViews or are 
there similar projects nowadays which I should look at first?


Regards,
Adrian

Re: Fetching lots of objects

[Musall, Maik]

Whoa. Parallel instantiation down to <2700 ms using multiple threads with a 
local ObjectContext each.

Well, if I need them all in the same context to work with after this, I would 
then need to localObject() them and be back at locking, this time against the 
graph manager. Dang. It would be nice if Cayenne would internally parallelize 
things like ObjectResolver.objectsFromDataRows() and use lock-free strategies 
to deal with the caching.


> Am 08.03.2017 um 14:17 schrieb John Huss :
> 
> If parallel is going to have any benefit you have to be using separate
> object contexts to avoid locking the same DataRow cache.
> On Wed, Mar 8, 2017 at 5:59 AM Musall, Maik  wrote:
> 
>> 
>>> Am 08.03.2017 um 10:56 schrieb Aristedes Maniatis :
>>> 
>>> On 8/3/17 6:54pm, Musall, Maik wrote:
>>> 
 regular SelectQuery: 25888 ms for 1291644 objects
 DataRowQuery alone: 14289 ms for 1291644 rows
 DataRowQuery sequential instantiation: 6878 ms for 1291644 objects, sum
>> = 21167
 DataRowQuery parallel instantiation: 7351 ms for 1291644 objects, sum =
>> 21640
 DataRowQuery with iterator: 22484 ms for 1291644 objects
 DataRowQuery with batch iterator of 100 each: 21219 ms for 1291644
>> objects
>>> 
>>> What about trying the new M5 release from yesterday and its ability to
>> select just the columns you need. You'll just get a list of column data
>> instead of a simpler object model, but it might be faster.
>>> 
>> 
>> This is M5 already (M6-SNAPSHOT really). But I need the full objects
>> because I need to do computations on them using the business logic
>> implemented in the DataObject class.
>> 
>> Maik
>> 
>> 

Re: Fetching lots of objects

[John Huss]

If parallel is going to have any benefit you have to be using separate
object contexts to avoid locking the same DataRow cache.
On Wed, Mar 8, 2017 at 5:59 AM Musall, Maik  wrote:

>
> > Am 08.03.2017 um 10:56 schrieb Aristedes Maniatis :
> >
> > On 8/3/17 6:54pm, Musall, Maik wrote:
> >
> >> regular SelectQuery: 25888 ms for 1291644 objects
> >> DataRowQuery alone: 14289 ms for 1291644 rows
> >> DataRowQuery sequential instantiation: 6878 ms for 1291644 objects, sum
> = 21167
> >> DataRowQuery parallel instantiation: 7351 ms for 1291644 objects, sum =
> 21640
> >> DataRowQuery with iterator: 22484 ms for 1291644 objects
> >> DataRowQuery with batch iterator of 100 each: 21219 ms for 1291644
> objects
> >
> > What about trying the new M5 release from yesterday and its ability to
> select just the columns you need. You'll just get a list of column data
> instead of a simpler object model, but it might be faster.
> >
>
> This is M5 already (M6-SNAPSHOT really). But I need the full objects
> because I need to do computations on them using the business logic
> implemented in the DataObject class.
>
> Maik
>
>

Re: Fetching lots of objects

[Musall, Maik]

> Am 08.03.2017 um 10:56 schrieb Aristedes Maniatis :
> 
> On 8/3/17 6:54pm, Musall, Maik wrote:
> 
>> regular SelectQuery: 25888 ms for 1291644 objects
>> DataRowQuery alone: 14289 ms for 1291644 rows
>> DataRowQuery sequential instantiation: 6878 ms for 1291644 objects, sum = 
>> 21167
>> DataRowQuery parallel instantiation: 7351 ms for 1291644 objects, sum = 21640
>> DataRowQuery with iterator: 22484 ms for 1291644 objects
>> DataRowQuery with batch iterator of 100 each: 21219 ms for 1291644 objects
> 
> What about trying the new M5 release from yesterday and its ability to select 
> just the columns you need. You'll just get a list of column data instead of a 
> simpler object model, but it might be faster.
> 

This is M5 already (M6-SNAPSHOT really). But I need the full objects because I 
need to do computations on them using the business logic implemented in the 
DataObject class.

Maik

Re: Fetching lots of objects

[Aristedes Maniatis]

On 8/3/17 6:54pm, Musall, Maik wrote:

> regular SelectQuery: 25888 ms for 1291644 objects
> DataRowQuery alone: 14289 ms for 1291644 rows
> DataRowQuery sequential instantiation: 6878 ms for 1291644 objects, sum = 
> 21167
> DataRowQuery parallel instantiation: 7351 ms for 1291644 objects, sum = 21640
> DataRowQuery with iterator: 22484 ms for 1291644 objects
> DataRowQuery with batch iterator of 100 each: 21219 ms for 1291644 objects

What about trying the new M5 release from yesterday and its ability to select 
just the columns you need. You'll just get a list of column data instead of a 
simpler object model, but it might be faster.

Ari



-- 
-->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A