Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
I don't understand this code at all. You're querying the data and then iterating over a separate call? And what does the extra call to list.size() have anything to do with this? The code linked from http://slim3demo.appspot.com/performance/ has just gone back to calling list.size() without

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Yasuo Higa
It's *NO* bogus benchmark because the sample iterates the results. http://code.google.com/p/slim3/source/browse/trunk/slim3demo/src/slim3/demo/controller/performance/GetLLController.java Yasuo Higa The code linked from http://slim3demo.appspot.com/performance/ has just gone back to calling

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
Ok - so what you're saying is that the extra call to list.size() before iterating through the list makes list iteration faster? Oddly enough, this does seem to make a difference. This looks like some sort of performance bug in the Low-Level API. It's clearly not related to Slim3... except in as

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
Star this issue: http://code.google.com/p/googleappengine/issues/detail?id=5167 I'm willing to bet I know exactly what's going on. When you call size() first, the backing ArrayList is being created initially with the proper size. If you don't call size() first, the backing ArrayList is

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Dennis Peterson
Haha, excellent. I studied cargo cults a bit in anthropology classes, long ago, and never suspected how relevant they would be. You would probably enjoy this: http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Gal Dolber
Ok, you actually got me to waste 1 hour on benchmarking and I hate to say that you are right. The use of reflexion seems not to be heavy enough to make the difference. Some extra features on slim3 make it slower when you don't make use them, I almost have a working path to avoid this problem, but

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
Don't feel bad - I've now wasted over a day on this! Sorry if I've come across as grumpy in previous emails... but I've wasted over a day on this :-( Jeff On Thu, Jun 9, 2011 at 4:25 PM, Gal Dolber gal.dol...@gmail.com wrote: Ok, you actually got me to waste 1 hour on benchmarking and I hate

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jay Young
When frameworks compete, everyone wins! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/3zCQizDuxPIJ. To post to this group, send

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Gal Dolber
No problem... I say what I think, but have no shame to admit when I'm wrong On Thu, Jun 9, 2011 at 8:55 PM, Jay Young jayyoung9...@gmail.com wrote: When frameworks compete, everyone wins! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java

[appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Erwin Streur
Indeed Dennis's measurements are very suspicious. First you should do a couple of warming ups on each of the implementations to prevent pollution like the JDO classpath scan for enhanced classes (which is one of the reasons for the high initial run). Then do a couple of run to determine a range of

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Dennis Peterson
It's not my benchmark, it's Slim3's :) ...but you're right, it's bogus. I asked on the main appengine group too, and it turns out the low-level benchmark is doing lazy loading. With that fixed, their numbers come out like yours. I found this one too, which also gets results like yours:

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Yasuo Higa
It is not bogus. LazyList#size() fetches all data as follows: public int size() { resolveAllData(); return results.size(); } Yasuo Higa On Wed, Jun 8, 2011 at 11:32 PM, Dennis Peterson dennisbpeter...@gmail.com wrote: It's not my benchmark, it's Slim3's :) ...but you're right,

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Dennis Peterson
Apologies, no offense meant. My impression was that if you wanted to, say, display all that data, it's going to take around 1000 ms to get it, not 1 ms. On Wed, Jun 8, 2011 at 10:55 AM, Yasuo Higa higaya...@gmail.com wrote: It is not bogus. LazyList#size() fetches all data as follows: public

[appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Mike Lawrence
I get... The number of entities: 1 low level 1717 millis slim3 1502 millis objectify 2970 millis jdo 3485 millis probably should modify this example to do an average of several runs one important thing to note, is slim3 allows you to update multiple entity types in a single transaction. not

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Yasuo Higa
Hi Dennis, You can see all sources. http://slim3demo.appspot.com/performance/ Java runtime reflections are very very slow. If you don't think so, please try it by you. Yasuo Higa On Thu, Jun 9, 2011 at 12:00 AM, Dennis Peterson dennisbpeter...@gmail.com wrote: Apologies, no offense meant. My

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Yasuo Higa
Hi Dennis, The following document will help you about global transactions: http://sites.google.com/site/slim3appengine/#gtx Yasuo Higa On Thu, Jun 9, 2011 at 12:33 AM, Dennis Peterson dennisbpeter...@gmail.com wrote: Those multi-entity transactions are definitely interesting to me. There's

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Dennis Peterson
Those multi-entity transactions are definitely interesting to me. There's some overhead but no getting around that. A while back I was playing around with some adhoc methods to do it in a specific case, but I suspect Slim3 is more solid and maybe faster than what I was doing. Definitely easier.

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Dennis Peterson
No I agree, and slim3 looks very interesting to me. It was just the very fast low-level times I was wondering about, but it looks like normally slim3 and low-level will be about the same speed. On Wed, Jun 8, 2011 at 11:42 AM, Yasuo Higa higaya...@gmail.com wrote: Hi Dennis, You can see all

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Jeff Schnitzer
You are wrong. Try adding getProperty() calls to your LL performance test, and the speed advantage of the LL API goes away. I don't know what to say about Slim3, but here's my test case: http://code.google.com/p/scratchmonkey/source/browse/#svn%2Fappengine%2Fperformance-test I created 10,000

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Gal Dolber
Slim3 is not only fast, the api is completely awesome. It has been my choice for a year now for all gae projects. It includes name safety and and amazing querying utils. Very recommendable! On Wed, Jun 8, 2011 at 3:41 PM, Jeff Schnitzer j...@infohazard.org wrote: You are wrong. Try adding

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Jeff Schnitzer
Slim3 may be a nice piece of software, but it has not been demonstrated to be faster than anything (including JDO). It might or might not be faster - I don't know - but based on the sloppy benchmarking, I'm pretty certain that the people making this claim don't know either. There's another

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Gal Dolber
Jeff, Objectify is a great project, I have used it a lot before slim3. I didn't use slim3 global transactions yet, but I don't understand why you attack that when that is the one feature that jdo/jpa/objectify/twig cannot deliver. Slim3 is indeed faster than any other because of the simple fact

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Yasuo Higa
What I want to provide is a fair and casual benchmark. As jeff advised, I modified samples as follows: for (Entity e : service.getBarListUsingLL()) { e.getKey(); e.getProperty(sortValue); } for (Bar bar : service.getBarListUsingSlim3()) { bar.getKey(); bar.getSortValue(); } for

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Jeff Schnitzer
On Wed, Jun 8, 2011 at 4:26 PM, Gal Dolber gal.dol...@gmail.com wrote: Slim3 is indeed faster than any other because of the simple fact that it uses apt(code generation) instead of reflexion, the generated code it's almost the same that you'll write by-hand to wrap the low-level api. I'm

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Jeff Schnitzer
Thank you for fixing the benchmark. I am very curious. According to this new benchmark - it's hard to tell without pushing the buttons a lot of times, but there seems to be a trend - Slim3 is somewhat faster than the Low Level API. Doesn't Slim3 use the Low Level API underneath? How can it

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Yasuo Higa
Slim3 uses LL API. To resolve a strange issue that slim3 is faster than LL, I tried the following samples: One: AsyncDatastoreService ds = DatastoreServiceFactory.getAsyncDatastoreService(); Query q = new Query(Bar); PreparedQuery pq = ds.prepare(q); ListEntity list =

[appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-07 Thread Ian Marshall
The low-level API does indeed look very fast. Just a comment on JDO: repeat runs roughly halve the JDO run time. I presume that this is because for repeat runs the JDO persistence manager factory has already been constructed. On Jun 6, 8:44 pm, DennisP dennisbpeter...@gmail.com wrote: I'm