Re: [web2py] Re: Appengine CPU cycles

2012-08-15 Thread Derek
Yes, I tried it on the model, it is true on every request. I don't understand why. On Tuesday, August 14, 2012 3:07:43 PM UTC-7, Anthony wrote: Where would you put that? If in a model or controller, won't the condition be true on every request? Anthony On Tuesday, August 14, 2012 4:44:24

Re: [web2py] Re: Appengine CPU cycles

2012-08-15 Thread Massimo Di Pierro
That is by design. model is not a module. this of db.define_table('whetever',Field('ts','datetime',default=request.now)) request.now is the date of the request. For this default to be properly set this MUST run at every request. This is the price you pay for the simple web2py syntax. If you

Re: [web2py] Re: Appengine CPU cycles

2012-08-14 Thread Khalil KHAMLICHI
This is what I am talking about, this functionality/dependency on requests needs to be moved to controllers, leaving models as a run once file. as some model code does in fact depend on the request and must be run each request). Anthony -- --

Re: [web2py] Re: Appengine CPU cycles

2012-08-14 Thread Anthony
On Tuesday, August 14, 2012 11:17:57 AM UTC-4, Khalil KHAMLICHI wrote: This is what I am talking about, this functionality/dependency on requests needs to be moved to controllers, leaving models as a run once file. Moving part of the model definitions to the controller is counter to the M/C

Re: [web2py] Re: Appengine CPU cycles

2012-08-14 Thread Derek
Would it not be possible to do something like this: if 'db' not in dir(): db = DAL(...) On Monday, August 13, 2012 6:43:34 PM UTC-7, rochacbruno wrote: Thinking again... It could be better if define_my_tables run once on the very first request and table keeps defined forever from

Re: [web2py] Re: Appengine CPU cycles

2012-08-14 Thread Anthony
Where would you put that? If in a model or controller, won't the condition be true on every request? Anthony On Tuesday, August 14, 2012 4:44:24 PM UTC-4, Derek wrote: Would it not be possible to do something like this: if 'db' not in dir(): db = DAL(...) On Monday, August 13, 2012

Re: [web2py] Re: Appengine CPU cycles

2012-08-13 Thread Khalil KHAMLICHI
I know, when I wrote that, I only meant to express my hope that this subject be included in web2py's roadmap. cloud hosting is the future and cpu cycles are its units for payment, web2py is so beautiful to work with, let's let this beauty fly to the clouds. On Mon, Aug 13, 2012 at 1:41 AM,

Re: [web2py] Re: Appengine CPU cycles

2012-08-13 Thread Anthony
I know, when I wrote that, I only meant to express my hope that this subject be included in web2py's roadmap. OK, that sounds more reasonable than I could clearly see that web2py was too heavy to be profitable on gae. :-) So, we already have conditional models as well as the option to

Re: [web2py] Re: Appengine CPU cycles

2012-08-13 Thread Khalil KHAMLICHI
I suppose a static file that is run once on server start-up and never again would be a good thing. On Mon, Aug 13, 2012 at 1:22 PM, Anthony abasta...@gmail.com wrote: I know, when I wrote that, I only meant to express my hope that this subject be included in web2py's roadmap. OK, that

Re: [web2py] Re: Appengine CPU cycles

2012-08-13 Thread Anthony
I suppose a static file that is run once on server start-up and never again would be a good thing. Are you talking about for model definitions? I think there was some discussion of something like this in the past (there would be some limitations, as some model code does in fact depend on

Re: [web2py] Re: Appengine CPU cycles

2012-08-13 Thread Bruno Rocha
The easy way is /models/ . nothing here ... /modules/mymodels.py from gluon import current from gluon.dal import DAL, Field def define_my_tables(*table_list): db = DAL() tables_definitions = { owner: {fields: [Field(name), Field(gender)], format: %(name)s, migrate:

Re: [web2py] Re: Appengine CPU cycles

2012-08-13 Thread Bruno Rocha
Thinking again... It could be better if define_my_tables run once on the very first request and table keeps defined forever from there --

Re: [web2py] Re: Appengine CPU cycles

2012-08-12 Thread Felipe Meirelles
Just a last update about the tests, my overall application cost (in dolars from appengine) droped 25% with the change, so, at least in appengine lazy models are a must. On Wednesday, August 8, 2012 12:24:37 PM UTC-3, Jonathan Lundell wrote: On 8 Aug 2012, at 7:59 AM, Massimo Di Pierro

Re: [web2py] Re: Appengine CPU cycles

2012-08-12 Thread Khalil KHAMLICHI
I have only made very little tests on gae, but I could clearly see that web2py was too heavy to be profitable on gae, it just consumes too much resources, running models.py on every request looks like a crazy thing to me.. --

Re: [web2py] Re: Appengine CPU cycles

2012-08-12 Thread Anthony
I have only made very little tests on gae, but I could clearly see that web2py was too heavy to be profitable on gae, it just consumes too much resources, running models.py on every request looks like a crazy thing to me.. Which is why this discussion has largely been about ways to

Re: [web2py] Re: Appengine CPU cycles

2012-08-08 Thread Felipe Meirelles
Well, after 24h running the new model less code I think I can post some data: First of all I'll explain my application archtecture: I have a cloud server gattering information from around 150 gps on some asserts. Each assert sends information every 30s, and some sends additional reading

Re: [web2py] Re: Appengine CPU cycles

2012-08-08 Thread Massimo Di Pierro
I do not understand. When you compare Django vs web2p[y performance, you seem to be comparing the MCycles for

Re: [web2py] Re: Appengine CPU cycles

2012-08-08 Thread Felipe Meirelles
Sorry about the URL confusion. The processing on web2py version is done on /queues/position, while in the django one is done in /api/position as I didnt implemented queues on the django version. Json dumps and loads are only used on web2py version, but not using the shipped simplejson (it realy

Re: [web2py] Re: Appengine CPU cycles

2012-08-08 Thread Jonathan Lundell
On 8 Aug 2012, at 7:59 AM, Massimo Di Pierro massimo.dipie...@gmail.com wrote: It is likely Django uses simplejson binary while web2py (for portability) only ships with simplejson in pure python. This may be another performance loss if you use json a lot. We need to fix this at the web2py

Re: [web2py] Re: Appengine CPU cycles

2012-08-08 Thread Massimo Di Pierro
Thanks for the clarification. On Wednesday, 8 August 2012 10:09:27 UTC-5, Felipe Meirelles wrote: Sorry about the URL confusion. The processing on web2py version is done on /queues/position, while in the django one is done in /api/position as I didnt implemented queues on the django

Re: [web2py] Re: Appengine CPU cycles

2012-08-07 Thread Alexei Vinidiktov
The app I'm developing is highly dependent on auth. Does it mean that any request that uses auth won't be faster than 200ms on GAE? On Tue, Aug 7, 2012 at 7:30 AM, howesc how...@umich.edu wrote: are you using auth? auth takes around 200ms to load i think on GAE. i avoid initializing auth

Re: [web2py] Re: Appengine CPU cycles

2012-08-07 Thread Christian Foster Howes
when i last tested it it was with a very complex custom auth + validators. there are probably ways to trim it down. one thing that i did was only init mail and attach mail to auth if i was doing something that required mail. cfh On 8/7/12 6:45 , Alexei Vinidiktov wrote: The app I'm

Re: [web2py] Re: Appengine CPU cycles

2012-08-07 Thread Anthony
The app I'm developing is highly dependent on auth. Does it mean that any request that uses auth won't be faster than 200ms on GAE? Note, if you're just checking for login, I don't think you need to call auth.define_tables() (that should only be needed for registration, login, and

Re: [web2py] Re: Appengine CPU cycles

2012-08-07 Thread Felipe Meirelles
Well, I'm still in the middle of tests but all points to a huge drop on cpu usage using a model less aproch. I've made, in a few days ill post numbers, but the cpu_ms is arround half of it was. I've set all db.define_table() in separete models, and just include them where i use them. Also, all

Re: [web2py] Re: Appengine CPU cycles

2012-08-07 Thread Massimo Di Pierro
Please share you benchmarks when you think you are done. On Tuesday, 7 August 2012 14:29:51 UTC-5, Felipe Meirelles wrote: Well, I'm still in the middle of tests but all points to a huge drop on cpu usage using a model less aproch. I've made, in a few days ill post numbers, but the cpu_ms

[web2py] Re: Appengine CPU cycles

2012-08-06 Thread Derek
Have you compiled your app? On Monday, August 6, 2012 10:46:33 AM UTC-7, Felipe Meirelles wrote: Hi, I'm using web2py in a project that stands on top of a REST api. The project is write intensive (around 110k requests a day, with all the requests making at least 2 updates to the

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Felipe Meirelles
Pre compiled by the admin interface? Not yet, is there some impove with the precompiled code? On Mon, Aug 6, 2012 at 3:16 PM, Derek sp1d...@gmail.com wrote: Have you compiled your app? On Monday, August 6, 2012 10:46:33 AM UTC-7, Felipe Meirelles wrote: Hi, I'm using web2py in a project

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Felipe Meirelles
Just compiled the app and deployed again, seems to make no diference. App engine uses the bytecode generated on web2py? Did I need to change some config at app.yaml? On Monday, August 6, 2012 3:21:29 PM UTC-3, Felipe Meirelles wrote: Pre compiled by the admin interface? Not yet, is there some

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Felipe Meirelles
Even the simplest controller give me a high cpu usage: import logging def test(): logging.info(Just making a performace test) ms=559 cpu_ms=612 api_cpu_ms=245 On Monday, August 6, 2012 3:37:38 PM UTC-3, Felipe Meirelles wrote: Just compiled the app and deployed again, seems to make no

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Bruno Rocha
What do you have in models? All model files runs alphabetically for each request, so we need to know what are you doing on model files. Can you try to test with an empty brand new app, remove all files from models folder and try your simple controller import logging def test():

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Felipe Meirelles
Without any model I have a huge drop on the cpu use, from around 300ms to around 60ms. Still higher than with Django, but its acceptable by the concept of the framework. Ill make some debug on my models and update the topic. On Monday, August 6, 2012 4:05:56 PM UTC-3, rochacbruno wrote: What

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Bruno Rocha
On Mon, Aug 6, 2012 at 4:19 PM, Felipe Meirelles fel...@felipemeirelles.com.br wrote: Without any model I have a huge drop on the cpu use, from around 300ms to around 60ms. Still higher than with Django, but its acceptable by the concept of the framework. Because of that I am using a

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Felipe Meirelles
With the basic models (i have 10+ tables on my app) the cpu time jumps from 60ms to 200ms. I have some grids and menus too, if I add that, it goes to my 300ms for a basic controller. Ill try the model less aproch and post the results again. Thanks. On Monday, August 6, 2012 4:27:41 PM UTC-3,

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Pystar
but doesnt going your model less route go against web2py tenet of convention over configuration? although it looks interesting On Monday, August 6, 2012 8:27:41 PM UTC+1, rochacbruno wrote: On Mon, Aug 6, 2012 at 4:19 PM, Felipe Meirelles wrote: Without any model I have a huge drop on the

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Bruno Rocha
On Mon, Aug 6, 2012 at 5:43 PM, Pystar aitoehi...@gmail.com wrote: but doesnt going your model less route go against web2py tenet of convention over configuration? although it looks interesting Yes, my sample code is very complex to implement. And this is why I am insisting on tests. If

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Pystar
+1 to making it an inbuilt feature and creating an easy to use API. I am not a python newbie, but your code is very tough meat to swallow. On Monday, August 6, 2012 9:53:02 PM UTC+1, rochacbruno wrote: On Mon, Aug 6, 2012 at 5:43 PM, Pystar aitoehi...@gmail.com wrote: but doesnt going your

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Massimo Di Pierro
You should check the {{=response.toolbar()}} because there may be more DB io than you think in the models. You should also try remove the setting of model attributes (requires=, models=, ...) and move them in the controllers that need them. Also mind that web2py on GAE has sessions turned on

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Felipe Meirelles
Actualy Django uses the datastore too. I'm using the memcache for all things its possible (it already reduce the old django version database access on 90% in this new web2py version). I think the overhead is realy from loading all models, grids and menus I've defined for my app. I'll try a

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread Anthony
+1 to making it an inbuilt feature and creating an easy to use API. I am not a python newbie, but your code is very tough meat to swallow. I've mentioned this before, but model-less just means that your db table definitions are moved into modules instead of being executed in files in the

Re: [web2py] Re: Appengine CPU cycles

2012-08-06 Thread howesc
are you using auth? auth takes around 200ms to load i think on GAE. i avoid initializing auth except for the controllers that use it, as well as having my model definitions in modules. cfh On Monday, August 6, 2012 3:57:51 PM UTC-7, Felipe Meirelles wrote: Actualy Django uses the datastore