Thanks guys!
So for next level strategy I will:
1) Determine and cache all queries and calculations that are non user
spefic (e.g. ranking of all participants)
- This can be done either with cached select(), or using
cache.ram/memcache/redis directly
- These could be done in a model level to make sure that they are updated
regularly or use a cron/scheduler to do that in a background
2) Make session specific calculations and queries lazy and cache those that
are required to be done multiple times in a session
- This can be done with @cache_property and/or storing results in a session
variable (e.g. participant ranking in different competitions)
3) Use iterator/generator with those methods/functions that require only
end result and most likely only once
- These could be also stored in a session for reuse if necessary
And of course it's a good idea to analyze code and decide the indexing of
the database tables based on what is used heaviest.
I think it's easier to get the right strategy now when the code is simple,
than later. Although I find Python flexible to change architecture also
later.
Ykä
On Saturday, August 17, 2013 12:59:13 AM UTC+3, Massimo Di Pierro wrote:
>
>
>
> On Friday, 16 August 2013 06:58:49 UTC-5, Ykä Marjanen wrote:
>>
>> I read about cached select a bit more. So basically I could cache
>> especially the queries that are non user dependent (e.g. when calculating
>> all participant rankings) and the query would be cached to all session.
>> Right?
>>
>
> yes
>
>
>>
>> I'm yet a bit early in the process to worry about too much optimization,
>> but it's good to get a right understanding from the beginning, as I will
>> soon have zillions of users :)
>>
>> Ykä
>>
>> On Friday, August 16, 2013 12:39:20 PM UTC+3, Niphlod wrote:
>>>
>>> That's a good way to eat up memory in the python process. I would have
>>> saved a lot of headaches and just used select() with cache, possibly
>>> exploiting memcache or redis to do the hard job. Don't take this the wrong
>>> way, the cached_property is a neat trick, it's just not that suitable for
>>> storing your zillions participant when you'll grow big.
>>>
>>> On Friday, August 16, 2013 10:25:32 AM UTC+2, Ykä Marjanen wrote:
>>>>
>>>> Hi,
>>>>
>>>> After learning Python and web2py deeper, I've restructured my web2py
>>>> application so that all functions and data are now in classes with lazy
>>>> methods in my own module. The lazy property class turns any method into a
>>>> "cached attribute", thus I can make a db query that returns rows and if it
>>>> is called again it uses the previous result.
>>>>
>>>> What this has allowed me to do, is to clean practically all code in my
>>>> controller as the logic and variables are in a class. Now I just pass the
>>>> class to the view and the view prints the "attributes" directly without
>>>> separate call and saving to dict. I've read that some don't like any logic
>>>> in the view, but that's unavoidable as in the view you need to print stuff
>>>> depending who the user is and is he logged in or not.
>>>>
>>>> I'd like your opinions and comments on my strategy. Here are few
>>>> simplified examples from my code:
>>>>
>>>> Module:
>>>>
>>>> class MyClass(object):
>>>> def __init__(self):
>>>> self.db = current.db
>>>>
>>>> @cached_property
>>>> def all_participants(self):
>>>> return self.db(self.db.participant.id>0).select()
>>>>
>>>> class MyClassParticipant(MyClass):
>>>> "Inherits base class"
>>>> def __init__(self, auth_user_id=None):
>>>> self.auth_user_id = auth_user_id
>>>>
>>>> super(MyClassParticipant, self).__init__()
>>>>
>>>> @cached_property
>>>> def participant_stats(self):
>>>> return
>>>> self.db(self.db.participant.auth_user_id==self.auth_user_id).select()
>>>>
>>>> Controller:
>>>>
>>>> def index():
>>>> if auth.user:
>>>> xclassinstance =
>>>> module.MyClassParticipant(auth_user_id=auth.user_id)
>>>> else:
>>>> xclassinstance = module.MyClass()
>>>>
>>>> return dict(xclassinstance=xclassinstance)
>>>>
>>>> View:
>>>>
>>>> {{if auth.user:}}
>>>> Your ranking is {{=xclassinstance.participant_stats.ranking}}
>>>>
>>>> Rankings of all participants
>>>> <br />
>>>> {{for participant in xclassinstance.all_participants:}}
>>>> {{=participant.name}} ranking is {{=participant.ranking}}
>>>> <br />
>>>>
>>>> Ykä
>>>>
>>>>
>>>>
--
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
