Re: Pylons: pros and cons
Let me thank all of you who took part in this discussion. I'm glad that I put my remarks about Pylons and related components here. I gained some new knowledge and updated existing one. Thanks :) and see you around.. T. -- _i__'simplicity_is_the_key'__tomasz_nazar _ii'i_am_concern_oriented'JKM-UPR _iii__'patsystem.sf.net'___linux_user _'aspectized.com'___prevayler --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
-- _i__'simplicity_is_the_key'__tomasz_nazar _ii'i_am_concern_oriented'JKM-UPR _iii__'patsystem.sf.net'___linux_user _'aspectized.com'___prevayler --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~--- pycon.tar.gz Description: GNU Zip compressed data
Re: Pylons: pros and cons
On Thu, Oct 16, 2008 at 2:08 AM, Mike Orr [EMAIL PROTECTED] wrote: On Wed, Oct 15, 2008 at 4:35 PM, Tomasz Nazar [EMAIL PROTECTED] wrote: On Tue, Oct 14, 2008 at 5:20 AM, Michael Bayer [EMAIL PROTECTED] wrote: On Oct 13, 12:14 pm, Tomasz Nazar [EMAIL PROTECTED] wrote: 1) all model classes are defined in 1 file together with database mapping like everyone is saying, you can roll this however you want and however is appropriate to the type of app you're building. Michael and Mike - I've just found your new tutorial on how to use SQLA and how to setup classes in separate files! That's it! I wasn't looking into docs for a while :( http://wiki.pylonshq.com/display/pylonsdocs/Using+SQLAlchemy+with+Pylons It looks I can split my big file - I dreamed about that. That is great! One stuff that doesn't work for me is autoload a table definition. I almost copied your MyApp.tar.gz sources and adjusted to autoload. But doesn't work. -- sqlalchemy.exceptions.UnboundExecutionError: The MetaData is not bound to an Engine or Connection. Execution can not proceed without a database to execute against. Either execute with an explicit connection or assign the MetaData's .bind to enable implicit execution. I tried to move all mappings + t_tables to init_model (as engine is only available then), but still fail. Is it too much asking for similar source code, but with classes split into files and working with 'autoload'? Or point to a right way at least :) I can upload source when I'm done with it.. I think you need to read the SQLAlchemy manual about how engines propagate and autoloading works. Then you'll be able to do it yourself. In summary, SQL queries (including ORM operations) need a database connection on which to operate. The engine gives out connections in a thread-safe manner, so you can just pretend the engine is a connection and everything works. The engine is necessary only when you *execute* a query; e.g., SQL.execute(), Session.execute(), Session.commit(). query.all(), using a query as an iterator, etc. Most execution methods take a bind= argument to make it use that engine. Autoloading is an exception because its argument is autoload_with= . (I think it was a bad design choice to not call it bind=, but that's the way it is.) If you don't use bind=, a Session-based query will look for an engine bound to the session, and then for an engine bound to the metadata. If neither exists the query will fail with the error you're seeing. A non-session-based SQL query will look in the metadata for an engine, and fail in the same way if there is none. So you should bind your engine to the session, use Session.execute() for any non-ORM operations you have to do, and pass bind= and autoload_with= to the remaining methods which don't fit into either category: table(autoload_with=), meta.create_all(bind=). You can also bind the engine to the metadata to avoid the inconvenience of those bind= and autoload_with= arguments. Bound metadata is no longer recommended for beginners because it's a bit magical, and because you can get into trouble if you have an ORM transaction open and simultaneously modify the database at the SQL level without going through Session.execute() -- this may give the ORM an incorrect impression of the database state, or the database may implicitly commit the pending transaction without the ORM realizing it. But it does work. -- Mike Orr [EMAIL PROTECTED] OK. I have clean Pylons 0.9.6.2 app working with 3 classes into separate modules. Works with MySql. Sqlite seems not to store objects (code the same, just exchanging sqlalchemy.url). Why? Cyclic dependencies do not work just that simple. Cyclic imports unfortunately have to placed inside a method - not at the top of module. I'd still use some help here. I'll try to attach full source - hope googlegroups allows that.. For impatient sample code below: app=pycon, requires mysql or sqlite, 3 classes defined # pycon.model.phone.py: class Phone(object): pass pycon.model.participant.py from pycon.model.conference import Conference class Participant(object): pass pycon.model.conference.py: from datetime import datetime class Conference(object): def __init__(self, subject=None): self.subject = subject self.date = datetime.now() def __str__(self): return %s, s: %s % (self.__class__.__name__, self.subject) def cyclic_dependency(self): #importing at the top of module causes cyclic import issue :( from pycon.model.participant import Participant pass pycon.model.__init__.py: import sqlalchemy as sa from sqlalchemy import orm from pycon.model import meta from pycon.model.phone import Phone from pycon.model.participant import Participant from pycon.model.conference import Conference def init_model(engine): Call me before using any of the tables or classes in the
Re: Pylons: pros and cons
wrt to organizing the model into different files, I've changed the
scheme that's is autogenerated in 0.9.7rc2.
My __init__.py looks like
##
from sqlalchemy import MetaData
from sqlalchemy.orm import scoped_session, sessionmaker
engine = None
meta = MetaData()
def init_model(engine):
Call me before using any of the tables or classes in the
model
sm = sessionmaker(autoflush=True, autocommit=False, bind=engine)
meta.engine = engine
meta.Session = scoped_session(sm)
# below are all individual classes
from user import *
from item import *
from folder import *
from page import *
###
every class file contains the tabel definition, the class defintion
and the mapper. example
page.py, containing, Page a derived class from Item
from sqlalchemy import MetaData, Column, Table, types, ForeignKey
from sqlalchemy.orm import mapper, relation, backref
from project.model import *
page = Table('page', meta.metadata,
Column('id', types.Integer, ForeignKey('item.id'),
primary_key=True),
Column('path', types.String(255)),
Column('language', types.String(2)),
Column('label', types.String(255)),
)
class Page(Item):
def __init__(self, identifier, **arg):
self.identifier = identifier
for (k,v) in arg.iteritems():
setattr(self,k,v)
mapper(Page, page, inherits=Item, polymorphic_identity='page')
###
but of course this all is a personal preference
Ruben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Thu, Oct 16, 2008 at 2:41 PM, cropr [EMAIL PROTECTED] wrote:
wrt to organizing the model into different files, I've changed the
scheme that's is autogenerated in 0.9.7rc2.
My __init__.py looks like
##
from sqlalchemy import MetaData
from sqlalchemy.orm import scoped_session, sessionmaker
engine = None
meta = MetaData()
def init_model(engine):
Call me before using any of the tables or classes in the
model
sm = sessionmaker(autoflush=True, autocommit=False, bind=engine)
meta.engine = engine
meta.Session = scoped_session(sm)
# below are all individual classes
from user import *
from item import *
from folder import *
from page import *
###
every class file contains the tabel definition, the class defintion
and the mapper. example
page.py, containing, Page a derived class from Item
from sqlalchemy import MetaData, Column, Table, types, ForeignKey
from sqlalchemy.orm import mapper, relation, backref
from project.model import *
page = Table('page', meta.metadata,
Column('id', types.Integer, ForeignKey('item.id'),
primary_key=True),
Column('path', types.String(255)),
Column('language', types.String(2)),
Column('label', types.String(255)),
)
class Page(Item):
def __init__(self, identifier, **arg):
self.identifier = identifier
for (k,v) in arg.iteritems():
setattr(self,k,v)
mapper(Page, page, inherits=Item, polymorphic_identity='page')
###
but of course this all is a personal preference
Ruben
I wouldn't mind to also have table + mapping inside a file with class.
But there are some issues with mapping using 'autoload' - engine does
not exist yet. Isn't it?
T.
--
_i__'simplicity_is_the_key'__tomasz_nazar
_ii'i_am_concern_oriented'JKM-UPR
_iii__'patsystem.sf.net'___linux_user
_'aspectized.com'___prevayler
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Thu, Oct 16, 2008 at 8:02 AM, Tomasz Nazar [EMAIL PROTECTED] wrote: I wouldn't mind to also have table + mapping inside a file with class. But there are some issues with mapping using 'autoload' - engine does not exist yet. Isn't it? You can't do autoloading or map to an autoloaded table at the top level. You have to do that inside init_model(). The reason is that the top-level code is executed when the module is first imported, and the engine does not exist at that point (or at least the module doesn't know about it). -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Thu, Oct 16, 2008 at 4:12 AM, Tomasz Nazar [EMAIL PROTECTED] wrote: OK. I have clean Pylons 0.9.6.2 app working with 3 classes into separate modules. Works with MySql. Sqlite seems not to store objects (code the same, just exchanging sqlalchemy.url). Why? That's not something we can troubleshoot. It could be a broken SQLite installation, a configuration error, or the database being in a different directory than you intended. Cyclic dependencies do not work just that simple. Cyclic imports unfortunately have to placed inside a method - not at the top of module. Importing in a function is the best way to deal with cyclic imports, yes. Although even better is to avoid them entirely. Why do conference and participant have to import each other? Neither is using the other as its base class. If you have a tight relationship between two classes like this, it's a good reason to put them in the same module. -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Thu, Oct 16, 2008 at 8:55 PM, Mike Orr [EMAIL PROTECTED] wrote: On Thu, Oct 16, 2008 at 4:12 AM, Tomasz Nazar [EMAIL PROTECTED] wrote: Cyclic dependencies do not work just that simple. Cyclic imports unfortunately have to placed inside a method - not at the top of module. Importing in a function is the best way to deal with cyclic imports, yes. Although even better is to avoid them entirely. Yes. Of course it is :) It's a matter of taste. I don't like having imports inside a method, though sometimes that's the only way.. Why do conference and participant have to import each other? Neither is using the other as its base class. If you have a tight relationship between two classes like this, it's a good reason to put them in the same module. That was just a fake example to test if that works. In real examples I have few such situations and I won't elaborate on that. I feel sorry that Python import works this way. It's just the way it is.. So I look for best method to work round that. I do not have and don't want to design my classes and modules in a tree like graph. IMHO Python forces me here too much - and my nature is to resist when pushed too much ;) T. -- _i__'simplicity_is_the_key'__tomasz_nazar _ii'i_am_concern_oriented'JKM-UPR _iii__'patsystem.sf.net'___linux_user _'aspectized.com'___prevayler --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Oct 16, 2:13 pm, Tomasz Nazar [EMAIL PROTECTED] wrote: On Thu, Oct 16, 2008 at 8:55 PM, Mike Orr [EMAIL PROTECTED] wrote: On Thu, Oct 16, 2008 at 4:12 AM, Tomasz Nazar [EMAIL PROTECTED] wrote: Cyclic dependencies do not work just that simple. Cyclic imports unfortunately have to placed inside a method - not at the top of module. Importing in a function is the best way to deal with cyclic imports, yes. Although even better is to avoid them entirely. Yes. Of course it is :) It's a matter of taste. I don't like having imports inside a method, though sometimes that's the only way.. Why do conference and participant have to import each other? Neither is using the other as its base class. If you have a tight relationship between two classes like this, it's a good reason to put them in the same module. That was just a fake example to test if that works. In real examples I have few such situations and I won't elaborate on that. I feel sorry that Python import works this way. It's just the way it is.. So I look for best method to work round that. I do not have and don't want to design my classes and modules in a tree like graph. IMHO Python forces me here too much - and my nature is to resist when pushed too much ;) I get tripped up by cyclic imports once in a while, but I think once you understand how imports work in general, it's really no big deal to work with/around their quirks. Every language has these kinds of quirks--I'm pretty sure it's unavoidable. I found this article helpful for coming to terms with how imports work in Python: http://effbot.org/zone/import-confusion.htm -wlb --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Wed, Oct 15, 2008 at 2:11 PM, Mike Orr [EMAIL PROTECTED] wrote: On Wed, Oct 15, 2008 at 10:25 AM, Jorge Vargas [EMAIL PROTECTED] wrote: On Mon, Oct 13, 2008 at 10:11 PM, Mike Orr [EMAIL PROTECTED] wrote: On Mon, Oct 13, 2008 at 9:14 AM, Tomasz Nazar [EMAIL PROTECTED] wrote: Is reason other than problems with pure Python imports (which don't work well for recursive dependencies)? Circular imports work but you have to put the lines in a certain order. If module A imports module B, and module B imports module A *and* in its top level code accesses something in module A, that something has to be created in module A *above* the import B statement. Otherwise, at the moment module B needs it, it doesn't exist yet. If you do all your attribute access inside functions, this doesn't matter because both modules are fully initialized by the time the function is called (as long as it isn't called at the top level of the module). Rather than having to carefully order their lines of code, most people prefer to just avoid circular imports. So they make small base modules like 'meta' and 'errors' that everybody can import, but which themselves import only the Python standard library. the underlaying problem is that if you want to make your models available as model.Obj instead of model.obj.Obj you need to provide this magic with the meta.py and __init__.py if you will reference everything in a java-like form, then you can get rid of the imports in model and everything works fine, but then you don't have the flexibility of reorganizing stuff inside the model package without breaking your client code. Bottom line it's a very handy feature that require a little obscure code, and by obscure I mean it's not logical at first glance not that is complex code. Forgot to address the issue of how Python imports differ from Java. In Java the import is just a syntax shortcut to avoid having to type the module prefix. If the module (class) is used anywhere, the compiler automatically loads it, with or without import. In Python, the module is not accessible unless you import it in every module that depends on it. import is also an executable statement (as opposed to a compiler directive like global), which means that it matters where you put it: the item is not accessible before you execute the import but it is accessible afterward. Technically, the first time a module is imported, it's loaded and assigned a key in sys.modules. If you import it multiple times, the memory copy is used directly rather than reloading the code. But even if the module is loaded, it's not *accessible* unless you import it . (Or sneak into it with 'module = sys.modules['the_module]'.) ok I think you are mixing two things here. #1 crud - as in code generation #2 admin interface - as in a real way to update models I was really talking about an admin interface, or something that could be plugged into a program, not code generation. BTW, a complete Admin interface should also stop and restart the server, view logs, manage users and permissions, etc. Managing users is only feasable if the application uses a standardized user model, but the other features are generic to any Pylons application. Zope has this kind of admin interface, and ideally it should be available for Pylons too. That's what I thought TurboGears was interested in building. That's a very nice long term plan. And I think rum is a good candidate for it. the user permissions is what I find the hardest as a standardized model is something that isn't possible in most applications. At most I'll say provide something that will inspect your model classes and work from there, which is what rum does but currently doesn't handles it any different from any other table. Stopping/restarting the server would have to be done in Paste, so it would need a method in Paste that Pylons could call. that could be interesting, although it means the admin will have to run standalone in it's own port, again something rum is able to do. I think most of the parts you suggest are already in place it's just a matter of sitting down and add the glue, although I do think that ones it happens people will complain that it isn't flexible enough. According to Adrian's talk at PyCon this year, Django's CRUD interface should theoretically run on Pylons. Somebody just needs to test it and write a HOWTO. Ben and I thought it would be coming soon, but it got lost among all the other work finishing up 0.9.7. django-admin? how will it know about all the model configuration stuff? is there any links anywhere that suggest how it should be done? last time I heard django's sqlalchemy compatibility was in an abandoned branch. Also I assume you still need to install the whole tarball to get it running. I don't know how since I've never used Django. But Ben thought it would be possible with some work. Me neither I have just play with it. but it
Re: Pylons: pros and cons
Thanks for such detailed answer Michael.
On Tue, Oct 14, 2008 at 5:20 AM, Michael Bayer [EMAIL PROTECTED] wrote:
On Oct 13, 12:14 pm, Tomasz Nazar [EMAIL PROTECTED] wrote:
2) SQLAlchemy 2nd level cache
That is most frustrating for me coming from Hibernate, where it's
built in. You may point to memcached or whatever, but the truth is it
improves performance and coding style hugely!
I would love to optimize later and be a bit lazy.
Maybe the authors do not have resources -- it's a great soft anyway --
but that should be one of the 1st features on the roadmap.
pull from the giant vat of identifiers when needed. This was one of
the original issues SQLA sought to solve, and makes it a whole lot
easier to load several levels of objects, with or without a LIMIT/
OFFSET applied to the overall results, with only one round trip.
I started to play with SQLa joins and 'add_entity' feature lately, so
if I would get all objects in one query that would be not so bad.
However, to do that one needs to learn SQLa query language a lot, and
build some advanced queries (and there aren't lots of examples in the
documentation :( ).
And also with 'add_entity' I loose my OO, and relations between
objects. Query result is not just a list of objects, but a touple of
all the different objects from the query...
Or one can write custom SQL query instead of SQLA query. Then we have
a list of kind of DTO objects. Also not that pretty.
Both ways (I use them) but have disadvantages.
Having 2nd level cache would allow a developer to write short and easy code.
I could write basic SQLA queries for objects and iterate through their
relations (even when few levels deep).
Of course, for the first time it would take long to hit DB with many
queries, but -- with cache -- further requests do not exist, cause
cache is hit instead.
That has advantage in early (even after 1 year) stage of project, so I
do not have to worry about fancy SQL queries or about learning SQLA
query language. I worry more about other stuff (understanding business
domain, etc..) So as developer spend precious time elsewhere.
Also code would be a lot simpler:
confs = dbsession().query(Conference).filter(XXX).all()
%for conf in confs:
${conf} ${conf.author} ${conf.author.phone} etc...
It's that simple with cache being used.
Without that... code is more complicated.
But eager loading isn't as good as no loading at all - so onto caching
for SQLA. There's some reasons they're inconvenient. ORM mapped
objects usually need to load additional data (i.e. lazyloading) which
implies an association with an active database connection. Caching
the ORM objects means you have to dance around all the lazy loaders to
ensure none fire off, since there is no active database connection -
furthermore, even if you re-associated them with one, now you have a
concurrency issue.
Secondly, its incredibly common to get some ORM mapped objects from
somewhere and start using them in a transaction. If you get them from
a cache, again you lose because they're global objects - you can't map
them back to your transaction. So ORM caching which seems very simple
and automatic quickly leads to some very broken use cases for those
who don't understand what they're doing.
I'd live with that. I mostly need cached objects for read-only view
layer, to iterate over objects and their relations..
Hibernate guys did that and I didn't had any problems with that cache.
For updating object from cache, I will load it from db anyway (or merge).
Pylons and Mako offer some of the most flexible view level caching
around - you can cache controller methods, functions, template
renders, and individual components (defs) of templates. I recently
just wrote a small %def which sits in the globally inherited layout
template, wraps the middle of the page in a conditional cache which is
enabled and configured by a method called in the controller - so that
context-sensitive info in the header (like login info) stays context
sensitive, the content-heavy middle is optionally cached, all the
parameters and zoning of the cache is configured at the controller
level, the ORM fetches data purely from the DB (but less often), and
none of the templates have any idea whether or not they're cached or
for how long. This is likely the subject of my next Pylons/Mako blog
entry.
I was thinking about your idea of caching view layer, instead of
having 2nd level object cache.
Yes - it is new for me. In Java it might be it is not so common.
I will give it a try..
Please, do write a blog then. It would be nice to listen to.
With examples please :)
T.
--
_i__'simplicity_is_the_key'__tomasz_nazar
_ii'i_am_concern_oriented'JKM-UPR
_iii__'patsystem.sf.net'___linux_user
_'aspectized.com'___prevayler
--~--~-~--~~~---~--~~
You received this message because you are
Re: Pylons: pros and cons
On Oct 15, 7:40 am, Tomasz Nazar [EMAIL PROTECTED] wrote:
Also code would be a lot simpler:
confs = dbsession().query(Conference).filter(XXX).all()
%for conf in confs:
${conf} ${conf.author} ${conf.author.phone} etc...
It's that simple with cache being used.
Without that... code is more complicated.
Well I think the gains with SQL level caching become problematic down
the road in lots of cases, but try out that example I gave you. It'll
give you exactly that behavior. It can be expanded in many ways,
such as to produce regioning behavior, i.e.
sess.query(Conference).cache(somekey,
region=short_term).filter(XXX).all()
where short_term points to some regioning object with timeout
params, etc.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---
Re: Pylons: pros and cons
W liście Tomasz Nazar z dnia środa 15 października 2008: I started to play with SQLa joins and 'add_entity' feature lately, so if I would get all objects in one query that would be not so bad. However, to do that one needs to learn SQLa query language a lot, and build some advanced queries (and there aren't lots of examples in the documentation :( ). And also with 'add_entity' I loose my OO, and relations between objects. Query result is not just a list of objects, but a touple of all the different objects from the query... Why not just use eager loading (well documented in sqlalchemy docs)? This uses joins and loads the related objects automatically while still returning object graph, not tuples. Also, when an object was already loaded, then future requests for this object will not hit the db at all. -- Paweł Stradomski --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Mon, Oct 13, 2008 at 10:11 PM, Mike Orr [EMAIL PROTECTED] wrote: On Mon, Oct 13, 2008 at 9:14 AM, Tomasz Nazar [EMAIL PROTECTED] wrote: Hi there! I've been a Pylons user for more than a year already. Few other people in the neighbourhood ask me for Pylons advantages often, so I also talk about Pylons disadvantages also. Here I'd like to point out some of them and ask you for your opinion/answer.. (basically I'm a happy Pylons user :) ) It may also be hard to define things below as related to Pylons only, as Pylons is often just a glue to other frameworks (aka SQLA..). Nevertheless full Pylons books contains several chapters treat about these, so let me also point them out. Most of the feature you suggest aren't part of pylons because they depend on a style of programming that may not fit everyone. But most of the things you need are actually build into TG2, except the SA catching, which as Mike suggested isn't really needed in the SA/web world. Here goes: 1) all model classes are defined in 1 file together with database mapping That is 2nd most frustrating on my daily work. Even in Pylons tutorial you've advised to mix all the code together in models/__init__.py. Business logic in my small app contains more than 3K of code and is in a single file! I already moved out DB SQLA mappings into separate file, but that's just a tip of the iceberg. So I ask: why is that? Each model class should be in a separate file (aka Rails, java frameworks, etc) It was not the developers' intention to force everything into one module. The default configuration was chosen to address a wide variety of needs both large and small, and to be a starting point for making your own customizations. The 'meta' module was created to make it easier to keep your classes in separate files, so that they could all import 'meta' and avoid importing 'model' circularly. SQLAlchemy models are intrinsically difficult to organize because there are three different parts (tables, classes, mappings) -- and Session configuration is a fourth part. Which module structure is best? People have different ideas and we couldn't come to a decision, so we just made the simplest configuration that would work. The next version of the SQLAlchemy configuration will default to SQLAlchemy 0.5's Declarative structure. This will bring the three parts together again, and perhaps make it easier to organize your classes per module without having to import as much. TG2 default model templates work like this, it does a little magic in order to prevent the circular imports issue but the code is really simple. And can be ported almost directly to pylons http://svn.turbogears.org/projects/tg.devtools/trunk/devtools/templates/turbogears/+package+/model/ Is reason other than problems with pure Python imports (which don't work well for recursive dependencies)? Circular imports work but you have to put the lines in a certain order. If module A imports module B, and module B imports module A *and* in its top level code accesses something in module A, that something has to be created in module A *above* the import B statement. Otherwise, at the moment module B needs it, it doesn't exist yet. If you do all your attribute access inside functions, this doesn't matter because both modules are fully initialized by the time the function is called (as long as it isn't called at the top level of the module). Rather than having to carefully order their lines of code, most people prefer to just avoid circular imports. So they make small base modules like 'meta' and 'errors' that everybody can import, but which themselves import only the Python standard library. the underlaying problem is that if you want to make your models available as model.Obj instead of model.obj.Obj you need to provide this magic with the meta.py and __init__.py if you will reference everything in a java-like form, then you can get rid of the imports in model and everything works fine, but then you don't have the flexibility of reorganizing stuff inside the model package without breaking your client code. Bottom line it's a very handy feature that require a little obscure code, and by obscure I mean it's not logical at first glance not that is complex code. Why: __init__.py -- there are lot's better names. Also, when one uses IDE and tries to find (by incremental file search) __init* there are many of them. Name it domain.py or model.py And finally: any Python ways to split _my_ big file? __init__.py is a Python language feature, and there's nothing the Pylons developers can do about it. (Except switch to Ruby :). hehehe rulons 3) flash messages : data persisted somewhere between http session and http request, which lives through one session of http redirects, but then is purged. I often need that, as I place some common controller logic in some controllers. Say: home.py:home which redirects to home.mako, and loads complete
Re: Pylons: pros and cons
On Wed, Oct 15, 2008 at 10:25 AM, Jorge Vargas [EMAIL PROTECTED] wrote: On Mon, Oct 13, 2008 at 10:11 PM, Mike Orr [EMAIL PROTECTED] wrote: On Mon, Oct 13, 2008 at 9:14 AM, Tomasz Nazar [EMAIL PROTECTED] wrote: Is reason other than problems with pure Python imports (which don't work well for recursive dependencies)? Circular imports work but you have to put the lines in a certain order. If module A imports module B, and module B imports module A *and* in its top level code accesses something in module A, that something has to be created in module A *above* the import B statement. Otherwise, at the moment module B needs it, it doesn't exist yet. If you do all your attribute access inside functions, this doesn't matter because both modules are fully initialized by the time the function is called (as long as it isn't called at the top level of the module). Rather than having to carefully order their lines of code, most people prefer to just avoid circular imports. So they make small base modules like 'meta' and 'errors' that everybody can import, but which themselves import only the Python standard library. the underlaying problem is that if you want to make your models available as model.Obj instead of model.obj.Obj you need to provide this magic with the meta.py and __init__.py if you will reference everything in a java-like form, then you can get rid of the imports in model and everything works fine, but then you don't have the flexibility of reorganizing stuff inside the model package without breaking your client code. Bottom line it's a very handy feature that require a little obscure code, and by obscure I mean it's not logical at first glance not that is complex code. Forgot to address the issue of how Python imports differ from Java. In Java the import is just a syntax shortcut to avoid having to type the module prefix. If the module (class) is used anywhere, the compiler automatically loads it, with or without import. In Python, the module is not accessible unless you import it in every module that depends on it. import is also an executable statement (as opposed to a compiler directive like global), which means that it matters where you put it: the item is not accessible before you execute the import but it is accessible afterward. Technically, the first time a module is imported, it's loaded and assigned a key in sys.modules. If you import it multiple times, the memory copy is used directly rather than reloading the code. But even if the module is loaded, it's not *accessible* unless you import it . (Or sneak into it with 'module = sys.modules['the_module]'.) ok I think you are mixing two things here. #1 crud - as in code generation #2 admin interface - as in a real way to update models I was really talking about an admin interface, or something that could be plugged into a program, not code generation. BTW, a complete Admin interface should also stop and restart the server, view logs, manage users and permissions, etc. Managing users is only feasable if the application uses a standardized user model, but the other features are generic to any Pylons application. Zope has this kind of admin interface, and ideally it should be available for Pylons too. That's what I thought TurboGears was interested in building. Stopping/restarting the server would have to be done in Paste, so it would need a method in Paste that Pylons could call. According to Adrian's talk at PyCon this year, Django's CRUD interface should theoretically run on Pylons. Somebody just needs to test it and write a HOWTO. Ben and I thought it would be coming soon, but it got lost among all the other work finishing up 0.9.7. django-admin? how will it know about all the model configuration stuff? is there any links anywhere that suggest how it should be done? last time I heard django's sqlalchemy compatibility was in an abandoned branch. Also I assume you still need to install the whole tarball to get it running. I don't know how since I've never used Django. But Ben thought it would be possible with some work. -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Tue, Oct 14, 2008 at 5:20 AM, Michael Bayer [EMAIL PROTECTED] wrote: On Oct 13, 12:14 pm, Tomasz Nazar [EMAIL PROTECTED] wrote: 1) all model classes are defined in 1 file together with database mapping like everyone is saying, you can roll this however you want and however is appropriate to the type of app you're building. Michael and Mike - I've just found your new tutorial on how to use SQLA and how to setup classes in separate files! That's it! I wasn't looking into docs for a while :( http://wiki.pylonshq.com/display/pylonsdocs/Using+SQLAlchemy+with+Pylons It looks I can split my big file - I dreamed about that. That is great! One stuff that doesn't work for me is autoload a table definition. I almost copied your MyApp.tar.gz sources and adjusted to autoload. But doesn't work. -- sqlalchemy.exceptions.UnboundExecutionError: The MetaData is not bound to an Engine or Connection. Execution can not proceed without a database to execute against. Either execute with an explicit connection or assign the MetaData's .bind to enable implicit execution. I tried to move all mappings + t_tables to init_model (as engine is only available then), but still fail. Is it too much asking for similar source code, but with classes split into files and working with 'autoload'? Or point to a right way at least :) I can upload source when I'm done with it.. T. -- _i__'simplicity_is_the_key'__tomasz_nazar _ii'i_am_concern_oriented'JKM-UPR _iii__'patsystem.sf.net'___linux_user _'aspectized.com'___prevayler --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Wed, Oct 15, 2008 at 4:35 PM, Tomasz Nazar [EMAIL PROTECTED] wrote: On Tue, Oct 14, 2008 at 5:20 AM, Michael Bayer [EMAIL PROTECTED] wrote: On Oct 13, 12:14 pm, Tomasz Nazar [EMAIL PROTECTED] wrote: 1) all model classes are defined in 1 file together with database mapping like everyone is saying, you can roll this however you want and however is appropriate to the type of app you're building. Michael and Mike - I've just found your new tutorial on how to use SQLA and how to setup classes in separate files! That's it! I wasn't looking into docs for a while :( http://wiki.pylonshq.com/display/pylonsdocs/Using+SQLAlchemy+with+Pylons It looks I can split my big file - I dreamed about that. That is great! One stuff that doesn't work for me is autoload a table definition. I almost copied your MyApp.tar.gz sources and adjusted to autoload. But doesn't work. -- sqlalchemy.exceptions.UnboundExecutionError: The MetaData is not bound to an Engine or Connection. Execution can not proceed without a database to execute against. Either execute with an explicit connection or assign the MetaData's .bind to enable implicit execution. I tried to move all mappings + t_tables to init_model (as engine is only available then), but still fail. Is it too much asking for similar source code, but with classes split into files and working with 'autoload'? Or point to a right way at least :) I can upload source when I'm done with it.. I think you need to read the SQLAlchemy manual about how engines propagate and autoloading works. Then you'll be able to do it yourself. In summary, SQL queries (including ORM operations) need a database connection on which to operate. The engine gives out connections in a thread-safe manner, so you can just pretend the engine is a connection and everything works. The engine is necessary only when you *execute* a query; e.g., SQL.execute(), Session.execute(), Session.commit(). query.all(), using a query as an iterator, etc. Most execution methods take a bind= argument to make it use that engine. Autoloading is an exception because its argument is autoload_with= . (I think it was a bad design choice to not call it bind=, but that's the way it is.) If you don't use bind=, a Session-based query will look for an engine bound to the session, and then for an engine bound to the metadata. If neither exists the query will fail with the error you're seeing. A non-session-based SQL query will look in the metadata for an engine, and fail in the same way if there is none. So you should bind your engine to the session, use Session.execute() for any non-ORM operations you have to do, and pass bind= and autoload_with= to the remaining methods which don't fit into either category: table(autoload_with=), meta.create_all(bind=). You can also bind the engine to the metadata to avoid the inconvenience of those bind= and autoload_with= arguments. Bound metadata is no longer recommended for beginners because it's a bit magical, and because you can get into trouble if you have an ORM transaction open and simultaneously modify the database at the SQL level without going through Session.execute() -- this may give the ORM an incorrect impression of the database state, or the database may implicitly commit the pending transaction without the ORM realizing it. But it does work. -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
Here goes: 1) all model classes are defined in 1 file together with database mapping For this particular gripe I think you're a little off-base. This is all personal preference, and the bigger your model gets, the better off you are breaking it up. Google around a bit and you'll find a few different strategies for splitting the model up into more logical chunks -- it's a fundamental feature of SQLA. All you have to do is import the relevant modules into your model/__init__.py and you're in business. And frankly, you're not married to that either -- it's just a convenience. You could import your model modules one at a time so long as you take care to set up the session and meta. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Oct 13, 12:14 pm, Tomasz Nazar [EMAIL PROTECTED] wrote:
Hi there!
I've been a Pylons user for more than a year already. Few other people
in the neighbourhood ask me for Pylons advantages often, so I also
talk about Pylons disadvantages also.
Here I'd like to point out some of them and ask you for your
opinion/answer.. (basically I'm a happy Pylons user :) )
It may also be hard to define things below as related to Pylons only,
as Pylons is often just a glue to other frameworks (aka SQLA..).
Nevertheless full Pylons books contains several chapters treat about
these, so let me also point them out.
Here goes:
1) all model classes are defined in 1 file together with database mapping
That is 2nd most frustrating on my daily work. Even in Pylons tutorial
you've advised to mix all the code together in models/__init__.py.
Business logic in my small app contains more than 3K of code and is in
a single file!
I already moved out DB SQLA mappings into separate file, but that's
just a tip of the iceberg.
So I ask: why is that? Each model class should be in a separate file
(aka Rails, java frameworks, etc)
Is reason other than problems with pure Python imports (which don't
work well for recursive dependencies)?
Why: __init__.py -- there are lot's better names.
Also, when one uses IDE and tries to find (by incremental file search)
__init* there are many of them. Name it domain.py or model.py
And finally: any Python ways to split _my_ big file?
You can take this as far as you would like. I have my init_model
method and mappers in model/__init__.py
My tables and their classes and logic are each in their own file in
model/tables
from project.model.tables import table1
__init__.py is a Python convention and really has nothing to do with
Pylons.
2) SQLAlchemy 2nd level cache
That is most frustrating for me coming from Hibernate, where it's
built in. You may point to memcached or whatever, but the truth is it
improves performance and coding style hugely!
I would love to optimize later and be a bit lazy.
Maybe the authors do not have resources -- it's a great soft anyway --
but that should be one of the 1st features on the roadmap.
Memcached works great. Why implement an maintain another 2nd level
caching system when a good one already exists and is very easy to
plugin to SA?
mapper(Foo, foo_table, extension=MemCachedMapper(mc, timeout=35))
3) flash messages : data persisted somewhere between http session
and http request, which lives through one session of http redirects,
but then is purged.
I often need that, as I place some common controller logic in some
controllers. Say: home.py:home which redirects to home.mako, and loads
complete user profile. I often use
h.redirect_to(cont=home,action=home) from inside other controllers.
I think such feature should be built in, similar to c, and session
variables.
Make yourself a component template and include it on pages you need to
present a flash
%def name=flash()
% if session.has_key('flash'):
div id=flashp${session.get('flash')}/p/div
%
del session['flash']
session.save()
%
% endif
/%def
4) CRUD: generate view + controller methods
Having controller file with CRUD methods already defined (using SQLA)
and redirecting to template files would be nice to have.
I don't need that at this stage of project, where I use custom views
all over the place. But for beginners it would be nice to have
CRUD*.mako(s) created (as option?) and to have working application
almost instantly.
It's often said that Django does that, and Pylons not :(
But what if I don't want that? .. 2-way street. Take a look at the
template for RESTful controllers. You could easily implement yourself
a CRUD controller to do just this. Then make a patch and open a
feature ticket and attach your patch. Who knows :)
5) easy_install pylons - Beta
It's a minor, but let me say that: When last time I installed Pylons
in Windows for a friend, it downloaded some eggs which were beta stage
(sqlalchemy?). I just know that proper 0.9.6 app was not working.. We
had to downgrade few eggs to make it work.
I can see the point here, I've done many a easy_install which needed a
rollback because the latest version of Foo was borked.
I usually find myself freezing my Python installation once development
starts on the application and doing sprints to phase in new product
versions.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---
Re: Pylons: pros and cons
Thanks both of you for answers...
On Mon, Oct 13, 2008 at 7:39 PM, Wayne Witzel [EMAIL PROTECTED] wrote:
On Oct 13, 12:14 pm, Tomasz Nazar [EMAIL PROTECTED] wrote:
Hi there!
1) all model classes are defined in 1 file together with database mapping
That is 2nd most frustrating on my daily work. Even in Pylons tutorial
you've advised to mix all the code together in models/__init__.py.
Business logic in my small app contains more than 3K of code and is in
a single file!
I already moved out DB SQLA mappings into separate file, but that's
just a tip of the iceberg.
So I ask: why is that? Each model class should be in a separate file
(aka Rails, java frameworks, etc)
Is reason other than problems with pure Python imports (which don't
work well for recursive dependencies)?
Why: __init__.py -- there are lot's better names.
Also, when one uses IDE and tries to find (by incremental file search)
__init* there are many of them. Name it domain.py or model.py
And finally: any Python ways to split _my_ big file?
You can take this as far as you would like. I have my init_model
method and mappers in model/__init__.py
My tables and their classes and logic are each in their own file in
model/tables
from project.model.tables import table1
__init__.py is a Python convention and really has nothing to do with
Pylons.
Table definitions might be in one place. I don't bother about that.
It's around 50 lines.
What I want is to separate classes definition into separate files / modules.
I always hit the wall with Python saying 'cannot import name ..' --
cause of recursive dependencies.
And for that I see place of having a doc for that.
I'm influenced a bit by Java, so I might not fully understand Python modules.
If modules are the only way to separate into files - I'm fine.
I will try again to split with latest Pylons code, maybe I'm having
some stupid bug. But I was trying 2 times and failed. Let's make it 3
then.
2) SQLAlchemy 2nd level cache
That is most frustrating for me coming from Hibernate, where it's
built in. You may point to memcached or whatever, but the truth is it
improves performance and coding style hugely!
I would love to optimize later and be a bit lazy.
Maybe the authors do not have resources -- it's a great soft anyway --
but that should be one of the 1st features on the roadmap.
Memcached works great. Why implement an maintain another 2nd level
caching system when a good one already exists and is very easy to
plugin to SA?
mapper(Foo, foo_table, extension=MemCachedMapper(mc, timeout=35))
I don't bother if it's implemented with memcached or any other. I
bother about having it at all.
Don't say to me it's easy to implement. If it's easy then as an author
I'd feel obliged to provide that. Other ORMs in the world have that.
Should I say more.. (?)
OK. Back to your code? What is this MemCachedMapper.. google 2 hits
only. Is it your own solution, does it work, can you share?
3) flash messages : data persisted somewhere between http session
and http request, which lives through one session of http redirects,
but then is purged.
I often need that, as I place some common controller logic in some
controllers. Say: home.py:home which redirects to home.mako, and loads
complete user profile. I often use
h.redirect_to(cont=home,action=home) from inside other controllers.
I think such feature should be built in, similar to c, and session
variables.
Make yourself a component template and include it on pages you need to
present a flash
%def name=flash()
% if session.has_key('flash'):
div id=flashp${session.get('flash')}/p/div
%
del session['flash']
session.save()
%
% endif
/%def
Yeah. I already did that months ago. I just feel it's good to propose
to include in Pylons, so you know one would make use of that..
4) CRUD: generate view + controller methods
Having controller file with CRUD methods already defined (using SQLA)
and redirecting to template files would be nice to have.
I don't need that at this stage of project, where I use custom views
all over the place. But for beginners it would be nice to have
CRUD*.mako(s) created (as option?) and to have working application
almost instantly.
It's often said that Django does that, and Pylons not :(
But what if I don't want that? .. 2-way street. Take a look at the
template for RESTful controllers. You could easily implement yourself
a CRUD controller to do just this. Then make a patch and open a
feature ticket and attach your patch. Who knows :)
If it's 2-way you don't go it. Someone else could go..
I don't insist on that. It'd be nice to have. I will have look at what
you propose. Thanks.
5) easy_install pylons - Beta
It's a minor, but let me say that: When last time I installed Pylons
in Windows for a friend, it downloaded some eggs which were beta stage
(sqlalchemy?). I just know that proper 0.9.6 app was not working.. We
had to downgrade few
Re: Pylons: pros and cons
On Oct 13, 5:33 pm, Tomasz Nazar [EMAIL PROTECTED] wrote: Thanks both of you for answers... On Mon, Oct 13, 2008 at 7:39 PM, Wayne Witzel [EMAIL PROTECTED] wrote: On Oct 13, 12:14 pm, Tomasz Nazar [EMAIL PROTECTED] wrote: Hi there! 1) all model classes are defined in 1 file together with database mapping That is 2nd most frustrating on my daily work. Even in Pylons tutorial you've advised to mix all the code together in models/__init__.py. Business logic in my small app contains more than 3K of code and is in a single file! I already moved out DB SQLA mappings into separate file, but that's just a tip of the iceberg. So I ask: why is that? Each model class should be in a separate file (aka Rails, java frameworks, etc) Is reason other than problems with pure Python imports (which don't work well for recursive dependencies)? Why: __init__.py -- there are lot's better names. Also, when one uses IDE and tries to find (by incremental file search) __init* there are many of them. Name it domain.py or model.py And finally: any Python ways to split _my_ big file? You can take this as far as you would like. I have my init_model method and mappers in model/__init__.py My tables and their classes and logic are each in their own file in model/tables from project.model.tables import table1 __init__.py is a Python convention and really has nothing to do with Pylons. Table definitions might be in one place. I don't bother about that. It's around 50 lines. What I want is to separate classes definition into separate files / modules. I always hit the wall with Python saying 'cannot import name ..' -- cause of recursive dependencies. And for that I see place of having a doc for that. I'm influenced a bit by Java, so I might not fully understand Python modules. If modules are the only way to separate into files - I'm fine. I will try again to split with latest Pylons code, maybe I'm having some stupid bug. But I was trying 2 times and failed. Let's make it 3 then. http://docs.python.org/reference/simple_stmts.html#import Basically, __init__.py turns your directory into a package. Which you can then qualify to get your classes and tables. model __init__.py foo.py class Foo and sa.Table foo_table are both in foo.py __init__.py contains: from model import foo Then you can map them in __init__.py with mapper(foo.Foo, foo.foo_table) 2) SQLAlchemy 2nd level cache That is most frustrating for me coming from Hibernate, where it's built in. You may point to memcached or whatever, but the truth is it improves performance and coding style hugely! I would love to optimize later and be a bit lazy. Maybe the authors do not have resources -- it's a great soft anyway -- but that should be one of the 1st features on the roadmap. Memcached works great. Why implement an maintain another 2nd level caching system when a good one already exists and is very easy to plugin to SA? mapper(Foo, foo_table, extension=MemCachedMapper(mc, timeout=35)) I don't bother if it's implemented with memcached or any other. I bother about having it at all. Don't say to me it's easy to implement. If it's easy then as an author I'd feel obliged to provide that. Other ORMs in the world have that. Should I say more.. (?) I said it was easy to plugin, not implement. OK. Back to your code? What is this MemCachedMapper.. google 2 hits only. Is it your own solution, does it work, can you share? Not my solution, was a solution presented a while back on the SA mailing list. See http://groups.google.com/group/sqlalchemy/msg/5d505529ee157162 I've used it in albeit non-critical, production sites. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Oct 13, 12:14 pm, Tomasz Nazar [EMAIL PROTECTED] wrote: 1) all model classes are defined in 1 file together with database mapping like everyone is saying, you can roll this however you want and however is appropriate to the type of app you're building. 2) SQLAlchemy 2nd level cache That is most frustrating for me coming from Hibernate, where it's built in. You may point to memcached or whatever, but the truth is it improves performance and coding style hugely! I would love to optimize later and be a bit lazy. Maybe the authors do not have resources -- it's a great soft anyway -- but that should be one of the 1st features on the roadmap. Hibernate makes a big deal out of second level cache because it makes too small a deal out of eager loading. Hibernate's eager loading is fundamentally broken since you can't use it on a result set that has LIMIT/OFFSET applied to it - the outer join gets wrapped inside the LIMIT. Their own docs think eager loading is a fairly rare use case, because they're coming from that old EJB mindset of entity beans stored in a giant in-memory hashtable against their primary keys, which assumes that most relations are many-to-one's which can just pull from the giant vat of identifiers when needed. This was one of the original issues SQLA sought to solve, and makes it a whole lot easier to load several levels of objects, with or without a LIMIT/ OFFSET applied to the overall results, with only one round trip. But eager loading isn't as good as no loading at all - so onto caching for SQLA. There's some reasons they're inconvenient. ORM mapped objects usually need to load additional data (i.e. lazyloading) which implies an association with an active database connection. Caching the ORM objects means you have to dance around all the lazy loaders to ensure none fire off, since there is no active database connection - furthermore, even if you re-associated them with one, now you have a concurrency issue. Secondly, its incredibly common to get some ORM mapped objects from somewhere and start using them in a transaction. If you get them from a cache, again you lose because they're global objects - you can't map them back to your transaction. So ORM caching which seems very simple and automatic quickly leads to some very broken use cases for those who don't understand what they're doing. To fix the above two issues, SQLA offers the merge(dont_load=True) method, which allows you to take your cached objects and associate a copy of them with the current Session. Though the overhead of copying objects from the cache into the Session might not be that much cheaper than the typical SQL query (and certainly doesn't save you anywhere near the overhead that view layer caching does). Here's a real simple 2nd level ORM query cache for SQLAlchemy which does the merge thing and provides an easy Query interface: http://www.sqlalchemy.org/trac/browser/sqlalchemy/trunk/examples/query_caching/query_caching.py . As of yet, nobody has sought to embark upon a SQLAlchemy 2nd Level Caching project, but if there were this might be where they'd start. If you swap out the dictionary there for a Beaker cache, there's your 2nd level Hibernate-style cache sans XML pushups. But concerning that method, I've built SQL statement caches in the past (caching objects which represent result sets mapped to SQL strings, without the ORM issues mentioned above) and not had good results due to the huge tree of SQL statements and result sets which get generated. Caches at this layer are complicated to manage, zone, and expire, and ultimately don't address enough of the performance overhead of the full request-to-render process. So the biggest reason that ORM caches aren't so critical is that caching for a web application is extremely effective at the view layer.If you've spent a lot time with Java/JSP, this might not be apparent since view layer caching support is abysmal in servlets/jsp/struts/taglibs/tiles/ etc. (more EJB fallout). Pylons and Mako offer some of the most flexible view level caching around - you can cache controller methods, functions, template renders, and individual components (defs) of templates. I recently just wrote a small %def which sits in the globally inherited layout template, wraps the middle of the page in a conditional cache which is enabled and configured by a method called in the controller - so that context-sensitive info in the header (like login info) stays context sensitive, the content-heavy middle is optionally cached, all the parameters and zoning of the cache is configured at the controller level, the ORM fetches data purely from the DB (but less often), and none of the templates have any idea whether or not they're cached or for how long. This is likely the subject of my next Pylons/Mako blog entry. View level caching usually reduces the size of data which is cached, since you are caching just the HTML, not fully pickled class instances, their
Re: Pylons: pros and cons
On Oct 13, 9:54 pm, Wayne Witzel [EMAIL PROTECTED] wrote: OK. Back to your code? What is this MemCachedMapper.. google 2 hits only. Is it your own solution, does it work, can you share? Not my solution, was a solution presented a while back on the SA mailing list. Seehttp://groups.google.com/group/sqlalchemy/msg/5d505529ee157162 I've used it in albeit non-critical, production sites. This cache has some drawbacks - it seems to be reinventing session.merge() somehow, and also only takes effect for straight get()s. A cache needs to cache the results of any SELECT statement to really be useful. The cache I've added to SQLA's examples does this but doesn't have the on-update expiration feature this one has - its much more involved to implement for a general SELECT cache and I would usually leave expiration as a manual affair. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons: pros and cons
On Mon, Oct 13, 2008 at 9:14 AM, Tomasz Nazar [EMAIL PROTECTED] wrote: Hi there! I've been a Pylons user for more than a year already. Few other people in the neighbourhood ask me for Pylons advantages often, so I also talk about Pylons disadvantages also. Here I'd like to point out some of them and ask you for your opinion/answer.. (basically I'm a happy Pylons user :) ) It may also be hard to define things below as related to Pylons only, as Pylons is often just a glue to other frameworks (aka SQLA..). Nevertheless full Pylons books contains several chapters treat about these, so let me also point them out. Here goes: 1) all model classes are defined in 1 file together with database mapping That is 2nd most frustrating on my daily work. Even in Pylons tutorial you've advised to mix all the code together in models/__init__.py. Business logic in my small app contains more than 3K of code and is in a single file! I already moved out DB SQLA mappings into separate file, but that's just a tip of the iceberg. So I ask: why is that? Each model class should be in a separate file (aka Rails, java frameworks, etc) It was not the developers' intention to force everything into one module. The default configuration was chosen to address a wide variety of needs both large and small, and to be a starting point for making your own customizations. The 'meta' module was created to make it easier to keep your classes in separate files, so that they could all import 'meta' and avoid importing 'model' circularly. SQLAlchemy models are intrinsically difficult to organize because there are three different parts (tables, classes, mappings) -- and Session configuration is a fourth part. Which module structure is best? People have different ideas and we couldn't come to a decision, so we just made the simplest configuration that would work. The next version of the SQLAlchemy configuration will default to SQLAlchemy 0.5's Declarative structure. This will bring the three parts together again, and perhaps make it easier to organize your classes per module without having to import as much. Is reason other than problems with pure Python imports (which don't work well for recursive dependencies)? Circular imports work but you have to put the lines in a certain order. If module A imports module B, and module B imports module A *and* in its top level code accesses something in module A, that something has to be created in module A *above* the import B statement. Otherwise, at the moment module B needs it, it doesn't exist yet. If you do all your attribute access inside functions, this doesn't matter because both modules are fully initialized by the time the function is called (as long as it isn't called at the top level of the module). Rather than having to carefully order their lines of code, most people prefer to just avoid circular imports. So they make small base modules like 'meta' and 'errors' that everybody can import, but which themselves import only the Python standard library. Why: __init__.py -- there are lot's better names. Also, when one uses IDE and tries to find (by incremental file search) __init* there are many of them. Name it domain.py or model.py And finally: any Python ways to split _my_ big file? __init__.py is a Python language feature, and there's nothing the Pylons developers can do about it. (Except switch to Ruby :). 3) flash messages : data persisted somewhere between http session and http request, which lives through one session of http redirects, but then is purged. I often need that, as I place some common controller logic in some controllers. Say: home.py:home which redirects to home.mako, and loads complete user profile. I often use h.redirect_to(cont=home,action=home) from inside other controllers. I think such feature should be built in, similar to c, and session variables. As MikeB said, this is in webhelpers.pylonslib.Flash in WebHelpers 0.6. It was decided not to put it into core Pylons because it's a non-essential feature, and Pylons wants to be small. It's in 'pylonslib' rather than in a regular helper module because it depends on pylons.session. 4) CRUD: generate view + controller methods Having controller file with CRUD methods already defined (using SQLA) and redirecting to template files would be nice to have. I don't need that at this stage of project, where I use custom views all over the place. But for beginners it would be nice to have CRUD*.mako(s) created (as option?) and to have working application almost instantly. It's often said that Django does that, and Pylons not :( We've been hoping TurboGears would build this for us. They had an old admin interface (Catwalk) and said they were going to build a new one. When Pylons and TurboGears teamed up, it was agreed that Pylons would concentrate on the core functionality (the essentials), and TG would build the extra stuff like ToscaWidgets, Javascript interfaces,
