Re: Turbogears now based on Pylons!

[Noah Gift]

On 6/29/07, Mike Orr [EMAIL PROTECTED] wrote:


 On 6/28/07, Noah Gift [EMAIL PROTECTED] wrote:
  I am pretty sure Turbogears riding on Pylons will be on a vengeance to
  document things correctly and not just the API, but usage scenarios for
  everything.

 Joining forces will make a big difference.  TG and Pylons combined
 probably account for half of the non-Zope Python web applications
 written in the past two years, give or take ten percent, and roughly
 similar for the amount of developer activity.  Having twice as many
 people working on the same codebase and docs rather than two separate
 ones should accellerate developement.  There may still be two wiki
 demos but they'll be able to borrow half their code from each other
 rather than just the look  feel.  Our skills are also complementary.
 TG developers do a better job of documentation and newbie handholding,
 while Pylons has attracted developers who are obsessed with modularity
 and reuse to a greater degree.  (TG developers may doubt this, but
 many people came to Pylons because they found TG more monolithic than
 they liked.)  Nine months till the next PyCon: it'll be exciting to
 see where we're at then, and what sprints we'll be having.


Yes, this is really exciting stuff.  I vote we capitalize on the energy will
it is hot and start making subprojects and delegating things.  One excellent
strategy is to use community user groups like the one I help
organizehttp://www.pyatl.org.  Give us some group projects and then do
that same thing for groups throughout the world.  Stuff will get done very
quickly!

Lets face it.  Django has killer documentation!  We could do worse than
emulate that!








--
 Mike Orr [EMAIL PROTECTED]

 



-- 
http://www.blog.noahgift.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Noah Gift]

On 6/29/07, Mike Orr [EMAIL PROTECTED] wrote:


 On 6/29/07, Noah Gift [EMAIL PROTECTED] wrote:
  Yes, this is really exciting stuff.  I vote we capitalize on the energy
 will
  it is hot and start making subprojects and delegating things.  One
 excellent
  strategy is to use community user groups like the one I help
  organizehttp://www.pyatl.org.  Give us some group
  projects and then do that same thing for groups throughout the world.

 The Seattle Python group would certainly be interested, any excuse for
 pair programming.

 I started a wiki page for this.
 http://docs.pythonweb.org/display/pylonscommunity
 /Community+Development+Wishlist


Awesome Idea!  I will announce this to our group.  I agree pair programming
rocks!



--
 Mike Orr [EMAIL PROTECTED]

 



-- 
http://www.blog.noahgift.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mike Orr]

On 6/27/07, Michael Bayer [EMAIL PROTECTED] wrote:
  Of course the engine must be initialized before any tables can be
  defined or any work done.

 since ive seen people get the wrong idea about this, let me
 reiterate:  *you can define all your tables and mappers with no
 database connection defined whatsoever*,

Not if you're using autoloading or want to be able to switch
autoloading on or off for any table without having to modify the rest
of the application.  To me that's important, especially when you have
to start changing the schema of an existing table (adding a field,
changing its type, etc), or if the schema is modified externally to
the application.

  SAContext initializes the default engine in
  the constructor.  Is there a need to push that back to some later
  stage?  Which stage?  Or can Ian's ideal wrapper simply delay creating
  the 'sac' until it has to?

 easy enough to make uri optional in SAContext (the way it is for
 everything else in SA) and just allow binding it later via
 add_engine() or whatever.   SAContext is designed the way it is right
 now because Pylons defines the default engine in the .ini file; theres
 no need for a delayed connection in the usual case.

I would probably want this optional URI in a subclass rather than in
SAContext itself.  The reason 'uri' is a required argument is to
guarantee that the default engine is initialized at all times.  When
we designed SAContext we (Mike  I) agreed that the bound metadata
strategy was the most straightforward and adequate for most user apps:
the 80/20 rule.  Hiding the engine (connectable) as much as possible
in the metadata, while still making it accessible when you really need
it.  Now we're adding one unbound metadata strategy after another.
That's fine as log as it doesn't detract from SAContex's primary
commitment to its primary userbase.  I'd say a mandatory URI and bound
metadata is part of its primary commitment for an easy-to-use front
end: a normal Python TypeError for You forgot the 'uri' argument is
about as straightforward as one can get.  Or is that changing too?
Are you having second thoughts about using bound metadata by default?

-- 
Mike Orr [EMAIL PROTECTED]

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mike Orr]

On 6/27/07, Uwe C. Schroeder [EMAIL PROTECTED] wrote:
  if you
  really want COMMIT for every SELECT, i'd favor it being enabled via an
  option passed to create_engine().

 Not every select, every transaction that didn't roll back.
 I just think the default of rollback on every transaction is wrong - a
 rollback should occur when there is a problem, not when the transaction was
 fine. But that may just be me.

But the framework doesn't know whether there's a problem, or whether
some uncommitted changes exist (maybe you called a function that
changed the database but didn't commit).  It certainly doesn't know
whether it should commit or roll back these changes if they do
exist.   But it DOES know that stray uncommitted changes should not
leak into the next web request or all hell will break loose.  So it
has to commit or roll back.  Rolling back is the prudent thing because
if the function really wanted to save those changes permanently, well,
it should've committed them.

  Beyond that I think the model
  implemented as stored procedure style of development is much in the
  minority these days, particularly within the lightweight/open-source
  development community.   just that the stored procedure changes the
  semantics of SELECT to be a write operation reminds me of the
  RESTful sin of using GET to post data.

 Probably because a lot of people can't figure out how to use stored procedures
 and triggers, since the lightweight/open-source programming is often done
 on a database that has very limited support for both :-)

 Personally I'm not a big fan of handling database integrity outside the
 database. Take a simple portal as example, where every new user who signs up
 gets a couple of mailboxes by default (inbox, outbox,drafts,trash). I could
 handle that in the db structure itself, but that would make the db model
 overly complicated for such a simple thing. The proper way is to handle that
 in a trigger, or I can handle it in a stored procedure that just makes sure
 the mailboxes are created. In my case I chose a stored procedure because it's
 actually a select on the user's profile, but the stored procedure also does
 certain sanity checks and inserts/updates stuff according to its findings.

 Handling the same in the application code is IMHO the least desirable
 solution, simply because I can test the trigger/stored procedure once and I
 know everything works until I change the database structure. Application code
 is meddled around with much more, so the chance of introducing a bug is
 higher on that end.
 You could argue that using SA with stored procedures defeats the portability
 approach. I agree that you can't easily move a Oracle or PostgreSQL or DB2
 based application with stored procedures to a different database, but how
 many people do that? If you build a generic application the argument is
 valid, but if you build something for a specific purpose, the database will
 never be changed for the life of the application (because generally special
 purpose applications are funded by someone and those people usually don't
 invest another couple grand just to make it work with some other database).

I'd argue the opposite, that people don't use stored procedures and
triggers because that ties them to a certain database, and that's the
opposite of why they chose SQLAlchemy in the first place.  Not only
that, you have to learn a second language that's limited to that
database, with its own quirks and limitations, and the language is
much less convenient/featureful than Python (it looks like a fossil
from 20 years ago), which is why we're using Python in the first
place.  I would be more likely to change my database than change my
application, or at least just as likely.  It's not that big a deal to
copy everything into a new database and adjust the program for it --
provided you don't have huge databases or stored procedures.  The
huge database part is outside your control, but the stored procedure
part is.  Tying myself to one database product gives me bad memories
of the vendor lock-in problems that used to be more prevelant in the
past, but are thankfully becoming rarer now that interoperability has
gotten better.

I actually have a practical example of this.  I may have to switch one
application from MySQL to PostgreSQL in order to access its PostGIS
geographical functions, which I hear are more advanced than MySQL's.
So if we decide we need this functionality we'll have to switch.
Without stored procedures I can just change my DBURI string and voila,
just write the SQL expressions.  With stored procedures I'd have to
port all those to the new database.

Having said all this, I have had to break down and start using views
because it made performance acceptable in one case (a mapped object on
a SELECT was just too slow).  Someday I may have to write stored
procedures for that reason.  But so far I haven't needed to.

-- 
Mike Orr [EMAIL PROTECTED]

Re: Turbogears now based on Pylons!

[Uwe C. Schroeder]

On Wednesday 27 June 2007, Mike Orr wrote:
 On 6/27/07, Uwe C. Schroeder [EMAIL PROTECTED] wrote:
   if you
   really want COMMIT for every SELECT, i'd favor it being enabled via an
   option passed to create_engine().
 
  Not every select, every transaction that didn't roll back.
  I just think the default of rollback on every transaction is wrong - a
  rollback should occur when there is a problem, not when the transaction
  was fine. But that may just be me.

 But the framework doesn't know whether there's a problem, or whether
 some uncommitted changes exist (maybe you called a function that
 changed the database but didn't commit).  It certainly doesn't know
 whether it should commit or roll back these changes if they do
 exist.   But it DOES know that stray uncommitted changes should not
 leak into the next web request or all hell will break loose.  So it
 has to commit or roll back.  Rolling back is the prudent thing because
 if the function really wanted to save those changes permanently, well,
 it should've committed them.

If it could, yes. Right now it looks to me like SA automatically rolls every 
select back without a way to tell it not to (and no, explicit transaction 
didn't do the trick, but then I'm not an expert on SA, so there may be a 
better way)

   Beyond that I think the model
   implemented as stored procedure style of development is much in the
   minority these days, particularly within the lightweight/open-source
   development community.   just that the stored procedure changes the
   semantics of SELECT to be a write operation reminds me of the
   RESTful sin of using GET to post data.
 
  Probably because a lot of people can't figure out how to use stored
  procedures and triggers, since the lightweight/open-source programming
  is often done on a database that has very limited support for both :-)
 
  Personally I'm not a big fan of handling database integrity outside the
  database. Take a simple portal as example, where every new user who signs
  up gets a couple of mailboxes by default (inbox, outbox,drafts,trash). I
  could handle that in the db structure itself, but that would make the db
  model overly complicated for such a simple thing. The proper way is to
  handle that in a trigger, or I can handle it in a stored procedure that
  just makes sure the mailboxes are created. In my case I chose a stored
  procedure because it's actually a select on the user's profile, but the
  stored procedure also does certain sanity checks and inserts/updates
  stuff according to its findings.
 
  Handling the same in the application code is IMHO the least desirable
  solution, simply because I can test the trigger/stored procedure once and
  I know everything works until I change the database structure.
  Application code is meddled around with much more, so the chance of
  introducing a bug is higher on that end.
  You could argue that using SA with stored procedures defeats the
  portability approach. I agree that you can't easily move a Oracle or
  PostgreSQL or DB2 based application with stored procedures to a different
  database, but how many people do that? If you build a generic application
  the argument is valid, but if you build something for a specific purpose,
  the database will never be changed for the life of the application
  (because generally special purpose applications are funded by someone
  and those people usually don't invest another couple grand just to make
  it work with some other database).

 I'd argue the opposite, that people don't use stored procedures and
 triggers because that ties them to a certain database, and that's the
 opposite of why they chose SQLAlchemy in the first place.  Not only
 that, you have to learn a second language that's limited to that
 database, with its own quirks and limitations, and the language is
 much less convenient/featureful than Python (it looks like a fossil
 from 20 years ago), which is why we're using Python in the first
 place.  I would be more likely to change my database than change my
 application, or at least just as likely.  It's not that big a deal to
 copy everything into a new database and adjust the program for it --
 provided you don't have huge databases or stored procedures.  The
 huge database part is outside your control, but the stored procedure
 part is.  Tying myself to one database product gives me bad memories
 of the vendor lock-in problems that used to be more prevelant in the
 past, but are thankfully becoming rarer now that interoperability has
 gotten better.

Interoperability has gotten better? A bit probably, but not all that much :-(
On the other hand, if you're as likely to change your DB model as you are to 
modify your application, your app and db model are either very simple, or the 
db model wasn't designed all that well. My current application has 513 tables 
and been in production for about 5 years. The application running on top of 
it was changed numerous times, the DB 

Re: Turbogears now based on Pylons!

[Wichert Akkerman]

Previously Michael Bayer wrote:
 On Jun 27, 6:42 pm, Ian Bicking [EMAIL PROTECTED] wrote:
  The way I see this working is something like (vaguely):
 
  def transaction_middleware(app):
   def wrapper(environ, start_response):
   manager = TransactionManager()
   environ['transaction.manager'] = manager
   try:
   app_iter = app(environ, start_response)
   except:
   manager.rollback()
   raise
   else:
   manager.commit()
 
  The manager is basically a container of *actual* transactions, and
  calling rollback or commit on it gets passed on to all the transactions
  in the manager.
 
 this is fine.  we're *really* starting to imitate J2EE in some ways.
 but its not a bad thing.

For what it's worth: Zope has a similar structure. The Zope publisher
starts a transaction when it receives a request and commits it when the
request processing has finished, unless an exception has been thrown or
someone did an explicit rollback. Rollback tends to be very rare so
it seems that it is not very useful to optimize for that case. The Zope
machinery allows you to add your own commit/rollback hooks, which is
used to tie SA into Zope.

Wichert.

-- 
Wichert Akkerman [EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/   It is hard to make things simple.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Uwe C. Schroeder]

On Thursday 28 June 2007, Mike Orr wrote:

 And this.  If you have a 50-column table, by gosh SQLAlchemy is going
 to select all 50 fields in full `TableName`.columnName AS columnName
 notation, which makes the query hard to read in the log.

M - a 50 column table? My old CS teacher will wiggle in his grave like a 
nightcrawler. What happened to normalizing tables? :-)
But then - I've done pretty awful things myself :-)

Uwe

--
Open Source Solutions 4U, LLC   1618 Kelly St
Phone:  +1 707 568 3056 Santa Rosa, CA 95401
Cell:   +1 650 302 2405 United States
Fax:+1 707 568 6416

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Jonathan LaCour]

Michael Bayer wrote:

 im not exactly sure why Jonathan thinks that the url forces one
 to define my connection up front. if youre saying, i want to
 distribute the application without an .ini file thats fine...but when
 the thing is actually *used*, there will be an .ini file or other
 configuration defined when things run. if you want to define your
 model independently of connection, just use an empty MetaData.  Set
 up the SAContext to bind the engine or individual connections to the
 session instead...or give it your MetaData for it to bind when youre
 ready. theres really nothing to it.

I just didn't like having my model get its URI directly from the Pylons
config, for the reasons I stated previously.  However, you just solved
my issue :) I had no idea that I was able to do this with SAContext, and
effectively use it just as I want to.

Thanks for the explanation Mike!

--
Jonathan LaCour
http://cleverdevil.org



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Jonathan LaCour]

Michael Bayer wrote:

 just to clarify, however this works, it should be *really easy* for
 individual controller methods to be marked as transactional or not.
 I have written decorators like these already, in the style of:

 class MyController(...):
  def index_page(self):
  .

  @transactional
  def post_message(self):
  .

  def display_message(self):
  .

 so id want that kind of thing to be readily available, i suppose it
 would communicate with the WSGI middleware on a per-request basis.

The only issue I can see with this is that having the transaction begin
in the controller method itself might be too late if you have other
things going on in middleware or elsewhere and something goes wrong
before you even get to the controller method -- say in object dispatch,
the new TG controller's lookup method, or even some middleware that
stores sessions in a database for example.

Transactions should start at the earliest possible point, IMO, so that
you can handle all of these cases.

 but another thing i like, is that of the response to the request being
 delivered *after* the transaction commits. i have raised this issue
 on the pylons list before but i dont think i impressed anyone. if you
 render your response template, then your transaction fails, you have
 the potential for that response template to still be delivered with
 the success! message. anyway, might be nice for the middleware to
 address this if possible.

I am totally with you here, as this is the other side of the coin that
I brought up above.  Open transactions shouldn't be committed until the
latest possible point.

I spent some dark days working on J2EE applications, and while there
isn't a whole lot that is worth emulating in J2EE, the transaction
handling is something that I actually liked.

 also i think this is all outside of the scope of SAContext.  SAContext
 should remain as a facade to SA-specific elements; it can accept
 strategy objects which control its internal workings, so if need be
 particular strategies can be delivered in order to function with the
 transaction manager, without changing the external view of SAContext.

Yes, I think agree here too, as long as there is a way to hook SAContext
into whatever ends up getting decided in this (really long) thread.  I
played a bit more with SAContext after these discussions, and it is
indeed very nice.  Good work Mike and Mike.


--
Jonathan LaCour
http://cleverdevil.org


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 28, 1:40 am, Uwe C. Schroeder [EMAIL PROTECTED] wrote:
 On Wednesday 27 June 2007, Michael Bayer wrote:

  this issue can be worked around by using explicit transactions.

 actually no, it can't. Maybe I don't get it right, but the only way for me to
 get a commit was actually to modify Connection._autocommit in
 sqlalchemy.engine.base.

either TG is getting in the way, or youre not getting it right.  if
anyone ever needs to modify the internals of SA to get something to
work, i would *highly* prefer if they could email the ML or post a
trac ticket with their issue so that it may be handled properly.


 Obviously SA thinks there is no transaction in TG,
 so it just wraps one around it.

if TG actually has a transaction going on, theyd certainly have to
configure SA to be aware of it (most likely via SessionTransaction).
if not, then yes things arent going to work at all (though still, an
explicit SA transaction should work all by itself).

 I agree, and that is certainly DB dependent. Personally I can't imagine that
 an automatically issued rollback for every select transaction is in any way
 more overhead than issuing a commit. Not wrapping a select in a transaction
 will definitely be the least overhead.

we dont issue a rollback for every select transaction.  we issue a
rollback when a connection is returned to the pool.  you can check out
a connection explicitly and perform any number of selects on it
without any rollbacks or commits.

because the rollback is at the connection-pool checkin level, it
should be more apparent how inappropriate it would be to issue a
*commit* every time a connection is returned to the pool, an operation
that knows nothing about what just happened with that connection.  the
rollback is to release database locks.

im thinking that it might be time to allow an option in SA that just
turns the DBAPI's autocommit flag on.  that way you can just blame
the DBAPI for whatever issues arise.

its not always possible to not wrap a select in a transaction.
oracle for example *always* has a transaction going on, so everything
is in a transaction in all cases.

  that a stored-procedure-oriented application is far more efficient
  is *extremely* debatable and database-dependent as well.

 I doubt it's *extremely* debatable.

its extremely debatable:

http://www.google.com/search?q=stored+procedures+vs

 Just issue 100 inserts from inside a
 stored procedure (or 100 updates) and do the same discretely with any kind of
 db interface. In case of the interface every statement has to be parsed by
 the db, whereas in a stored procedure the statement is already compiled of
 sorts (at least Oracle and PostgreSQL do that).

the debate over SP's is about a larger issue than is an SP faster
than 5 separate INSERT statements.  SP's are of course much better
for micro-benchmarks like that.  its their impact on application
development and architecture where the debate comes in (read some of
the googled articles).

I am certainly not anti-SP, ive done pure SP applications before (on
projects where the DBAs controlled the SPs)...I just dont want to
start hardwiring SQLAlchemy to expect that sort of application.   I
think 80/20 as applied to SELECT is that 80% of SELECTs are for read
operations and a COMMIT is inappropriate.

  if you
  really want COMMIT for every SELECT, i'd favor it being enabled via an
  option passed to create_engine().

 Not every select, every transaction that didn't roll back.
 I just think the default of rollback on every transaction is wrong - a
 rollback should occur when there is a problem, not when the transaction was
 fine. But that may just be me.

this is the use case:

c1 = pool.connect()
row = c1.cursor().execute(select * from sometable).fetchone()
pool.return_connection(c1)

c2 = pool.connect()  # (returns a connection that is not c1)
c2.cursor().execute(drop sometable)  # -- deadlock

if DBAPI supported a release_locks() method, we'd be using that.

 Probably because a lot of people can't figure out how to use stored procedures
 and triggers, since the lightweight/open-source programming is often done
 on a database that has very limited support for both :-)

keep in mind youre including the vast Hibernate community, including
its creators, etc.  im not sure if the im to dum to use stored
procedures argument can fully explain why the SP-architecture remains
a minority use case.  i think the overall inconvenience of it, the
clunky old languages you have to use on DBs like Oracle and SQL
Server, as well as the harsh resistance it puts up to so-called agile
development methods are better reasons.

 Personally I'm not a big fan of handling database integrity outside the
 database.
  continue SP arguments.

thats great, you can have your preferences..the google link above
should reveal that quite a few people have established their
preferences in this matter.  If you are truly writing an SP-only
application which prevents direct SQL access (that was the 

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 28, 2:09 am, Mike Orr [EMAIL PROTECTED] wrote:

 I would probably want this optional URI in a subclass rather than in
 SAContext itself.  The reason 'uri' is a required argument is to
 guarantee that the default engine is initialized at all times.  When
 we designed SAContext we (Mike  I) agreed that the bound metadata
 strategy was the most straightforward and adequate for most user apps:
 the 80/20 rule.  Hiding the engine (connectable) as much as possible
 in the metadata, while still making it accessible when you really need
 it.  Now we're adding one unbound metadata strategy after another.
 That's fine as log as it doesn't detract from SAContex's primary
 commitment to its primary userbase.  I'd say a mandatory URI and bound
 metadata is part of its primary commitment for an easy-to-use front
 end: a normal Python TypeError for You forgot the 'uri' argument is
 about as straightforward as one can get.  Or is that changing too?
 Are you having second thoughts about using bound metadata by default?


OK just FTR heres why SQLAlchemy is not a framework and why in order
to do things without the advent of a framework you work with all these
little highly granular components, and also why im still antsy
including SAContext in SA.  just trying to establish the simplest
heres your connection object spawns all this debate over how it
should look/work/act/etc.  I would encourage those here to look at
SAContext, see that its a dead-simple piece of code, and just work up
various preferred versions of it to taste.  My goal with it was more
to illustrate *how* you can build these things.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 27, 5:56 pm, Ian Bicking [EMAIL PROTECTED] wrote:
 * Way for a library to get the transaction manager.
 * Interface for that transaction manager, maybe copied from Zope.
 * Single convention for how to specify a database connection (ideally
 string-based).
 * Probably a way to get the configuration, so you can lazily get a
 connection based on the configured connection.

just as a point of reference, here is zalchemy's integration of Zope
datamanager:

http://svn.zope.org/z3c.zalchemy/trunk/src/z3c/zalchemy/datamanager.py?rev=77165view=auto

I would favor that the interface supports two-phase semantics like
zope's.  sqlalchemy's flush() model already fits in pretty well with
the two-phase model, and will eventually be building explicit two-
phase hooks into SQLAlchemy's engine and session, with real
implementations for postgres to start with.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 28, 2007, at 9:58 AM, Jonathan LaCour wrote:


 Michael Bayer wrote:

 just to clarify, however this works, it should be *really easy* for
 individual controller methods to be marked as transactional or not.
 I have written decorators like these already, in the style of:

 class MyController(...):
  def index_page(self):
  .

  @transactional
  def post_message(self):
  .

  def display_message(self):
  .

 so id want that kind of thing to be readily available, i suppose it
 would communicate with the WSGI middleware on a per-request basis.

 The only issue I can see with this is that having the transaction  
 begin
 in the controller method itself might be too late if you have other
 things going on in middleware or elsewhere and something goes wrong
 before you even get to the controller method -- say in object  
 dispatch,
 the new TG controller's lookup method, or even some middleware that
 stores sessions in a database for example.

the way this idea works is that if some wrapping component knows it  
needs a transaction, it opens one.  then the controller method  
participates in that transaction, regardless of whether or not its  
marked as @transactional.

 but another thing i like, is that of the response to the request  
 being
 delivered *after* the transaction commits. i have raised this issue
 on the pylons list before but i dont think i impressed anyone. if you
 render your response template, then your transaction fails, you have
 the potential for that response template to still be delivered with
 the success! message. anyway, might be nice for the middleware to
 address this if possible.

 I am totally with you here, as this is the other side of the coin that
 I brought up above.  Open transactions shouldn't be committed until  
 the
 latest possible point.

yes.  im talking specifcally about the way Pylons passes off from  
controller to view...right now you call render_response(), which  
typically renders the entire template in a buffer before the  
controller method has finished.  id favor changing the behavior of  
render_response() to creating a deferred render object which Pylons  
explcitly places outside the scope of the controller call (however,  
probably still within the scope of enclosing middleware).


 I spent some dark days working on J2EE applications, and while there
 isn't a whole lot that is worth emulating in J2EE, the transaction
 handling is something that I actually liked.

thats where im getting this from.  theres subtleties to it that you  
cant emulate casually.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Jonathan LaCour]

Michael Bayer wrote:

 the way this idea works is that if some wrapping component knows
 it needs a transaction, it opens one. then the controller method
 participates in that transaction, regardless of whether or not its
 marked as @transactional.

Of course, that makes complete sense.  As long at the wrapping component
knows about the transaction manager (which, clearly, I think it should).

 I am totally with you here, as this is the other side of the coin
 that I brought up above.  Open transactions shouldn't be committed
 until the latest possible point.

 yes. im talking specifcally about the way Pylons passes off from
 controller to view...right now you call render_response(), which
 typically renders the entire template in a buffer before the
 controller method has finished. id favor changing the behavior of
 render_response() to creating a deferred render object which Pylons
 explcitly places outside the scope of the controller call (however,
 probably still within the scope of enclosing middleware).

Makes sense to me.

 I spent some dark days working on J2EE applications, and while there
 isn't a whole lot that is worth emulating in J2EE, the transaction
 handling is something that I actually liked.

 thats where im getting this from. theres subtleties to it that you
 cant emulate casually.

Yes, the further down the rabbit hole I go, the more clear this is
becoming.  That being said, I think I am sold on the concept that we
have hashed out here.

I need to take some time to look at Zope's transaction manager at some
point, and see how hard it would be to extract it from Zope, and get rid
of any dependencies, so it can easily be packaged up as its own, simple
egg.  Has anyone done this before?  Is anyone interested in taking this
up as a project?

--
Jonathan LaCour
http://cleverdevil.org


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Ian Bicking]

Michael Bayer wrote:
 On Jun 27, 6:42 pm, Ian Bicking [EMAIL PROTECTED] wrote:
 The way I see this working is something like (vaguely):

 def transaction_middleware(app):
  def wrapper(environ, start_response):
  manager = TransactionManager()
  environ['transaction.manager'] = manager
  try:
  app_iter = app(environ, start_response)
  except:
  manager.rollback()
  raise
  else:
  manager.commit()

 The manager is basically a container of *actual* transactions, and
 calling rollback or commit on it gets passed on to all the transactions
 in the manager.
 
 this is fine.  we're *really* starting to imitate J2EE in some ways.
 but its not a bad thing.

At least someone else thought it was a good idea too ;).  It's also all 
very similar to Zope.

 If you don't do anything that needs a transaction (e.g., read-only), you
 shouldn't put your transaction in the manager.
 
 just to clarify, however this works, it should be *really easy* for
 individual controller methods to be marked as transactional or not.
 I have written decorators like these already, in the style of:
 
 class MyController(...):
  def index_page(self):
  .
 
  @transactional
  def post_message(self):
  .
 
  def display_message(self):
  .
 
 so id want that kind of thing to be readily available, i suppose it
 would communicate with the WSGI middleware on a per-request basis.

I think that would be easy enough.  All that is setup early is the 
transaction container; nothing is automatically added to it.  So 
@transactional could just add a transaction to the container (presumably 
everything would otherwise be autocommit, or perhaps automatically 
rolled back?)

Presumably what you really want to do is 
add-this-transaction-if-it-isn't-there-already all over the place.  For 
instance, if you've already started a transaction in middleware, I don't 
think you'll need to start another transaction.  Unless you have 
subtransactions, which I suppose is possible.

 but another thing i like, is that of the response to the request being
 delivered *after* the transaction commits.  i have raised this issue
 on the pylons list before but i dont think i impressed anyone.  if you
 render your response template, then your transaction fails, you have
 the potential for that response template to still be delivered with
 the success! message.  anyway, might be nice for the middleware to
 address this if possible.

Hmm... I believe paste.exceptions should do this already, even if an 
exception is raised after start_response is called, as long as it is 
raised before the app_iter is returned.  If it's after the app_iter, 
then you've somehow escaped the stack, which doesn't much concern me ;)

So as middleware I think it should be fine.  If you want to catch a 
problem with the commit and handle it gracefully, I think you can just 
do your own commit in your code (and the middleware commit will just 
commit what little happens after that).

And of course, what you put in as a transaction is up to you.  You could 
put in some object which uses whatever flags you want to actually pass 
through a commit or else just ignore it.

 also i think this is all outside of the scope of SAContext.  SAContext
 should remain as a facade to SA-specific elements; it can accept
 strategy objects which control its internal workings, so if need be
 particular strategies can be delivered in order to function with the
 transaction manager, without changing the external view of SAContext.

I dunno... I think if we can get this in place, then a whole lot of 
things are going to become simpler.  But there also still has to be 
wrappers like SAContext from this fairly low-level stuff to a particular 
framework.


-- 
Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org
 | Write code, do good | http://topp.openplans.org/careers

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 28, 11:46 am, Ian Bicking [EMAIL PROTECTED] wrote:

 So as middleware I think it should be fine.  If you want to catch a
 problem with the commit and handle it gracefully, I think you can just
 do your own commit in your code (and the middleware commit will just
 commit what little happens after that).

 And of course, what you put in as a transaction is up to you.  You could
 put in some object which uses whatever flags you want to actually pass
 through a commit or else just ignore it.

usually, applications want a single big transaction for the whole
request.  if an app wants something that will commit independently of
the enclosing transaction, that can be an option to the decorator:

@transactional(independent=True)
def do_something(self):

the term independent is made up and maybe not so clear...but again,
this is straight out of EJB.  they let you annotate session beans in
an XML file using these terms: required, requiresNew, Supports,
Mandatory, NotSupported, Never.  I think we should consider each
of these cases and determine if we think we need to support that case,
and if so how. (also does Zope express this many variants?)


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Alberto Valverde]

On Jun 28, 2007, at 12:42 AM, Ian Bicking wrote:


 Jonathan LaCour wrote:
 Ian Bicking wrote:

 Personally I'd like to see something a bit different:

 * Way for a library to get the transaction manager.
 * Interface for that transaction manager, maybe copied from Zope.
 * Single convention for how to specify a database connection  
 (ideally
   string-based).
 * Probably a way to get the configuration, so you can lazily get
   a connection based on the configured connection.

 The frameworks would setup the transaction manager and  
 configuration,
 and database frameworks would basically consume this information.

 This would be pretty much ideal, and would satisfy my use cases very
 well.  It would also allow framework authors to build in support  
 for any
 ORM / database layer in an agnostic way, sort of like the Buffet API
 does for templating systems.

 Then, we could also create a simple WSGI middleware for Pylons that
 gives it TurboGears-like automatic transaction start/commit/rollback
 on a per request basis.  Only, we could make it configurable so that
 it didn't happen on read-only operations.  All of this would be ORM /
 database layer agnostic, which would be very nice indeed.

 The way I see this working is something like (vaguely):

 def transaction_middleware(app):
  def wrapper(environ, start_response):
  manager = TransactionManager()
  environ['transaction.manager'] = manager
  try:
  app_iter = app(environ, start_response)
  except:
  manager.rollback()
  raise
  else:
  manager.commit()

I see you've mentioned vaguely and that paste.transaction  
implements it properly... but I just had to say this. An almost exact  
implementation of that pseudocode caused me a very embarrassing bug  
in which transactions were committed after the authorization  
middleware trapped security exceptions and turned them into 403s.  
Luckily it was only an internal app and the bug was noticed in- 
house... :)

Bottom line is that the transaction middleware should rollback any  
transaction if the status code is = 400 to handle case like this  
(and that using existing, well tested code is usually better than  
rolling your own no matter how fun it is :)

Alberto

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Jonathan LaCour]

Alberto Valverde wrote:

 I'd like to add here that I've got use-cases for the transaction
 boundary and session initialization being closer to the server than
 what the __before__ and __after__ hooks provide.

 For example, the authentication function I feed to AuthKit which uses
 a DB connection to retrieve the user and it's associated roles and
 permissions.

 I can think of other cases where other layers of middleware (more
 precisely, Middleware FrameWork Components) might want to access the
 database. Having all these layers plus the controller method (or other
 WSGI which sit below PylonsWSGIApp) wrapped in the same transaction
 sounds like a good idea to me.

Fair enough to all of the above, however all the feedback from Mike and
Ian has made it clear to me that we need a few different things:

1. A standard transaction manager that can be plugged into any
   application, WSGI or not.  It should be similar in scope and API
   to Zope's transaction manager, but free of Zope dependancies.

2. A piece of middleware that uses the transaction manager to  
rollback
   any active transactions in the case of an exception, or commit  
them
   in the other case.

3. Either some middleware or just a few lines inside the  
BaseController
   of the TurboGears template that starts a transaction on every
   request and puts it into the transaction manager.

The first and second item are generally useful, and will allow you to
manage transactions manually if you so desire.  Any general-purpose
middleware out there that needs to use a transaction can simply use the
transaction manager, and all will be well.  TurboGears will have the
same behavior as it has had for a while, and since it happens in the
__call__ of the BaseController, you can tweak it to your heart's content
based upon the URI, HTTP method, or whatever you want.

And, in my opinion, items #1 and #2 should be in Pylons by default, and
the third item should be in the TurboGears template by default.

There is, of course, some extra stuff we could add for nested
transactions, and all of the cases he enumerated in his last email
(required, requiresNew, supports, mandatory, etc), but I think the three
things above will give a good set of building blocks for making those
things easy to deal with.

--
Jonathan LaCour
http://cleverdevil.org

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Ian Bicking]

Alberto Valverde wrote:
 
 On Jun 28, 2007, at 12:42 AM, Ian Bicking wrote:
 
 Jonathan LaCour wrote:
 Ian Bicking wrote:

 Personally I'd like to see something a bit different:

 * Way for a library to get the transaction manager.
 * Interface for that transaction manager, maybe copied from Zope.
 * Single convention for how to specify a database connection  
 (ideally
   string-based).
 * Probably a way to get the configuration, so you can lazily get
   a connection based on the configured connection.

 The frameworks would setup the transaction manager and  
 configuration,
 and database frameworks would basically consume this information.
 This would be pretty much ideal, and would satisfy my use cases very
 well.  It would also allow framework authors to build in support  
 for any
 ORM / database layer in an agnostic way, sort of like the Buffet API
 does for templating systems.

 Then, we could also create a simple WSGI middleware for Pylons that
 gives it TurboGears-like automatic transaction start/commit/rollback
 on a per request basis.  Only, we could make it configurable so that
 it didn't happen on read-only operations.  All of this would be ORM /
 database layer agnostic, which would be very nice indeed.
 The way I see this working is something like (vaguely):

 def transaction_middleware(app):
  def wrapper(environ, start_response):
  manager = TransactionManager()
  environ['transaction.manager'] = manager
  try:
  app_iter = app(environ, start_response)
  except:
  manager.rollback()
  raise
  else:
  manager.commit()
 
 I see you've mentioned vaguely and that paste.transaction  
 implements it properly... but I just had to say this. An almost exact  
 implementation of that pseudocode caused me a very embarrassing bug  
 in which transactions were committed after the authorization  
 middleware trapped security exceptions and turned them into 403s.  
 Luckily it was only an internal app and the bug was noticed in- 
 house... :)

paste.transaction itself is something I should have really removed, as 
it's so incomplete.  But yes, it's the basic pattern, but without a 
clear concept of a TransactionManager.

 Bottom line is that the transaction middleware should rollback any  
 transaction if the status code is = 400 to handle case like this  
 (and that using existing, well tested code is usually better than  
 rolling your own no matter how fun it is :)

Er... I'm not sure about this.  I'm not sure =400 is really a failure. 
  Maybe it is; I just don't feel comfortable making that claim generally.

If you have a security exception that is caught, you could also look for 
the manager and roll it back exactly then, instead of expecting the 
middleware to do it.  The middleware is just the furthest-out boundary 
for transactions, it's not the only place where you can control the 
transactions.

-- 
Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org
 | Write code, do good | http://topp.openplans.org/careers

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Alberto Valverde]

On Jun 28, 2007, at 7:33 PM, Jonathan LaCour wrote:


 Alberto Valverde wrote:

 I'd like to add here that I've got use-cases for the transaction
 boundary and session initialization being closer to the server than
 what the __before__ and __after__ hooks provide.

 For example, the authentication function I feed to AuthKit which uses
 a DB connection to retrieve the user and it's associated roles and
 permissions.

 I can think of other cases where other layers of middleware (more
 precisely, Middleware FrameWork Components) might want to access the
 database. Having all these layers plus the controller method (or  
 other
 WSGI which sit below PylonsWSGIApp) wrapped in the same transaction
 sounds like a good idea to me.

 Fair enough to all of the above, however all the feedback from Mike  
 and
 Ian has made it clear to me that we need a few different things:

 1. A standard transaction manager that can be plugged into any
application, WSGI or not.  It should be similar in scope and  
 API
to Zope's transaction manager, but free of Zope dependancies.

 2. A piece of middleware that uses the transaction manager to
 rollback
any active transactions in the case of an exception, or commit
 them
in the other case.

(I'm assuming this middleware above the PylonsWSGIApp...)


 3. Either some middleware or just a few lines inside the
 BaseController
of the TurboGears template that starts a transaction on every
request and puts it into the transaction manager.

Hmmm, wouldn't having two transaction boundaries at 2 and 3 be  
redundant? Or maybe both boundaries will use the same manager in a  
similar way paste.registry has only one Registry for the whole stack?  
(albeit different context boundaries every time a RegistryManager is  
stacked)

One problem I see is that this might cause trouble in cases like this:

Request
|
   V
db transaction middleware  - creates session A
   |
   V
authentication middleware - authenticates user and keeps ref to User  
instance (attached to session A)
   |
   V
other middleware and PylonsApp
   |
   V
TG controller  - creates session B inside __call__
   |
   V
authorization library tries to check permissions for mapped Foo  
instance (attached to session B) which has a owner attribute:

foo.owner (sess B) == logged_in_user (sess A)

I haven't actually tried it so I'm not sure how to objects attached  
to different sessions compare as or interact, so this might be a non- 
issue (maybe Mike can shed some light on this?)

Anyway, this might be a non-issue on the whole if the transaction  
middleware does not handle session creation at all.

OTOH, I do see the difficulty of having just one boundary in a single  
TG/Pylons app in a layer of middleware above PylonsApp since I don't  
see a clear way of making parameters passed via a decorator reach the  
middleware *before* the method is actually called (which will be  
needed to signal that a BEGIN should not be issued)

Alberto

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Alberto Valverde]

On Jun 28, 2007, at 7:45 PM, Ian Bicking wrote:


 Alberto Valverde wrote:

 On Jun 28, 2007, at 12:42 AM, Ian Bicking wrote:

 Jonathan LaCour wrote:
 Ian Bicking wrote:

 Personally I'd like to see something a bit different:

 * Way for a library to get the transaction manager.
 * Interface for that transaction manager, maybe copied from Zope.
 * Single convention for how to specify a database connection
 (ideally
   string-based).
 * Probably a way to get the configuration, so you can lazily get
   a connection based on the configured connection.

 The frameworks would setup the transaction manager and
 configuration,
 and database frameworks would basically consume this information.
 This would be pretty much ideal, and would satisfy my use cases  
 very
 well.  It would also allow framework authors to build in support
 for any
 ORM / database layer in an agnostic way, sort of like the Buffet  
 API
 does for templating systems.

 Then, we could also create a simple WSGI middleware for Pylons that
 gives it TurboGears-like automatic transaction start/commit/ 
 rollback
 on a per request basis.  Only, we could make it configurable so  
 that
 it didn't happen on read-only operations.  All of this would be  
 ORM /
 database layer agnostic, which would be very nice indeed.
 The way I see this working is something like (vaguely):

 def transaction_middleware(app):
  def wrapper(environ, start_response):
  manager = TransactionManager()
  environ['transaction.manager'] = manager
  try:
  app_iter = app(environ, start_response)
  except:
  manager.rollback()
  raise
  else:
  manager.commit()

 I see you've mentioned vaguely and that paste.transaction
 implements it properly... but I just had to say this. An almost exact
 implementation of that pseudocode caused me a very embarrassing bug
 in which transactions were committed after the authorization
 middleware trapped security exceptions and turned them into 403s.
 Luckily it was only an internal app and the bug was noticed in-
 house... :)

 paste.transaction itself is something I should have really removed, as
 it's so incomplete.  But yes, it's the basic pattern, but without a
 clear concept of a TransactionManager.

 Bottom line is that the transaction middleware should rollback any
 transaction if the status code is = 400 to handle case like this
 (and that using existing, well tested code is usually better than
 rolling your own no matter how fun it is :)

 Er... I'm not sure about this.  I'm not sure =400 is really a  
 failure.
   Maybe it is; I just don't feel comfortable making that claim  
 generally.

I can understand this... at least we're sure we shouldn't implicitly  
*commit* the transaction

 If you have a security exception that is caught, you could also  
 look for
 the manager and roll it back exactly then, instead of expecting the
 middleware to do it.  The middleware is just the furthest-out boundary
 for transactions, it's not the only place where you can control the
 transactions.

However, this would couple the security-exception-trapping middleware  
with the transaction manager. Right now, the mentioned middleware's  
responsability is very simple: Trap security exceptions and turn them  
into 403s or 401s (depending if there's a user logged in). it doesn't  
even handle authentication (which is handled above to provide a  
REMOTE_USER and/or wsgi_org.user_info and trapping 401s). I can also  
think of other layers which could change status codes, expecting all  
this layers to be aware that a transaction manager is in the stack  
and explicitly rollback any transaction doesn't favor loose-coupling  
from my POV.

I believe that in this scenario the most straightforward way to  
signal that an error (or non OK) condition has ocurred is to through  
the HTTP status code hence the db transaction middleware should take  
it into account. The question of what status codes should be treated  
as errors is still open though...

Alberto

 -- 
 Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org
  | Write code, do good | http://topp.openplans.org/careers

 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Uwe C. Schroeder]

On Thursday 28 June 2007, Michael Bayer wrote:
 On Jun 28, 1:40 am, Uwe C. Schroeder [EMAIL PROTECTED] wrote:
  On Wednesday 27 June 2007, Michael Bayer wrote:
   this issue can be worked around by using explicit transactions.
 
  actually no, it can't. Maybe I don't get it right, but the only way for
  me to get a commit was actually to modify Connection._autocommit in
  sqlalchemy.engine.base.

 either TG is getting in the way, or youre not getting it right.  if
 anyone ever needs to modify the internals of SA to get something to
 work, i would *highly* prefer if they could email the ML or post a
 trac ticket with their issue so that it may be handled properly.

I agree. I didn't put this on the ML or trac because I just don't know why it 
behaves like that and frankly I just gave up on it after a couple of hours 
and took the shortcut. 
Obviously TG wraps every request into a transaction and commits/rollback's  
that when the controller method returns the response.
There is a valid transaction object in the tg.sa_transaction variable which 
should allow to end the outer transaction and start a new one on your own.
Although I can do that, the result was always the same - SA doesn't see the 
transaction and runs into the base._autocommit method (which I think it 
shouldn't if the transaction was handled properly before that). Tracking what 
the DB gets, it never sees a rollback/commit when I tell the original 
transaction object in tg.sa_transaction to rollback. The command is just 
ignored.

When it comes to submitting a patch: I do have code changes for SA to handle 
arrays and user defined types for postgresql in the autodetect code. So I 
might roll that up and send it over when I have a minute.


  Obviously SA thinks there is no transaction in TG,
  so it just wraps one around it.

 if TG actually has a transaction going on, theyd certainly have to
 configure SA to be aware of it (most likely via SessionTransaction).
 if not, then yes things arent going to work at all (though still, an
 explicit SA transaction should work all by itself).

The tg.sa_transaction variable contains a SessionTransaction object at that 
time. So they obviously do something to get that.

  I agree, and that is certainly DB dependent. Personally I can't imagine
  that an automatically issued rollback for every select transaction is in
  any way more overhead than issuing a commit. Not wrapping a select in a
  transaction will definitely be the least overhead.

 we dont issue a rollback for every select transaction.  we issue a
 rollback when a connection is returned to the pool.  you can check out
 a connection explicitly and perform any number of selects on it
 without any rollbacks or commits.

 because the rollback is at the connection-pool checkin level, it
 should be more apparent how inappropriate it would be to issue a
 *commit* every time a connection is returned to the pool, an operation
 that knows nothing about what just happened with that connection.  the
 rollback is to release database locks.

 im thinking that it might be time to allow an option in SA that just
 turns the DBAPI's autocommit flag on.  that way you can just blame
 the DBAPI for whatever issues arise.

:-) Yes. I bet the result would be the same in my case. 


 its not always possible to not wrap a select in a transaction.
 oracle for example *always* has a transaction going on, so everything
 is in a transaction in all cases.

   that a stored-procedure-oriented application is far more efficient
   is *extremely* debatable and database-dependent as well.
 
  I doubt it's *extremely* debatable.

 its extremely debatable:

 http://www.google.com/search?q=stored+procedures+vs

  Just issue 100 inserts from inside a
  stored procedure (or 100 updates) and do the same discretely with any
  kind of db interface. In case of the interface every statement has to be
  parsed by the db, whereas in a stored procedure the statement is already
  compiled of sorts (at least Oracle and PostgreSQL do that).

 the debate over SP's is about a larger issue than is an SP faster
 than 5 separate INSERT statements.  SP's are of course much better
 for micro-benchmarks like that.  its their impact on application
 development and architecture where the debate comes in (read some of
 the googled articles).

 I am certainly not anti-SP, ive done pure SP applications before (on
 projects where the DBAs controlled the SPs)...I just dont want to
 start hardwiring SQLAlchemy to expect that sort of application.   I
 think 80/20 as applied to SELECT is that 80% of SELECTs are for read
 operations and a COMMIT is inappropriate.

I agree with your goals and I also agree that a lot of people don't use stored 
procedures and similar db-dependent things. All I'd need is a way to tell SA 
that this one select is something worth committing.

   if you
   really want COMMIT for every SELECT, i'd favor it being enabled via an
   option passed to create_engine().
 
  Not every 

Re: Turbogears now based on Pylons!

[Mike Orr]

On 6/28/07, Jonathan LaCour [EMAIL PROTECTED] wrote:
 3. Either some middleware or just a few lines inside the
 BaseController
of the TurboGears template that starts a transaction on every
request and puts it into the transaction manager.

 And, in my opinion, items #1 and #2 should be in Pylons by default, and
 the third item should be in the TurboGears template by default.

... and available to non-TG Pylons applications.  The @transactional
decorator sounds like a good idea, possibly with a False flag if the
default is transactional.

This is a good point to ask, how will the TG template differ from the
standard Pylons template?  Will there be a base controller, or at what
point will they diverge?  Where will CherryPy-style dispatching be
implemented?  I imagine you will override PylonsApp and/or
PylonsBaseWSGIApp in pylons.wsgiapp, no?

-- 
Mike Orr [EMAIL PROTECTED]

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Alberto Valverde]

On Jun 28, 2007, at 8:52 PM, Mike Orr wrote:


 On 6/28/07, Jonathan LaCour [EMAIL PROTECTED] wrote:
 3. Either some middleware or just a few lines inside the
 BaseController
of the TurboGears template that starts a transaction on every
request and puts it into the transaction manager.

 And, in my opinion, items #1 and #2 should be in Pylons by  
 default, and
 the third item should be in the TurboGears template by default.

 ... and available to non-TG Pylons applications.  The @transactional
 decorator sounds like a good idea, possibly with a False flag if the
 default is transactional.

 This is a good point to ask, how will the TG template differ from the
 standard Pylons template?  Will there be a base controller, or at what
 point will they diverge?  Where will CherryPy-style dispatching be
 implemented?  I imagine you will override PylonsApp and/or
 PylonsBaseWSGIApp in pylons.wsgiapp, no?

The dispatching is currently implemented in a route method in a  
PylonsWSGIController subclass. There's a route defined like:

map.connect('*url', controller='root', action='route')

That handles it url to further dispatch it, cherrypy style. Here's  
all the meat:

http://trac.turbogears.org/browser/trunk/tg/controllers.py#L58

Alberto

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Noah Gift]

On 6/28/07, Alberto Valverde [EMAIL PROTECTED] wrote:



 On Jun 28, 2007, at 8:52 PM, Mike Orr wrote:

 
  On 6/28/07, Jonathan LaCour [EMAIL PROTECTED] wrote:
  3. Either some middleware or just a few lines inside the
  BaseController
 of the TurboGears template that starts a transaction on every
 request and puts it into the transaction manager.
 
  And, in my opinion, items #1 and #2 should be in Pylons by
  default, and
  the third item should be in the TurboGears template by default.
 
  ... and available to non-TG Pylons applications.  The @transactional
  decorator sounds like a good idea, possibly with a False flag if the
  default is transactional.
 
  This is a good point to ask, how will the TG template differ from the
  standard Pylons template?  Will there be a base controller, or at what
  point will they diverge?  Where will CherryPy-style dispatching be
  implemented?  I imagine you will override PylonsApp and/or
  PylonsBaseWSGIApp in pylons.wsgiapp, no?

 The dispatching is currently implemented in a route method in a
 PylonsWSGIController subclass. There's a route defined like:

 map.connect('*url', controller='root', action='route')

 That handles it url to further dispatch it, cherrypy style. Here's
 all the meat:

 http://trac.turbogears.org/browser/trunk/tg/controllers.py#L58


The controller is also slightly different as it is based a bit on Rick
Copeland's q-lookup Traversable system to a degree:

http://code.feestjeproject.com/feestje/browser/trunk/pyatl/lib/generic.py





Alberto

 



-- 
http://www.blog.noahgift.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Jonathan LaCour]

Mike Orr wrote:

 ... and available to non-TG Pylons applications.  The @transactional
 decorator sounds like a good idea, possibly with a False flag if the
 default is transactional.

Agreed.

 This is a good point to ask, how will the TG template differ from
 the standard Pylons template?  Will there be a base controller,
 or at what point will they diverge?  Where will CherryPy-style
 dispatching be implemented?  I imagine you will override PylonsApp
 and/or PylonsBaseWSGIApp in pylons.wsgiapp, no?

The TurboGears template, as of right now, is basically the same as
a Pylons template, only it includes a default route to a special
WSGIController subclass called TurboGearsController that implements
all sorts of nice things, like object-dispatch, content-negotation,
validation, and TG style @expose decoration.  This controller is called
the Root controller and is preconfigured in the routes configuration:

 map.connect('*url', controller='root', action='route')

You can find the controller here:

 http://trac.turbogears.org/browser/trunk/tg/controllers.py

The route method of the controller probably needs to be split up
into some smaller, overridable chunks.  The nice thing about the
approach that we have taken thus far is that you can mix TG-style
object-dispatching controllers and regular Pylons controllers if you
like.

--
Jonathan LaCour
http://cleverdevil.org


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mike Orr]

On 6/28/07, Jonathan LaCour [EMAIL PROTECTED] wrote:
 The TurboGears template, as of right now, is basically the same as
 a Pylons template, only it includes a default route to a special
 WSGIController subclass called TurboGearsController that implements
 all sorts of nice things, like object-dispatch, content-negotation,
 validation, and TG style @expose decoration.  This controller is called
 the Root controller and is preconfigured in the routes configuration:

  map.connect('*url', controller='root', action='route')

 You can find the controller here:

  http://trac.turbogears.org/browser/trunk/tg/controllers.py

 The route method of the controller probably needs to be split up
 into some smaller, overridable chunks.  The nice thing about the
 approach that we have taken thus far is that you can mix TG-style
 object-dispatching controllers and regular Pylons controllers if you
 like.

So if i want to use this for all URLs under a certain prefix, I would
direct the route to my subclass of TurboGearsController, using action
route (always?).
Line 66 (object_dispatch) returns my TG action method in all its
decorated glory, line 85 calls it and receives a dict, and line 88
calls a special render_response that invokes the template.
render_response is copying things out of the 'c' variable, so
templates would use '${foo}' rather than '${c.foo}' for data values.
Is this right?

I have long thought about making a TG-style dispatcher for Pylons.
It's great that somebody with more expertise has actually done it.

Noah wrote:
 The controller is also slightly different as it is based a bit on Rick 
 Copeland's  q-lookup Traversable system to a degree:

 http://code.feestjeproject.com/feestje/browser/trunk/pyatl/lib/generic.py

This reminds me of Quixote, especially the '._q_lookup' method.  Did
they borrow it from him or did he borrow it from them?

-- 
Mike Orr [EMAIL PROTECTED]

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Jonathan LaCour]

Mike Orr wrote:

 So if i want to use this for all URLs under a certain prefix, I would
 direct the route to my subclass of TurboGearsController, using action
 route (always?).

Well, thats the idea.  It needs lots of love and testing, though.

 Line 66 (object_dispatch) returns my TG action method in all its
 decorated glory, line 85 calls it and receives a dict, and line
 88 calls a special render_response that invokes the template.
 render_response is copying things out of the 'c' variable, so
 templates would use '${foo}' rather than '${c.foo}' for data values.
 Is this right?

Yes.  Most of us TurboGears folks are not big fans of single
character globals that represent stacked object proxies :) The
TurboGearsController will work like TurboGears 1.0 controllers in that
methods return dicts and those dicts are passed into the namespace of
the template.

 I have long thought about making a TG-style dispatcher for Pylons.
 It's great that somebody with more expertise has actually done it.

Well, most of the credit goes to Rick Copeland, since he hacked most of
the code for the new controller.

 This reminds me of Quixote, especially the '._q_lookup' method.  Did
 they borrow it from him or did he borrow it from them?

You'll have to ask Rick where he came up with the idea, all I know is
that I find it to be extremely useful!

--
Jonathan LaCour
http://cleverdevil.org


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Noah Gift]

On 6/28/07, Jonathan LaCour [EMAIL PROTECTED] wrote:


 Mike Orr wrote:

  So if i want to use this for all URLs under a certain prefix, I would
  direct the route to my subclass of TurboGearsController, using action
  route (always?).

 Well, thats the idea.  It needs lots of love and testing, though.

  Line 66 (object_dispatch) returns my TG action method in all its
  decorated glory, line 85 calls it and receives a dict, and line
  88 calls a special render_response that invokes the template.
  render_response is copying things out of the 'c' variable, so
  templates would use '${foo}' rather than '${c.foo}' for data values.
  Is this right?

 Yes.  Most of us TurboGears folks are not big fans of single
 character globals that represent stacked object proxies :) The
 TurboGearsController will work like TurboGears 1.0 controllers in that
 methods return dicts and those dicts are passed into the namespace of
 the template.

  I have long thought about making a TG-style dispatcher for Pylons.
  It's great that somebody with more expertise has actually done it.

 Well, most of the credit goes to Rick Copeland, since he hacked most of
 the code for the new controller.

  This reminds me of Quixote, especially the '._q_lookup' method.  Did
  they borrow it from him or did he borrow it from them?

 You'll have to ask Rick where he came up with the idea, all I know is
 that I find it to be extremely useful!


Yes, Rick is a big fan of Quixote... :)  I am sure he will get in on this
thread at some point...


--
 Jonathan LaCour
 http://cleverdevil.org


 



-- 
http://www.blog.noahgift.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 28, 2:32 pm, Uwe C. Schroeder [EMAIL PROTECTED] wrote:

 If I could figure out how, I probably would. The code from the docs doesn't
 work, because TG does something around that. I guess one of the major
 drawbacks of TG is it's documentation (or lack thereof) when it comes to
 using SA together with it. I guess that will change once TG uses SA by
 default.

my random guess is that they create a SessionTransaction, then they
forget about it.  if you then go and do some straight SQL like
select(foo).execute(), that goes against the engine, but that engine
needs to be set up explicitly within the SessionTransaction:

http://www.sqlalchemy.org/docs/unitofwork.html#unitofwork_transaction_sql

alternatively, you can pull out the connection straight from the
session and use that, via session.connection() - this also registers
the engine with the current transaction.

when using just the ORM for Query/flush(), the Session does this for
you when it first uses an engine while the transaction is present...so
it would seem like it just works, when really it only just works
for ORM logic.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 28, 2:01 pm, Alberto Valverde [EMAIL PROTECTED] wrote:
 One problem I see is that this might cause trouble in cases like this:

 Request
 |
V
 db transaction middleware  - creates session A
|
V
 authentication middleware - authenticates user and keeps ref to User
 instance (attached to session A)
|
V
 other middleware and PylonsApp
|
V
 TG controller  - creates session B inside __call__
|
V
 authorization library tries to check permissions for mapped Foo
 instance (attached to session B) which has a owner attribute:

 foo.owner (sess B) == logged_in_user (sess A)

you generally should not share objects between sessions, things will
break.  for example, the right way to use the user from session a into
session b would be :

foo.owner = sessionb.merge(user_from_sessiona)


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Noah Gift]

  drawbacks of TG is it's documentation (or lack thereof) when it comes to
  using SA together with it.


I actually asked Michael Bayer a couple of dumb questions as result of not
knowing what the controller did in a SQLAlchemy turbogears project.  I am
glad I wasn't the only one who was confused.  The automatic transactions in
the controller were unexpected as I went through the SQLAlchemy tutorial
thinking I had to control my transactions...which I liked!

That statement is very important though.  I think that Django's
documentation is something to aspire to and that if Turbogears doesn't reach
that level then it will turn into Twisted.  Every time I see someone give a
demo on Twisted I thinkwow...that is crazy, but then I start getting
into the documentation and I think...huh?

I am pretty sure Turbogears riding on Pylons will be on a vengeance to
document things correctly and not just the API, but usage scenarios for
everything.  Something can be the most innovative solution in the world, but
if no one knows about it, it is a complete waste


I guess that will change once TG uses SA by
  default.


I





http://www.blog.noahgift.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[[EMAIL PROTECTED]]

For me TG on Pylons would be a set of  similar packages-components
that make thinks easier and quicker
- wrappers for various backends (tg-user for authkit and others)
- set of helpers (toscawidgets, webhelpers)


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Uwe C. Schroeder]

Funny you mention Twisted. May app is actually Twisted based - and yes, I 
agree. I had some good (and some bad) experiences with Glyph emailing back 
and forth to get things resolved :-)


On Thursday 28 June 2007, Noah Gift wrote:
   drawbacks of TG is it's documentation (or lack thereof) when it comes to
 
   using SA together with it.

 I actually asked Michael Bayer a couple of dumb questions as result of not
 knowing what the controller did in a SQLAlchemy turbogears project.  I am
 glad I wasn't the only one who was confused.  The automatic transactions in
 the controller were unexpected as I went through the SQLAlchemy tutorial
 thinking I had to control my transactions...which I liked!

 That statement is very important though.  I think that Django's
 documentation is something to aspire to and that if Turbogears doesn't
 reach that level then it will turn into Twisted.  Every time I see someone
 give a demo on Twisted I thinkwow...that is crazy, but then I start
 getting into the documentation and I think...huh?

 I am pretty sure Turbogears riding on Pylons will be on a vengeance to
 document things correctly and not just the API, but usage scenarios for
 everything.  Something can be the most innovative solution in the world,
 but if no one knows about it, it is a complete waste


 I guess that will change once TG uses SA by

   default.

 I





 http://www.blog.noahgift.com

 


-- 
UC

--
Open Source Solutions 4U, LLC   1618 Kelly St
Phone:  +1 707 568 3056 Santa Rosa, CA 95401
Cell:   +1 650 302 2405 United States
Fax:+1 707 568 6416

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mike Orr]

On 6/28/07, Noah Gift [EMAIL PROTECTED] wrote:
 I am pretty sure Turbogears riding on Pylons will be on a vengeance to
 document things correctly and not just the API, but usage scenarios for
 everything.

Joining forces will make a big difference.  TG and Pylons combined
probably account for half of the non-Zope Python web applications
written in the past two years, give or take ten percent, and roughly
similar for the amount of developer activity.  Having twice as many
people working on the same codebase and docs rather than two separate
ones should accellerate developement.  There may still be two wiki
demos but they'll be able to borrow half their code from each other
rather than just the look  feel.  Our skills are also complementary.
TG developers do a better job of documentation and newbie handholding,
while Pylons has attracted developers who are obsessed with modularity
and reuse to a greater degree.  (TG developers may doubt this, but
many people came to Pylons because they found TG more monolithic than
they liked.)  Nine months till the next PyCon: it'll be exciting to
see where we're at then, and what sprints we'll be having.

-- 
Mike Orr [EMAIL PROTECTED]

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Turbogears now based on Pylons!

[Spider]

And in breaking news ...

Jonathan LaCour writing at http://cleverdevil.org/ says :
This past weekend, Mark Ramm and I held a sprint to experiment on
implementing a TurboGears like API on top of the excellent Pylons web
framework. The sprint was hugely successful, and as a result,
TurboGears 2.0 will be based upon our work.

He links to a longer article by Mark Ramm at http://compoundthinking.com/blog/
which describes the work that they did.

I think this reflects very positively on the WSGI-based approach of
Pylons.

Would the Pylons core developers care to comment?


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[anderbubble]

I couldn't be happier with this news. I've been using TurboGears for
almost a year now, and had been frustrated with the stagnation in
development and the myriad of Python web frameworks. I'm glad to see
two of the top contenders combining, rather than duplicating, their
efforts.

Keep up the good work, both of you! Best of luck.

~jon

On Jun 27, 10:10 am, Jonathan LaCour [EMAIL PROTECTED]
wrote:
 Spider wrote:
  I think this reflects very positively on the WSGI-based approach of
  Pylons.

 We do too!  TurboGears started a bit before WSGI really took hold, and
 as we all got further along, it seemed clear to most of us that we
 really needed to latch onto WSGI and build it into the foundation.  The
 more we looked, the more we realized that Pylons was so very similar to
 TurboGears, with a few differences, except it already had WSGI at its
 core.  Putting WSGI at the core of TurboGears after the fact would be
 much harder than just making a thin-layer on top of Pylons.

 Plus, now we get all the great benefits of Pylons itself, and get to
 participate in the great community that is growing around WSGI and
 Pylons.

 Very exciting stuff!

 --
 Jonathan LaCourhttp://cleverdevil.org


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mark Ramm]

 I couldn't be happier with this news. I've been using TurboGears for
 almost a year now, and had been frustrated with the stagnation in
 development and the myriad of Python web frameworks. I'm glad to see
 two of the top contenders combining, rather than duplicating, their
 efforts.

Pylons really is a good tool for building web framework type stuff.
We were able to get a lot done very quickly, and we're very excited
about the benefits of closer collaboration, and we hope that we're not
the only ones who reallize how usefull it is to have a solid, stable,
core and to innovate on top of that.

I'm looking forward to our new pylons powered future.


--Mark

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mike Orr]

[checking my watch]   It's not April 1st so this must be true.
Reminds me of ZDjangoGears and the joke that ultimately became Parrot.
 This is great news for Python web interpoerability!  Pylons users
will benefit from having access to a TG-style dispatcher and
decorators for validation, JSON, etc., without having to make a
wholesale choice between one framework or the other.  I do think
Django is on a dead-end road with its not invented here structure,
and eventually it will have to become more interoperable and
component-sharing somehow.  But that's up to the Django developers to
decide.

Mark and John, the two things I'm most concerned about are the
SQLAlchemy and templating interfaces, both of which are in flux in
Pylons.I just want to make sure we're on the same page as far as
their direction goes.  Have you seen my work on SAContext and
Smorgasbord, and do they look like something TG can build upon?  What
compatibility issues do you see, and how much compatibility with TG 1
are you willing to break?

SAContext has been performing well and I think it's close to
stability.  It hasn't been officially accepted into Pylons yet but
there's consensus that pylons.templating is inadequate and I haven't
seen any other proposal.  It's proven trivial to drop SAContext into
an existing Pylons application (one variable definition and two
search-and-replaces) so I expect the same will be true for TG
applications.  I'm not sure whether the module will be called
pylons.database or pylons.database.sqlalchemy or something like that,
but that'll be just a matter of changing imports.

Templating is a bigger issue so I'll start a separate thread for that.

-- 
Mike Orr [EMAIL PROTECTED]

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mark Ramm]

 It's proven trivial to drop SAContext into
 an existing Pylons application (one variable definition and two
 search-and-replaces) so I expect the same will be true for TG
 applications.

Yea, Jonathan has stuff to talk to you about.   I think SAContext
could be pretty easily modified to meet his needs.  (He wants to share
models between Pylons and other command line tools), but I think
that's a use case we should try to address if we can.

 I'm not sure whether the module will be called
 pylons.database or pylons.database.sqlalchemy or something like that,
 but that'll be just a matter of changing imports.

I had this crazy idea this morning when talking to Jonathan.  The idea
would be to create a more generic database interface go in (perhaps on
top of SA Context.)

I'd like there to be an entry point for database engines (SA, SO,
DejaVu, or whatever) that offer a very minimal set of functions in a
database agnostic way.

* connect
* start_transaction
* end_transaction
* rollback

This would make it possible for Evelind or others to write a DejaVu
engine for Pylons/TG that just works when installed.  Actual database
manipulations would happen in application code in whatever way users
want and would not touch any of these functions.   But we can call
connect to initalize the engine, and we can call the other methods
from a new generic transaction middleware component that we need to
write for TG on pylons.

But Jonathan has more details on the actual use case for this than I
do.   So, hopefully he'll chime in on this too.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Ian Bicking]

Mark Ramm wrote:
 I'd like there to be an entry point for database engines (SA, SO,
 DejaVu, or whatever) that offer a very minimal set of functions in a
 database agnostic way.
 
 * connect
 * start_transaction
 * end_transaction
 * rollback

Personally I'd like to see something a bit different:

* Way for a library to get the transaction manager.
* Interface for that transaction manager, maybe copied from Zope.
* Single convention for how to specify a database connection (ideally 
string-based).
* Probably a way to get the configuration, so you can lazily get a 
connection based on the configured connection.

The frameworks would setup the transaction manager and configuration, 
and database frameworks would basically consume this information.

-- 
Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org
 | Write code, do good | http://topp.openplans.org/careers

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Jonathan LaCour]

Ian Bicking wrote:

 Personally I'd like to see something a bit different:

 * Way for a library to get the transaction manager.
 * Interface for that transaction manager, maybe copied from Zope.
 * Single convention for how to specify a database connection (ideally
   string-based).
 * Probably a way to get the configuration, so you can lazily get
   a connection based on the configured connection.

 The frameworks would setup the transaction manager and configuration,
 and database frameworks would basically consume this information.

This would be pretty much ideal, and would satisfy my use cases very
well.  It would also allow framework authors to build in support for any
ORM / database layer in an agnostic way, sort of like the Buffet API
does for templating systems.

Then, we could also create a simple WSGI middleware for Pylons that
gives it TurboGears-like automatic transaction start/commit/rollback
on a per request basis.  Only, we could make it configurable so that
it didn't happen on read-only operations.  All of this would be ORM /
database layer agnostic, which would be very nice indeed.

The big issue for me with SAContext right now is that it uses a
BoundMetaData and expects you to know your dburi up-front as it
were.  I don't like having my model tied to my configuration system.
I'd rather have my model fetch its configuration from some third party
source, such as this new package we are discussing, so that I can
create desktop applications, command line utilities, etc. that share my
pylons application's model, without having to ship pylons or a pylons
configuration file.  The last thing that Ian requests here would let me
do that to a certain extent:

 from db_buffet_api import config
 from sacontext import SAContext

 sac = SAContext(dburi=config.dburi)

So, I am a big +1 on such a beast.

--
Jonathan LaCour
http://cleverdevil.org


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Noah Gift]

On 6/27/07, Jonathan LaCour [EMAIL PROTECTED] wrote:


 Ian Bicking wrote:

  Personally I'd like to see something a bit different:
 
  * Way for a library to get the transaction manager.
  * Interface for that transaction manager, maybe copied from Zope.
  * Single convention for how to specify a database connection (ideally
string-based).
  * Probably a way to get the configuration, so you can lazily get
a connection based on the configured connection.
 
  The frameworks would setup the transaction manager and configuration,
  and database frameworks would basically consume this information.

 This would be pretty much ideal, and would satisfy my use cases very
 well.  It would also allow framework authors to build in support for any
 ORM / database layer in an agnostic way, sort of like the Buffet API
 does for templating systems.

 Then, we could also create a simple WSGI middleware for Pylons that
 gives it TurboGears-like automatic transaction start/commit/rollback
 on a per request basis.  Only, we could make it configurable so that
 it didn't happen on read-only operations.  All of this would be ORM /
 database layer agnostic, which would be very nice indeed.

 The big issue for me with SAContext right now is that it uses a
 BoundMetaData and expects you to know your dburi up-front as it
 were.  I don't like having my model tied to my configuration system.
 I'd rather have my model fetch its configuration from some third party
 source, such as this new package we are discussing, so that I can
 create desktop applications, command line utilities, etc. that share my
 pylons application's model, without having to ship pylons or a pylons
 configuration file.  The last thing that Ian requests here would let me
 do that to a certain extent:

  from db_buffet_api import config
  from sacontext import SAContext

  sac = SAContext(dburi=config.dburi)

 So, I am a big +1 on such a beast.


I agree on being able to control the transactions!

My two cents to add are that it would be great to have control over
transactions being turned on or off on a per controller basis and to have a
way to allow Turbogears style validation that is database specific.  For
example...check to make sure this isn't already in the database for
me...(might be too crazy to implement though...)







--
 Jonathan LaCour
 http://cleverdevil.org


 



-- 
http://www.blog.noahgift.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Ian Bicking]

Jonathan LaCour wrote:
 Ian Bicking wrote:
 
 Personally I'd like to see something a bit different:

 * Way for a library to get the transaction manager.
 * Interface for that transaction manager, maybe copied from Zope.
 * Single convention for how to specify a database connection (ideally
   string-based).
 * Probably a way to get the configuration, so you can lazily get
   a connection based on the configured connection.

 The frameworks would setup the transaction manager and configuration,
 and database frameworks would basically consume this information.
 
 This would be pretty much ideal, and would satisfy my use cases very
 well.  It would also allow framework authors to build in support for any
 ORM / database layer in an agnostic way, sort of like the Buffet API
 does for templating systems.
 
 Then, we could also create a simple WSGI middleware for Pylons that
 gives it TurboGears-like automatic transaction start/commit/rollback
 on a per request basis.  Only, we could make it configurable so that
 it didn't happen on read-only operations.  All of this would be ORM /
 database layer agnostic, which would be very nice indeed.

The way I see this working is something like (vaguely):

def transaction_middleware(app):
 def wrapper(environ, start_response):
 manager = TransactionManager()
 environ['transaction.manager'] = manager
 try:
 app_iter = app(environ, start_response)
 except:
 manager.rollback()
 raise
 else:
 manager.commit()

The manager is basically a container of *actual* transactions, and 
calling rollback or commit on it gets passed on to all the transactions 
in the manager.

If you don't do anything that needs a transaction (e.g., read-only), you 
shouldn't put your transaction in the manager.

 The big issue for me with SAContext right now is that it uses a
 BoundMetaData and expects you to know your dburi up-front as it
 were.  I don't like having my model tied to my configuration system.
 I'd rather have my model fetch its configuration from some third party
 source, such as this new package we are discussing, so that I can
 create desktop applications, command line utilities, etc. that share my
 pylons application's model, without having to ship pylons or a pylons
 configuration file.  The last thing that Ian requests here would let me
 do that to a certain extent:
 
  from db_buffet_api import config
  from sacontext import SAContext
 
  sac = SAContext(dburi=config.dburi)
 
 So, I am a big +1 on such a beast.

I'd kind of like a way of getting the current WSGI environment.  Then 
one possible implementation of this config-getter is:

   def get_dburi():
   environ = get_environ()
   return environ['paste.config']['dburi']

Or it could get it out of another key, of course.  And in a non-web 
context you have a different get_dburi() implementation.  The only 
annoying part is actually figuring out how you get that function, and 
how you provide that function.

The way they do it in Zope, which is similar in ways to Paste's 
StackedObjectProxy and paste.registry, is basically something like:

   dburi = getUtility('get_dburi')()

Except in Zope they use an interface instead of 'get_dburi'.  But I 
think we should use a string.

We might want to look at PEAK's contextual stuff, as this is basically 
addressing the same problem.  I believe Phillip recently extracted that 
from PEAK.  (And all this config stuff is exactly the same issue as 
getting the current transaction manager.)  Personally I'm not terribly 
comfortable with paste.registry and StackedObjectProxy, as I feel it 
pretends to be more transparent than it really is; I prefer something 
more explicit like getUtility().


-- 
Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org
 | Write code, do good | http://topp.openplans.org/careers

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mike Orr]

On 6/27/07, Mark Ramm [EMAIL PROTECTED] wrote:
 Yea, Jonathan has stuff to talk to you about.   I think SAContext
 could be pretty easily modified to meet his needs.  (He wants to share
 models between Pylons and other command line tools), but I think
 that's a use case we should try to address if we can.

That's a good idea.  I also will increasingly need to access my web
applications' data from command-line tools, such as usage-report
generators and quasi-static webpages.

  I'm not sure whether the module will be called
  pylons.database or pylons.database.sqlalchemy or something like that,
  but that'll be just a matter of changing imports.

 I had this crazy idea this morning when talking to Jonathan.  The idea
 would be to create a more generic database interface go in (perhaps on
 top of SA Context.)

It sounds a bit abstract for me to work on, but let me know if it will
require changes to SAContext.  I envision SAContext can be a plugin
to this system.  I also use Durus so I'd like to get it working with
pylons.templating autoconfiguration at some point.  I have an idea
even crazier than yours: a SQLAlchemy engine for Durus, which may
support only some SQLAlchemy features but would at least explore the
possibility of managing a SQL and non-SQL database from a common front
end, for possible use in interoperability or database-copying
scenarios.  But it's still just an idea.

 I'd like there to be an entry point for database engines (SA, SO,
 DejaVu, or whatever) that offer a very minimal set of functions in a
 database agnostic way.

 * connect
 * start_transaction
 * end_transaction
 * rollback

 This would make it possible for Evelind or others to write a DejaVu
 engine for Pylons/TG that just works when installed.  Actual database
 manipulations would happen in application code in whatever way users
 want and would not touch any of these functions.   But we can call
 connect to initalize the engine, and we can call the other methods
 from a new generic transaction middleware component that we need to
 write for TG on pylons.

I hadn't heard of DejaVu until now.  One issue with SQLAlchemy is
connect/rollback are buried in the SQL and ORM APIs, so I'm not sure
how useful it would be to shoehorn them into this API which is out of
band from the controller code.  Is this a full-request transaction
the controller is expected to use?  Can we assume that every
controller has no more than one transaction, and that rollback always
corresponds to certain HTTP status codes?

-- 
Mike Orr [EMAIL PROTECTED]

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Jonathan LaCour]

Ian Bicking wrote:

 The way I see this working is something like (vaguely):

 def transaction_middleware(app):
  def wrapper(environ, start_response):
  manager = TransactionManager()
  environ['transaction.manager'] = manager
  try:
  app_iter = app(environ, start_response)
  except:
  manager.rollback()
  raise
  else:
  manager.commit()

 The manager is basically a container of *actual* transactions,
 and calling rollback or commit on it gets passed on to all the
 transactions in the manager.

 If you don't do anything that needs a transaction (e.g., read-only),
 you shouldn't put your transaction in the manager.

Okay, I am mostly with you, but then you end up with a lot of
boilerplate elsewhere wherever you start a transaction and throw it
into the manager.  I think we can address this in the TurboGears pylons
template somehow and automatically start a transaction and put it into
the manager on request by request basis, and provide some way to disable
it for read-only requests.

There have also been discussions of allowing you to turn it off for
specific HTTP methods, so you would never have a transaction for GET
unless you created it yourself, but POST, DELETE, etc. would usually
have transactions.  This feels a touch too magical to me, since the
whole point of the automatic transaction-per-request in TurboGears was
to make things easy by default, and not difficult to understand.

I wouldn't have a problem if there were two separate pieces of
middleware though: one for rolling back active transactions on
exceptions, and another for setting up when you want those transactions
to be created automatically.

 The last thing that Ian requests here would let me do that to a
 certain extent:

  from db_buffet_api import config
  from sacontext import SAContext

  sac = SAContext(dburi=config.dburi)

 I'd kind of like a way of getting the current WSGI environment.
 Then one possible implementation of this config-getter is:

def get_dburi():
environ = get_environ()
return environ['paste.config']['dburi']

 Or it could get it out of another key, of course.  And in a non-web
 context you have a different get_dburi() implementation.  The only
 annoying part is actually figuring out how you get that function, and
 how you provide that function.

This database layer is acting as a middleman between the configuration
system of the framework (Pylons, etc.) and the model itself, which may
not live inside the framework.  Why not just have the web framework tell
the middleman how to get the dburi from the configuration, and the model
can ask for the dburi from the middleman?

 The way they do it in Zope, which is similar in ways to Paste's
 StackedObjectProxy and paste.registry, is basically something like:

dburi = getUtility('get_dburi')()

 Except in Zope they use an interface instead of 'get_dburi'.  But I
 think we should use a string.

Agreed, we should use a string.

 We might want to look at PEAK's contextual stuff, as this is basically
 addressing the same problem.  I believe Phillip recently extracted
 that from PEAK.  (And all this config stuff is exactly the same issue
 as getting the current transaction manager.)  Personally I'm not
 terribly comfortable with paste.registry and StackedObjectProxy, as I
 feel it pretends to be more transparent than it really is; I prefer
 something more explicit like getUtility().

I'll take a look at the stuff in PEAK, but it usually breaks my brain
for at least two days before I finally grok it.  But, to be fair,
StackedObjectProxy does the same thing ;)

How should we move forward with this?  Is this the type of thing that
the DB-SIG cares about?  Or should the discussion remain here?

--
Jonathan LaCour
http://cleverdevil.org



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mike Orr]

On 6/27/07, Jonathan LaCour [EMAIL PROTECTED] wrote:

 Ian Bicking wrote:

  Personally I'd like to see something a bit different:
 
  * Way for a library to get the transaction manager.
  * Interface for that transaction manager, maybe copied from Zope.
  * Single convention for how to specify a database connection (ideally
string-based).
  * Probably a way to get the configuration, so you can lazily get
a connection based on the configured connection.
 
  The frameworks would setup the transaction manager and configuration,
  and database frameworks would basically consume this information.

 This would be pretty much ideal, and would satisfy my use cases very
 well.  It would also allow framework authors to build in support for any
 ORM / database layer in an agnostic way, sort of like the Buffet API
 does for templating systems.

 Then, we could also create a simple WSGI middleware for Pylons that
 gives it TurboGears-like automatic transaction start/commit/rollback
 on a per request basis.  Only, we could make it configurable so that
 it didn't happen on read-only operations.  All of this would be ORM /
 database layer agnostic, which would be very nice indeed.

All this is to avoid a try-commit block in the controller method or
model function?  Is it really that important?  Is this what Noah means
by controlling transactions?

The normal Pylons strategy is to clear the SQLAlchemy session in the
base controller before calling the action method.  This is effectively
a rollback, but since all the changes are in memory and haven't been
compiled to SQL yet, no rollback is necessary.  Users do
sac.session.flush() to write their changes to the database, which is
essentially a commit.  I'm more comfortable with that happening
explicitly in the controller method rather than vaguely in some
middleware (especially since the application is not supposed to know
or care whether a certain middleware is active).

As for direct SQL statements in SQLAlchemy, I just assume they're
autocommit.  Maybe someday I'll have to get more sophistocated about
them.

 The big issue for me with SAContext right now is that it uses a
 BoundMetaData and expects you to know your dburi up-front as it
 were.  I don't like having my model tied to my configuration system.
 I'd rather have my model fetch its configuration from some third party
 source, such as this new package we are discussing, so that I can
 create desktop applications, command line utilities, etc. that share my
 pylons application's model, without having to ship pylons or a pylons
 configuration file.  The last thing that Ian requests here would let me
 do that to a certain extent:

  from db_buffet_api import config
  from sacontext import SAContext

  sac = SAContext(dburi=config.dburi)

If that's all you want, it already does this.  SAContext requires an
explicit URI and options as arguments, and doesn't know about
anybody's configuration system.  My hope is that it will be included
in SQLAlchemy at some point, for many diverse applications.

PylonsSAContext addresses the needs Pylons users have right now: a
standard way to configure an engine/metadata/session that's better
than pylons.database.
The overriding concerns are:
- Correctness per SQLAlchemy's recommendation (which favors
BoundMetaData, even though the SQLAlchemy manual hasn't gone that far
yet).
- Easy for users to put into their model (just a couple short lines of code).
- Automatically reads all engine options from the config file that can
be specified in a scalar format.
- Accepts other engine options from the developer which can't be
expressed in a scalar format.

If your concept requires something different than PylonsSAContext,
perhaps a different subclass, there's no reason the two can't live
side by side.

The model is tied to the configuration such that you can't import it
standalone, at least if you follow the SAC_Demo example.  Instead you
have to use the module under paster shell, paster setup-app, or
manually prepare the environment for the import (which is not well
documented but websetup.py gives some hints).  This is *no worse* than
existing Pylons usage, which also has the same limitation.  We have
thought long about how to improve this but have not come up with a
better way.  Some people put all their tables in an init function in
the model so that it can be called sometime after the actual import,
and with arguments.  This is a little better in some ways but still
not a robust solution.  Do you set global variables for the tables or
pass them back as the return value?  Either way is messy.

Of course the engine must be initialized before any tables can be
defined or any work done.  SAContext initializes the default engine in
the constructor.  Is there a need to push that back to some later
stage?  Which stage?  Or can Ian's ideal wrapper simply delay creating
the 'sac' until it has to?

-- 
Mike Orr [EMAIL PROTECTED]

Re: Turbogears now based on Pylons!

[Mike Orr]

On 6/27/07, Jonathan LaCour [EMAIL PROTECTED] wrote:
 Okay, I am mostly with you, but then you end up with a lot of
 boilerplate elsewhere wherever you start a transaction and throw it
 into the manager.  I think we can address this in the TurboGears pylons
 template somehow and automatically start a transaction and put it into
 the manager on request by request basis, and provide some way to disable
 it for read-only requests.

 There have also been discussions of allowing you to turn it off for
 specific HTTP methods, so you would never have a transaction for GET
 unless you created it yourself, but POST, DELETE, etc. would usually
 have transactions.  This feels a touch too magical to me, since the
 whole point of the automatic transaction-per-request in TurboGears was
 to make things easy by default, and not difficult to understand.

 I wouldn't have a problem if there were two separate pieces of
 middleware though: one for rolling back active transactions on
 exceptions, and another for setting up when you want those transactions
 to be created automatically.

It sounds like Pylons and TurboGears have very different paradigms
about how transactions are handled.  Could somebody explain them side
by side, showing how the controller method interacts with the
middleware under various read-only, write-commit, write-rollback
scenarios?  Especially taking into account the various reasons one
might want to roll back.  As a starting point, i tried to explain the
typical Pylons usage in my previous email.

Then we can see whether SAContext or something else can meet all the needs.

-- 
Mike Orr [EMAIL PROTECTED]

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Mark Ramm]

 It sounds like Pylons and TurboGears have very different paradigms
 about how transactions are handled.

I can't do a side-by-side comparison, because I am not 100% clear on
an example of the right way to handle transactions in Pylons.

But I can describe what TurboGears 1.0 does.

TG currently has an automatic transaction per request feature, which
is very widely used.

If a request fails for any reason, at any point the exception is
propigated out, and causes a transaction rollback.

We want to set up the transaction as early as possible.  So that if
you have validators which hit the DB (which many do) and you want that
wrapped in a transaction that is done for you.  And this is important
because you can't do that in the controller method itself since
validation happens in the decorator -- before you enter the controller
method itself.  And we want the transaction closed as late as possible
so that if your internationalization fails or some other post
controller action fails, you can rollback the transaction.

Of course the problem with the current implementation of this feature
is that all kinds of pages which don't actually change anything get
transaction overhead that they don't need.   So we are looking for the
best way to control transactions on a per controller basis.

--Mark Ramm

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Uwe C. Schroeder]

On Wednesday 27 June 2007, Mark Ramm wrote:
  It sounds like Pylons and TurboGears have very different paradigms
  about how transactions are handled.

 I can't do a side-by-side comparison, because I am not 100% clear on
 an example of the right way to handle transactions in Pylons.

 But I can describe what TurboGears 1.0 does.

 TG currently has an automatic transaction per request feature, which
 is very widely used.

 If a request fails for any reason, at any point the exception is
 propigated out, and causes a transaction rollback.

 We want to set up the transaction as early as possible.  So that if
 you have validators which hit the DB (which many do) and you want that
 wrapped in a transaction that is done for you.  And this is important
 because you can't do that in the controller method itself since
 validation happens in the decorator -- before you enter the controller
 method itself.  And we want the transaction closed as late as possible
 so that if your internationalization fails or some other post
 controller action fails, you can rollback the transaction.

 Of course the problem with the current implementation of this feature
 is that all kinds of pages which don't actually change anything get
 transaction overhead that they don't need.   So we are looking for the
 best way to control transactions on a per controller basis.


And on that note: if you're using SA with TG, SA issues a rollback on every 
transaction that is not an insert or update. So if you're having a stored 
procedure (which you trigger with select * from stored_proc() and that 
stored procedure actually does updates or inserts, you're going to lose 
changes - simply because SA issues a rollback on Select statements.
Quick fix for this is to modify SA to just issue a commit on every statement, 
so the TG transaction can roll back or commit without being affected. IMHO 
issuing a commit on a select shouldn't be more overhead than issuing a 
rollback - because the db should know what to do (in this case nothing)

Maybe something to think about too, because I can't be the only one making 
heavy use of stored procedures (which are far more efficient than 
controller/model side code)

Uwe


--
Open Source Solutions 4U, LLC   1618 Kelly St
Phone:  +1 707 568 3056 Santa Rosa, CA 95401
Cell:   +1 650 302 2405 United States
Fax:+1 707 568 6416

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

 I hadn't heard of DejaVu until now.  One issue with SQLAlchemy is
 connect/rollback are buried in the SQL and ORM APIs, so I'm not sure
 how useful it would be to shoehorn them into this API which is out of
 band from the controller code.


just to clarify, not really.  the entire point of the explicit
connection API, which 99% of the time people either arent aware of,
or find it to be too verbose when they see it, is that the
transaction going on can be completely extracted to be external to
everything SQLAlchemy.  if we are looking for an ORM-agnostic API that
just wraps all internal operations in a transaction, the SAContext
need not even be involved, youd use SessionTransaction against its
current session.

Also SAContext has no dependence on bound metadata as it supports
alternative strategies which define the usage of engines, and that it
takes the database URI in its constructor is just a small
implementation detail thats easily changed.  Bound metadata in its
old sense no longer exists in SQLAlchemy, as all metadata objects
can have a new Engine assigned to them at any time (or not at all).

 Is this a full-request transaction
 the controller is expected to use?  Can we assume that every
 controller has no more than one transaction, and that rollback always
 corresponds to certain HTTP status codes?

it seems we are talking about just the case where, the entire request
is inside of a transaction, and it seems like this is mainly for the
purposes of WSGI middleware...having any fine-grained access to the
ORM in an agnostic way (i.e. anything beyond define connection
parameters and transaction boundaries) would be unworkable.



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 27, 8:24 pm, Uwe C. Schroeder [EMAIL PROTECTED] wrote:

 And on that note: if you're using SA with TG, SA issues a rollback on every
 transaction that is not an insert or update. So if you're having a stored
 procedure (which you trigger with select * from stored_proc() and that
 stored procedure actually does updates or inserts, you're going to lose
 changes - simply because SA issues a rollback on Select statements.

this issue can be worked around by using explicit transactions.  i
also have a notion of SQL functions being marked as transactional to
help this issue.  but this thread so far seems to be about the notion
of an entire request being marked as transactional, which as it
turns out is a central concept of J2EE and others, which would also
eliminate the issue youre having.

 Quick fix for this is to modify SA to just issue a commit on every statement,
 so the TG transaction can roll back or commit without being affected. IMHO
 issuing a commit on a select shouldn't be more overhead than issuing a
 rollback - because the db should know what to do (in this case nothing)

this would be something that would need to be benchmarked.  i do think
it would add some overhead.  but beyond that, i dont like the idea of
unnecessary COMMITs for every SELECT statement at all.



 Maybe something to think about too, because I can't be the only one making
 heavy use of stored procedures (which are far more efficient than
 controller/model side code)

that a stored-procedure-oriented application is far more efficient
is *extremely* debatable and database-dependent as well.  if you
really want COMMIT for every SELECT, i'd favor it being enabled via an
option passed to create_engine().  Beyond that I think the model
implemented as stored procedure style of development is much in the
minority these days, particularly within the lightweight/open-source
development community.   just that the stored procedure changes the
semantics of SELECT to be a write operation reminds me of the
RESTful sin of using GET to post data.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 27, 6:42 pm, Ian Bicking [EMAIL PROTECTED] wrote:
 The way I see this working is something like (vaguely):

 def transaction_middleware(app):
  def wrapper(environ, start_response):
  manager = TransactionManager()
  environ['transaction.manager'] = manager
  try:
  app_iter = app(environ, start_response)
  except:
  manager.rollback()
  raise
  else:
  manager.commit()

 The manager is basically a container of *actual* transactions, and
 calling rollback or commit on it gets passed on to all the transactions
 in the manager.

this is fine.  we're *really* starting to imitate J2EE in some ways.
but its not a bad thing.


 If you don't do anything that needs a transaction (e.g., read-only), you
 shouldn't put your transaction in the manager.

just to clarify, however this works, it should be *really easy* for
individual controller methods to be marked as transactional or not.
I have written decorators like these already, in the style of:

class MyController(...):
 def index_page(self):
 .

 @transactional
 def post_message(self):
 .

 def display_message(self):
 .

so id want that kind of thing to be readily available, i suppose it
would communicate with the WSGI middleware on a per-request basis.

but another thing i like, is that of the response to the request being
delivered *after* the transaction commits.  i have raised this issue
on the pylons list before but i dont think i impressed anyone.  if you
render your response template, then your transaction fails, you have
the potential for that response template to still be delivered with
the success! message.  anyway, might be nice for the middleware to
address this if possible.

also i think this is all outside of the scope of SAContext.  SAContext
should remain as a facade to SA-specific elements; it can accept
strategy objects which control its internal workings, so if need be
particular strategies can be delivered in order to function with the
transaction manager, without changing the external view of SAContext.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Michael Bayer]

On Jun 27, 7:44 pm, Mike Orr [EMAIL PROTECTED] wrote:
 The normal Pylons strategy is to clear the SQLAlchemy session in the
 base controller before calling the action method.  This is effectively
 a rollback, but since all the changes are in memory and haven't been
 compiled to SQL yet, no rollback is necessary.

theres a rollback() going on ;  SQLAlchemy's connection pool issues a
rollback() on all connections before theyre returned to the pool.
this is to release any locks that might be present on the connection
(and its definitely quite necessary...unless we went with the COMMIT
on every statement/connection returned idea someone has mentioned).

 Of course the engine must be initialized before any tables can be
 defined or any work done.

since ive seen people get the wrong idea about this, let me
reiterate:  *you can define all your tables and mappers with no
database connection defined whatsoever*, and you can even start
creating mapped objects.   SQLAlchemy does not care a whiff about your
database connection until the moment you tell it to execute a SQL
statement...and when the statement is done, it goes back to not caring
at all about your database connection; you can tear it down or modify
it to be something else all you like.

 SAContext initializes the default engine in
 the constructor.  Is there a need to push that back to some later
 stage?  Which stage?  Or can Ian's ideal wrapper simply delay creating
 the 'sac' until it has to?

easy enough to make uri optional in SAContext (the way it is for
everything else in SA) and just allow binding it later via
add_engine() or whatever.   SAContext is designed the way it is right
now because Pylons defines the default engine in the .ini file; theres
no need for a delayed connection in the usual case.  im not exactly
sure why Jonathan thinks that the url forces one to define my
connection up front.  if youre saying, i want to distribute the
application without an .ini file thats fine...but when the thing is
actually *used*, there will be an .ini file or other configuration
defined when things run.  if you want to define your model
independently of connection, just use an empty MetaData.  Set up the
SAContext to bind the engine or individual connections to the session
instead...or give it your MetaData for it to bind when youre ready.
theres really nothing to it.



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---

Re: Turbogears now based on Pylons!

[Uwe C. Schroeder]

On Wednesday 27 June 2007, Michael Bayer wrote:
 On Jun 27, 8:24 pm, Uwe C. Schroeder [EMAIL PROTECTED] wrote:
  And on that note: if you're using SA with TG, SA issues a rollback on
  every transaction that is not an insert or update. So if you're having a
  stored procedure (which you trigger with select * from stored_proc()
  and that stored procedure actually does updates or inserts, you're going
  to lose changes - simply because SA issues a rollback on Select
  statements.

 this issue can be worked around by using explicit transactions. 

actually no, it can't. Maybe I don't get it right, but the only way for me to 
get a commit was actually to modify Connection._autocommit in 
sqlalchemy.engine.base.  Obviously SA thinks there is no transaction in TG, 
so it just wraps one around it. After I got tired of searching for the 
problem I just added SELECT to the above method and now get my commit.
I'm sure either I do something wrong or there's a bug in the db implementation 
of TG. All I could find is that the transaction (since 1.0.2 available in 
tg.sa_transaction) is just a subtransaction from the autocommit code - thus 
when the outer transaction issues a rollback the inner transaction that was 
committed will be rolled back too.


 i also have a notion of SQL functions being marked as transactional to
 help this issue.  but this thread so far seems to be about the notion
 of an entire request being marked as transactional, which as it
 turns out is a central concept of J2EE and others, which would also
 eliminate the issue youre having.

Yes - see below on the every select statement.


  Quick fix for this is to modify SA to just issue a commit on every
  statement, so the TG transaction can roll back or commit without being
  affected. IMHO issuing a commit on a select shouldn't be more overhead
  than issuing a rollback - because the db should know what to do (in this
  case nothing)

 this would be something that would need to be benchmarked.  i do think
 it would add some overhead.  but beyond that, i dont like the idea of
 unnecessary COMMITs for every SELECT statement at all.

I agree, and that is certainly DB dependent. Personally I can't imagine that 
an automatically issued rollback for every select transaction is in any way 
more overhead than issuing a commit. Not wrapping a select in a transaction 
will definitely be the least overhead.


  Maybe something to think about too, because I can't be the only one
  making heavy use of stored procedures (which are far more efficient than
  controller/model side code)

 that a stored-procedure-oriented application is far more efficient
 is *extremely* debatable and database-dependent as well.

I doubt it's *extremely* debatable. Just issue 100 inserts from inside a 
stored procedure (or 100 updates) and do the same discretely with any kind of 
db interface. In case of the interface every statement has to be parsed by 
the db, whereas in a stored procedure the statement is already compiled of 
sorts (at least Oracle and PostgreSQL do that). I had cases where moving the 
application code (standard DBAPI calls, no ORM) to a stored procedure reduced 
the execution time from 27 seconds to 2 seconds without changing the database 
structure (ok, extreme case, but handling 1500 inserts discretely in the 
application is just a lot of remote overhead, particularly when the requests 
come over the network where you get an extra delay for network operations)

 if you 
 really want COMMIT for every SELECT, i'd favor it being enabled via an
 option passed to create_engine(). 

Not every select, every transaction that didn't roll back.
I just think the default of rollback on every transaction is wrong - a 
rollback should occur when there is a problem, not when the transaction was 
fine. But that may just be me.

 Beyond that I think the model 
 implemented as stored procedure style of development is much in the
 minority these days, particularly within the lightweight/open-source
 development community.   just that the stored procedure changes the
 semantics of SELECT to be a write operation reminds me of the
 RESTful sin of using GET to post data.

Probably because a lot of people can't figure out how to use stored procedures 
and triggers, since the lightweight/open-source programming is often done 
on a database that has very limited support for both :-)
 
Personally I'm not a big fan of handling database integrity outside the 
database. Take a simple portal as example, where every new user who signs up 
gets a couple of mailboxes by default (inbox, outbox,drafts,trash). I could 
handle that in the db structure itself, but that would make the db model 
overly complicated for such a simple thing. The proper way is to handle that 
in a trigger, or I can handle it in a stored procedure that just makes sure 
the mailboxes are created. In my case I chose a stored procedure because it's 
actually a select on the user's profile, but the stored procedure also