Re: [Zope] SQLSession vs FSSession for ultra-high scalability and speed.
>>> Pavlos Christoforou wrote > OTO an RDBMS maybe a more reliable datastore than the filesystem with > better consistency, recovery tools etc etc. The other bigwin for an RDBMS-based one is, of course, that you can share it between zopes. And for me, it's easier to manage a DBMS table than a chunk of filesystem somewhere on a machine. > Also check whether SQLSession caches info. It used to hit the RDBMs for > every variable access which was very prohibitive for us, but Anthony > mentioned that he was changing that. As of 0.2.x it sometimes does, depending on whether you've turned on caching - but 0.3.0 will fix it entirely, I plan to remove the non-cached option, and do it a whole lot smarter. 0.2.9 will be out shortly (tonight, if I can stop this sodding PC from crashing every 10 minutes) and will be the last of the current architecture. 0.3.0 is moving to a more sane system - one that will stop my DBA from hurting me. :) Anthony, who's really not a fan of PC hardware. -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] SQLSession vs FSSession for ultra-high scalability and speed.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article <[EMAIL PROTECTED]>, chas <[EMAIL PROTECTED]> writes >>Graham Chiu wrote: >>It may be the way I designed my site, but if the database serving >>SQLSessions had to be taken down for maintenance, my site became >>inaccessible. > >Same would happen to my site. Rather than try to build around >that, I think I'll try to keep the DB up instead - since I won't >have much functionality without sessions. What I meant was that it should be easy to plug in another database engine for the session serving since there should be nothing database specific about the interface to SQLSession. ( As it happens, there is :-( ) So, taking down the database for Session management should not take down the site. - -- Regards, Graham Chiu gchiucompkarori.co.nz http://www.compkarori.co.nz/index.php Powered by Interbase and Zope -BEGIN PGP SIGNATURE- Version: PGPsdk version 1.7.1 iQA/AwUBOSew+7TRdIWzaLpMEQI03gCfU+VlPOn/x3xXkcJaN8Pc0fVtlH0An1YK 2pdxpkp1E9Fy5aDPUdP2qp20 =8kZ/ -END PGP SIGNATURE- ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] SQLSession vs FSSession for ultra-high scalability and speed.
Thank you very much for the comprehensive and speedy reply, J. >Let me give it a try (or just dump all of what I know). Much more than I expected - and greatly appreciated. Thank you. >First of all it looks like you are going to want ZEO. If you don't >know ZEO, get to know ZEO, it fits perfectly with what you want for >the frontline servers. Wow, has it been released for public consumption yet ? (I've been out of the Zope-loop for a while unfortunately) But yes, I had ZEO in mind for keeping the application logic on the frontline servers in sync. >As for the database, MySQL is hands down the fastest, but certainly >not the most robust in terms of features. Oracle is great if you have >$10,000 and a full time DBA but PostgreSQL 7.0 is really your best >bet. You might also want to take the money you were going to spend on >the DB2 and put it right back into DB1 and house everything there. >Just get a beefy machine, RAID it up for redundancy and call it a day. Will definitely be getting RAID for the application data. >The other idea that I have been playing around with in terms of >redundancy is to build something into the Zope code for the ZSQL >Methods that can go into a "cached" mode if your database dies. I >have only begun to play with this, but in theory since, Zope can >cache all your common data that comes from SQL you could, in effect, >bring down the SQL database and have Zope continue to work with the >cached data. I think all that it would involve (and I am sure that I >am simplifying the hell out of this) is to have Zope know that the >SQL database is no longer there (look for an error) and just turn the >"Maximum time (seconds) to cache results" to unlimited until the >database was restored. > >Has anyone else thought about this or played with this idea? I think in most cases, if the DB dies, it's game over until it goes up again. >Graham Chiu wrote: >It may be the way I designed my site, but if the database serving >SQLSessions had to be taken down for maintenance, my site became >inaccessible. Same would happen to my site. Rather than try to build around that, I think I'll try to keep the DB up instead - since I won't have much functionality without sessions. >Based upon this experience, I would therefore now prefer to use >different database engines for the main database and for the session >tables. And since for sessions, all you really want is speed without >any fancy SQL transactions, then MySQL for the session data seems an >appropriate use. Exactly what I was hoping - thank you for the confirmation :) >Graham Chiu wrote: >> >>Steve Alexander wrote: >>I seem to recall from the MySQL vs Postgres thread a while ago that >>MySQL was fastest in mostly-reading situations, whereas Postgres was far >>better when there is quite a lot of writing as well as reading. Interesting. I would have thought that MySQL won in writes b/c it doesn't have the transaction overhead. Then again, Postgres 7 seems to have updated the ante a lot. >>If this is true, the Postgres should be a better choice for a >>session-data DBMS. > > For SQLSessions, you only write the session data once, and read multiple > times. So, that would suit MySQL ? I guess it depends what you're doing with your session object. We'll be changing the properties of the session object as the user traverses the site, so the session object will have to be updated (ie. updated on the RDB). Thank you all for your help. chas ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] SQLSession vs FSSession for ultra-high scalability and speed.
In article <[EMAIL PROTECTED]>, Steve Alexander writes >> And since for sessions, all you really want is speed without >> any fancy SQL transactions, then MySQL for the session data seems an >> appropriate use. > >I seem to recall from the MySQL vs Postgres thread a while ago that >MySQL was fastest in mostly-reading situations, whereas Postgres was far >better when there is quite a lot of writing as well as reading. > >If this is true, the Postgres should be a better choice for a >session-data DBMS. For SQLSessions, you only write the session data once, and read multiple times. So, that would suit MySQL ? -- Regards, Graham Chiu gchiucompkarori.co.nz http://www.compkarori.co.nz/index.php Powered by Interbase and Zope ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] SQLSession vs FSSession for ultra-high scalability and speed.
Graham Chiu wrote: > > In article, J. Atwood > <[EMAIL PROTECTED]> writes > >As for the database, MySQL is hands down the fastest, but certainly not the most > >robust in terms of features. Oracle is great if you have $10,000 and a full time > >DBA but PostgreSQL 7.0 is really your best bet. You might also want to take the > >money you were going to spend on the DB2 and put it right back into DB1 and > >house everything there. Just get a beefy machine, RAID it up for redundancy and > >call it a day. > And since for sessions, all you really want is speed without > any fancy SQL transactions, then MySQL for the session data seems an > appropriate use. I seem to recall from the MySQL vs Postgres thread a while ago that MySQL was fastest in mostly-reading situations, whereas Postgres was far better when there is quite a lot of writing as well as reading. If this is true, the Postgres should be a better choice for a session-data DBMS. Of course, the *right* way to find out is to run a benchmark :-) -- Steve Alexander Software Engineer Cat-Box limited ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] SQLSession vs FSSession for ultra-high scalability and speed.
In article, J. Atwood <[EMAIL PROTECTED]> writes >As for the database, MySQL is hands down the fastest, but certainly not the most >robust in terms of features. Oracle is great if you have $10,000 and a full time >DBA but PostgreSQL 7.0 is really your best bet. You might also want to take the >money you were going to spend on the DB2 and put it right back into DB1 and >house everything there. Just get a beefy machine, RAID it up for redundancy and >call it a day. It may be the way I designed my site, but if the database serving SQLSessions had to be taken down for maintenance, my site became inaccessible. Based upon this experience, I would therefore now prefer to use different database engines for the main database and for the session tables. And since for sessions, all you really want is speed without any fancy SQL transactions, then MySQL for the session data seems an appropriate use. Session tables wouldn't have to be too big as you could run ZScheduler to remove expired sessions. -- Regards, Graham Chiu gchiucompkarori.co.nz http://www.compkarori.co.nz/index.php Powered by Interbase and Zope. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] SQLSession vs FSSession for ultra-high scalability and speed.
Hi Folks, Sorry if this has been asked before, but can anybody advise on FSSession vs SQLSession for: a) Speed. b) Scalability. Planned architecture employs multiple Zope/Apache frontline servers: +-+ | Z/A |---+ +-+ | | +-+ | | Z/A |---+ +- DB1 +-+ | | |--| +-+ | | | Z/A |---+ +- DB2 (optional) +-+ | | | Etc, etc. Z/A = 'front line' webservers running Zope/Apache. Contains application/business logic. We are NOT using the ZODB for data storage. DB1 = Data storage. eg. Oracle/Postgres DB2 = potentially just for managing the sessions, if we use SQLSession. Would MySQL be best ? Just how scalable can it really get ? We need it to at least be able to scale to 12 million sessions per day (and substant- -ially more writes to DB1 per day), even if we will be starting out much more modestly than that. Realistically, should I be looking at something like ATG Dynamo instead ? Thank you very much, chas ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )