On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: > 2009/12/26 OvermindDL1 <[email protected]>: >> I have started looking at Wt::DBO to replace my sqlite marshaller and >> I notice something odd right off the bat. >> It seems there can only be one session at a time? > > Not at all, a bit unexpectedly, the blog on our homepage (which uses > Wt::Dbo) was hit by slashdot and handled the many concurrent sessions > very well.
Very nice to know, a slashdotting is certainly one of the best ways to test a site uptime capability under load. :) On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: >> Rather, more specifically, I would have to use some type of locking >> queue of connections, and pull from it each time I wish to create a >> session, and it kind of looks like it would be rather heavy to create >> a session object each time as well. So I can either use a pool of >> connections and create sessions on demand from a connection in that >> pool (with the running of many mapClass's), or I could make a pool of >> sessions, each attached to its own connections, and use the sessions >> from the pool as necessary, and so on and so forth. The two files I > > The idea is to create a dbo::Session per user session. What is > currently lacking is a ConnectionPool object (as it is only an > optimization), which allows sessions to only use a connection during > an active transaction. But everything has been implemented with a > connection pool in mind. In the future we might also consider sharing > a kind of ObjectRegistry between sessions for read-only objects. So I gathered that after I read the code. My first emails were mostly how everything looked before I learned how it implemented everything. What I write in this email is after I read through just about all of DBO's code. I like many of the design decisions. On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: > W.r.t. locking: the current implementation only implements optimistic > database locking, which is what in our experience is what is most > often used in web applications since it does not require native > database locks (making is scalable). Yep, I do agree with that very much, I like. :) On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: > We've been working on a tutorial on Dbo that motivates the current > design and our ideas on future improvements: > http://www.webtoolkit.eu/wt/doc/tutorial/dbo/tutorial.html Ah, very nice, I shall look at that here shortly, see how it compares with my initial jump-in experience. On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: >> found that demonstrate how to use the DBO are just test frameworks and >> it is quite obvious that this is not how it was designed to be used, >> so is there an actual example somewhere that demonstrates its real >> use, or should I just create a pool of session objects as I was >> initially thinking? > > See the blog/ example. I shall examine that closely. :) On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: >> Also, the createtables, does it destructively recreate the tables, as >> in deleting the existing data, or if the table already exists does it >> do nothing, or what if the table layout is different, does it do >> nothing, or does it create/remove columns as it needs, and is that >> destructive or not to the existing data, and is there a method to drop >> tables as well? > > Currently it only naively tries to create all tables. It will either > entirely succeed or entirely fail (because of the transaction). It is > just there to get going quickly, but is not usable for migrating > schemas. That is one place SqlAlchemy fails at too, so no real biggie, can always change the schema's manually. Actually in SqlAlchemy I usually had the tables be versioned based on their name, anytime I wanted to change something in a table then I would create a new table with the same name with a version number at the end incremented and created a function that migrated from the old table to the new at program launch (simple function, Python's reflection abilities made that a cinch), so you could quite literally take a very old version of the program and upgrade it to the latest and it would upgrade the database completely (maybe not the fastest way as it incremented each table version from one to the next, but it worked well with no surprises). On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: >> You might look at the Python library SQLAlchemy, it handles all of >> this extremely well (and I have actually linked in Python into some of >> my C++ projects *just* so I could use sqlalchemy, it is that nice to >> use, although a slight bit of an overhead since it is, well, python). > > I looked at SQL alchemy just recently, and I agree it deserves very > well the praise it receives! I would not mind Wt::Dbo being compared > with that library some day :-) It very well could be in time. :) On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: >> I started looking at how DBO was implemented. >> >> In Cession.c:194, the parseSql function. The parsing seems a bit >> heavy. If you do not mind a requirement on a more recent version of > > As you point out,, that check is wrong. It should read: > > std::size_t selectPos = sql.find("select "); > if (selectPos != 0) > throw std::logic_error("Session::query(): query should start with 'select > '" > " (sql='" + sql + "'"); > > Thus even a boost::starts_with() would be more appropriate here. > > The rest of the 'parsing' is indeed very ad-hoc, and especially when > start wanting to understand the queries better, an approach based on > Boost.Spirit will be preferred. A spirit version using a lazy symbol table rule parser would allow you to add future rules should you wish to add new commands later on. On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: >> Boost then I can Spirit'ify it, which can fill the final values with >> what they should be directly, instead of filling up temporaries and >> multiple finds over a string. Also it mentions that it should start >> with select (would be nice to have a way to mutate values without >> needing to fetch them... A counter is a good example of why this >> would be good), but the select test just looks to see if "select " > > True, and there is in fact little reason why we require a "select " to > be there (but we can only bind result values if it is a select query). Might add commands in the future though, so having a starting keyword like 'select' is good for future proofing it anyway. On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: >> appears anywhere in the sql string at all, what if it was in a string >> blob, or someone had a cell name of myselect, that would mess it up >> pretty well. If I wrote a full Spirit.Qi parser then there would be >> no such issue, but that requires a newer version of Boost then what >> your current requirements state. > > I think with what we currently need to parse (and taking into account > the correction of above), there is little need for a more involved > parsing. We currently require boost 1.35 or higher. How would that > change if we want to use spirit for query parsing later on (since Wt > already uses boost.spirit) ? The Spirit in 1.35 is ancient, it is slow, difficult to use, etc... Spirit 2.1 and higher (introduced in 1.41, Spirit2.2 is in 1.42, Spirit2.3 is in the trunk, 2.1 would be fine for us) would be a great deal better as it is vastly easier to use (much less code to write), it generates much faster code, and requires no temporaries as it is capable of parsing into your own structure formats, whatever they may be. As stated a while back, you do not actually need to require the user to get boost anyway, we can use the bcp tool to include the parts of boost that Wt uses, and using CMake you can set it up so that if boost is not detected on the system then we can use the built-in version that just includes what Wt needs (and maybe a library or two extra that is very useful to general programming if you wish), thus you could even remove your boost dependency, but if the user has it they would have access to more things. I can work on that if you are curious. On Mon, Dec 28, 2009 at 2:34 AM, Koen Deforche <[email protected]> wrote: >> Also, SqlConnection:69, the line: >> typedef std::map<std::string, SqlStatement *> StatementMap; >> With potentially a lot of entries (as I plan to do, no point >> recreating often-used statements), would not boost::unordered_map be >> faster? Perhaps even a ternary tree? Boost.Spirit has a wonderful >> ternary tree that would also work quite well (wish it was in the main >> boost namespace, but boost does not have a generic tree container yet >> like it has unordered_map and such). >> >> At the very least, perhaps change it to boost::unordered_map (while >> including boost/unordered_map.hpp)? It provides a speed boost over >> std::map and is identical to tr1::unordered_map (as it was modeled on >> boost's version). I think boost's unordered_map is a using >> declaration of tr1::unordered_map if your compiler has tr1. > > Note that the prepared statement map is maintained behind the scenes > automatically for you (when using Session::query() or Session::find() > methods), It also already takes into account later usage of a session > pool by not assuming that a statement that was already prepared will > be prepared in the next transaction because it may (in general) be > using a different connection. > > Potentially, yes, at least it seems unordered_map will do the job. In > general, I do prefer to do optimizations like these based on profiling > run results... True, we can profile first of course, but in general that is why unordered_map is chosen over map, you losing ordering guarantees, but it allows for a more efficient container. I use a nice statistical profiling library (currently in the Boost sandbox) that is exceedingly accurate to within nano-seconds (it even tests the timing inaccuracies on the processor it is running on first) that I can use if you want me to setup a test framework to compare std::map and std::tr1::unordered_map, would probably only take 30 minutes or so to set it up to test all the standard methods of both (then probably another half hour or so to run the tests...). ------------------------------------------------------------------------------ This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev _______________________________________________ witty-interest mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/witty-interest
