Steve, On Tue, Feb 8, 2011 at 9:33 PM, Steve Pinkham <steve.pink...@gmail.com> wrote: > On 02/08/2011 07:07 PM, Andres Riancho wrote: >> Steve, >> >> On Tue, Feb 8, 2011 at 8:45 PM, Steve Pinkham <steve.pink...@gmail.com> >> wrote: >>> On 02/03/2011 12:04 PM, Andres Riancho wrote: >>>> Do we know about any noSQL database that's "file based" like sqlite? >>>> Maybe we could use this small rewrite to compare the performance of >>>> those backends. >>>> >>>> Regards, >>>> >>> >>> I'm somewhat at a loss of what you think "noSQL" will buy you. It's >>> useful in distributed, massively parallel systems, but offers no real >>> benefit for single user databases. >> >> I disagree. I've seen how sqlite3's performance impacted in the >> framework's performance before mainly because of its slow access >> (SELECT). For what I can understand from the noSQL databases, the >> access to any row should be ultra fast, even if we save whole HTTP >> requests and responses to it. > > noSQL servers are usually fast because they are in-memory systems. > sqlite can be used in that mode also if you like.
mongodb is not an in-memory db! > If select is your problem, you're probably not indexing properly or your > selects are waiting on writes. Both are fixable. Could be, I don't deny that I could have overlooked something related to indexing, cache, locks, etc. > That said, if all you ever want is a key value store and you never see > yourself use any more complicated searches than that, maybe a key value > store is for you. > > Otherwise writing better selects and tuning your indexing is probably a > bigger win. We usually just perform queries like "SELECT * FROM table WHERE id = ?". You may take a look at line 168 of history.py [0] and line 118 of db.py [1]. As far as I can see, we didn't have any "CREATE INDEX" statements in our code, so I added the following statement "CREATE INDEX table_index ON table( primary_key )". I suppose this should increase the SELECT queries a little bit (or a lot?). [0] http://sourceforge.net/apps/trac/w3af/browser/trunk/core/data/db/history.py [1] http://sourceforge.net/apps/trac/w3af/browser/trunk/core/data/db/db.py -- Andrés Riancho Director of Web Security at Rapid7 LLC Founder at Bonsai Information Security Project Leader at w3af ------------------------------------------------------------------------------ The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
