That's the principle I am currently using, but that functionality is extended and packaged in a Perl module. There is not one store per user, data has to be shared between users. Each file is named by a multi-part index with other non-indexed data also inside the file. Files in multiple directories are used. For instance, in one example one directory has 743 files, another 715 and another only 1 file.
The file separation is half the effort for storing the data in this solution. I don't think it's worth extending that into SQLite. I am looking at SQLite as a solution to replace that effort. Thanks Eric. Anil. -----Original Message----- From: Eric Pankoke [mailto:[EMAIL PROTECTED] Sent: Friday, 2 February 2007 1:08 PM To: sqlite-users@sqlite.org Subject: RE: [sqlite] Appropriate uses for SQLite By separate files, do you mean that each user has their own data store? If so, why not do the same thing with SQLite? If each user has a unique name or ID, append that to a generic file name and you have a separate instance for each user. I don't believe there's that much overhead to the system tables that SQLite uses, and you solve your concurrency issues. Eric Pankoke Founder Point Of Light Software http://www.polsoftware.com/ -----Original Message----- From: Anil Gulati -X (agulati - Michael Page at Cisco) [mailto:[EMAIL PROTECTED] Sent: Thursday, February 01, 2007 8:27 PM To: sqlite-users@sqlite.org Subject: RE: [sqlite] Appropriate uses for SQLite That's what I'm talking about! It's good to get the perspective from your setup, Phil. I'm beginning to get the picture. I am starting to think that I should stick to an enhancement of my current system. It's *very* basic, even more basic than SQLite, but corruption seems almost impossible when data is in separate files with no compression, etc. Thanks for taking the time. Anil. -----Original Message----- From: Philip Butler [mailto:[EMAIL PROTECTED] Sent: Friday, 2 February 2007 12:16 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Appropriate uses for SQLite And this is when I'll step back and listen to the experts... Since it is a low-load situation, file/record locking on SQLite seems like it would be acceptable to me. As for data corruption - that's bad -- very bad. However, with automated backups some degree of comfort may be realized. With the db systems that I have designed, I have an automatic process that dumps the db to a text file every 4 hours or so. These are kept for a couple of days. I sleep easy at night knowing this... Phil On Feb 1, 2007, at 7:59 PM, Anil Gulati -X ((agulati - Michael Page at Cisco)) wrote: > Thanks for replying Phil... > > Actually I am not running separate websites - but I have to deploy to > multiple webservers which will all serve the same pages. Each > webserver will have their own copy of the SQLite code, but they need > to load the data from a network file server to share the same data. > > I guess this is why I am asking for feedback: it seems that this case > is a marginal case for SQLite and I am just trying to assess > performance and corruption possibilities in more detail than is > presented on the SQLite web pages. > > The main point that encourages me to try SQLite is that it is > recommended for 99.9% of websites. I believe my traffic is very low > and SQLite should be recommended from that point of view. > > However, although the likelihood of two users simultaneously updating > a particular record is going to be very low I believe it is going to > happen that two users will try to update the database simultaneously. > > I know that SQLite has some file locking features that have even been > improved in v 3.0. So: > - will simultaneous *database* access result in corruption? > - will simultaneous *record* access result in corruption? > - if not, when *can* corruption occur? > > I don't mind making the users wait in the unlikely event of a record > collision, or even drop data once in a blue moon, but corruption is > not acceptable. > > Thanks again. > Anil. > > -----Original Message----- > From: Philip Butler [mailto:[EMAIL PROTECTED] > Sent: Friday, 2 February 2007 11:39 AM > To: sqlite-users@sqlite.org > Subject: Re: [sqlite] Appropriate uses for SQLite > > I am not an expert on SQLite - but if you are running separate > websites from your multiple servers, then why not use 4 instances of > SQLite ?? > That is unless the websites need to share the same database/tables. > > If they do need to share the same database/tables, then PostgreSQL or > MySQL may be the more appropriate choice. They are designed to be > distributed (hence their increased overhead) while SQLite is designed > to be lean-and-mean. > > Just my 2 cents worth... > > Phil > > On Feb 1, 2007, at 7:03 PM, Anil Gulati -X ((agulati - Michael Page at > Cisco)) wrote: > >> Hi SQLite users >> >> Thank you for your attention - I am just hoping for some >> clarification > >> of usability of SQLite. >> Referring to: http://www.sqlite.org/whentouse.html >> - SQLite works well in websites >> - Other RDBMS may work better for Client/Server applications >> - SQLite will work over a network file system, but because of the >> latency associated with most network file systems, performance will >> not be great >> >> I am trying to decide whether I can use SQLite for a website that >> runs > >> on 4 load-balanced servers using networked file storage mounted by >> all > >> servers for common data access. SQLite will have to load its database >> from the network file space. >> >> These servers run multiple web-sites, hence the additional servers, >> but my pages are not high hit-rate. The sites I am planning >> anticipate > >> a maximum of 200 users altogether. Current raw, uncompressed data >> (mostly >> text) is about 2MB growing to around 4MB. The current starter >> database > >> of 1.6MB raw compresses to 963KB. >> >> My concerns are: >> >> 1. Network file system >> How bad is the latency introduced from using a network file system? >> >> 2. Concurrent access >> I can't understand how SQLite is recommended for 99.9% of websites >> but > >> only *high* concurrency is not recommended? I currently use a flat- >> file system which uses a single file per record. If users happen to >> write to the same record simultaneously one of the updates will be >> lost but corruption is highly unlikely, if not impossible. It seems >> that for SQLite the risk for concurrent access is always data >> corruption, which would be unacceptable for me. >> >> The issue is that there may be short periods where multiple users >> will > >> be updating around the same time and I want to make sure that the >> possibility of corruption is extremely low. I am asking for more >> detailed information on the above issues to clarify my decision. >> >> All feedback gratefully received. >> Thanks. >> Anil. ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
