Re: [PHP-DB] Performance (lots of tables / databases...)
Thanks Tim, this helped me a lot. Martin Tim Hawkins napsal(a): If you are looking at future expansion then the separate DB per blog is defiantly the way to go. Some notes: 1) Avoid joins like the plague, in fact most operations on a blog application would consist of getting primary record and then decorating it with secondary data, for example getting a blog post and then getting the comments associated with it. Do two queries for this, dont join them. 2) Consider putting some intelligence into the database selection, have a master table/db that holds the database/server details for each blog, It will allow you to spread your blog application across multiple DB servers, as load goes up you can reassign blogs to different machines. You can memcache the data from the master db on a per blog basis, so you wont take a hit on accessing it, but it gives you great flexibility of redistributing data to different machines/clusters. 3) Use memcache, its a life saver. 4) If user registrations are common across all blogs, have a separate db for the users, again you can "shard" this, use a hashing algorithm to allow sections of the user database to be split across multiple user databases on multiple db servers. Again memcache the hell out of the user lookup, its a fixed id=>db/datarecord mapping so its great for using memcache against as the mapping never changes. 5) Use an external indexer for any search functionality such as sphinx (http://www.sphinxsearch.com/), sphinx can index separate databases and join the indices together to form a single distributed search, it also supports incremental indexing. Dont be tempted to use the mysql query system for searches. On 28 Sep 2008, at 00:22, Jack van Zanen wrote: If it were Oracle I'd go with one database and separate schema for each blog. For Mysql I think I'd go for a database each blog. Jack 2008/9/28 Martin Zvarík <[EMAIL PROTECTED]> Hi, I am working on a blog system and I am currently thinking of what would be the best DB approach. I have read lots about wordpress and other blog's optimizations and DB structure, but I have not found any mention of having separate database for each blog/user. So, my question is, which one is performance better (talking about 1000 blogs): a) 1000 blogs * 5 (let's say we will have tables like comments, post... for each blog) = 5000 tables in one database ... this is Wordpress default b) 1000 databases (for each blog) each having 5 tables c) 5 databases by 1000 tables - in this case, won't this be an issue when SELECTing like this: [db_comments].testblog, [db_posts].testblog ? Is that a controversial topic? :-/ Thanks for ideas, Martin -- PHP Database Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- J.A. van Zanen -- PHP Database Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DB] Performance (lots of tables / databases...)
If you are looking at future expansion then the separate DB per blog is defiantly the way to go. Some notes: 1) Avoid joins like the plague, in fact most operations on a blog application would consist of getting primary record and then decorating it with secondary data, for example getting a blog post and then getting the comments associated with it. Do two queries for this, dont join them. 2) Consider putting some intelligence into the database selection, have a master table/db that holds the database/server details for each blog, It will allow you to spread your blog application across multiple DB servers, as load goes up you can reassign blogs to different machines. You can memcache the data from the master db on a per blog basis, so you wont take a hit on accessing it, but it gives you great flexibility of redistributing data to different machines/ clusters. 3) Use memcache, its a life saver. 4) If user registrations are common across all blogs, have a separate db for the users, again you can "shard" this, use a hashing algorithm to allow sections of the user database to be split across multiple user databases on multiple db servers. Again memcache the hell out of the user lookup, its a fixed id=>db/datarecord mapping so its great for using memcache against as the mapping never changes. 5) Use an external indexer for any search functionality such as sphinx (http://www.sphinxsearch.com/), sphinx can index separate databases and join the indices together to form a single distributed search, it also supports incremental indexing. Dont be tempted to use the mysql query system for searches. On 28 Sep 2008, at 00:22, Jack van Zanen wrote: If it were Oracle I'd go with one database and separate schema for each blog. For Mysql I think I'd go for a database each blog. Jack 2008/9/28 Martin Zvarík <[EMAIL PROTECTED]> Hi, I am working on a blog system and I am currently thinking of what would be the best DB approach. I have read lots about wordpress and other blog's optimizations and DB structure, but I have not found any mention of having separate database for each blog/user. So, my question is, which one is performance better (talking about 1000 blogs): a) 1000 blogs * 5 (let's say we will have tables like comments, post... for each blog) = 5000 tables in one database ... this is Wordpress default b) 1000 databases (for each blog) each having 5 tables c) 5 databases by 1000 tables - in this case, won't this be an issue when SELECTing like this: [db_comments].testblog, [db_posts].testblog ? Is that a controversial topic? :-/ Thanks for ideas, Martin -- PHP Database Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- J.A. van Zanen -- PHP Database Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DB] Performance (lots of tables / databases...)
If it were Oracle I'd go with one database and separate schema for each blog. For Mysql I think I'd go for a database each blog. Jack 2008/9/28 Martin Zvarík <[EMAIL PROTECTED]> > Hi, > I am working on a blog system and I am currently thinking of what would be > the best DB approach. > > I have read lots about wordpress and other blog's optimizations and DB > structure, but I have not found any mention of having separate database for > each blog/user. > > So, my question is, which one is performance better (talking about 1000 > blogs): > > a) 1000 blogs * 5 (let's say we will have tables like comments, post... for > each blog) = 5000 tables in one database > ... this is Wordpress default > > b) 1000 databases (for each blog) each having 5 tables > > c) 5 databases by 1000 tables - in this case, won't this be an issue when > SELECTing like this: [db_comments].testblog, [db_posts].testblog ? > > > Is that a controversial topic? :-/ > > Thanks for ideas, > Martin > > -- > PHP Database Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > > -- J.A. van Zanen
Re: [PHP-DB] Performance (lots of tables / databases...)
Lester Caine: danaketh wrote: Hi, the first choice is probably the best for you. When you think about second solution, it will be a nightmare when you have 1000+ databases and have to administrate them from one central system (if you're about to do it like this). The third solution looks little complicated to me - have one DB for comments, one for items etc. But you can do it also in one database and six tables. Make one table 'blogs' where the blogs names and ids will be stored. Then you can just add one more field 'blog_id' to every table and identify items, categories, whatever on this. However in your situation (1000+ blogs) it may be not the best solution. That alternate option is the easiest to manage, but since you make no mention of WHICH database engine which is best would be affected by that choice. And how you are hosting the site(s) may also limit your options. Properly indexed tables will be fast and a single connection accessing that data will be faster than having to make multiple connections to different databases, and if it only has to manage a small number of table most links would probably be cached. Sorry, it's MySQL. I am talking about ONE server => one connection and then switching between databases. Example: SELECT * FROM [db_comments].testblog, [db_posts].testblog Would this be a performance issue? Thanks for joining, Marti -- PHP Database Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DB] Performance (lots of tables / databases...)
Hello, the solution you mentioned came up on my mind too, but as you also said - it doesn't seem efficient on high load. Why do you think having 1000+ databases would be a nightmare? I think it would be easy to backup, fast to read/write... although I don't know what would that cause to the system - trying to imagine 1000 folders... is it a problem? I supposed that on the third solution is based the optimized wordpress (wordpress.com) - it does seem complicated, but better than having it all in one database. Martin danaketh napsal(a): Hi, the first choice is probably the best for you. When you think about second solution, it will be a nightmare when you have 1000+ databases and have to administrate them from one central system (if you're about to do it like this). The third solution looks little complicated to me - have one DB for comments, one for items etc. But you can do it also in one database and six tables. Make one table 'blogs' where the blogs names and ids will be stored. Then you can just add one more field 'blog_id' to every table and identify items, categories, whatever on this. However in your situation (1000+ blogs) it may be not the best solution. Martin Zvarík napsal(a): Hi, I am working on a blog system and I am currently thinking of what would be the best DB approach. I have read lots about wordpress and other blog's optimizations and DB structure, but I have not found any mention of having separate database for each blog/user. So, my question is, which one is performance better (talking about 1000 blogs): a) 1000 blogs * 5 (let's say we will have tables like comments, post... for each blog) = 5000 tables in one database ... this is Wordpress default b) 1000 databases (for each blog) each having 5 tables c) 5 databases by 1000 tables - in this case, won't this be an issue when SELECTing like this: [db_comments].testblog, [db_posts].testblog ? Is that a controversial topic? :-/ Thanks for ideas, Martin
Re: [PHP-DB] Performance (lots of tables / databases...)
danaketh wrote: Hi, the first choice is probably the best for you. When you think about second solution, it will be a nightmare when you have 1000+ databases and have to administrate them from one central system (if you're about to do it like this). The third solution looks little complicated to me - have one DB for comments, one for items etc. But you can do it also in one database and six tables. Make one table 'blogs' where the blogs names and ids will be stored. Then you can just add one more field 'blog_id' to every table and identify items, categories, whatever on this. However in your situation (1000+ blogs) it may be not the best solution. That alternate option is the easiest to manage, but since you make no mention of WHICH database engine which is best would be affected by that choice. And how you are hosting the site(s) may also limit your options. Properly indexed tables will be fast and a single connection accessing that data will be faster than having to make multiple connections to different databases, and if it only has to manage a small number of table most links would probably be cached. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/lsces/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php -- PHP Database Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DB] Performance (lots of tables / databases...)
Hi, the first choice is probably the best for you. When you think about second solution, it will be a nightmare when you have 1000+ databases and have to administrate them from one central system (if you're about to do it like this). The third solution looks little complicated to me - have one DB for comments, one for items etc. But you can do it also in one database and six tables. Make one table 'blogs' where the blogs names and ids will be stored. Then you can just add one more field 'blog_id' to every table and identify items, categories, whatever on this. However in your situation (1000+ blogs) it may be not the best solution. Martin Zvarík napsal(a): Hi, I am working on a blog system and I am currently thinking of what would be the best DB approach. I have read lots about wordpress and other blog's optimizations and DB structure, but I have not found any mention of having separate database for each blog/user. So, my question is, which one is performance better (talking about 1000 blogs): a) 1000 blogs * 5 (let's say we will have tables like comments, post... for each blog) = 5000 tables in one database ... this is Wordpress default b) 1000 databases (for each blog) each having 5 tables c) 5 databases by 1000 tables - in this case, won't this be an issue when SELECTing like this: [db_comments].testblog, [db_posts].testblog ? Is that a controversial topic? :-/ Thanks for ideas, Martin
[PHP-DB] Performance (lots of tables / databases...)
Hi, I am working on a blog system and I am currently thinking of what would be the best DB approach. I have read lots about wordpress and other blog's optimizations and DB structure, but I have not found any mention of having separate database for each blog/user. So, my question is, which one is performance better (talking about 1000 blogs): a) 1000 blogs * 5 (let's say we will have tables like comments, post... for each blog) = 5000 tables in one database ... this is Wordpress default b) 1000 databases (for each blog) each having 5 tables c) 5 databases by 1000 tables - in this case, won't this be an issue when SELECTing like this: [db_comments].testblog, [db_posts].testblog ? Is that a controversial topic? :-/ Thanks for ideas, Martin -- PHP Database Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
