I'm already looking at 100,000+ records, and I am somewhat nervous as to
how it will perform as the load begins to get a little bit higher. I
suppose I will have to do a little tweaking to make it work properly,
but it seems like this approach would lend itself a little more to
indexing rather than using the 2 layer approach.
Which leads me to my next question - Chris, in your 2 layer approach how
do you handle indexing? If you have a fairly sizable chunk of
YAML/XML/etc data, how do you handle searching it?
Rolan Yang wrote:
Brian Dailey wrote:
This looks like what I'll ultimately be doing. I played around with
some SQL queries and what I ended up with was similar to this -
joining the table for each data value that I wanted to look up (and
since I expect a matching data value, I don't even have to LEFT JOIN
everything, which speeds it up significantly).
Thanks for all of your help and suggestions!
You may eventually reach a point where your data volume becomes
monstrous and the queries start to take an unacceptably long time to
run, or consume too much CPU, or maybe even lock up the tables from
being written to. At that point (or before) you may want to consider
setting up a second mysql machine as as replicated slave - dedicated for
generating reports. Once you have that you can create triggers or run
periodic batch processes to index your data in more optimized tables so
the reports can be generated on the fly or at least more quickly.
~Rolan
_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk
NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com
Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php
--
Thanks!
- Brian Dailey
Software Developer
New York, NY
www.dailytechnology.net
begin:vcard
fn:Brian Dailey
n:Dailey;Brian
org:;Department of Geekery
adr;dom:;;;New York;NY
email;internet:[EMAIL PROTECTED]
title:Senior Software Engineer
tel;home:423 506 0349
x-mozilla-html:FALSE
url:http://dailytechnology.net
version:2.1
end:vcard
_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk
NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com
Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php
