Re: [PHP] Re: how to display images stored in DB
# martin@bugs.unl.edu.ar / 2007-03-03 09:18:08 -0300: We have a system (I didn't work on it, just maintaining it) that has about 1100 images in a directory. I think we aren't seen any problems just because it's on a 64bit system. You should test that assumption. -- How many Vietnam vets does it take to screw in a light bulb? You don't know, man. You don't KNOW. Cause you weren't THERE. http://bash.org/?255991 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
steve wrote: As a newbie, is storing an image in a dB a good thing or a bad thing? I tend to go with depends. We actually store files in a DB in development, as those machines are separate from the grid. Since some are windows, linux, and MacOS, it is far easier to store in a DB than have different code for each development machine depending on their filesystem. Otherwise, in production, neither way really works, so there is the hybrid way I alluded to before. Also, when you hit the 1024 image limit you have to think about directory schema to store the images, as the linux filesystem (and also on other 32 bit systems) will start getting slow, until things like ls will just give you an error. We have a system (I didn't work on it, just maintaining it) that has about 1100 images in a directory. I think we aren't seen any problems just because it's on a 64bit system. -- 21:50:04 up 2 days, 9:07, 0 users, load average: 0.92, 0.37, 0.18 - Lic. Martín Marqués | SELECT 'mmarques' || Centro de Telemática| '@' || 'unl.edu.ar'; Universidad Nacional| DBA, Programador, del Litoral | Administrador - -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
On Fri, 2007-03-02 at 22:53 -0500, markw@mohawksoft.com wrote: On Fri, 2007-03-02 at 17:31 -0500, markw@mohawksoft.com wrote: Your claim is that in ALL cases using a file system to store images is preferable to using a database. As such, you claim that using a dB for storing images is bad practice. That is fairly close to my professional opinion, yes. There is, of course, always the exception to any rule, but in general, I would call storing bitmaps in a database without a damn good reason, a bad practice, yes. Ah, you've changed your tune. Thanks for coming out. I haven't changed any tune what so ever. Uhm, ookaay dky then. I guess it's a good thing that the archives tell the real story... not to mention the readers of the list don't exactly have short memories. Cheers, Rob. Two things: First, I do want to apologize for being kind of a wise ass, I was just trying to be funny, but some of it proably came out as insulting. Anyway, to re-use an analogy. Using a wrench as a hammer is never the correct use of a wrench. Does that mean that I've never used a wrench as a hammer? No of course not. The same goes for the bitmap data in the database, I can't come up with a single valid engineering reason to do it, but there are exceptions that are still not good engineering solutions but given time constraints may be expedient. That being said, a good engineer will acknowledge that it is a sub-optimal implementation and fix it in the next rev. At its core, a lot of web guys don't understand databases or SQL in general. Using a database is different from designing one. Don't take this as a slam or an insult, fields of expertise force us all to specialize. Take me for instance, I know the HTML RFC, I know the PHP internals. (I've written a few extensions including a serializer), but the actual web code is something I'm not good at. I spend more time looking up function references than actually coding. I know a bit about what I'm talking about. SQL databases have some appalling behaviors under a highly active and dynamic data (a pretty low insert/update ratio) environment. As a rule, you should keep what ever does not need to be in a database out of the database. I've stated a good few reasons in the various postings. Take my advice or not, but rather than argue about it, take a good look at the internals of a database and really get to understand it, you'll come to the same conclusions. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
On Sat, 2007-03-03 at 08:22 -0500, markw@mohawksoft.com wrote: On Fri, 2007-03-02 at 22:53 -0500, markw@mohawksoft.com wrote: On Fri, 2007-03-02 at 17:31 -0500, markw@mohawksoft.com wrote: Your claim is that in ALL cases using a file system to store images is preferable to using a database. As such, you claim that using a dB for storing images is bad practice. That is fairly close to my professional opinion, yes. There is, of course, always the exception to any rule, but in general, I would call storing bitmaps in a database without a damn good reason, a bad practice, yes. Ah, you've changed your tune. Thanks for coming out. I haven't changed any tune what so ever. Uhm, ookaay dky then. I guess it's a good thing that the archives tell the real story... not to mention the readers of the list don't exactly have short memories. Two things: First, I do want to apologize for being kind of a wise ass, I was just trying to be funny, but some of it proably came out as insulting. Well then, we'll say no more :) FYI, though perhaps you know this and like me just forget sometimes. Various types of emoticons and parenthesized blurbs can help get across that you are making light of a point or trying to be generally funny. Without such helpful directives, one can only presume you are serious unless the context is completely obvious :B Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `' -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB**
At 10:01 AM -0500 3/1/07, markw@mohawksoft.com wrote: tedd said: Well... it's not just me, but from what I've learned and read over the years. Your claim is that in ALL cases using a file system to store images is preferable to using a database. As such, you claim that using a dB for storing images is bad practice. That is fairly close to my professional opinion, yes. There is, of course, always the exception to any rule, Good, then we are agreed. There are exceptions to every rule. See that was easy. :-) I realize that you obviously think in terms of absolute, but the world isn't like that. Everything, including the 1's and 0's in computing are not black and white, but rather meeting, or not, different thresholds. Bad practice or good practice depends upon the needs and one's perspective of the situation. Take this email communication for example, we say what we want and we make our points, or not. But, HOW we (as programmers) do it doesn't matter much from our perspective because it's basic communication. As long as one doesn't become too rude or present things that are totally wrong, everything (including making grammatical errors ) is considered good practice. Whereas if we were communicating on a writers list, then those types of errors would be consider bad practice. We live in a world of compromise. My claim is that both methods have their advantages and disadvantages, as I said in my opening post on this thread Both techniques have up-sides and down-sides -- and I use both depending upon needs. I would say that the error is that you weight each approach equally, and that one is clearly a bad practice for the various reasons I have posted. No my friend, you are the one placing weight. I'm simply stating that both have advantages and disadvantages and that is true. Much like saying that the rule one-way is always right has exceptions and thus is false by definition. - Part of the problem here is that we are using subjective terms such as bad practice and preferable methodologies. So a bit of objectivity is in order. Such terms can be viewed from different perspectives, such as time taken, cost in equipment, ease of use, and many other such concerns. One can not view the act of using a wrench as a hammer as anything but a bad practice. I'm sure at one time or another, we have all used a wrench as a hammer, but that in no way justifies an opinion that using a wrench as a hammer is appropriate. It may be necessary or convenient at some point in time, but not something one designs as a solution. We can argue semantics if you want, but a tool is still just a tool -- it's there to be used as we need. There are ways to optimize tool's use and one can define good or bad in those terms. But, those terms would are subject to the artificial perspective you are placing on their use. For example, if you were being attacked, couldn't find a gun, and used a wrench instead to defend yourself I think that would show good practice of a wrench's use -- would it not? However, I realize that what's preferable and bad practice for you may differ for me and therein provides an opportunity for you to pass on *proving* that none of the following fits within in your considerations and restrictions. But, that's Okay -- you can keep your opinion. I am only offering what I believe to be relevant because you said you would honestly consider it. Hiding behind opinion is too easy. There are things that are better than others. It's not a question of hiding behind opinion, but rather defining the rules. If you use subjective terms then by definition they are subjective and not absolute. Yes, a hammer is better than a wrench for driving nails, but the rule you are using is tool design. The hammer is better at driving nails because it was designed for driving nails -- this is a begging the question type of rule. However, if you change the rule, then the term good or bad changes as well. It's a matter of semantics. Quality sucks in the software world largely because proper use of technologies is not understood. You see, those are opinions. You think that the quality of software sucks, whatever that means. But I take it that it doesn't preform to your expectations -- that's OK, but it's your expectations. You also think that the reasons for this are that the use of technologies is not understood -- again, that's Okay, but it's still your opinion. Side note: Will Rogers once said We are all ignorant, only in different subjects. If you were to talk to the average computer user; or to the average investor who invests in technologies; or to the businessman who is paying the bills for programmers to build the product while trying to make money at the same time; they all might have different opinions as to the quality and state of software. Again, that would be their opinion -- and -- they would be correct from
Re: [PHP] Re: how to display images stored in DB
Also, when you hit the 1024 image limit you have to think about directory schema to store the images, as the linux filesystem (and also on other 32 bit systems) will start getting slow, until things like ls will just give you an error. We have a system (I didn't work on it, just maintaining it) that has about 1100 images in a directory. I think we aren't seen any problems just because it's on a 64bit system. What filesystem has a 1024 image (file) limit? Also, on ext3 you can add the dir_index option to a volume to use a btree index on directories (which, funny enough, is the base for some databases). Not that ext3 is great, mind you, though it is well tested. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Thu, 2007-03-01 at 19:42 -0500, markw@mohawksoft.com wrote: On Thu, 2007-03-01 at 21:08 +0100, Roman Neuhauser wrote: # [EMAIL PROTECTED] / 2007-03-01 12:46:09 -0500: At 10:01 AM -0500 3/1/07, markw@mohawksoft.com wrote: In this discussion I have stated reasons why it is a bad idea. No one has come up with a counter point which can only be served by a database and thus proves me wrong. I think that says something. Contradiction is not a sign of falsity, nor the lack of contradiction a sign of truth. I think no comment says that discussing this issue has problems. For example, this has been subject of many flame wars before and I'm sure that many just don't care to join in. If you want to claim something absurd, then who are they to correct you? And why should they care? Exactly, ted. markw is so obviously right, and he's presented the points so well, there's nothing to add, really. but since you asked: yeah, he's right. I'm going to skip his response to my previous comments and just add the following to this post: To follow up with Ted, nobody said using the filesystem is bad, No, it is the most efficient way. Prove it! Don't just claim it. what many of us are saying is that the database is not necessarily bad either. To make this claim, you need a rational argument to support it. Faith Based engineering is a por substitute. It really depends on what you're doing and how you choose to address the problem with all of your knowledge. Assuming your knowledge never expands when confronted with a problem which requires learning more. Many of us here are quite aware of the different technologies available to access shared binary data across some kind of network etc. But, given time constraints, budget constraints, and all manner of personal preference and training and ingenuity, we CHOOSE to use one solution over the other for any given problem space. The good enough fallacy. So far Mark has almost entirely focused on the performance and filesystems were made to do that argument... Um, yea. Who the hell cares??! Ahh, famous last words. You only need to get bitten once and you will change your tune very quickly. Oh, those aren't my last words, and I'm hardly famous. And why would I change my tune quickly? And wouldn't that depend on what bites me? I haven't heard anything from you yet that suggests you are right and I am wrong. I've heard anecdotal arguments in favour of the filesystem for some cases, but nothing that proves it is the flat out winner in all situations. If people stopped trying to use old ideas to solve novel problems then innovation and ingenuity would go out the window... I would argue that if people fail to research and devise better solutons, then innovation doesn't happen. Many new ideas come from people not versed in previous solutions. This happens because they don't get stuck in the mindset taught to them by the establishment. I think you're a bit stuck in a rut with horse blinders covering your face... nowhere to go but filesystem because you're to closed minded to appreciate novel solutions using databases *smirk*. Personally, I can appreciate both solutions. And I'll just reiterate, nobody is arguing databases are always better. But they are sometimes. and if that happens, then nothing advances, nothing is doscovered, we live a life of boring old filesystems that just do files. So, files are bad? No, but they may not be the best solution. Who knows, humans are a very young species and our knowledge has only begun. Why CAN'T a database be used as a filesystem? Because it is not well suited for the task for which you are trying to use it. Just because something isn't generally well suited doesn't mean it can't be used. You seem to think that everyone needs to follow your way. Wrong, we'll follow whichever way we think is better given our own experience. Mark said himself that filesystems pretty much are databases.. I said nothing of the sort. I forget the exact words, but yes there are conceptual similarities, but that's the limit of what I have said. And I quote: Rob said: In fact many newer bleeding edge filesystems are practically database implementations. Mark said: Not even bleeding edge. Think of the UNIX file systems, many use file names merely as indexes to inodes which point to files. Very database-esque. database-esque -- straight from the horses mouth :B why limit them to just doing what they do now? Why can't they do more? Because a database *is* designed to do more than a filesystem. Databases have a LOT of features and requirements that file systems do not have and things like bitmap binary objects don't need. They are fundamentally different things. Why can't they become more like fully fledged queryable databases? See above. What precisely
RE: [PHP] Re: how to display images stored in DB
I highly recommend a dark ale or two to bring down the core temperature :) Amen! -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
At 5:24 PM -0600 3/1/07, Richard Lynch wrote: On Wed, February 28, 2007 6:41 pm, Robert Cummings wrote: What about when you need to share those files across a 50 node network? I'd keep it in a database, then when I need it cache a local copy on the filesystem. Then I can just check the timestamp in the database to see if the file has changed. Voila, multi-node high availability images. Seems better than have a local copy of every single image. I guess the answer is... it depends on what you're doing! Add an image server (or 20) and change the HTML to point to the image server. Use rsync on the 20 image client servers from the master image server. Lots of ways to skin this cat. But the cat ain't going to like any of them. tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Re: how to display images stored in DB
I highly recommend a dark ale or two to bring down the core temperature :) A good Zinfendel, sit back and relax. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Fri, 2007-03-02 at 11:01 -0500, markw@mohawksoft.com wrote: I'm going to mostly skip your drivel... and cut straight to a short comment and an example proving the invalidity of your general argument that the filesystem is always a better place to store image content. There we go... you only think you are right. Thanks for clearing that up. When you have proof of being right I'll start taking notes. At this time I'd like to bring into play a line originally introduced to me by Mr. Tedd Sperling: Locus ab auctoritate est infirmissimus. Thanks tedd, I like that line. When you have something substantial to back up your claim, maybe proof as opposed to argument, I'll be all ears. Your education, your immersion in the field, even your hands on experience do not make you right. They increase the likelihood that you know your subject matter, but in absence of tangible evidence they are only theories. I have posted a good number of reasons why the position presented is correct. You have posted zero refutations to the reasons. It may not be irrefutable in your eyes, but funny how they have not been refuted. You have posted a good number of reasons why it is sometimes better, even more often than not better, you have not posted a single reason the indicates why it is ALWAYS better. Why'd you skip the second half of my post where I gave contradictory examples? Let me spell another out for you, I'll even do it in your language: User requests a page that requires the assimilation of N icons into a single larger image as uploaded previous by the user of the current session (n can be any number of uploaded icons -- all icons just so happen to fall under 2k in size): Database Storage: Get http request parse request check permissions init PHP request check session validity create sql request parse query execute query open file (if table not currently open) lock file for read grab N items of data based on user ID index field return data (local socket connection) unlock read lock format return data (in memory) create aggregate image based on image dims (in table fields) for i = 1 to n copy sub-image into aggregate image endloop set headers for browser send data Filesystem: Get http request parse request check permissions init PHP request check session validity open directory for user ID image path (not sure but I think you need a read lock) read directory (maybe release lock) close directory for i = 1 to n open image file (not sure but I think you need a read lock) read in image content (maybe release lock) close image file temporarily store image content endloop create aggregate image based on image dims for i = 1 to n copy sub-image into aggregate image endloop set headers for browser send data Now which one did you say was faster? Which did you say was more convenient? Which is more understandable by future developers? Which is more scalable 00 maybe my images aren't on just this one server maybe they're on 10 servers? By the way, maybe I also want descriptions for each image imported as part of the aggregated image. So I need to do a query for those anyways, or maybe you'd use files for that too -- after all don't files store data? Anyways, you are plainly wrong in the general case of filesystems being better than databases. They may be better often, but not always. Understandably you will argue against this with some artificial argument such as it's an artifical example, but really, who cares? You said never, so artificial examples count. You'll probably still argue (that's your closed mind getting in the way again), but I'll probably just ignore you now since I don't think you have anything to add that you haven't already stated, and you still haven't been able to prove the argument in general. I'll tell you what, I'll give you one last chance to prove it. Take the above example, implement your filesystem method for 1 icons and I'll do the same using a database. Then we'll benchmark them. And yes, I do want associated descriptions embedded in the aggregate image. If you are so right, then this should be trivial. Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating
Re: [PHP] Re: how to display images stored in DB*
At 10:01 AM -0500 3/1/07, markw@mohawksoft.com wrote: If you are open to honestly consider, then I shall provide a couple of examples. But if I do and you do not agree, then your only recourse will be to *prove* otherwise. So, what say you? Absolutely, I'm all about computer science. Give me some real world examples and I'll show you the flaws. -snip- What do *you* say? Mark: Well... it's not just me, but from what I've learned and read over the years. Your claim is that in ALL cases using a file system to store images is preferable to using a database. As such, you claim that using a dB for storing images is bad practice. My claim is that both methods have their advantages and disadvantages, as I said in my opening post on this thread Both techniques have up-sides and down-sides -- and I use both depending upon needs. Part of the problem here is that we are using subjective terms such as bad practice and preferable methodologies. So a bit of objectivity is in order. Such terms can be viewed from different perspectives, such as time taken, cost in equipment, ease of use, and many other such concerns. However, I realize that what's preferable and bad practice for you may differ for me and therein provides an opportunity for you to pass on *proving* that none of the following fits within in your considerations and restrictions. But, that's Okay -- you can keep your opinion. I am only offering what I believe to be relevant because you said you would honestly consider it. The following are the things I consider important for this discussion: Consideration 1 Storing images in a database bloats the database and you're more likely to approach memory limits of your operating system. However, storing images the file system, likewise may cause lookups to become slower on some operating systems than using a dB (Granted this is a weak consideration, but I read it, so it must be true.) :-) Consideration 2 If your project is to supply images across several web servers on different host, then you are caught in a sync problem. Images stored in a file system must be stored on a local host. In a multiple host situation you must have a duplicate set of images on each host. Synchronizing images across different files systems on different host is far more complicated and problematic than maintaining a single database from which all host can draw the most current images. Consideration 3 Storing images images in a file system and their location in a database is a two step process in all image operations. If your project requires a transactions, then it's obviously more difficult to accomplish for not only do you have to do two operations, but they must take place in two domains. Whereas, if you store images in a database it is a simple matter to add, alter, and delete because it's one operation and thus no need to confirm that the file system and database remain in sync. Consideration 4 Even without transactions, it's simpler to have all data in a single dB than it is two divide the data into two sets and then maintain a sync. Consideration 5 Backups. It is far easier, faster, more reliable, and less prone to error to perform a single mysql dump of all images than it is to preform a mysql dump of all image locations AND duplicate all image files. I even had a client who demanded that his image files be localized in such a manner. That's a real world example, where's the flaw in providing what the customer wants? Consideration 6 While this should be avoided through good coding practice, but if you screw-up a path, things can get chaotic real fast. Hard coding partial paths in your code and pulling the rest of the path out of the dB is prone to more error than just accessing the image directly from the database. Granted, coders should be aware that all of the path should be taken from the dB and not hard code any path information, but that practice isn't always followed. Unfortunately, we don't always work with just our code and preferences. Consideration 7 The future -- while programming styles and types differ, the trend is toward encapsulating (i.e., keeping all data, methods and functions together within an organization and hidden). From OOP to unobtrusive code, clearly this is the direction of today. My personal preference is to keep all my relative data together as well. If my data, along with other attributes, incorporates images then, I like the idea (but, don't always practice it myself) of having everything in one place and storing images along with other associated data in a database makes sense to me. To me, images are nothing more than large hunks of data. Why tell fat people to sit somewhere else? I certainly can understand and realize that images, because of their size (and size alone), can be problematic with respect to time and memory, however, both of those physical restraints are becoming less of a
Re: [PHP] Re: how to display images stored in DB
Not in my environment. All db servers have RAID 10 over 8 SCSI 15K disks. Pulling from them is always faster than a webserver pulling from its SATA drive. That's not exactly an apples to apples comparison. That is true, and sort of the point. However, the whole thing started because someone was asking a trivial question. The assumption there is that they are on a hosted environment, likely with the the web server and the database on the same server. Chances are the filesystem will be a better bet for them for speed. But on such a system, the speed difference may not be much of a concern. Where I am, I am forced to think about different things, so my viewpoint may be improperly colored. Its completely random that I'm posting to this list at all... Also, there is issue of going to the database and going to the filesystem. Timing wise, you Please explain. I'm making assumptions about the layout again. The database will likely already have the table files opened, but will need a seek to get the data. The webserver will have to do an additional seek (I was assuming on a far slower drive system, and likely twice for stat and read). Putting the files in the DB doesn't always remove extra cleanup/management stuff. Sometimes the database stores binary objects as files and sometimes they don't get deleted correctly. I'm dying to know what database you are using that puts every blob in a separate file these days. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
snip... For the record, I will *never* say one size fits all in the realm of computer programming... Consideration 2 If your project is to supply images across several web servers on different host, then you are caught in a sync problem. Images stored in a file system must be stored on a local host. In a multiple host situation you must have a duplicate set of images on each host. Synchronizing images across different files systems on different host is far more complicated and problematic than maintaining a single database from which all host can draw the most current images. What does the DB approach offer that a centralized image server could not? Still making network requests for some resource (in this case an image) Consideration 3 Storing images images in a file system and their location in a database is a two step process in all image operations. If your project requires a transactions, then it's obviously more difficult to accomplish for not only do you have to do two operations, but they must take place in two domains. Whereas, if you store images in a database it is a simple matter to add, alter, and delete because it's one operation and thus no need to confirm that the file system and database remain in sync. Very true, transactions and rollback capabilities would keep things in sync. Consideration 5 Backups. It is far easier, faster, more reliable, and less prone to error to perform a single mysql dump of all images than it is to preform a mysql dump of all image locations AND duplicate all image files. I even had a client who demanded that his image files be localized in such a manner. That's a real world example, where's the flaw in providing what the customer wants? Again, thinking of a RAID Array for a centralized image server, with potentially N hot standbys doing occasional rsyncs. So, this consideration is not quite as strong as others... And we all know customers can be idiots, so drawing conclusions on what they deem important is not a strong consideration either. Consideration 6 While this should be avoided through good coding practice, but if you screw-up a path, things can get chaotic real fast. Hard coding partial paths in your code and pulling the rest of the path out of the dB is prone to more error than just accessing the image directly from the database. Granted, coders should be aware that all of the path should be taken from the dB and not hard code any path information, but that practice isn't always followed. Unfortunately, we don't always work with just our code and preferences. Hmm... coder stupidity...can happen with any system you implement. So it's a draw on this one. Consideration 7 The future -- while programming styles and types differ, the trend is toward encapsulating (i.e., keeping all data, methods and functions together within an organization and hidden). From OOP to unobtrusive code, clearly this is the direction of today. Yeah, the future... I guess memory is getting cheaper every day. But there is still a capacity for a given machine. So, how easy would it be to scale DB servers across multiple machines as opposed to having a centralized DB server pointing to image locations potentially serviced by N image servers? It is without doubt, in my mind, that the ease that dB's offer in localizing data will prevail over distributive methodologies which are primarily based upon time and money concerns -- for time and money are lessening concerns. Yep, but how many companies do you know of *today* that would scrap $5 million in computers to upgrade them all at a cost $x as opposed to just spending another $25K to expand their current base? So as long as the best bang for the buck today line of thinking holds...distributed systems will remain dominant, especially in the area of data storage. Overall this has been a cool and interesting thread. For the most part, I would say I side with Mark, but again the fatal flaw appears to be the insistence that one approach is right in 100% of the time...which 99% of the time is false. -B -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
At 10:28 AM -0800 3/2/07, steve wrote: Putting the files in the DB doesn't always remove extra cleanup/management stuff. Sometimes the database stores binary objects as files and sometimes they don't get deleted correctly. I'm dying to know what database you are using that puts every blob in a separate file these days. People often look at 0's and 1's differently depending upon how they got there. There are some valid reasons to look at architecture, but it's still just 1's and 0's. But, all dB's basically use a read/write system that mimics the file system in some fashion -- so at some level the two are undistinguished. Cheers, tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
On Fri, 2007-03-02 at 13:49 -0500, Brad Bonkoski wrote: Overall this has been a cool and interesting thread. For the most part, I would say I side with Mark, but again the fatal flaw appears to be the insistence that one approach is right in 100% of the time...which 99% of the time is false. This is exactly the crux of my objection to Mark's posts. Not that the filesystem isn't usually a superior choice, but his claim that it is always the superior choice :/ Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `' -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Thursday 01 March 2007 22:24, Richard Lynch wrote: I have reflected upon this thread for awhile now. I believe that the only new thing I have to add is for newbies. I believe that for a newbie, it would be easier to use the filesystem rather than the DB. THAT on the other hand is BS! not to jerk around with you, but no and HELL no. Storing in the database was the easy way out, I started using the filesystem cuz of both a limitation in a forum AND a few mails from you. But easier... NO. True, you then have to do some extra cleanup/management work for deleted records, so that the related images go away. This is just deleting records... a record with a blob is just as easy/hard with/without a blob. But storing them in the DB invariably ends up with too many issues involving DB storage size and query buffer size, compounded by data escaping/security issues. This on the other hand... I'm a newbie, whatever your saying, BUT and it is a big one; The data has to be escaped anyway, the binary object has to be checked for security breaches with or without the database, so this is reason is by all means wrongly put. None of these are easy for a newbie to solve, and all too often, due to environmental limitations (read: shared hosting) it is impossible to fix the query buffer size limitation. If you know what you are doing, and have thought out the pros and cons, by all means, go ahead and put your images into the DB. If you have no idea what to do, choose the file system to get started, and then do more research when you have spare time. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- --- Børge Kennel Arivene http://www.arivene.net --- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Friday 02 March 2007 14:48, tedd wrote: At 5:24 PM -0600 3/1/07, Richard Lynch wrote: On Wed, February 28, 2007 6:41 pm, Robert Cummings wrote: What about when you need to share those files across a 50 node network? I'd keep it in a database, then when I need it cache a local copy on the filesystem. Then I can just check the timestamp in the database to see if the file has changed. Voila, multi-node high availability images. Seems better than have a local copy of every single image. I guess the answer is... it depends on what you're doing! Add an image server (or 20) and change the HTML to point to the image server. Use rsync on the 20 image client servers from the master image server. Lots of ways to skin this cat. But the cat ain't going to like any of them. I don't particulary like cats... tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- --- Børge Kennel Arivene http://www.arivene.net --- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
At 1:49 PM -0500 3/2/07, Brad Bonkoski wrote: Overall this has been a cool and interesting thread. For the most part, I would say I side with Mark, but again the fatal flaw appears to be the insistence that one approach is right in 100% of the time...which 99% of the time is false. -B Brad: You can't have it both ways -- either some of what I say is true, or none of it is true. If it's not true, then you can side with Mark. But, if it is true then you cannot side with Mark. That's the reason for this discussion, not to see where the majority of opinions falls. I seldom use dB's to store images either, but on occasion it is preferable and is certainly not bad practice to do so, as Mark claims. Cheers, tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
At 7:52 PM +0100 3/2/07, Børge Holen wrote: On Thursday 01 March 2007 22:24, Richard Lynch wrote: I believe that for a newbie, it would be easier to use the filesystem rather than the DB. THAT on the other hand is BS! not to jerk around with you, but no and HELL no. Storing in the database was the easy way out, I started using the filesystem cuz of both a limitation in a forum AND a few mails from you. But easier... NO. Børge: I'm lost. As a newbie, is storing an image in a dB a good thing or a bad thing? tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Friday 02 March 2007 21:13, tedd wrote: At 7:52 PM +0100 3/2/07, Børge Holen wrote: On Thursday 01 March 2007 22:24, Richard Lynch wrote: I believe that for a newbie, it would be easier to use the filesystem rather than the DB. THAT on the other hand is BS! not to jerk around with you, but no and HELL no. Storing in the database was the easy way out, I started using the filesystem cuz of both a limitation in a forum AND a few mails from you. But easier... NO. Børge: I'm lost. Yes, I was refering to that the reason Richard gave; that it is easier to use the filesystem rather than the database to store files. As a newbie, is storing an image in a dB a good thing or a bad thing? I would say, depends on yer project. In my case; it lean out towards 4000 records, close to 3000 x 2 (cuz of thumbnails) images (be that of any size beneath 640x480). Some of Richards earlier statements made me do some rather large forum applications, wheras I made the images go to the filesystem. Both because I didn't know how, and wanted each record inserted to hold up 20 images in one submit, this was the clearly logical solution. However... 6000 small files (and a sub if obliged) I cannot see one heck of a good reason NOT to hold storage in a database... Imagine the rotten backup cyclus. THAT particlulary mysqldump really make me feel good, each time the cron does an automated backup!!! As for answering yer question in a short matter. Good thing... Keeping track of authenticated users and watermarked images and keeping regular users at bay, how can we NOT take adventage of this magnificent tool? Yes I see how hardcore users working for yahoogoogleland and make a dumperload of money at each project delivery may choose well proved techiques... BUT HELL, I do help people who asks, last year I got a bottle of Pepsi. No complaints so far ;D, and btw: I like it fashionable, pretty, swell and fenomenal not just adequate. tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- --- Børge Kennel Arivene http://www.arivene.net --- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
But storing them in the DB invariably ends up with too many issues involving DB storage size and query buffer size, compounded by data escaping/security issues. This on the other hand... I'm a newbie, whatever your saying, BUT and it is a big one; The data has to be escaped anyway, the binary object has to be checked for security breaches with or without the database, so this is reason is by all means wrongly put. Depends. Use a prepared statement and mysqli for example, and there is no escaping. You should still do a virus scan first, however. Doesn't matter how you store things. As a newbie, is storing an image in a dB a good thing or a bad thing? I tend to go with depends. We actually store files in a DB in development, as those machines are separate from the grid. Since some are windows, linux, and MacOS, it is far easier to store in a DB than have different code for each development machine depending on their filesystem. Otherwise, in production, neither way really works, so there is the hybrid way I alluded to before. If you are on a hosted server, you may have different filesystem and database limits for storage. That could make the decision for you. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
At 10:01 AM -0500 3/1/07, markw@mohawksoft.com wrote: If you are open to honestly consider, then I shall provide a couple of examples. But if I do and you do not agree, then your only recourse will be to *prove* otherwise. So, what say you? Absolutely, I'm all about computer science. Give me some real world examples and I'll show you the flaws. -snip- What do *you* say? Mark: Well... it's not just me, but from what I've learned and read over the years. Your claim is that in ALL cases using a file system to store images is preferable to using a database. As such, you claim that using a dB for storing images is bad practice. That is fairly close to my professional opinion, yes. There is, of course, always the exception to any rule, but in general, I would call storing bitmaps in a database without a damn good reason, a bad practice, yes. My claim is that both methods have their advantages and disadvantages, as I said in my opening post on this thread Both techniques have up-sides and down-sides -- and I use both depending upon needs. I would say that the error is that you weight each approach equally, and that one is clearly a bad practice for the various reasons I have posted. Part of the problem here is that we are using subjective terms such as bad practice and preferable methodologies. So a bit of objectivity is in order. Such terms can be viewed from different perspectives, such as time taken, cost in equipment, ease of use, and many other such concerns. One can not view the act of using a wrench as a hammer as anything but a bad practice. I'm sure at one time or another, we have all used a wrench as a hammer, but that in no way justifies an opinion that using a wrench as a hammer is appropriate. It may be necessary or convenient at some point in time, but not something one designs as a solution. However, I realize that what's preferable and bad practice for you may differ for me and therein provides an opportunity for you to pass on *proving* that none of the following fits within in your considerations and restrictions. But, that's Okay -- you can keep your opinion. I am only offering what I believe to be relevant because you said you would honestly consider it. Hiding behind opinion is too easy. There are things that are better than others. Quality sucks in the software world largely because proper use of technologies is not understood. The following are the things I consider important for this discussion: Consideration 1 Storing images in a database bloats the database and you're more likely to approach memory limits of your operating system. However, storing images the file system, likewise may cause lookups to become slower on some operating systems than using a dB (Granted this is a weak consideration, but I read it, so it must be true.) :-) this is far too weak and unspecific to really address. Bloat is a subjective title. And lookups to become slower either encompasses failures of specific file systems or approach. Consideration 2 If your project is to supply images across several web servers on different host, then you are caught in a sync problem. Images stored in a file system must be stored on a local host. In a multiple host situation you must have a duplicate set of images on each host. Synchronizing images across different files systems on different host is far more complicated and problematic than maintaining a single database from which all host can draw the most current images. This is a complete fallacy. It fails on a few assumptions. (1) If you assume that you have one database supply all the images to all your servers, then you merely need to expose an images.domain.com URL that contains all your images. That will be faster and more efficient than a single database. (2) Again, assuming one machine supply all data, a back-end apache server serving images from one machine through a validated PHP pass through is still more efficient than a database. (3) Assuming that you want to distribute database images, there are many tools to accomplish this, not the least of which is rsync. Consideration 3 Storing images images in a file system and their location in a database is a two step process in all image operations. If your project requires a transactions, then it's obviously more difficult to accomplish for not only do you have to do two operations, but they must take place in two domains. Whereas, if you store images in a database it is a simple matter to add, alter, and delete because it's one operation and thus no need to confirm that the file system and database remain in sync. This is a purely hypothetical and easily solved problem, if you can even create a scenario that would cause it. First of all, assuming a requirement of ACID transactions, one create the bitmap file name using something like a UID, save the bitmap to the disk using fsync() to ensure it is committed to disk, then update
Re: [PHP] Re: how to display images stored in DB
Not in my environment. All db servers have RAID 10 over 8 SCSI 15K disks. Pulling from them is always faster than a webserver pulling from its SATA drive. That's not exactly an apples to apples comparison. That is true, and sort of the point. Not really, anything that you provide for the database, could equally be used for the image server. However, the whole thing started because someone was asking a trivial question. The assumption there is that they are on a hosted environment, likely with the the web server and the database on the same server. Chances are the filesystem will be a better bet for them for speed. But on such a system, the speed difference may not be much of a concern. Where I am, I am forced to think about different things, so my viewpoint may be improperly colored. Its completely random that I'm posting to this list at all... Also, there is issue of going to the database and going to the filesystem. Timing wise, you Please explain. I'm making assumptions about the layout again. The database will likely already have the table files opened, but will need a seek to get the data. The webserver will have to do an additional seek (I was assuming on a far slower drive system, and likely twice for stat and read). I'm not sure I trust this assessment.If it is an often used image it will be in OS level cache. If it is not, then you will have the I/O. Regardless, it will be using SQL database cache blocks and reduce the efficacy of the SQL cache. Putting the files in the DB doesn't always remove extra cleanup/management stuff. Sometimes the database stores binary objects as files and sometimes they don't get deleted correctly. I'm dying to know what database you are using that puts every blob in a separate file these days. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
It is funny, but most people don't get the fact that SQL databases are not the best way to store data. They are designed to select algebraic relationships from a data set. They are designed to ensure accuracy of the relationships and the integrity of the data. Like blobs in separate files, this is a very old perspective. Not to say incorrect, but they have evolved into more analytical information storage servers in the last 15 years. Hmm that sounded like marketing speak. !! I better get back to work... bye all! -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
I'm making assumptions about the layout again. The database will likely already have the table files opened, but will need a seek to get the data. The webserver will have to do an additional seek (I was assuming on a far slower drive system, and likely twice for stat and read). I'm not sure I trust this assessment.If it is an often used image it will be in OS level cache. If it is not, then you will have the I/O. Regardless, it will be using SQL database cache blocks and reduce the efficacy of the SQL cache. Remember, I don't use a db to store images, for a variety of reasons, mostly in that centralized repositories of anything don't scale. However, if your hardware is far more than than you userbase will need, the SQL server's caching won't matter, and it will be faster. Not that it matters, since it would be a more expensive setup than necessary. Ease of programming wise, or in the one case where we use it: ease of developer setups, it works better. Those are really just configuration issues and a syncing issues across platforms. Storing everything on one machine does not scale. And as to the slashdot effect -- snore. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
# [EMAIL PROTECTED] / 2007-03-02 21:37:48 +0100: However... 6000 small files (and a sub if obliged) I cannot see one heck of a good reason NOT to hold storage in a database... Imagine the rotten backup cyclus. I cannot imagine it. What was the problem? -- How many Vietnam vets does it take to screw in a light bulb? You don't know, man. You don't KNOW. Cause you weren't THERE. http://bash.org/?255991 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Saturday 03 March 2007 00:00, Roman Neuhauser wrote: # [EMAIL PROTECTED] / 2007-03-02 21:37:48 +0100: However... 6000 small files (and a sub if obliged) I cannot see one heck of a good reason NOT to hold storage in a database... Imagine the rotten backup cyclus. I cannot imagine it. What was the problem? No problem... Just ugliness... and some afterthought. The complete dataset spread out over more than 6000 files instead of one. Using tar and bunzip2 on the amount of files and at the same time.. also do a sqldump of all the rest of the information... glue together and send it off to backup storage... Just seems so someone put in a word. If something happens now, I just rollback, no fuzz. Backup is scheduled once a day and gives me all the safety needed to keep it live'n kicking. Last time this case was up for discussion, I got the lecture from Richard, whereas he gave me quite a few good pointers on the file storage subject, witch I've implemented in my present and future project (Thanks Richard), but see absolutely no reason to rollback over the previously mentioned project with the images. It just doesn't fit the task at hand. Use the tool fitted for the task and databases is _A_ tool, and a good one at that, but even I can see many ways to missuse it, and the reason is simple; The first thing one find if searching for many php file storage topics is easy and even easier solutions to store images in a blob rather than the filesystem witch actually requires some thinking. So one should probably STRONGLY suggest otherwise if someone asks, but one should not ban it, cuz is has its moments -- --- Børge Kennel Arivene http://www.arivene.net --- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
I'm making assumptions about the layout again. The database will likely already have the table files opened, but will need a seek to get the data. The webserver will have to do an additional seek (I was assuming on a far slower drive system, and likely twice for stat and read). I'm not sure I trust this assessment.If it is an often used image it will be in OS level cache. If it is not, then you will have the I/O. Regardless, it will be using SQL database cache blocks and reduce the efficacy of the SQL cache. Remember, I don't use a db to store images, for a variety of reasons, mostly in that centralized repositories of anything don't scale. Finally someone has something that sounds like a well constructed thought. However, if your hardware is far more than than you userbase will need, the SQL server's caching won't matter, and it will be faster. The good enough fallacy again. Not that it matters, since it would be a more expensive setup than necessary. Using hardware and resources more effectively is seldom more expensive. Ease of programming wise, or in the one case where we use it: ease of developer setups, it works better. I've never seen it in practice. A trivial procedural rule during development is not a big deal. Those are really just configuration issues and a syncing issues across platforms. Storing everything on one machine does not scale. And as to the slashdot effect -- snore. It is a handy short hand for heavy access, and it is a big concern if you have a popular site or hope to have one. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
It is funny, but most people don't get the fact that SQL databases are not the best way to store data. They are designed to select algebraic relationships from a data set. They are designed to ensure accuracy of the relationships and the integrity of the data. Like blobs in separate files, this is a very old perspective. Not as old as you may think. Most databases use fixed block sizes. This, obviously, allows an efficient buffer caching system. Be it MySQL, PostgreSQL, SQLite3, etc. BLOBS don't fit into the blocks well. So, some method must be used to store this data in a way that does not mess up the nice buffering system. PostgreSQL uses TOAST, which is kind of cool. It stores large data in a separate table file. Even allows compression. MySQL MyISAM, if I recall correctly, stores the binary data in a separate unique block for blobs. Oracle still allows you to store BLOBs outside of the tablespace. Not to say incorrect, but they have evolved into more analytical information storage servers in the last 15 years. People who think the past have no importance in the present are mistaken. My information was not old but your interpretation was ignorant. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
On Fri, 2007-03-02 at 17:31 -0500, markw@mohawksoft.com wrote: Your claim is that in ALL cases using a file system to store images is preferable to using a database. As such, you claim that using a dB for storing images is bad practice. That is fairly close to my professional opinion, yes. There is, of course, always the exception to any rule, but in general, I would call storing bitmaps in a database without a damn good reason, a bad practice, yes. Ah, you've changed your tune. Thanks for coming out. Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `' -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
On Fri, 2007-03-02 at 17:31 -0500, markw@mohawksoft.com wrote: Your claim is that in ALL cases using a file system to store images is preferable to using a database. As such, you claim that using a dB for storing images is bad practice. That is fairly close to my professional opinion, yes. There is, of course, always the exception to any rule, but in general, I would call storing bitmaps in a database without a damn good reason, a bad practice, yes. Ah, you've changed your tune. Thanks for coming out. I haven't changed any tune what so ever. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB*
On Fri, 2007-03-02 at 22:53 -0500, markw@mohawksoft.com wrote: On Fri, 2007-03-02 at 17:31 -0500, markw@mohawksoft.com wrote: Your claim is that in ALL cases using a file system to store images is preferable to using a database. As such, you claim that using a dB for storing images is bad practice. That is fairly close to my professional opinion, yes. There is, of course, always the exception to any rule, but in general, I would call storing bitmaps in a database without a damn good reason, a bad practice, yes. Ah, you've changed your tune. Thanks for coming out. I haven't changed any tune what so ever. Uhm, ookaay dky then. I guess it's a good thing that the archives tell the real story... not to mention the readers of the list don't exactly have short memories. Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `' -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Wed, 2007-02-28 at 22:08 -0500, markw@mohawksoft.com wrote: On Wed, 2007-02-28 at 17:04 -0500, Mark wrote: Kevin Waterson wrote: This one time, at band camp, zerof [EMAIL PROTECTED] wrote: It is not a good practice to store pictures in DataBases, use links, instead of. Rubbish, where are your benchmarks? It has almost nothing to do with benchmarks. Images are typically best supported in the form of files. They are more easily manipulated by external tools. The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. Image files do not need to be constrained by the rigid requirements of a relational database. I could go on, but it should be clear enough that putting images in a database is not a good idea. What about when you need to share those files across a 50 node network? Without more information about the nodes and the network design, I can't offer a good argument against it, Oh, let's just say 50 nodes across the internet -- no specific location. And remember, I don't want to have every file on every server, I may have petabytes of data. Feel free to comment. OK, sure. First, are you saying that 50 nodes, spread across the internet, all have access to your SQL database? Yikes. but I can say, that given any rational system, bitmap images are better as discrete files than contents of a database. I'd argue that's only true if you need to modify the contents, and you can't do so without a copy on the filesystem. Otherwise I'd have to disagree. Databases implement their own filesystem for the most parts. A data storage system within a file system. It is a subtle difference. In fact many newer bleeding edge filesystems are practically database implementations. Not even bleeding edge. Think of the UNIX file systems, many use file names merely as indexes to inodes which point to files. Very database-esque. As such, the OS and the Database are merely layers over the raw medium. To a point, this is true, but layers with different purposes. Sure the database is often layered over the OS's implementation of the filesystem, but that is not necessaril true since some databases support raw access Absolutly, raw access can speed up SQL databases with, of course, certain caveats. in which case they're performance is probably just as good as the OS (and quite probably better if you want to store meta information about the file and it's data). Well, here's where your probably wrong. Depending on the database, of course, as PostgreSQL, MySQL, Oracle, DB2, etc. all have different behavior characteristics. Now, first the only point in your argument thus far has been performance. There are more issues involved, but for the moment lets argue that simple point. A file system is designed to take a file name, find its location on disk or in cache, and return it. (Obviously simplified, but you get the point) A database is designed to accept a query, parse the query and produce a good access plan for the data requested, and return the data. In many databases, blobs or binary objects are merely stored as files outside the data tables. Even if they are stored in your database they are large and possibly compressed. Either way they are large. If the bitmaps are stored as external files, then performance wise, you have the overhead of the SQL query in top of your file access. If the bitmaps are stored in the database, then they take up cached blocks that the database manage and reduce overall performance of the database. If you give me more information, I can counter with more specifics. I don't have specifics, I was just giving a sample scenario from the top of my head. I'd keep it in a database, then when I need it cache a local copy on the filesystem. Then I can just check the timestamp in the database to see if the file has changed. Voila, multi-node high availability images. You can do that sort of operation with any number of other tools more efficiently. Maybe... and even so, are they just as convenient? Why implement multiple protocols when one will suffice? KISS! Scalability and reliability and performance all come from keeping logically separate functions being done separately. Seems better than have a local copy of every single image. I guess the answer is... it depends on what you're doing! No, it just seems like if the only tool you are comfortable with is a hammer, then every job is more or less exactly like a nail. I'm quite comfortable with many tools. But if it looks like a duck, and walks like a duck, then I'm sure the hammer will suffice. It only looks like a duck if you ignore the horns, hooves, and teeth. Databases are great tools, but there are many tasks which they can do, just not
Re: [PHP] Re: how to display images stored in DB
The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. In a simple setup, that is probably true. However, if you use PHP to do authentication or throttling, then the engine is already there. On the flip side, you can use sendfile() or on Lighhttpd you can push the sending of the file back to the webserver using x-sendfile. That's doesn't address the storage location of the file. Image files do not need to be constrained by the rigid requirements of a relational database. File systems are not immune to constraints. For example, ext3 only allows 32000 subdirectories. So if you gave each user a directory to upload files to, you would be stuck at a max of 32000 users. Or start going to silly things like /S/t/e/Steve.gif First of all, I wouldn't use EXT3 for anything. In is, in all my tests, the slowest file system. More constaints below.. What about when you need to share those files across a 50 node network? Webfarm scenarios do come to mind. There is an issue of how to sync all webservers to have all files. Then again, if you are using 50 webservers, the chances of them all being able to house all your files (1 petabyte, as an example given) is not very good. Again, the problem of replication or distribution does not require a database. If you are saying that your single database will contain all your bitmap files, then that's messed up and your database will be a bottleneck. You've stated a problem: A large amount of data spread across multiple machines, this is a real problem domain, but it absolutely does not say why a database is the right solution or even a solution at all. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
At 5:04 PM -0500 2/28/07, Mark wrote: Images are typically best supported in the form of files. They are more easily manipulated by external tools. -snip- I could go on, but it should be clear enough that putting images in a database is not a good idea. It's clear enough to me that it's not a bad idea -- it depends upon the purpose. You also said: At 10:08 PM -0500 2/28/07, markw@mohawksoft.com wrote: No, it just seems like if the only tool you are comfortable with is a hammer, then every job is more or less exactly like a nail. Interesting, someone usually makes that comment to one who has a limited repertoire. On one hand, I'm advocating using dB for image storage when the needs of the job are better served by doing that, while you only see only one way and that is to use the file system. Can you not imagine any scenario where storing images in a dB would be a better choice? Or is this the hammer and nail thing you stated but failed to realize you were practicing? I suggest, and it's only a suggestion, that before you condemn one practice and advocate another that you know both the good and bad of both. Cheers, tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
At 5:04 PM -0500 2/28/07, Mark wrote: Images are typically best supported in the form of files. They are more easily manipulated by external tools. -snip- I could go on, but it should be clear enough that putting images in a database is not a good idea. It's clear enough to me that it's not a bad idea -- it depends upon the purpose. Obviously, I don't agree. You also said: At 10:08 PM -0500 2/28/07, markw@mohawksoft.com wrote: No, it just seems like if the only tool you are comfortable with is a hammer, then every job is more or less exactly like a nail. Interesting, someone usually makes that comment to one who has a limited repertoire. Or needs a smart ass way of saying that someone is using the wrong tool for the job. On one hand, I'm advocating using dB for image storage when the needs of the job are better served by doing that, while you only see only one way and that is to use the file system. Actually, I think you would be hard pressed to make a viable case for storing bitmap images in a database. Just because you *can* do it, does not mean it is a good solution to the problem. Can you not imagine any scenario where storing images in a dB would be a better choice? Or is this the hammer and nail thing you stated but failed to realize you were practicing? Well, to be honest, I can't think of a single instance where storing bitmap data in a database is a good idea. I've done everything from DICOM systems to huge web sites, and it has never been a solution that did not introduce more problems than it solves. Storing a bitmap path in the database is far more flexible and efficient. I suggest, and it's only a suggestion, that before you condemn one practice and advocate another that you know both the good and bad of both. In this discussion I have stated reasons why it is a bad idea. No one has come up with a counter point which can only be served by a database and thus proves me wrong. I think that says something. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
Rubbish, where are your benchmarks? ^^ whoever wrote that needs to check the manual before you make bold statements and my friend it is not 'Rubbish' On 2/28/07, Richard Lynch [EMAIL PROTECTED] wrote: On Tue, February 27, 2007 8:03 pm, Kevin Waterson wrote: This one time, at band camp, Richard Lynch [EMAIL PROTECTED] wrote: *ALL* of the arguments on this topic, and benchmarks, are in the PHP General archives. I am not concerned with past benchmarks done by others, I am asking what current benchmarks this user has made to make his claim. Why? They'd be no more nor less meaningful than the previous benchmarks to you personally. Or maybe I've just lost the thread of conversation here... -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- -- Jack Gleeson Web Designer
Re: [PHP] Re: how to display images stored in DB
Information on image storage in DB's http://www.webmasterworld.com/forum88/9091.htm Quoting markw@mohawksoft.com: The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. In a simple setup, that is probably true. However, if you use PHP to do authentication or throttling, then the engine is already there. On the flip side, you can use sendfile() or on Lighhttpd you can push the sending of the file back to the webserver using x-sendfile. That's doesn't address the storage location of the file. Image files do not need to be constrained by the rigid requirements of a relational database. File systems are not immune to constraints. For example, ext3 only allows 32000 subdirectories. So if you gave each user a directory to upload files to, you would be stuck at a max of 32000 users. Or start going to silly things like /S/t/e/Steve.gif First of all, I wouldn't use EXT3 for anything. In is, in all my tests, the slowest file system. More constaints below.. What about when you need to share those files across a 50 node network? Webfarm scenarios do come to mind. There is an issue of how to sync all webservers to have all files. Then again, if you are using 50 webservers, the chances of them all being able to house all your files (1 petabyte, as an example given) is not very good. Again, the problem of replication or distribution does not require a database. If you are saying that your single database will contain all your bitmap files, then that's messed up and your database will be a bottleneck. You've stated a problem: A large amount of data spread across multiple machines, this is a real problem domain, but it absolutely does not say why a database is the right solution or even a solution at all. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
At 10:01 AM -0500 3/1/07, markw@mohawksoft.com wrote: In this discussion I have stated reasons why it is a bad idea. No one has come up with a counter point which can only be served by a database and thus proves me wrong. I think that says something. Contradiction is not a sign of falsity, nor the lack of contradiction a sign of truth. I think no comment says that discussing this issue has problems. For example, this has been subject of many flame wars before and I'm sure that many just don't care to join in. If you want to claim something absurd, then who are they to correct you? And why should they care? Also, it might be that if counter points are made, will they just fall on deaf ears and be dismissed as easy as you claimed that there are no good reasons for doing in the first place. Keep in mind that you admittedly don't know of any good reasons and thus are more of an advocate for one side than open to seeing both sides. If I were to provide you with good reasons, would you honestly consider them OR are you so set in your ways that no example would suffice. Remember, that this topic has started flames wars before where people, on both sides, would not admit that one technique might prove more appropriate for one condition over another. If you are open to honestly consider, then I shall provide a couple of examples. But if I do and you do not agree, then your only recourse will be to *prove* otherwise. So, what say you? tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
# [EMAIL PROTECTED] / 2007-03-01 12:46:09 -0500: At 10:01 AM -0500 3/1/07, markw@mohawksoft.com wrote: In this discussion I have stated reasons why it is a bad idea. No one has come up with a counter point which can only be served by a database and thus proves me wrong. I think that says something. Contradiction is not a sign of falsity, nor the lack of contradiction a sign of truth. I think no comment says that discussing this issue has problems. For example, this has been subject of many flame wars before and I'm sure that many just don't care to join in. If you want to claim something absurd, then who are they to correct you? And why should they care? Exactly, ted. markw is so obviously right, and he's presented the points so well, there's nothing to add, really. but since you asked: yeah, he's right. -- How many Vietnam vets does it take to screw in a light bulb? You don't know, man. You don't KNOW. Cause you weren't THERE. http://bash.org/?255991 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Thu, 2007-03-01 at 21:08 +0100, Roman Neuhauser wrote: # [EMAIL PROTECTED] / 2007-03-01 12:46:09 -0500: At 10:01 AM -0500 3/1/07, markw@mohawksoft.com wrote: In this discussion I have stated reasons why it is a bad idea. No one has come up with a counter point which can only be served by a database and thus proves me wrong. I think that says something. Contradiction is not a sign of falsity, nor the lack of contradiction a sign of truth. I think no comment says that discussing this issue has problems. For example, this has been subject of many flame wars before and I'm sure that many just don't care to join in. If you want to claim something absurd, then who are they to correct you? And why should they care? Exactly, ted. markw is so obviously right, and he's presented the points so well, there's nothing to add, really. but since you asked: yeah, he's right. I'm going to skip his response to my previous comments and just add the following to this post: To follow up with Ted, nobody said using the filesystem is bad, what many of us are saying is that the database is not necessarily bad either. It really depends on what you're doing and how you choose to address the problem with all of your knowledge. Many of us here are quite aware of the different technologies available to access shared binary data across some kind of network etc. But, given time constraints, budget constraints, and all manner of personal preference and training and ingenuity, we CHOOSE to use one solution over the other for any given problem space. So far Mark has almost entirely focused on the performance and filesystems were made to do that argument... Who the hell cares??! If people stopped trying to use old ideas to solve novel problems then innovation and ingenuity would go out the window... and if that happens, then nothing advances, nothing is doscovered, we live a life of boring old filesystems that just do files. Why CAN'T a database be used as a filesystem? Mark said himself that filesystems pretty much are databases.. why limit them to just doing what they do now? Why can't they do more? Why can't they become more like fully fledged queryable databases? Why does Mark think he's more right than the many people on this list? What evidence has he given that says filesystems are ALWAYS, IRREFUTABLY the best option? He cannot, anything he argues in favour of the irrefutability of his argument only needs one counter example. Unlike our argument, he must prove the all encompassing nature of his solution space. We only need to prove a single example. The onus is on Mark to prove his argument is irrefutable, not on us to prove that database are always better, because we have never made that claim. I now await the book from Mark describing how filesystem storage for every single binary storage problem ever encountered and ever to be encountered is the solution. Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `' -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
I have reflected upon this thread for awhile now. I believe that the only new thing I have to add is for newbies. I believe that for a newbie, it would be easier to use the filesystem rather than the DB. True, you then have to do some extra cleanup/management work for deleted records, so that the related images go away. But storing them in the DB invariably ends up with too many issues involving DB storage size and query buffer size, compounded by data escaping/security issues. None of these are easy for a newbie to solve, and all too often, due to environmental limitations (read: shared hosting) it is impossible to fix the query buffer size limitation. If you know what you are doing, and have thought out the pros and cons, by all means, go ahead and put your images into the DB. If you have no idea what to do, choose the file system to get started, and then do more research when you have spare time. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Wed, February 28, 2007 4:04 pm, Mark wrote: Kevin Waterson wrote: This one time, at band camp, zerof [EMAIL PROTECTED] wrote: It is not a good practice to store pictures in DataBases, use links, instead of. Rubbish, where are your benchmarks? It has almost nothing to do with benchmarks. Images are typically best supported in the form of files. They are more easily manipulated by external tools. There are definitely far more issues than performance to this question. I personally prefer to store images OUTSIDE the DB, for several reasons. The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. Even this is not ALWAYS true. If the image is tiny enough (think an icon), and the other data that you HAVE to fetch on the page is large enough, then the image itself may be small enough to be cheaper to pull through the DB rather than an extra fstat. This is an edge case in a niche situation, but it's there, and there's a benchmark in the archives proving it. Image files do not need to be constrained by the rigid requirements of a relational database. Unless you find that tracking down the image in the file system is more trouble than letting the DB nuke it if you have frequent record deletes. This could be particularly true in some weird case (again) where there are many inter-dependent records, where a simple trigger in a DB would nuke the file when the last reference goes away, but you'd have to write some gnarly query in your cleanup script to decide if you can remove the file system image or not. Again, it's an edge case, but there can be no question that for somebody somewhere, this is clearly a Better Way. [Not for me, and not for most newbies, but it's there.] I could go on, but it should be clear enough that putting images in a database is not a good idea. Add the word usually and I'd agree with you. Without it, you're just plain wrong, even if you are Mr. Mcache. :-) [right?] One also has to consider your backup strategy. Throwing images in the db makes backing them up easy, but quite expensive, as you'll probably be dumping them out all the time, and then archiving the dump, and using a LOT of backup space for something that, in MOST applications, changes seldom, if ever. Backing up the more or less static images (for MOST web apps) can be done more efficiently with a separate process, in MOST cases. It might require a quick check of the last few records and images added to be sure that there wasn't a hiccup that orphaned record/image, but that's probably a Good Idea if something went wrong, really, even if you mostly trust your DB with images to have done the right thing in a crash... Further, in the unlikely event of a database/hard-drive disaster, with backups also possibly lost/broken/whatever, re-building the DB without images will PROBABLY be less painful than the one with the images, assuming the whole thing isn't so corrupted, throughout all backups, as to be useless anyway. In other words, without the images in the DB, you're going to have an easier time restoring the data of the DB, and then dealing with any db-file-image relationships separately, than trying to restore a DB that's loaded up with large images. Finally. consider that the first easy major performance enhancement of a site that grows beyond the ability of hardware to handle it is to split the DB off to another box. Sucking down the image through the LAN from one box to another is going to hurt a lot more than adding a THIRD box, custom-tuned for images, or even going the Akami [et al] route. Of course, you may be running a site that is never going to have to worry about heavy traffic. I know I have lots of those. :-) Ultimately, this is a lot more complex than it first appears, and can only be answered correctly in the context of a specific web application and architecture, but I'm still sticking to my guns and saying that most newbies should be using the file system for images, until they have a really good reason to put them into the DB. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Wed, February 28, 2007 6:41 pm, Robert Cummings wrote: What about when you need to share those files across a 50 node network? I'd keep it in a database, then when I need it cache a local copy on the filesystem. Then I can just check the timestamp in the database to see if the file has changed. Voila, multi-node high availability images. Seems better than have a local copy of every single image. I guess the answer is... it depends on what you're doing! Add an image server (or 20) and change the HTML to point to the image server. Use rsync on the 20 image client servers from the master image server. Lots of ways to skin this cat. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Thu, 2007-03-01 at 21:08 +0100, Roman Neuhauser wrote: # [EMAIL PROTECTED] / 2007-03-01 12:46:09 -0500: At 10:01 AM -0500 3/1/07, markw@mohawksoft.com wrote: In this discussion I have stated reasons why it is a bad idea. No one has come up with a counter point which can only be served by a database and thus proves me wrong. I think that says something. Contradiction is not a sign of falsity, nor the lack of contradiction a sign of truth. I think no comment says that discussing this issue has problems. For example, this has been subject of many flame wars before and I'm sure that many just don't care to join in. If you want to claim something absurd, then who are they to correct you? And why should they care? Exactly, ted. markw is so obviously right, and he's presented the points so well, there's nothing to add, really. but since you asked: yeah, he's right. I'm going to skip his response to my previous comments and just add the following to this post: To follow up with Ted, nobody said using the filesystem is bad, No, it is the most efficient way. what many of us are saying is that the database is not necessarily bad either. To make this claim, you need a rational argument to support it. Faith Based engineering is a por substitute. It really depends on what you're doing and how you choose to address the problem with all of your knowledge. Assuming your knowledge never expands when confronted with a problem which requires learning more. Many of us here are quite aware of the different technologies available to access shared binary data across some kind of network etc. But, given time constraints, budget constraints, and all manner of personal preference and training and ingenuity, we CHOOSE to use one solution over the other for any given problem space. The good enough fallacy. So far Mark has almost entirely focused on the performance and filesystems were made to do that argument... Um, yea. Who the hell cares??! Ahh, famous last words. You only need to get bitten once and you will change your tune very quickly. If people stopped trying to use old ideas to solve novel problems then innovation and ingenuity would go out the window... I would argue that if people fail to research and devise better solutons, then innovation doesn't happen. and if that happens, then nothing advances, nothing is doscovered, we live a life of boring old filesystems that just do files. So, files are bad? Why CAN'T a database be used as a filesystem? Because it is not well suited for the task for which you are trying to use it. Mark said himself that filesystems pretty much are databases.. I said nothing of the sort. I forget the exact words, but yes there are conceptual similarities, but that's the limit of what I have said. why limit them to just doing what they do now? Why can't they do more? Because a database *is* designed to do more than a filesystem. Databases have a LOT of features and requirements that file systems do not have and things like bitmap binary objects don't need. They are fundamentally different things. Why can't they become more like fully fledged queryable databases? See above. Why does Mark think he's more right than the many people on this list? I think I'm right because I've been down this road a couple times, and have done a lot of research and can defend the position. What evidence has he given that says filesystems are ALWAYS, IRREFUTABLY the best option? I'll give you a few facts to think about: (1) If the binary object is stored as a separate file (as in a lot of databases) then there is no advantage to storing the binary data in the database. (2) If the database uses an internal block cache, the binary data will use up more blocks and cause the database to be less efficient. (3) On MySQL, with MyISAM, the table is readlocked when data is being read from a table. If you store bitmap data in the database you will have longer locks on your database and overall capacity will be diminished. (4) The data in a bitmap does not need to be evaluated for joining or selection, storing it in the database is an expense for no reason. The metadata of the bitmap, size, filename, datestamp, etc. are more usable and take up less space. He cannot, anything he argues in favour of the irrefutability of his argument only needs one counter example. Which, somehow, you've failed to provide. Unlike our argument, he must prove the all encompassing nature of his solution space. We only need to prove a single example. The onus is on Mark to prove his argument is irrefutable, not on us to prove that database are always better, because we have never made that claim. I now await the book from Mark describing how filesystem storage for every single binary storage problem ever encountered and ever to be encountered is the solution. Nice strawman, but the actual question is this: Where do you store bitmap
Re: [PHP] Re: how to display images stored in DB
To follow up with Ted, nobody said using the filesystem is bad, No, it is the most efficient way. Not in my environment. All db servers have RAID 10 over 8 SCSI 15K disks. Pulling from them is always faster than a webserver pulling from its SATA drive. Also, there is issue of going to the database and going to the filesystem. Timing wise, you Add an image server (or 20) and change the HTML to point to the image server. I can't imagine Flickr doing that. I believe that the only new thing I have to add is for newbies. I believe that for a newbie, it would be easier to use the filesystem rather than the DB. True, you then have to do some extra cleanup/management work for deleted records, so that the related images go away. But storing them in the DB invariably ends up with too many issues involving DB storage size and query buffer size, compounded by data escaping/security issues. Strange... I came to the opposite conclusion. Using prepared statements eliminates data escaping issues, etc. And putting the files in the db removes the extra cleanup/management stuff. And easier to backup (though not efficient). Again, the problem of replication or distribution does not require a database. If you are saying that your single database will contain all your bitmap files, then that's messed up and your database will be a bottleneck. You've stated a problem: A large amount of data spread across multiple machines, this is a real problem domain, but it absolutely does not say why a database is the right solution or even a solution at all. I guess you skimmed what I wrote. What I wrote was about using a database for meta data and server and file location. I was talking about using that date to intelligently know where the file was on a file system. Mostly because it is cheaper to scale that way as you can tune things to add more replicas for redundancy and performance. Then that can hit scalability problems with many hundred of servers. But easy to solve by breaking down meta data and storage parts. Have fewer storage servers talk to a meta data (database) server such that you can run the databases on the same cheap machines that you run the storage stuff. Then you have a task server that manages the meta data servers and storage servers. Chaining things this may see like a lot of steps to go through. But it can be very efficient throughput wise, which matters far more than a benchmark. To anyone that has designed CPUs, this will look a little familiar (though more flexible). At that point you also never have a complete backup on one machine. I remember that being a weird thing -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
To follow up with Ted, nobody said using the filesystem is bad, No, it is the most efficient way. Not in my environment. All db servers have RAID 10 over 8 SCSI 15K disks. Pulling from them is always faster than a webserver pulling from its SATA drive. That's not exactly an apples to apples comparison. Also, there is issue of going to the database and going to the filesystem. Timing wise, you Please explain. Add an image server (or 20) and change the HTML to point to the image server. I can't imagine Flickr doing that. I believe that the only new thing I have to add is for newbies. I believe that for a newbie, it would be easier to use the filesystem rather than the DB. True, you then have to do some extra cleanup/management work for deleted records, so that the related images go away. But storing them in the DB invariably ends up with too many issues involving DB storage size and query buffer size, compounded by data escaping/security issues. Strange... I came to the opposite conclusion. Using prepared statements eliminates data escaping issues, etc. And putting the files in the db removes the extra cleanup/management stuff. And easier to backup (though not efficient). Putting the files in the DB doesn't always remove extra cleanup/management stuff. Sometimes the database stores binary objects as files and sometimes they don't get deleted correctly. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Tue, February 27, 2007 8:03 pm, Kevin Waterson wrote: This one time, at band camp, Richard Lynch [EMAIL PROTECTED] wrote: *ALL* of the arguments on this topic, and benchmarks, are in the PHP General archives. I am not concerned with past benchmarks done by others, I am asking what current benchmarks this user has made to make his claim. Why? They'd be no more nor less meaningful than the previous benchmarks to you personally. Or maybe I've just lost the thread of conversation here... -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
Kevin Waterson wrote: This one time, at band camp, zerof [EMAIL PROTECTED] wrote: It is not a good practice to store pictures in DataBases, use links, instead of. Rubbish, where are your benchmarks? It has almost nothing to do with benchmarks. Images are typically best supported in the form of files. They are more easily manipulated by external tools. The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. Image files do not need to be constrained by the rigid requirements of a relational database. I could go on, but it should be clear enough that putting images in a database is not a good idea. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Wed, 2007-02-28 at 17:04 -0500, Mark wrote: Kevin Waterson wrote: This one time, at band camp, zerof [EMAIL PROTECTED] wrote: It is not a good practice to store pictures in DataBases, use links, instead of. Rubbish, where are your benchmarks? It has almost nothing to do with benchmarks. Images are typically best supported in the form of files. They are more easily manipulated by external tools. The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. Image files do not need to be constrained by the rigid requirements of a relational database. I could go on, but it should be clear enough that putting images in a database is not a good idea. What about when you need to share those files across a 50 node network? I'd keep it in a database, then when I need it cache a local copy on the filesystem. Then I can just check the timestamp in the database to see if the file has changed. Voila, multi-node high availability images. Seems better than have a local copy of every single image. I guess the answer is... it depends on what you're doing! Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `' -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Wed, 2007-02-28 at 17:04 -0500, Mark wrote: Kevin Waterson wrote: This one time, at band camp, zerof [EMAIL PROTECTED] wrote: It is not a good practice to store pictures in DataBases, use links, instead of. Rubbish, where are your benchmarks? It has almost nothing to do with benchmarks. Images are typically best supported in the form of files. They are more easily manipulated by external tools. The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. Image files do not need to be constrained by the rigid requirements of a relational database. I could go on, but it should be clear enough that putting images in a database is not a good idea. What about when you need to share those files across a 50 node network? Without more information about the nodes and the network design, I can't offer a good argument against it, but I can say, that given any rational system, bitmap images are better as discrete files than contents of a database. If you give me more information, I can counter with more specifics. I'd keep it in a database, then when I need it cache a local copy on the filesystem. Then I can just check the timestamp in the database to see if the file has changed. Voila, multi-node high availability images. You can do that sort of operation with any number of other tools more efficiently. Seems better than have a local copy of every single image. I guess the answer is... it depends on what you're doing! No, it just seems like if the only tool you are comfortable with is a hammer, then every job is more or less exactly like a nail. Databases are great tools, but there are many tasks which they can do, just not well. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Wed, 2007-02-28 at 22:08 -0500, markw@mohawksoft.com wrote: On Wed, 2007-02-28 at 17:04 -0500, Mark wrote: Kevin Waterson wrote: This one time, at band camp, zerof [EMAIL PROTECTED] wrote: It is not a good practice to store pictures in DataBases, use links, instead of. Rubbish, where are your benchmarks? It has almost nothing to do with benchmarks. Images are typically best supported in the form of files. They are more easily manipulated by external tools. The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. Image files do not need to be constrained by the rigid requirements of a relational database. I could go on, but it should be clear enough that putting images in a database is not a good idea. What about when you need to share those files across a 50 node network? Without more information about the nodes and the network design, I can't offer a good argument against it, Oh, let's just say 50 nodes across the internet -- no specific location. And remember, I don't want to have every file on every server, I may have petabytes of data. Feel free to comment. but I can say, that given any rational system, bitmap images are better as discrete files than contents of a database. I'd argue that's only true if you need to modify the contents, and you can't do so without a copy on the filesystem. Otherwise I'd have to disagree. Databases implement their own filesystem for the most parts. In fact many newer bleeding edge filesystems are practically database implementations. As such, the OS and the Database are merely layers over the raw medium. Sure the database is often layered over the OS's implementation of the filesystem, but that is not necessaril true since some databases support raw access in which case they're performance is probably just as good as the OS (and quite probably better if you want to store meta information about the file and it's data). If you give me more information, I can counter with more specifics. I don't have specifics, I was just giving a sample scenario from the top of my head. I'd keep it in a database, then when I need it cache a local copy on the filesystem. Then I can just check the timestamp in the database to see if the file has changed. Voila, multi-node high availability images. You can do that sort of operation with any number of other tools more efficiently. Maybe... and even so, are they just as convenient? Why implement multiple protocols when one will suffice? KISS! Seems better than have a local copy of every single image. I guess the answer is... it depends on what you're doing! No, it just seems like if the only tool you are comfortable with is a hammer, then every job is more or less exactly like a nail. I'm quite comfortable with many tools. But if it looks like a duck, and walks like a duck, then I'm sure the hammer will suffice. Databases are great tools, but there are many tasks which they can do, just not well. Well one thing I know the webserver itself and the filesystem can't do is check the credentials of a logged in user to see if they are allowed the file. Well, I guess the webserver could do it... sort of... with http auth. Wonder if that works with many custom authentication systems... I presume you would somehow cram that into the webserver? I guess if the only tool with which you're comfortable is a screwdriver, then every job is more or less exactly like a screw. As such, I think you're screwed. Remember, I said it depends on what you're doing, you said no, and well quite frankly you're wrong, because your screw and screwdriver mentality doesn't work in every imaginable scenario. Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `' -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
The web browser sees an image as a single HTTP request. Invoking the PHP script engine, parsing the script, and executing a SQL query to retrieve the image from the database is less efficient than letting the web server just send the file. In a simple setup, that is probably true. However, if you use PHP to do authentication or throttling, then the engine is already there. On the flip side, you can use sendfile() or on Lighhttpd you can push the sending of the file back to the webserver using x-sendfile. Image files do not need to be constrained by the rigid requirements of a relational database. File systems are not immune to constraints. For example, ext3 only allows 32000 subdirectories. So if you gave each user a directory to upload files to, you would be stuck at a max of 32000 users. Or start going to silly things like /S/t/e/Steve.gif More constaints below.. What about when you need to share those files across a 50 node network? Webfarm scenarios do come to mind. There is an issue of how to sync all webservers to have all files. Then again, if you are using 50 webservers, the chances of them all being able to house all your files (1 petabyte, as an example given) is not very good. some databases support raw access in which case they're performance is probably just as good as the OS (and quite probably better if you want to store meta information about the file and it's data). While the database method makes for a good way of doing offsite backups (replicating out to a slave), the database can easily become a choke point as well. I'd keep it in a database, then when I need it cache a local copy on the filesystem. Then I can just check the timestamp in the database to see if the file has changed. Voila, multi-node high availability images. You will need to keep information about what you are caching so that you can prune. Best choice I guess would be to keep a local sqlite db on the webserver to keep track. However, you had better understand your filesystem. In ext3, for example, if you have a lot of files in a folder you will likely use the dir_index option when creating the partition. But realize that deleting files does not delete leaf nodes of the btree, which can have all sorts of performance and disk usage effects that are non-obvious. There are hybid models, of course. mogilefs for example uses mysql to store data file info, but not the file itself. Instead a series of programs are used to spread the files across a farm of servers, using mutiple replicas for fault tolerance and performance reasons. Those files are stored using a combination of directory mazes and hashes to avoid typical filesystem issues. So there are lots of ways to deal with the issue. Depends on your constraints on time, complexity, and scalability. After all, if you only have 10,000 users, who cares! 100 million might be different. -s -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On Sun, February 25, 2007 8:27 pm, Kevin Waterson wrote: This one time, at band camp, zerof [EMAIL PROTECTED] wrote: It is not a good practice to store pictures in DataBases, use links, instead of. Rubbish, where are your benchmarks? In the archives. Unless your images are teeny-tiny and can take advantage of buffer sizes on MySQL client/server interaction, you'll get SLIGHTLY better performance with images from file system. *ALL* of the arguments on this topic, and benchmarks, are in the PHP General archives. Several times over. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
This one time, at band camp, Richard Lynch [EMAIL PROTECTED] wrote: *ALL* of the arguments on this topic, and benchmarks, are in the PHP General archives. I am not concerned with past benchmarks done by others, I am asking what current benchmarks this user has made to make his claim. Kevin -- Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: how to display images stored in DB
Alain Roger escreveu: Hi, i stored all my pictures in my PostgreSQL DB as bytea type. Now i would like to know how can i display them in my PHP pages ? where can i find some example ? thanks a lot, -- It is not a good practice to store pictures in DataBases, use links, instead of. -- zerof http://www.educar.pro.br/ Apache - PHP - MySQL - Boolean Logics - Project Management -- Você deve, sempre, consultar uma segunda opinião! -- Deixe todos saberem se esta informação foi-lhe útil. -- You must hear, always, one second opinion! In all cases. -- Let the people know if this info was useful for you! -- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
On 2/25/07, zerof [EMAIL PROTECTED] wrote: Alain Roger escreveu: Hi, i stored all my pictures in my PostgreSQL DB as bytea type. Now i would like to know how can i display them in my PHP pages ? where can i find some example ? thanks a lot, -- It is not a good practice to store pictures in DataBases, use links, instead of. -- zerof http://www.educar.pro.br/ Apache - PHP - MySQL - Boolean Logics - Project Management -- Você deve, sempre, consultar uma segunda opinião! -- Deixe todos saberem se esta informação foi-lhe útil. -- You must hear, always, one second opinion! In all cases. -- Let the people know if this info was useful for you! -- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php Indeed. I am building an events directory at the moment that allows posters to upload images and I store all the images on the file system and the links in the db. -- http://www.web-buddha.co.uk
[PHP] Re: how to display images stored in DB
At 3:53 PM -0300 2/25/07, zerof wrote: Alain Roger escreveu: Hi, i stored all my pictures in my PostgreSQL DB as bytea type. Now i would like to know how can i display them in my PHP pages ? where can i find some example ? thanks a lot, -- It is not a good practice to store pictures in DataBases, use links, instead of. Flame suit on Let's not go there again. Both techniques have up-sides and down-sides -- and I use both depending upon needs. For example, have you ever tried to back-up an image dB? It's a *lot* easier if the files are all stored in a dB rather than in a file system. One is saving just a dB and the other is saving both a dB and the file system (a lot can go wrong there). Plus, ask that question on a MySQL list and the MySQL experts will not advise you to stay away from storing images in MySQL. And every MySQL book I've ever read that address that issue says that both techniques have valid points. The *only* ones that I've ever heard complain about storing an image file in a dB have been a few php programmers on this list. And no one, to my knowledge, has ever demonstrated that saving an image in MySQL is poor practice, other than providing lip-service to that effect. My advice, if it works -- use it. Both techniques are valid, have their advantages, and are good practice. Flame suit off Cheers, tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: how to display images stored in DB
This one time, at band camp, zerof [EMAIL PROTECTED] wrote: It is not a good practice to store pictures in DataBases, use links, instead of. Rubbish, where are your benchmarks? kevin -- Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: How to display all keys in a multi-dim array?
Richard Davey [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Hi all, I am trying to work out how to display all of the keys in a multi-dimensional array (not their values). For example: $array = array(); $array['a'] = 1; $array['a']['b'] = 2; $array['a']['b']['c'] = 3; $array['a']['b']['c']['d'] = 4; What I need is to be able to somehow print out the values a,b,c and d based on the search result for 4. Could you describe a bit more in detail what you mean with based on search result for 4? Thanks and best regards, Torsten Roehr -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: How to display all keys in a multi-dim array?
Hello Torsten, Sunday, May 23, 2004, 6:44:08 PM, you wrote: $array = array(); $array['a'] = 1; $array['a']['b'] = 2; $array['a']['b']['c'] = 3; $array['a']['b']['c']['d'] = 4; TR Could you describe a bit more in detail what you mean with based on search TR result for 4? I know that 4 is a *unique value* in my multi-dim array, but I do not know the key names that exist under-neath it because they were created dynamically. In other words - how do I find out all of the keys that hold the value 4? Best regards, Richard Davey -- http://www.launchcode.co.uk - PHP Development Services I am not young enough to know everything. - Oscar Wilde -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: How to display all keys in a multi-dim array?
Richard Davey [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Hello Torsten, Sunday, May 23, 2004, 6:44:08 PM, you wrote: $array = array(); $array['a'] = 1; $array['a']['b'] = 2; $array['a']['b']['c'] = 3; $array['a']['b']['c']['d'] = 4; TR Could you describe a bit more in detail what you mean with based on search TR result for 4? I know that 4 is a *unique value* in my multi-dim array, but I do not know the key names that exist under-neath it because they were created dynamically. In other words - how do I find out all of the keys that hold the value 4? So you basically want to search for the value and get its key(s), right? Take a look at array_search(): http://de.php.net/manual/en/function.array-search.php There are also some user comments on multi-dim arrays that might help you. Hope this helps, Regards, Torsten -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: how to display a font with two words???
Rename the font... I think that's the only solution... -- // DvDmanDT MSN: dvdmandt¤hotmail.com Mail: dvdmandt¤telia.com Matt Hedges [EMAIL PROTECTED] skrev i meddelandet news:[EMAIL PROTECTED] Howdy, I'm trying to specify font face=Monotype Corsiva in my php document... but can't b/c it's two words... if it was html I could just put , but since I can't do that in php, does anyone know what to do? prob. a stupid question, thanks a lot, Matt -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: how to display a font with two words???
you can use single quotes ' ' (I didn't think it would work, but it does) around the font. matt Matt Hedges [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Howdy, I'm trying to specify font face=Monotype Corsiva in my php document... but can't b/c it's two words... if it was html I could just put , but since I can't do that in php, does anyone know what to do? prob. a stupid question, thanks a lot, Matt -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: how to display a font with two words???
I just realized my misstake.. In GD you can't, in HTML it's possible.. Read the other ppls replies.. -- // DvDmanDT MSN: dvdmandt¤hotmail.com Mail: dvdmandt¤telia.com Dvdmandt [EMAIL PROTECTED] skrev i meddelandet news:[EMAIL PROTECTED] Rename the font... I think that's the only solution... -- // DvDmanDT MSN: dvdmandt¤hotmail.com Mail: dvdmandt¤telia.com Matt Hedges [EMAIL PROTECTED] skrev i meddelandet news:[EMAIL PROTECTED] Howdy, I'm trying to specify font face=Monotype Corsiva in my php document... but can't b/c it's two words... if it was html I could just put , but since I can't do that in php, does anyone know what to do? prob. a stupid question, thanks a lot, Matt -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: How to display?
Here is small example on how to do it with PHP MySQL and TemplateTamer: http://www.templatetamer.org/index.php?SimpleMySqlRowList rush -- http://www.templatetamer.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: how to display a file's last updated time using php?
look here: http://www.php.net/manual/en/ref.filesystem.php and especially here: http://www.php.net/manual/en/function.filemtime.php Michael Rui Huang [EMAIL PROTECTED] schrieb im Newsbeitrag news:[EMAIL PROTECTED]... Hi, friends, I want to display the last updated time of a file using php, but I don't know which function I should use. It seems that the date() just show the current time. For html file, a simple javascript works well, but in php files that script works weired. How to solve that problem? Thanks a lot. _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: How to display one line from a file rather than the whole lot
Dirty trick:
?php
$line_I_want = '47';
// get a web page into an array and print it out
$fcontents = file ('http://www.php.net');
while (list ($line_num, $line) = each ($fcontents)) {
// echo bLine $line_num:/b . htmlspecialchars ($line) . br\n;
if ($line_I_want == $line_num) {
$dirty_trick = htmlspecialchars($line);
}
}
echo $dirty_trick;
?
No knowledge of what number is the line:
?php
// get a web page into an array and print it out
$filename = 'http://www.php.net;
$fd = fopen ($filename, 'r');
$read = fread($fd, 3);
fclose($fd);
$search = eregi('what_comes_before_the_line_you_want(.*)what_comes_after',
$read, $printing);
echo $printing[1];
?
--
Julio Nobrega
No matter where you go, this.
Kevin Garrett [EMAIL PROTECTED] wrote in message
[EMAIL PROTECTED]">news:[EMAIL PROTECTED]...
Hi guys,
I'm looking to edit this to just return one particular line rather than
the
whole file:
?php
// get a web page into an array and print it out
$fcontents = file ('http://www.php.net');
while (list ($line_num, $line) = each ($fcontents)) {
echo bLine $line_num:/b . htmlspecialchars ($line) . br\n;
}
?
Can anybody help me?
Kev
_
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]
