Malcolm, Been there, done that in terms of storing outside the database. Fine when it is small amount of data but a nightmare from a security point of view with large amounts of data.
We moved soma 600Gb of scanned & audio files from a VFP database (stored only a link to the file) because of thise. Unfortunately if your folder structure gets fouled up or slow to access say because you have more than about 5,000 files in a folder then it totally screws you. Also people are tempted to meddle with data in folders and a simple inadvertand rename of a folder or file can render your data useless. Now at least I sleep knowing the data is safe in SQL Server and is being mirrored, snapshot and replicated without me doing anything. Also access is of course quick and doesn't degrade. Currently the initial 600Gb has increased to over 3Tb as the amount of paperwork has been reduced in the company and it still runs as fast. Also, nobody can get at it from a security point of view without knowing how to access SQL, so our company security auditors are happy. Dave -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Malcolm Greene Sent: 24 September 2011 14:16 To: [email protected] Subject: Re: [NF] For those interested in Big Data Paul, > I think most people just store the image data, video data, whatever, as a node in the database. You want to limit the number of things to search for and pulling the pointer from the database and then the file from the filesystem wouldn't be as fast as just pulling the blob from the db to begin with. I respectfully disagree with your position. There was a recent thread on the SQLite mailing list that included some benchmarks that argued against use of blobs for storing files. While much of the discussion was SQLite specific, I think that many of the principles would also apply to other databases. Some generic reasons to store files outside of a database include the ability to use lower cost/speed file systems for file storage, faster database backups, and faster database maintenance activities when schemas change. Also much less database fragmentation. The only reason I can think to store files inside a database is for convenience, eg. having a single container for all your data. Curious to hear other opinions on this topic. Malcolm [excessive quoting removed by server] _______________________________________________ Post Messages to: [email protected] Subscription Maintenance: http://leafe.com/mailman/listinfo/profox OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech Searchable Archive: http://leafe.com/archives/search/profox This message: http://leafe.com/archives/byMID/profox/[email protected] ** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
