On Nov 9, 2008, at 11:41 PM, Brian Aker wrote:
On Nov 10, 2008, at 7:27 AM, Jim Starkey wrote:
Philosophically, I'm not sure than inconvenience should be a major
consideration when considering architectures. If drizzle could
delay blob materialization to the last possible nanosecond in hope
it would need to, drizzle would be a great deal more application
friendly.
This is something I find more interesting because if we can delay
the last moment (or just pass in a fd/offset/length) we can do
sendfile tricks to just map from disk to card bypassing memory
entirely. This would mean though that an engine laid out a blob on
disk as a single object.
Part of he logic in the original protocol was:
SELECT a, b, c
Is that if I asked for the values in that order, I don't want to
have to make a second request for a particular member of that set. I
have already said "this is what I want, give it to me". The blob
fetching has always implied a sort of "but do you really want it?".
The blob fetching in Oracle was an original limitations of their
design, it just happened to be in MySQL that it didn't have that
limitation (though the current twice copy of blob data is ridiculous).
[ Little plug for one of our projects, but I think it is quite
relevant to this discussion. ]
The functionality of the BLOB streaming engine (www.blobstreaming.org)
already covers quite a bit of this stuff.
The engine:
- Maintains a BLOB repository where BLOBs are stored sequentially on
disk (sendfile() possible).
- Issues BLOB locators which can be stored in the rows instead of the
data.
- Allows BLOBs to be streamed in and out of the database using HTTP
PUT and GET.
- Automatically delays deletion of BLOBs so that they can be "grabbed
later".
Replication of the BLOB data is possible in a similar manor described
by Mats. However, the BLOB data need not be written to the binary log
because it can be retrieved from the repository.
Engines call the BLOB streaming engine's "server-side" streaming API
in order to support this functionality. Consistency between databases
tables and the repository is maintained using reference counting calls
in the API.
Maybe this is an option, and we could create a Drizzle tree with the
functionality for testing...
Best regards,
Paul
--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
