On Jul 1, 2010, at 4:35 AM, Samuel GARCIA wrote: > Hi list, > I am new user of sqlalchemy. I play with python and MySQL for several years > now and I discover sqlalchemy 1 year ago, it change my life. > So thank you for this fantastic work. > > I develop a project of analysing big experimental dataset of intra/extra > cellular recordings : OpenElectrophy. > I highly use database for this, schema are really simple but table and row > are really heavy because I use BLOB to store big raw data signal. > I usually play with 1Go to 300Go databases. > Playing with SQL is very util but problems are coming because signals get > bigger and bigger. > > After a walk on google, a solution could come with streaming Binary field. > In short to be able to take a field chunk by chunk. > > PBXT engine for mysql seems to umplement it: http://www.blobstreaming.org/ > > Postgree SQL seems to natively support sort of streaming. > > I don't known for SQLite. > > It seems that there is not an official way to "stream" big binary fields in > SQL world. Correct me if I am wrong I am a newbie in this fields. > > > At the end my questions : > 1 - Does sqlalchemy plan to implement a unified layer for playing with > stream-able field ? > I really can't figure out the amount of work to achieve that, sorry if it is > naive. > > 2 - Could you give me somes tips to deal with streaming with theses backend : > MySQL, SQLite, postgree
its not something DBAPI supports, and to my knowledge only cx_oracle has an actual "streamable" blob type - im not aware of one for MySQL, PG, etc. So there are no plans for cross-DBAPI support of "streaming" blobs in SQLAlchemy right now since DBAPI doesn't provide it for us (we have some support for cx_oracle's feature). My advice for now would be that storing BLOBs are probably not a good idea for streams of data larger than a few megs - I'd use the filesystem instead. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
