Following MySQL's BLOB field design, can Paimon also support streaming write capabilities for BLOB fields? MySQL Large Object Storage
1. BINARY vs BLOB *Note: MySQL supports both BINARY and BLOB types, whereas Paimon currently only supports Binary* Type Description BINARY Fixed-length binary string type, similar to CHAR, but stores bytes instead of characters. BLOB Variable-length binary large object type, used to store large amounts of binary data (e.g., images, audio, files). ------------------------------ 2. Operation InterfacesInput Streams (Writing Data) Category Method Purpose Statement setBinaryStream(int index, InputStream x, int length) Writes binary stream data into a BLOB field; used for inserting or updating BLOB data. Recommended for streaming writes. setBlob(int index, InputStream inputStream) Writes BLOB data using an input stream (JDBC 4.0+). A more modern approach that does not require specifying the length. setBytes(int index, byte[] bytes) Directly writes a byte array. Suitable only for small files (<1MB); be cautious about memory usage. Output Streams (Reading Data) Category Method Purpose ResultSet getBinaryStream(String columnName) Reads BLOB data as an input stream. Recommended for streaming large files to avoid OOM. getBinaryStream(int index) Same as above, but accesses by column index. Equivalent to using column name, useful when column order is known. Large Object Handling (Blob) Category Method Purpose Blob ResultSet.getBlob(String columnName) Retrieves a java.sql.Blob object, which provides additional methods for manipulation. Blob.getBinaryStream() Gets an input stream from the Blob object. Used in conjunction with ResultSet.getBlob(). Blob.length() Returns the size (length) of the BLOB data. Useful for determining file size or allocating buffers. Byte Array Access Category Method Purpose Bytes ResultSet.getBytes(String columnName) Reads the entire BLOB directly into a byte array. Only suitable for small files, as large files may cause OutOfMemoryError (OOM). ------------------------------ This comparison highlights that MySQL provides robust streaming I/O support for BLOBs, enabling efficient handling of large binary objects without loading them entirely into memory — a capability that could be valuable to implement in Paimon for better multimodal data management.
