Viraj Jasani created PHOENIX-8036:
-------------------------------------
Summary: Protobuf Data type
Key: PHOENIX-8036
URL: https://issues.apache.org/jira/browse/PHOENIX-8036
Project: Phoenix
Issue Type: New Feature
Reporter: Viraj Jasani
Phoenix supports two document data types: JSON and BSON. Both data types help
store JSON documents in encoded formats, where the document does not need to
follow any schema patterns. However, storing JSON documents requires storing
all the key-value pairs of the document. Some use cases do not always need
flexible structure to store the document data in a single column.
Protocol buffers (or protobuf) is a binary serialization format that uses
schema. Protobufs are language-neutral, platform-neutral extensible mechanisms
for serializing structured data. Users can describe the protobuf messages in a
.proto file. The protoc compiler generates typed classes for Java, C++, Go,
Python and other languages. Those classes encode messages into compact bytes
and decode them back.
On the wire, each field is written as a pair: a field number plus a type tag,
followed by the value. Integers use varints (small numbers take fewer bytes).
Field names are not sent on the wire, and therefore protobuf is quite compact
than JSON or BSON serialized formats. Protobufs are also faster to parse and
serialize than JSON/BSON formats.
Under normal circumstances, protobuf serialized bytes can be stored as
VARBINARY format in Phoenix or any other SQL databases. However, it does not
work when the use case requires creating a global secondary index on one or
more fields of the protobuf data, or retrieving values of one or more fields
rather than retrieving full binary data to the SQL client. Therefore, Phoenix
is introducing a new data type PROTOBUF, with the data stored as variable
length binary, and certain functions to retrieve the field values to clients
and allow creating secondary indexes on the fields of the protobuf data.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
