[
https://issues.apache.org/jira/browse/PHOENIX-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas D'Silva updated PHOENIX-3586:
------------------------------------
Attachment: PHOENIX-3586-v2.patch
[~jamestaylor]
Thanks for the review, I have attached a v2 patch. I refactored the
ImmutableStorageScheme enum, previously the was array data type builder was an
instance variable which was not thread safe.
Now the enum implements the following interface
{code}
+ interface ColumnValueEncoderDecoderSupplier {
+ ColumnValueEncoder getEncoder(int numElements);
+ ColumnValueDecoder getDecoder();
}
+ public enum ImmutableStorageScheme implements
ColumnValueEncoderDecoderSupplier {
+ ONE_CELL_PER_COLUMN((byte)1) {
+ @Override
+ public ColumnValueEncoder getEncoder(int numElements) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public ColumnValueDecoder getDecoder() {
+ throw new UnsupportedOperationException();
+ }
+ },
+ // stores a single cell per column family that contains all serialized
column values
+ SINGLE_CELL_ARRAY_WITH_OFFSETS((byte)2) {
+ @Override
+ public ColumnValueEncoder getEncoder(int numElements) {
+ PDataType type = PVarbinary.INSTANCE;
+ int estimatedSize = PArrayDataType.estimateSize(numElements,
type);
+ TrustedByteArrayOutputStream byteStream = new
TrustedByteArrayOutputStream(estimatedSize);
+ DataOutputStream oStream = new DataOutputStream(byteStream);
+ return new PArrayDataTypeEncoder(byteStream, oStream,
numElements, type, SortOrder.ASC, false,
PArrayDataType.IMMUTABLE_SERIALIZATION_VERSION);
+ }
+
+ @Override
+ public ColumnValueDecoder getDecoder() {
+ return new PArrayDataTypeDecoder();
+ }
+ };
{code}
Many of the test changes were because in PHOENIX-3519 I changed the default to
not use encoded column qualifiers and tSINGLE_CELL_ARRAY_WITH_OFFSETS storage
scheme. I reverted the change in this patch. This caused changes to
AlterMultiTenantTableWithViewsIT, StatsCollectorIT and MutationStateTest.
The following tests were parameterized : AlterTableIT, AlterTableWithViewsIT,
StoreNullsIT, StatsCollectorIT
The following tests changed because I renamed StorageScheme to
ImmutableStorageScheme : CorrelatePlanTest and LiteralResultIteratorPlanTest
I refactored ArrayConstructorExpressionTest and moved some other the tests into
a new test ImmutableStorageSchemeTest.
I renamed PArrayDataTypeBytesArrayBuilder to PArrayDataTypeEncoder and so
PDataTypeForArraysTest changed.
[~samarthjain] is going to add tests for different # of bytes for the column
qualifier.
COLUMN_ENCODED_BYTES does have a "none" option in the patch I attached to
PHOENIX-3519.
If we derive the immutableStorageScheme from the serialized array, then all
future serialization formats will need to store the enum ordinal in the
serialized array. It is only written and read once.
WritableUtils.writeEnum serializes the enum using the name, in the v2 patch I
modified the code to use the enum ordinal.
Samarth handled adding the IMMUTABLE_STORAGE_SCHEME column during the schema
upgrade in PHOENIX-3447.
> Add StorageScheme table property to allow users to specify their custom
> storage schemes
> ---------------------------------------------------------------------------------------
>
> Key: PHOENIX-3586
> URL: https://issues.apache.org/jira/browse/PHOENIX-3586
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: Thomas D'Silva
> Assignee: Thomas D'Silva
> Attachments: PHOENIX-3586.patch, PHOENIX-3586-v2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
