Re: [DISCUSS] Move Type out of KeyValue

2017-10-03 Thread Stack
On Mon, Oct 2, 2017 at 11:46 PM, Chia-Ping Tsai wrote: > ... > > > We'd still be stuck at read time when all we had was Cell#getTypeByte > > returning a byte. > > > > We could add a CellType with the KV#Type static utility codeToType but > > would be sweet if we could do

Re: [DISCUSS] Move Type out of KeyValue

2017-10-03 Thread Chia-Ping Tsai
(reply again...) > You could add to CellBuilder methods to do setPutType, setDeleteType, etc., > which would work for build time allowing you could keep Type byte private. > This would seem to be enough for your case Chia-Ping? Yep, i will file a issue to address it. > We could add a CellType

Re: [DISCUSS] Move Type out of KeyValue

2017-10-03 Thread Chia-Ping Tsai
> You could add to CellBuilder methods to do setPutType, setDeleteType, etc., > which would work for build time allowing you could keep Type byte private. > This would seem to be enough for your case Chia-Ping? Yep, i will file a issue to address it. > We'd still be stuck at read time when all we

Re: [DISCUSS] Move Type out of KeyValue

2017-10-02 Thread Stack
On Mon, Oct 2, 2017 at 1:54 AM, Chia-Ping Tsai wrote: > How about introducing an new enum "CellType" which is subset of > KeyValue#Type? It will be exposed as IA.Public to end user for helping > build the custom cell (via CellBuilder). The types which "CellType" should >

Re: [DISCUSS] Move Type out of KeyValue

2017-10-02 Thread Chia-Ping Tsai
I don't want to impact the Cell...Maybe the new class "CellType" should be added into CellBuilderType. The Cell#getTypeByte() won't be changed. On 2017-10-02 21:50, Ted Yu wrote: > If CellType is added, should the following method of Cell be affected ? > > /** > >

Re: [DISCUSS] Move Type out of KeyValue

2017-10-02 Thread Ted Yu
If CellType is added, should the following method of Cell be affected ? /** * @return The byte representation of the KeyValue.TYPE of this cell: one of Put, Delete, etc */ byte getTypeByte(); Ideally CellType should be returned instead of a byte. On Mon, Oct 2, 2017 at 1:54 AM,

Re: [DISCUSS] Move Type out of KeyValue

2017-10-02 Thread Chia-Ping Tsai
How about introducing an new enum "CellType" which is subset of KeyValue#Type? It will be exposed as IA.Public to end user for helping build the custom cell (via CellBuilder). The types which "CellType" should have are shown below. 1) Put 2) Delete 3) DeleteFamilyVersion 4) DeleteColumn 5)

Re: [DISCUSS] Move Type out of KeyValue

2017-10-01 Thread Andrew Purtell
Ok, thanks. I understand now. +1 > On Sep 30, 2017, at 9:28 PM, Chia-Ping Tsai wrote: > > The "custom cell type" never exists in the story. (Sorry for misleading you) > > Here is the story. i add some custom cells (for saving memory) to Put via > Put#add(Cell). The

Re: [DISCUSS] Move Type out of KeyValue

2017-09-30 Thread Andrew Purtell
Thanks for sharing these details. They are intriguing. If possible could you explain why the custom type is needed? Something has to be deployed on the server or the custom cell type isn’t guaranteed to be handled correctly. It may work now by accident. I’m a little surprised a custom cell

Re: [DISCUSS] Move Type out of KeyValue

2017-09-30 Thread Chia-Ping Tsai
Thanks for the nice suggestions. Andrew. Sorry for delay response. Busy today. The root reason we must build own Cell on client side is that the data are located on shared memory which is similar with MSLAB. You are right. We can use attribute to carry our data but the byte[] is not acceptable

Re: [DISCUSS] Move Type out of KeyValue

2017-09-29 Thread Andrew Purtell
​Construct a normal put or delete or batch mutation, add whatever extra state you need in one or more operation attributes, and use a regionobserver to extend normal processing to handle the extra state. I'm curious what dispatching to extension code because of a custom cell type buys you over

Re: [DISCUSS] Move Type out of KeyValue

2017-09-29 Thread Chia-Ping Tsai
> Instead of a custom cell, could you use a regular cell with a custom > operation attribute (see OperationWithAttributes). Pardon me, I didn't get what you said. On 2017-09-30 04:31, Andrew Purtell wrote: > Instead of a custom cell, could you use a regular cell with a

Re: [DISCUSS] Move Type out of KeyValue

2017-09-29 Thread Andrew Purtell
Instead of a custom cell, could you use a regular cell with a custom operation attribute (see OperationWithAttributes). On Fri, Sep 29, 2017 at 1:28 PM, Chia-Ping Tsai wrote: > The custom cell help us to save memory consumption. We don't have own >

Re: [DISCUSS] Move Type out of KeyValue

2017-09-29 Thread Chia-Ping Tsai
The custom cell help us to save memory consumption. We don't have own serialization/deserialization mechanism, hence to transform data from client to server needs many conversion phase (user data -> Put/Cell -> pb object). The cost of conversion is large in transferring bulk data. In fact, we

Re: [DISCUSS] Move Type out of KeyValue

2017-09-29 Thread Andrew Purtell
What are the use cases for a custom cell? It seems a dangerously low level thing to attempt and perhaps we should unwind support for it. But perhaps there is a compelling justification. On Thu, Sep 28, 2017 at 10:20 PM, Chia-Ping Tsai wrote: > Thanks for all comment. > >

Re: [DISCUSS] Move Type out of KeyValue

2017-09-29 Thread Anoop John
Ya as Chia-Ping said, the problem he is trying to solve is very basic one. As long as we allow custom Cell creation (Via CellBuilder API) and allow Mutations to be added with Cells and pass that from client side APIs, we have to make the Type public accessible. Or else the Cell building APIs

Re: [DISCUSS] Move Type out of KeyValue

2017-09-29 Thread ramkrishna vasudevan
Even if we are trying to move out I think only few of the types are really user readable. So we should be very careful here. So since we have CellBuilder way it is better we check what type of cells a user can build. I think for now the Cellbuilder is not client exposed? But again moving to Cell

Re: [DISCUSS] Move Type out of KeyValue

2017-09-28 Thread Chia-Ping Tsai
Thanks for all comment. The problem i want to resolve is the valid code should be exposed as IA.Public. Otherwise, end user have to access the IA.Private class to build the custom cell. For example, I have a use case which plays a streaming role in our appliaction. It applies the

Re: [DISCUSS] Move Type out of KeyValue

2017-09-28 Thread Andrew Purtell
I agree with Stack. Was typing up a reply to Anoop but let me move it down here. The type code exposes some low level details of how our current stores are architected. But what if in the future you could swap out HStore implements Store with PStore implements Store, where HStore is backed by

Re: [DISCUSS] Move Type out of KeyValue

2017-09-28 Thread Stack
On Thu, Sep 28, 2017 at 2:25 AM, Chia-Ping Tsai wrote: > hi folks, > > User is allowed to create custom cell but the valid code of type - > KeyValue#Type - is declared as IA.Private. As i see it, we should expose > KeyValue#Type as Public Client. Three possible ways are

Re: [DISCUSS] Move Type out of KeyValue

2017-09-28 Thread Anoop John
Moving out of KV means it is BC break right? Even though KV is Private, using that only client side can create cell objects (At least Type specify) . Should we have a deprecation cycle? Deprecate KV.Type and add Cell.Type (?) -Anoop- On Thu, Sep 28, 2017 at 5:14 PM, Ted Yu

Re: [DISCUSS] Move Type out of KeyValue

2017-09-28 Thread Ted Yu
How about choosing option #2 ? Cell is annotated Public. On Thu, Sep 28, 2017 at 2:25 AM, Chia-Ping Tsai wrote: > hi folks, > > User is allowed to create custom cell but the valid code of type - > KeyValue#Type - is declared as IA.Private. As i see it, we should expose >

[DISCUSS] Move Type out of KeyValue

2017-09-28 Thread Chia-Ping Tsai
hi folks, User is allowed to create custom cell but the valid code of type - KeyValue#Type - is declared as IA.Private. As i see it, we should expose KeyValue#Type as Public Client. Three possible ways are shown below: 1) Change declaration of KeyValue#Type from IA.Private to IA.Public 2) Move