From: Jesse Yates <jesse.k.ya...@gmail.com> Date: Wed, 5 Mar 2014 18:15:59 -0800
bq. I'd rather do that one layer up as I proposed in the PhoenixWriterCommiter - it helps separate concerns for cases where users want to use the indexing but don't want all the rest of the phoenix stuff. Would you mind commenting on the JIRA instead of the dev list? On Wed, Mar 5, 2014 at 6:15 PM, Jesse Yates <jesse.k.ya...@gmail.com> wrote: > I'd rather do that one layer up as I proposed in the PhoenixWriterCommiter > - it helps separate concerns for cases where users want to use the indexing > but don't want all the rest of the phoenix stuff. > > ------------------- > Jesse Yates > @jesse_yates > jyates.github.com > > > On Wed, Mar 5, 2014 at 6:09 PM, James Taylor (JIRA) <j...@apache.org> > wrote: > > > > > [ > > > https://issues.apache.org/jira/browse/PHOENIX-113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921900#comment-13921900 > ] > > > > James Taylor commented on PHOENIX-113: > > -------------------------------------- > > > > My proposal for 3.0 is to just go through a new KeyValueBuilder that > > potentially just makes a copy of the Mutation with a real KeyValue in the > > case of ClientKeyValue. In ParallelWriterIndexCommitter:114: > > {code} > > final List<Mutation> mutations = (List<Mutation>) entry.getValue(); > > // Pass the list through to the KeyValueBuilder to potentially make > > copies of them, as > > // the memstore requires the KeyValue to have a backing buffer > > mutations = kvBuilder.copyMutationsIfNecessary(mutations); > > {code} > > > > Seems worth the effort, given that we've done all this work for > > KeyValueBuilder and we're getting very little benefit from it. > > > > > Enable usage of ClientKeyValue on for indexing on server > > > -------------------------------------------------------- > > > > > > Key: PHOENIX-113 > > > URL: https://issues.apache.org/jira/browse/PHOENIX-113 > > > Project: Phoenix > > > Issue Type: Bug > > > Affects Versions: 3.0.0 > > > Reporter: James Taylor > > > Fix For: 3.0.0 > > > > > > Attachments: phoenix-113-sketch.txt > > > > > > > > > Modify code to go through KeyValueBuilder where necessary. Adding > > abstraction for having different KVComparator on each KeyValueBuilder > impl. > > > Still doesn't work on the server-side due to memstore needing a backing > > buffer for the KeyValue. > > > According to [~jesse_yates], the options are: > > > 1. Update the ClientKeyValues on the way out via a new hook in the > > codec, e.g. "preWriteToIndexTable(List<Mutation>, byte[] indexTable)", > > > Wrap all the clientkvs in something like an ImmutableClientKeyValue > that > > does the right thing for getBuffer, getOffset, etc, but without doing all > > the copying. This may be easier to just do as modifications to > > ClientKeyValue, but seems easier to start as a new class first > > > 2. Automatically check all the mutations being written in the > > ParallelIndexWriter for being clientKeyvalues > > > A bit more painful than the above, since you could short circuit there, > > but easier in terms of code complexity > > > 3. Check to see if the write is going to a local region and then > > transform the ClientKeyValues when necessary > > > Most difficult as it reaches down into the guts of the coprocessor > logic > > for determining region location. Unfortunately, it is already being done > by > > the CoprocessorHConnection, but we would need to recreate it for us > before > > it hits the HTable. > > > You could do it by creating a CoprocessorHConnection and then wrapping > > that in a ClientKeyValueCheckingHConnection and then passing that into > the > > constructor for HTable creation. > > > This is even more onerous as that constructor is only available on an > > HTable directly, not via the CoprocessorEnvironment, meaning we have to > > manually close tables. We have pretty good exception wrapping, so its not > > terrible to implement, but still a pain to do. > > > > > > > > -- > > This message was sent by Atlassian JIRA > > (v6.2#6252) > > >
