[ https://issues.apache.org/jira/browse/HBASE-22623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902136#comment-16902136 ]
Geoffrey Jacoby commented on HBASE-22623: ----------------------------------------- So, as things stand now, [~apurtell] is -1 to a previous version of the patch that didn't contain WALEdit in the hook, and [~Apache9] is -1 to the current version that does contain it, leaving the null set. So where do we go from here to resolve things? I think [~anoop.hbase]'s last point is critical -- it's theoretically possible to accomplish "adding a Cell to the WALEdit of every mutation", but it's both 1. Incredibly complicated AND 2. Impossible to add to the WALEdit but not the MemStore / HFiles, which would be desirable in cases where you want to annotate metadata for consumption by a replication endpoint but not end-users of the client API. (Please correct me if I'm wrong) 3. Brittle anytime anyone changes any of the affected coprocessor hooks Spreading the code around also violates the single responsibility principle -- it's best to have all the code for a particular purpose in a single particular place. Between having to write over half a dozen coprocessor hooks with very different semantics (batch mutate hooks look different from single mutation hooks, append and increment look different from Put, etc), our hypothetical coproc developer needs to know and understand an enormous variety of concepts, not to mention the ordering of the entire write pipeline. Note how the discussion between Duo and Anoop above is finding it non-trivial, and you guys are among the foremost experts in the world on this. What exactly are the expectations for coproc developers? You seem to be assuming simultaneously that coproc developers need to be heavily constrained because their lack of knowledge is dangerous, but at the same time they are wise to be able to figure out complex workarounds to those constraints. (And to know when those complex workarounds are good (your proposal for WAL cell addition above) vs bad (you dislike the local index implementation).) Just exposing the WALEdit gives a simple, clean API that can be implemented in a few lines of code and isn't any more dangerous than any other existing coproc hook. > Add RegionObserver coprocessor hook for preWALAppend > ---------------------------------------------------- > > Key: HBASE-22623 > URL: https://issues.apache.org/jira/browse/HBASE-22623 > Project: HBase > Issue Type: New Feature > Reporter: Geoffrey Jacoby > Assignee: Geoffrey Jacoby > Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0 > > > While many coprocessor hooks expose the WALEdit to implementing coprocs, > there aren't any that expose the WALKey before it's created and added to the > WALEntry. > It's sometimes useful for coprocessors to be able to edit the WALKey, for > example to add extended attributes using the fields to be added in > HBASE-22622. -- This message was sent by Atlassian JIRA (v7.6.14#76016)