> On 2012-05-09 05:04:12, Brock Noland wrote: > > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, > > line 132 > > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line132> > > > > Wish: It'd be ideal if we could use mockito to pass it a fake HTable > > object and then test that transactions are handled correctly if Error and > > RuntimeException are thrown. > > Hari Shreedharan wrote: > I am not too familiar with mockito. I will do that in a different patch, > when I have time to pick up Mockito.
Sounds good, for a future reference FLUME-1131 uses Mockito. > On 2012-05-09 05:04:12, Brock Noland wrote: > > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, > > line 231 > > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line231> > > > > Maybe we should use a different default row key? I am guessing the row > > key prefix is supposed to be used to get around hot spotting due to the > > timestamp. Maybe UUID would be a better default? > > Hari Shreedharan wrote: > The idea is that the user gives an initial prefix in the conf. This way > they can supply different prefixes for different sinks, within the same > agent(and later identify which sink each of the rows came from). I agree that > using uuid is a better default, but the concerns I have are its size, and > also that scans will return the rows in a different order than inserted, > while inserting it using timestamps will guarantee that values inserted in a > specific order will be returned together. I would like your feedback on that, > if that is not a major use case, then I will change it to uuid, since the > implementation is also cleaner. Please let me know. Keys are so important, I wonder if this isn't an area where we should provide options via an interface and then three default implementations: 1) Prefixed timestamp (current) 2) Prefixed reverse timestamp (Long.MAX_VALUE - timestamp) (sorts newest to the top) 3) uuid/md5sum (randomly distributed keys) 4) User can extend the interface and generate their own keys based on the headers/body. - Brock ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7720 ----------------------------------------------------------- On 2012-05-09 03:04:07, Hari Shreedharan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/5073/ > ----------------------------------------------------------- > > (Updated 2012-05-09 03:04:07) > > > Review request for Flume. > > > Summary > ------- > > Hbase sink. > > > This addresses bug FLUME-1183. > https://issues.apache.org/jira/browse/FLUME-1183 > > > Diffs > ----- > > > flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java > PRE-CREATION > flume-ng-sinks/pom.xml acb3087 > pom.xml 8c11a2d > flume-ng-dist/pom.xml 5bdcfe7 > flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION > > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java > PRE-CREATION > bin/flume-ng 0108997 > > Diff: https://reviews.apache.org/r/5073/diff > > > Testing > ------- > > Unit tests added > > > Thanks, > > Hari > >
