Hi Duo, I was wondering, as per the comments in ticket [1] has mentioned NEW_VERSION_BEHAVIOR resolves previous known limitation, I have doubts how it resolves, may be I am missing some point here. We also do have similar scenario in our implementation, currently we separate delete and put using different time stamps. But in our tests there are intermittent failures even in that approach. We don't see these intermittent failures if we do flush() on buffered mutator per each delete and put call. In that case I think it violates buffered mutator purpose I guess. I would like to know, what is the recommended approach to handle such case ( Delete followed by put ) to prevent any masks? You input from HBase community is highly appreciated.
[1] https://issues.apache.org/jira/browse/HBASE-2256 Regards Kevin On Tue, Apr 16, 2019 at 2:50 PM 张铎(Duo Zhang) <palomino...@gmail.com> wrote: > This is good point... IIRC, we haven't considered the scenario where > we bundle several updates for the same column in a multi... > > Kevin Ratnasekera <djkevincr1...@gmail.com> 于2019年4月16日周二 下午5:03写道： > >> Hi Duo, >> >> Thank you for the response. As per the design, Can I please know, where >> this NEW_VERSION_BEHAVIOR is intended to work with Buffered Mutator? For >> example DELETE PUT PUT sequence ( in Buffered Mutator ) for same row, >> column will it work without any mask issues as previously considered as >> limitations HBase? >> >> Regards >> Kevin >> >> On Tue, Apr 16, 2019 at 2:03 PM 张铎(Duo Zhang) <palomino...@gmail.com> >> wrote: >> >>> Sorry, this behavior has not been tested critically yet... >>> >>> The guy who developed this feature has left and we haven't enabled this >>> feature in our cluster yet. >>> >>> So feel free to open issues if you find some bugs of it. We will try to >>> fix >>> it if possible. >>> >>> Thanks. >>> >>> Kevin Ratnasekera <djkevincr1...@gmail.com> 于2019年4月16日周二 上午3:09写道： >>> >>> > Code for delete row issue in cases of qualifier = null, even after row >>> is >>> > deleted it will return non null value, >>> > >>> > @Test >>> > public void testing() throws IOException { >>> > >>> > Connection conn = ConnectionFactory.createConnection(conf); >>> > >>> > TableName test = TableName.valueOf("Test"); >>> > TableDescriptorBuilder tableDescBuilder = >>> > TableDescriptorBuilder.newBuilder(test); >>> > ColumnFamilyDescriptorBuilder columnDescBuilder = >>> > ColumnFamilyDescriptorBuilder >>> > .newBuilder(Bytes.toBytes("test-family")); >>> > columnDescBuilder.setNewVersionBehavior(true); >>> > ColumnFamilyDescriptor columnDescriptor = columnDescBuilder.build(); >>> > tableDescBuilder.addColumnFamily(columnDescriptor); >>> > TableDescriptor tableDescriptor = tableDescBuilder.build(); >>> > >>> > conn.getAdmin().createTable(tableDescriptor); >>> > >>> > Table table = conn.getTable(test); >>> > >>> > Put put = new Put(Bytes.toBytes("com.example/http")); >>> > put.addColumn(Bytes.toBytes("test-family"), null, >>> > Bytes.toBytes("test-value")); >>> > table.put(put); >>> > >>> > Delete del = new Delete(Bytes.toBytes("com.example/http")); >>> > table.delete(del); >>> > >>> > Get get = new Get(Bytes.toBytes("com.example/http")); >>> > //get.addColumn(Bytes.toBytes("test-family"), null); >>> > Result result = table.get(get); >>> > byte [] value = result.getValue(Bytes.toBytes("test-family"), null); >>> > >>> > if(value == null) { >>> > // Testing purposes >>> > } >>> > } >>> > >>> > >>> > On Mon, Apr 15, 2019 at 10:51 PM Kevin Ratnasekera < >>> > djkevincr1...@gmail.com> >>> > wrote: >>> > >>> > > Hi all, >>> > > >>> > > Here's test code I used for testing purposes and I would like to >>> know the >>> > > behavior, >>> > > >>> > > >>> > > @Test >>> > > public void testing() throws IOException { >>> > > >>> > > Connection conn = ConnectionFactory.createConnection(conf); >>> > > >>> > > TableName test = TableName.valueOf("Test"); >>> > > TableDescriptorBuilder tableDescBuilder = >>> > TableDescriptorBuilder.newBuilder(test); >>> > > ColumnFamilyDescriptorBuilder columnDescBuilder = >>> > ColumnFamilyDescriptorBuilder >>> > > .newBuilder(Bytes.toBytes("test-family")); >>> > > columnDescBuilder.setNewVersionBehavior(true); >>> > > ColumnFamilyDescriptor columnDescriptor = >>> columnDescBuilder.build(); >>> > > tableDescBuilder.addColumnFamily(columnDescriptor); >>> > > TableDescriptor tableDescriptor = tableDescBuilder.build(); >>> > > >>> > > conn.getAdmin().createTable(tableDescriptor); >>> > > >>> > > Table table = conn.getTable(test); >>> > > BufferedMutator bm = conn.getBufferedMutator(test); >>> > > >>> > > Delete del = new Delete(Bytes.toBytes("com.example/http")); >>> > > del.addColumn(Bytes.toBytes("test-family"), >>> > Bytes.toBytes("test-qualifier")); >>> > > bm.mutate(del); >>> > > >>> > > Put put = new Put(Bytes.toBytes("com.example/http")); >>> > > put.addColumn(Bytes.toBytes("test-family"), >>> > Bytes.toBytes("test-qualifier"), Bytes.toBytes("test-value")); >>> > > bm.mutate(put); >>> > > >>> > > put = new Put(Bytes.toBytes("com.example/http")); >>> > > put.addColumn(Bytes.toBytes("test-family"), >>> > Bytes.toBytes("test-qualifier"), Bytes.toBytes("test-value-2")); >>> > > bm.mutate(put); >>> > > >>> > > bm.flush(); >>> > > bm.close(); >>> > > >>> > > Get get = new Get(Bytes.toBytes("com.example/http")); >>> > > get.addColumn(Bytes.toBytes("test-family"), >>> > Bytes.toBytes("test-qualifier")); >>> > > Result result = table.get(get); >>> > > byte [] val = result.getValue(Bytes.toBytes("test-family"), >>> > Bytes.toBytes("test-qualifier")); >>> > > >>> > > if(val == null) { >>> > > // Testing purposes >>> > > } >>> > > } >>> > > >>> > > I would like to know the exact behavior of value val variable here, >>> when >>> > NEW_VERSION_BEHAVIOR >>> > > is true. Above code when it is executed returns null as value for val >>> > > variable. As I can see, delete overshadows the subsequent puts. >>> > > >>> > > Regards >>> > > Kevin >>> > > >>> > > On Mon, Apr 15, 2019 at 2:05 PM Kevin Ratnasekera < >>> > djkevincr1...@gmail.com> >>> > > wrote: >>> > > >>> > >> Update: >>> > >> Found out that above delete row works all columns which have non >>> null >>> > >> qualifier. Delete doesn't work for puts with column qualifier is >>> null. >>> > >> >>> > >> On related note, >>> > >> >>> > >> Delete delete = new Delete(keyRaw); >>> > >> >>> > >> delete.addFamily(hcol.getFamily()); >>> > >> >>> > >> Let's say I need to perform delete on all columns created inside a >>> > family >>> > >> in a particular row, once I execute above delete operation, I am >>> not be >>> > >> able to do put operation on the same row same column family. >>> > >> >>> > >> Is there something I am missing here? >>> > >> >>> > >> Regards >>> > >> >>> > >> Kevin >>> > >> >>> > >> >>> > >> On Mon, Apr 15, 2019 at 12:20 AM Kevin Ratnasekera < >>> > >> djkevincr1...@gmail.com> wrote: >>> > >> >>> > >>> Hi all, >>> > >>> >>> > >>> I am from Apache Gora project. I currently in the process of >>> upgrade >>> > our >>> > >>> HBase dependencies to 2.0.5. We faced the issue described in [1] >>> and >>> > had to >>> > >>> put workarounds to get through it some time back. ( By timestamps ) >>> > Now I >>> > >>> started to test NEW_VERSION_BEHAVIOR and I noticed even trivial row >>> > deletes >>> > >>> fails when NEW_VERSION_BEHAVIOR is set to true. >>> > >>> >>> > >>> Delete del = new Delete(toBytes(key)); >>> > >>> >>> > >>> table = connection.getTable(tableName); >>> > >>> >>> > >>> table.delete(del); >>> > >>> Result result = table.get(new Get(toBytes(key))); >>> > >>> >>> > >>> Here Get returns some cells which are marked for delete. ( Some >>> columns >>> > >>> are deleted and some are not ) These deletes works fine when >>> > >>> NEW_VERSION_BEHAVIOR is set to false. Please note I do this tests >>> > using HBaseTestingUtility >>> > >>> minicluster. >>> > >>> >>> > >>> Are there any known issues with NEW_VERSION_BEHAVIOR with deletes? >>> It >>> > >>> would be really helpful if someone can give some pointers to debug >>> the >>> > root >>> > >>> cause of this. >>> > >>> >>> > >>> [1] https://issues.apache.org/jira/browse/HBASE-2256 >>> > >>> >>> > >>> Regards >>> > >>> Kevin >>> > >>> >>> > >> >>> > >>> >>
