How does HBase treat end keys?

[Lewis John Mcgibbney]

Hi,

Whilst working on some tests for Apache Gora, we've discovered a problem
with one of them. The following test [1], which I have also pasted below
(I've made the area if code we are concerned with *bold* to try and point
it out clearly), expects the last key in a range that was deleted to be
present. The developer that reported the issue believes that the end key in
a query should be inclusive, but our test treats as exclusive. Having
searched the mailing lists [2] I am still not 100% certain what HBase's
behaviour is... I wonder if someone can clarify this for me and I can make
the commit accordingly to get the test working properly.

Thank you very much in advance for any information/direction on this one.

Kind Regards

Lewis

[1]
http://svn.apache.org/viewvc/incubator/gora/trunk/gora-core/src/test/java/org/apache/gora/store/DataStoreTestUtil.java?view=markup
[2]
http://article.gmane.org/gmane.comp.java.hadoop.hbase.user/7017/match=endkey+exclusive
----------------------------------------------

 public static void testDeleteByQueryFields(DataStore<String, WebPage>
store)
 throws IOException {

   Query<String, WebPage> query;

   //test 5 - delete all with some fields
   WebPageDataCreator.createWebPageData(store);

   query = store.newQuery();
   query.setFields(WebPage.Field.OUTLINKS.getName()
       , WebPage.Field.PARSED_CONTENT.getName(),
WebPage.Field.CONTENT.getName());

   assertNumResults(store.newQuery(), URLS.length);
   store.deleteByQuery(query);
   store.deleteByQuery(query);
   store.deleteByQuery(query);//don't you love that HBase sometimes does
not delete arbitrarily

   store.flush();

   assertNumResults(store.newQuery(), URLS.length);

   //assert that data is deleted
   for (int i = 0; i < SORTED_URLS.length; i++) {
     WebPage page = store.get(SORTED_URLS[i]);
     Assert.assertNotNull(page);

     Assert.assertNotNull(page.getUrl());
     Assert.assertEquals(page.getUrl().toString(), SORTED_URLS[i]);
     Assert.assertEquals(0, page.getOutlinks().size());
     Assert.assertEquals(0, page.getParsedContent().size());
     if(page.getContent() != null) {
       System.out.println("url:" + page.getUrl().toString());
       System.out.println( "limit:" + page.getContent().limit());
     } else {
       Assert.assertNull(page.getContent());
     }
   }

   //test 6 - delete some with some fields
   WebPageDataCreator.createWebPageData(store);

   query = store.newQuery();
   query.setFields(WebPage.Field.URL.getName());
   String startKey = SORTED_URLS[NUM_KEYS];
   String endKey = SORTED_URLS[SORTED_URLS.length - NUM_KEYS];
   query.setStartKey(startKey);
   query.setEndKey(endKey);

   assertNumResults(store.newQuery(), URLS.length);
   store.deleteByQuery(query);
   store.deleteByQuery(query);
   store.deleteByQuery(query);//don't you love that HBase sometimes does
not delete arbitrarily

   store.flush();

   assertNumResults(store.newQuery(), URLS.length);

   //assert that data is deleted
   for (int i = 0; i < URLS.length; i++) {
     WebPage page = store.get(URLS[i]);
     Assert.assertNotNull(page);
*     if( URLS[i].compareTo(startKey) < 0 || URLS[i].compareTo(endKey) > 0)
{ *
       //not deleted
       assertWebPage(page, i);
     } else {
       //deleted
       Assert.assertNull(page.getUrl());
       Assert.assertNotNull(page.getOutlinks());
       Assert.assertNotNull(page.getParsedContent());
       Assert.assertNotNull(page.getContent());
       Assert.assertTrue(page.getOutlinks().size() > 0);
       Assert.assertTrue(page.getParsedContent().size() > 0);
     }
   }



-- 
Lewis