Hi Ted,
Just to tell that unfortunately i wasn't seeing any INCLUDE_AND_SEEK_NEXT_ROW nor INCLUDE_AND_NEXT_COL when debuging so adding them were not going to make any difference. Nevetherless i implement De Morgan's law as you second suggestion (it was easier as I expected) and it's working, so thanks again for that! Best, ________________________________ De: Ted Yu <[email protected]> Enviado: lunes, 02 de enero de 2017 11:36 p.m. Para: [email protected] Asunto: Re: Is it possible to implement a NOT filter in Hbase? There is INCLUDE_AND_SEEK_NEXT_ROW which was not accounted for in the nested if statement. Is it possible for you to come up with unit test showing what you observe ? If you don't have time, please consider using De Morgan's law. Cheers On Mon, Jan 2, 2017 at 5:19 PM, Carl M <[email protected]> wrote: > Thanks for your response Ted. > > > I did the change, unfortunately it doesn't make any difference. > > > Best, > > ________________________________ > De: Ted Yu <[email protected]> > Enviado: lunes, 02 de enero de 2017 07:58 p.m. > Para: [email protected] > Asunto: Re: Is it possible to implement a NOT filter in Hbase? > > In the nested if statement, can you also handle NEXT_COL return code ? > > It should be translated to INCLUDE_AND_NEXT_COL > > Cheers > > > On Jan 2, 2017, at 1:54 PM, Carl M <[email protected]> wrote: > > > > Sorry for not being clear enough. > > > > > > Maybe i misunderstand your suggestion, but what I've done in implement > my own filter, wrapping a FilterList > > > > public class NotFilterList extends Filter { > > private FilterList filter; > > > > public NotFilterList(final List<Filter> rowFilters) { > > this.filter = new FilterList(rowFilters); > > } > > > > public void reset() throws IOException { > > this.filter.reset(); > > } > > > > public boolean filterRowKey(byte[] rowKey, int offset, int length) > throws IOException { > > return this.filter.filterRowKey(rowKey, offset, length); > > } > > > > public boolean filterAllRemaining() throws IOException { > > return this.filter.filterAllRemaining(); > > } > > > > public ReturnCode filterKeyValue(Cell v) throws IOException { > > ReturnCode code = this.filter.filterKeyValue(v); > > > > if (code == ReturnCode.INCLUDE) > > code = ReturnCode.SKIP; > > else if (code == ReturnCode.INCLUDE_AND_NEXT_COL) > > code = ReturnCode.NEXT_COL; > > else if (code == ReturnCode.NEXT_ROW) > > code = ReturnCode.INCLUDE_AND_NEXT_COL; > > else if (code == ReturnCode.SKIP) { > > code = ReturnCode.INCLUDE; > > } > > > > return code; > > } > > > > public Cell transformCell(Cell v) throws IOException { > > return this.filter.transformCell(v); > > } > > > > public KeyValue transform(KeyValue currentKV) throws IOException { > > return this.filter.transform(currentKV); > > } > > > > public void filterRowCells(List<Cell> kvs) throws IOException { > > this.filter.filterRowCells(kvs); > > } > > > > public boolean hasFilterRow() { > > return this.filter.hasFilterRow(); > > } > > > > public boolean filterRow() throws IOException { > > return this.filter.filterRow(); > > } > > > > public KeyValue getNextKeyHint(KeyValue currentKV) throws IOException > { > > return this.filter.getNextKeyHint(currentKV); > > } > > > > public Cell getNextCellHint(Cell currentKV) throws IOException { > > return this.filter.getNextCellHint(currentKV); > > } > > > > public boolean isFamilyEssential(byte[] name) throws IOException { > > return this.filter.isFamilyEssential(name); > > } > > > > public byte[] toByteArray() throws IOException { > > return this.filter.toByteArray(); > > } > > > > public static NotFilterList parseFrom(final byte [] pbBytes) throws > DeserializationException { > > FilterList filterList = FilterList.parseFrom(pbBytes); > > return new NotFilterList(filterList.getFilters()); > > } > > > > boolean areSerializedFieldsEqual(Filter other) { > > return this.filter.areSerializedFieldsEqual(other); > > } > > > > } > > > > > > What I mean is that filterKeyValue in the way i have it now, return the > right results but only the fields that were not originally skipped. > > > > > > So for example, if i have two rows each one with two fields > > > > Row 1 > > > > Name: Bill > > > > Surname: Gates > > > > > > Row 2 > > > > Name: Steve > > > > Surname: Jobs > > > > > > And I want to query for Rows that doesn't have Name 'Bill' > > > > NOT (Name='Bill') > > > > > > What I get as result from Hbase with this NotFilter is > > > > Row 2 > > > > Surname: Jobs > > > > > > I suppose it's related to the cell "Name: Steve" skipped in the first > place (before reversing the ReturnCode). > > > > > > Best, > > > > > > > > ________________________________ > > De: Ted Yu <[email protected]> > > Enviado: lunes, 02 de enero de 2017 06:31 p.m. > > Para: [email protected] > > Asunto: Re: Is it possible to implement a NOT filter in Hbase? > > > > bq. the cell/value that was originally skip is not return > > > > Can you be a bit more specific (with a concrete example) : the skipped > cell > > would not be returned (as indicated by the ReturnCode). > > > > Thanks > > > >> On Mon, Jan 2, 2017 at 1:06 PM, Carl M <[email protected]> wrote: > >> > >> Hi Ted, > >> > >> > >> I tried your suggestion, unfortunately it doesn't work as expected. I > >> don't fully understand FilterList, but if a cell value was skip and I > >> reverse the ReturnCode, i get the right row but the cell/value that was > >> originally skip is not return. > >> > >> I also tried reversing only filterRow() method of FilterList, but I got > >> the same behaviour (the original cell/value missing). > >> > >> > >> Best, > >> > >> > >> ________________________________ > >> De: Ted Yu <[email protected]> > >> Enviado: viernes, 30 de diciembre de 2016 12:56 p.m. > >> Para: [email protected] > >> Asunto: Re: Is it possible to implement a NOT filter in Hbase? > >> > >> I think the ReturnCode opposite INCLUDE_AND_NEXT_COL is NEXT_COL : > you're > >> not interested in any version of the current Cell. > >> > >> Cheers > >> > >>> On Fri, Dec 30, 2016 at 4:53 AM, Carl M <[email protected]> > wrote: > >>> > >>> Thanks Ted! Great idea replacing the value in filterKeyValue. Although > >> I'm > >>> not quite sure looking at FilterList code if only INCLUDE/SKIP should > be > >>> replaced, and which should be the correct replacement for > >>> INCLUDE_AND_NEXT_COL. What do you think? If not maybe i should try to > >>> implement DeMorgan's law but I think it would be harder. > >>> > >>> > >>> Best, > >>> > >>> ________________________________ > >>> De: Ted Yu <[email protected]> > >>> Enviado: jueves, 29 de diciembre de 2016 06:10 p.m. > >>> Para: [email protected] > >>> Asunto: Re: Is it possible to implement a NOT filter in Hbase? > >>> > >>> You can try negating the ReturnCode from filterKeyValue() (at the root > of > >>> FilterList): > >>> > >>> abstract public ReturnCode filterKeyValue(final Cell v) throws > >>> IOException; > >>> > >>> INCLUDE -> SKIP > >>> > >>> SKIP -> INCLUDE > >>> > >>> Alternatively, you can use De Morgan's law to transfer the condition: > >>> > >>> NOT (a = '123' AND b = '456') becomes > >>> > >>> (NOT a = '123') OR (b = '456') > >>> > >>>> On Thu, Dec 29, 2016 at 12:56 PM, Carl M <[email protected]> > >>> wrote: > >>> > >>>> Hi guys > >>>> > >>>> > >>>> I'm trying to implement some kind of NOT filter in Hbase, but don't > >> know > >>>> if possible, I'm playing with FilterIfMissing and FilterList.Operator > >> but > >>>> without luck. > >>>> > >>>> > >>>> I know how to return rows not having a specific column, but I mean > >>>> something like returning rows NOT fullfilling a condition, where > >>> condition > >>>> could be not only a SingleColumnValueFilter but a combined condition > >> with > >>>> FilterList. In SQL would be something like this for example > >>>> > >>>> > >>>> SELECT * FROM table WHERE NOT (a = '123' AND b = '456'); > >>>> > >>>> > >>>> Thanks in advance, > >> >
