All:
I am using accumulo 1.6.1. I am using a sharded index to search for
data
that matches the values of certain fields. Here is the situation:
I have four fields EntityId, EntityIdType, EntityName, EntitySource.
Sometimes I need all records which match EntityId and EntityIdType
Othertimes I need all records which match all four fields.
The plan was use uses ranges in the scanner to determine which fields to
match against. I have tried both subsets of ranges, setting all ranges, and
gotten the same result.
I created an Index as follows:
RowId = fieldname
ColumnFamily = fieldvalue
ColumnQualifier = the overall record id (RowID) of my main record in
another table.
Here is the output of a scan of my index table:
entityid
1707945d-34d8-455d-85b1-55610739ce62:1707945d-34d8-455d-85b1-55610739ce62
[]
entityidtype GUID:1707945d-34d8-455d-85b1-55610739ce62 []
name TestEntity:1707945d-34d8-455d-85b1-55610739ce62 []
source Unit Test:1707945d-34d8-455d-85b1-55610739ce62 []
NOTE: While in this case the entityid equals the overall RowID from the
other table that is not always true
When I run the code below it does not return any rows in the scanner.
In the debugger when running the code below terms show as follows:
[1707945d-34d8-455d-85b1-55610739ce62, GUID, TestEntity, Unit Test]
I have tried both IntersectingIterator and IndexDocIterator both have
the same results.
For whatever reason the API docs for these classes is not showing up on
the Apache
Accumulo site.
Am I missing the purpose/function of this iterator?
Do I have to call IndexedDocIterator.setColfs with some values so I get
the column qualifiers back?
Below is my code:
public List<String> getCoalesceEntityKeysForEntityId(String entityId,
String
entityIdType,
String entityName,
String
entitySource) throws CoalescePersistorException
{
// Use are sharded term index to find the merged keys
Connector dbConnector = null;
ArrayList<String> keys = new ArrayList<String>();
Text[] terms = {new Text(entityId), new Text(entityIdType),
new Text(entityName), new Text(entitySource)};
try {
dbConnector = AccumuloDataConnector.getDBConnector();
BatchScanner keyscanner =
dbConnector.createBatchScanner(AccumuloDataConnector.coalesceEntityIndex,
Authorizations.EMPTY, 4);
// Set up an IntersectingIterator for the values
IteratorSetting iter = new IteratorSetting(1, "intersect",
IndexedDocIterator.class);
IndexedDocIterator.setColumnFamilies(iter,terms);
keyscanner.addScanIterator(iter);
// Use ranges to limit the bins searched
//ArrayList<Range> ranges = new ArrayList<Range>();
// May not be necessary to restrict ranges but will do it
to be safe
//ranges.add(new Range("entityid"));
//ranges.add(new Range("entityitype"));
//ranges.add(new Range("entityname"));
// ranges.add(new Range("source"));
//keyscanner.setRanges(ranges);
keyscanner.setRanges(Collections.singleton(new Range()));
// Return the list of keys
for(Entry<Key,Value> entry : keyscanner) {
keys.add(entry.getKey().getColumnQualifier().toString());
}
} catch (TableNotFoundException ex) {
System.err.println(ex.getLocalizedMessage());
return null;
}
return keys;
}
--
========= mailto:[email protected] ============
David W. Boyd
VP, Data Solutions
10432 Balls Ford, Suite 240
Manassas, VA 20109
office: +1-703-552-2862
cell: +1-703-402-7908
============== http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC - www.iliterobotics.org
Board Member- USSTEM Foundation - www.usstem.org
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.
