Keith, If an iterator has state that it is maintaining, what is the best way to transfer that state to the new iterator after a tear down? For example, MyIterator might have a Boolean flag of some sort. After tear down, is there a way to copy that state to the new iterator before it starts seeking again?
Roshan On Wed, Jan 4, 2017 at 10:33 AM Keith Turner <ke...@deenlo.com> wrote: > Josh, > > Deepcopy is not called when an iterator is torn down. It has an > entirely different use. Deepcopy allows cloning of an iterator during > init(). The clones allow you to have multiple pointers into a tablets > data which allows things like server side joins. > > Keith > > On Wed, Dec 28, 2016 at 12:50 PM, Josh Clum <joshc...@gmail.com> wrote: > > Hi, > > > > I have a question about iterator teardown. It seems from > > > https://github.com/apache/accumulo/blob/master/docs/src/main/asciidoc/chapters/iterator_design.txt#L383-L390 > > that deepCopy should be called when an iterator is torn down. I'm not > seeing > > that behavior. Below is a test that sets table.scan.max.memory to 1 which > > should force a tear down for each kv returned. I should see deepCopy > being > > called 3 times but when I tail the Tserver logs I'm not seeing it being > > called. Below is the test and the Tserver output. > > > > What am I missing here? > > > > Josh > > > > ➜ tail -f -n200 ...../accumulo/logs/TabletServer_*.out | grep MyIterator > > MyIterator: init > > MyIterator: seek > > MyIterator: hasTop > > MyIterator: getTopKey > > MyIterator: getTopValue > > MyIterator: init > > MyIterator: seek > > MyIterator: hasTop > > MyIterator: getTopKey > > MyIterator: getTopValue > > MyIterator: init > > MyIterator: seek > > MyIterator: hasTop > > MyIterator: getTopKey > > MyIterator: getTopValue > > MyIterator: init > > MyIterator: seek > > MyIterator: hasTop > > > > public static class MyIterator implements SortedKeyValueIterator<Key, > Value> > > { > > > > private SortedKeyValueIterator<Key, Value> source; > > > > public MyIterator() { } > > > > @Override > > public void init(SortedKeyValueIterator<Key, Value> source, > > Map<String, String> options, > > IteratorEnvironment env) throws IOException { > > System.out.println("MyIterator: init"); > > this.source = source; > > } > > > > @Override > > public boolean hasTop() { > > System.out.println("MyIterator: hasTop"); > > return source.hasTop(); > > } > > > > @Override > > public void next() throws IOException { > > System.out.println("MyIterator: next"); > > source.next(); > > } > > > > @Override > > public void seek(Range range, Collection<ByteSequence> > columnFamilies, > > boolean inclusive) throws IOException { > > System.out.println("MyIterator: seek"); > > source.seek(range, columnFamilies, inclusive); > > } > > > > @Override > > public Key getTopKey() { > > System.out.println("MyIterator: getTopKey"); > > return source.getTopKey(); > > } > > > > @Override > > public Value getTopValue() { > > System.out.println("MyIterator: getTopValue"); > > return source.getTopValue(); > > } > > > > @Override > > public SortedKeyValueIterator<Key, Value> > deepCopy(IteratorEnvironment > > env) { > > System.out.println("MyIterator: deepCopy"); > > return source.deepCopy(env); > > } > > } > > > > @Test > > public void testTearDown() throws Exception { > > String table = "test"; > > Connector conn = cluster.getConnector("root", "secret"); > > conn.tableOperations().create(table); > > conn.tableOperations().attachIterator(table, new IteratorSetting(25, > > MyIterator.class)); > > conn.tableOperations().setProperty(table, "table.scan.max.memory", > "1"); > > > > BatchWriter writer = conn.createBatchWriter(table, new > > BatchWriterConfig()); > > > > Mutation m1 = new Mutation("row"); > > m1.put("f1", "q1", 1, "val1"); > > writer.addMutation(m1); > > > > Mutation m2 = new Mutation("row"); > > m2.put("f2", "q2", 1, "val2"); > > writer.addMutation(m2); > > > > Mutation m3 = new Mutation("row"); > > m3.put("f3", "q3", 1, "val3"); > > writer.addMutation(m3); > > > > writer.flush(); > > writer.close(); > > > > BatchScanner scanner = conn.createBatchScanner(table, new > > Authorizations(), 3); > > scanner.setRanges(Collections.singletonList(new Range())); > > for(Map.Entry<Key, Value> entry : scanner) { > > System.out.println(entry.getKey() + " : " + entry.getValue()); > > } > > System.out.println("Results complete!"); > > } >
