Github user DLotts commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/149#discussion_r135072243
--- Diff:
extras/indexing/src/main/java/org/apache/rya/indexing/accumulo/temporal/AccumuloTemporalIndexer.java
---
@@ -102,42 +103,50 @@
private boolean isInit = false;
+ /**
+ * intilize the temporal index.
+ * This is dependent on a few set method calls before init:
+ * > Connector = ConfigUtils.getConnector(conf);
+ * > MultiTableBatchWriter mtbw =
connector.createMultiTableBatchWriter(new BatchWriterConfig());
+ * > // optional: temporal.setConnector(connector);
+ * > temporal.setMultiTableBatchWriter(mtbw);
+ * > temporal.init();
+ */
+ @Override
+ public void init() {
+ if (!isInit) {
+ try {
+ initInternal();
+ isInit = true;
+ } catch (final AccumuloException | AccumuloSecurityException |
TableNotFoundException | TableExistsException | RyaClientException e) {
+ logger.warn("Unable to initialize index. Throwing Runtime
Exception. ", e);
+ throw new RuntimeException(e);
+ }
+ }
+ }
private void initInternal() throws AccumuloException,
AccumuloSecurityException, TableNotFoundException,
- TableExistsException {
+ TableExistsException, RyaClientException {
+ if (mtbw == null)
+ throw new RyaClientException("Failed to initialize temporal
index, setMultiTableBatchWriter() was not set.");
+ if (conf == null)
+ throw new RyaClientException("Failed to initialize temporal
index, setConf() was not set.");
+
temporalIndexTableName = getTableName();
// Create one index table on first run.
- ConfigUtils.createTableIfNotExists(conf, temporalIndexTableName);
-
- mtbw = ConfigUtils.createMultitableBatchWriter(conf);
-
+ Boolean isCreated = ConfigUtils.createTableIfNotExists(conf,
temporalIndexTableName);
+ if (isCreated) {
+ logger.info("First run, created temporal index table: " +
temporalIndexTableName);
+ }
temporalIndexBatchWriter =
mtbw.getBatchWriter(temporalIndexTableName);
-
validPredicates = ConfigUtils.getTemporalPredicates(conf);
}
//initialization occurs in setConf because index is created using
reflection
@Override
public void setConf(final Configuration conf) {
this.conf = conf;
- if (!isInit) {
--- End diff --
In AccumuloRyaDAO, for each secondary index, it calls setConf, then creates
a batchwriter, then sets the batchwriter, then calls init() last. Indexes
should not create there own batchwriter because it does not get flushed
properly.
If we call init() from setConf(), then the batchwriter does not exist yet.
I think several other indexes have this issue.
This was discovered by someone outside our team because it does not fail
for unit tests. I discussed it with Aaron. He said that Indexes should accept
a batchwriter , and then implement init() to be called by AccumuloRyaDAO.
Am I missing something? Here is the code from AccumuloRyaDAO.
```
for (AccumuloIndexer index : secondaryIndexers) {
index.setConf(conf);
}
... Create a batch writer used by all indexes here ...
...
for (AccumuloIndexer index : secondaryIndexers) {
index.setConnector(connector);
index.setMultiTableBatchWriter(mt_bw);
index.init();
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
