[
https://issues.apache.org/jira/browse/METRON-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699932#comment-16699932
]
ASF GitHub Bot commented on METRON-1845:
----------------------------------------
Github user mmiklavc commented on a diff in the pull request:
https://github.com/apache/metron/pull/1247#discussion_r236521617
--- Diff:
metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/components/ElasticSearchComponent.java
---
@@ -194,35 +215,41 @@ public Client getClient() {
return client;
}
- public BulkResponse add(String indexName, String sensorType, String...
docs) throws IOException {
+ public void add(UpdateDao updateDao, String indexName, String
sensorType, String... docs)
+ throws IOException, ParseException {
List<String> d = new ArrayList<>();
Collections.addAll(d, docs);
- return add(indexName, sensorType, d);
+ add(updateDao, indexName, sensorType, d);
}
- public BulkResponse add(String indexName, String sensorType,
Iterable<String> docs)
- throws IOException {
- BulkRequestBuilder bulkRequest = getClient().prepareBulk();
- for (String doc : docs) {
- IndexRequestBuilder indexRequestBuilder = getClient()
- .prepareIndex(indexName, sensorType + "_doc");
-
- indexRequestBuilder = indexRequestBuilder.setSource(doc);
- Map<String, Object> esDoc = JSONUtils.INSTANCE
- .load(doc, JSONUtils.MAP_SUPPLIER);
- indexRequestBuilder.setId((String) esDoc.get(Constants.GUID));
- Object ts = esDoc.get("timestamp");
- if (ts != null) {
- indexRequestBuilder =
indexRequestBuilder.setTimestamp(ts.toString());
- }
- bulkRequest.add(indexRequestBuilder);
- }
+ public void add(UpdateDao updateDao, String indexName, String
sensorType, Iterable<String> docs)
--- End diff --
Might it be better to just use IndexDao, which `extends UpdateDao,
SearchDao, RetrieveLatestDao, ColumnMetadataDao`? To that end, if we're looking
to route all of this through the ES component in that fashion, it might make
sense to simply replace the internal `private Client client;` and instead use
the new desired IndexDao for the proxied calls to ES. It could be setup at
construction time of the component and remove the need to do the same thing for
every test that uses the component class, unless they actually want to do
something custom and pass in their own dao during init.
> Correct Test Data Load in Elasticsearch Integration Tests
> ---------------------------------------------------------
>
> Key: METRON-1845
> URL: https://issues.apache.org/jira/browse/METRON-1845
> Project: Metron
> Issue Type: Sub-task
> Reporter: Nick Allen
> Assignee: Nick Allen
> Priority: Major
>
> The Elasticsearch integration tests use the legacy Transport client to load
> test data into the search indexes before running the tests. Loading the test
> data like this does not accurately reflect how the indices will appear in a
> production environment.
> This should be changed to use our existing ElasticsearchUpdateDao to write
> the test data. This ensures that any changes made to the 'write' portion of
> our Elasticsearch code will function correctly with the 'read' portion. This
> ensures that telemetry written into Elasticsearch by 'Indexing' can be read
> correctly by the Alerts UI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
