[
https://issues.apache.org/jira/browse/RANGER-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Petr Fedchenkov updated RANGER-4676:
------------------------------------
Description:
OpenSearch has its own java library to connect with and is not fully compatible
with bulk requests using elasticsearch library (at least v7). So let's
implement another destination.
With current elasticsearch destination I can see problems with response parsing:
```
23/09/22 10:13:20 ERROR BaseAuditHandler: Error sending message to ElasticSearch
java.io.IOException: Unable to parse response body for
Response{requestLine=POST /_bulk?timeout=1m HTTP/1.1,
host=[http://nia-spark-fedchenkov.ru-central1.internal:9200|http://nia-spark-fedchenkov.ru-central1.internal:9200/],
response=HTTP/1.1 200 OK}
at
org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1651)
at
org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1602)
at
org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1572)
at
org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:537)
at
org.apache.ranger.audit.destination.ElasticSearchAuditDestination.log(ElasticSearchAuditDestination.java:141)
at
org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
at org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NullPointerException
at java.base/java.util.Objects.requireNonNull(Objects.java:221)
at org.elasticsearch.action.DocWriteResponse.<init>(DocWriteResponse.java:127)
at org.elasticsearch.action.index.IndexResponse.<init>(IndexResponse.java:54)
at org.elasticsearch.action.index.IndexResponse.<init>(IndexResponse.java:39)
at
org.elasticsearch.action.index.IndexResponse$Builder.build(IndexResponse.java:107)
at
org.elasticsearch.action.index.IndexResponse$Builder.build(IndexResponse.java:104)
at
org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:159)
at
org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:188)
at
org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1911)
at
org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$8(RestHighLevelClient.java:1573)
at
org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1649)
... 7 more
```
was:
OpenSearch has its own java library to connect with and is not fully compatible
with bulk requests using elasticsearch library (at least v7). So let's
implement another destination.
With current elasticsearch destination I can see problems with response parsing:
23/09/22 10:13:20 ERROR BaseAuditHandler: Error sending message to ElasticSearch
java.io.IOException: Unable to parse response body for
Response\{requestLine=POST /_bulk?timeout=1m HTTP/1.1,
host=http://nia-spark-fedchenkov.ru-central1.internal:9200, response=HTTP/1.1
200 OK}
at
org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1651)
at
org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1602)
at
org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1572)
at
org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:537)
at
org.apache.ranger.audit.destination.ElasticSearchAuditDestination.log(ElasticSearchAuditDestination.java:141)
at
org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
at
org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NullPointerException
at java.base/java.util.Objects.requireNonNull(Objects.java:221)
at
org.elasticsearch.action.DocWriteResponse.<init>(DocWriteResponse.java:127)
at
org.elasticsearch.action.index.IndexResponse.<init>(IndexResponse.java:54)
at
org.elasticsearch.action.index.IndexResponse.<init>(IndexResponse.java:39)
at
org.elasticsearch.action.index.IndexResponse$Builder.build(IndexResponse.java:107)
at
org.elasticsearch.action.index.IndexResponse$Builder.build(IndexResponse.java:104)
at
org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:159)
at
org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:188)
at
org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1911)
at
org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$8(RestHighLevelClient.java:1573)
at
org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1649)
... 7 more
> OpenSearch destination implementation
> -------------------------------------
>
> Key: RANGER-4676
> URL: https://issues.apache.org/jira/browse/RANGER-4676
> Project: Ranger
> Issue Type: Improvement
> Components: audit
> Reporter: Petr Fedchenkov
> Priority: Trivial
>
> OpenSearch has its own java library to connect with and is not fully
> compatible with bulk requests using elasticsearch library (at least v7). So
> let's implement another destination.
> With current elasticsearch destination I can see problems with response
> parsing:
> ```
> 23/09/22 10:13:20 ERROR BaseAuditHandler: Error sending message to
> ElasticSearch
> java.io.IOException: Unable to parse response body for
> Response{requestLine=POST /_bulk?timeout=1m HTTP/1.1,
> host=[http://nia-spark-fedchenkov.ru-central1.internal:9200|http://nia-spark-fedchenkov.ru-central1.internal:9200/],
> response=HTTP/1.1 200 OK}
> at
> org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1651)
> at
> org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1602)
> at
> org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1572)
> at
> org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:537)
> at
> org.apache.ranger.audit.destination.ElasticSearchAuditDestination.log(ElasticSearchAuditDestination.java:141)
> at
> org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
> at org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: java.lang.NullPointerException
> at java.base/java.util.Objects.requireNonNull(Objects.java:221)
> at org.elasticsearch.action.DocWriteResponse.<init>(DocWriteResponse.java:127)
> at org.elasticsearch.action.index.IndexResponse.<init>(IndexResponse.java:54)
> at org.elasticsearch.action.index.IndexResponse.<init>(IndexResponse.java:39)
> at
> org.elasticsearch.action.index.IndexResponse$Builder.build(IndexResponse.java:107)
> at
> org.elasticsearch.action.index.IndexResponse$Builder.build(IndexResponse.java:104)
> at
> org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:159)
> at
> org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:188)
> at
> org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1911)
> at
> org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$8(RestHighLevelClient.java:1573)
> at
> org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1649)
> ... 7 more
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
