shuaijinchao opened a new issue #5636:
URL: https://github.com/apache/apisix/issues/5636
### Issue description
Splunk is a machine data search engine. According to the latest database
search engine ranking [1], Splunk has been ranked second, and Splunk can be
used to collect, index and retrieve various application data. Splunk, like
ElasticSearch, is quasi-real-time and can provide an uninterrupted data stream
of search results. I think Apache APISIX can provide a plugin that supports
pushing request logs to Splunk, expands the data observability of APISIX, and
reduces the cost of Splunk users using APISIX as a gateway.
### Name
- splunk-hec-logging
### Configuration
```json
{
"endpoint":{
"uri":"https://hec-splunk.company.com/services/collector";,
"token":"BD274822-96AA-4DA6-90EC-18940FB2414C",
"channel":"FE0ECFAD-13D5-401B-847D-77833BD77131",
"ssl":true
},
"inactive_timeout":10,
"max_retry_count":0,
"buffer_duration":60,
"retry_delay":1,
"batch_max_size":1
}
```
- `endpoint` Splunk HTTP Event Collector(HEC) endpoint
- `endpoint.uri` Splunk HTTP Event Collector(HEC) endpoint request uri
- `endpoint.token` Splunk HTTP Event Collector(HEC) endpoint request
access token [2]
- `endpoint.channel` Splunk HTTP Event Collector(HEC) endpoint channel
identifier (GUID) [3]
- `endpoint.ssl` Enable SSL Verify
- `max_retry_count` max number of retries before removing from the
processing pipe line
- `retry_delay` number of seconds the process execution should be
delayed if the execution fails
- `buffer_duration` max age in seconds of the oldest entry in a batch
before the batch must be processed
- `inactive_timeout` max age in seconds when the buffer will be flushed
if inactive
- `batch_max_size` max size of each batch
### Details
#### Configuration process
1. Add and set up the Http Event Controller (HEC) through the Splunk console
and get the access token.
2. If HEC enable the indexer to confirm, you must specify a channel and
obtain the channel ID.
3. Set the request URI, access token, and channel ID of HEC to the plugin
configuration
#### HTTP Request process
1. Obtain and assemble request information data in the APISIX Log stage,
data format refer to [4]
2. Add the assembled request data to the batch queue
3. When the threshold of the batch queue is triggered, the requested data is
submitted in batches to Splunk's HEC
[1] https://db-engines.com/en/ranking/search+engine
[2]
https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector#Create_an_Event_Collector_token_on_Splunk_Enterprise
[3]
https://docs.splunk.com/Documentation/Splunk/latest/Data/AboutHECIDXAck#About_channels_and_sending_data
[4]
https://docs.splunk.com/Documentation/Splunk/latest/Data/FormateventsforHTTPEventCollector#Event_metadata
### Environment
- apisix version (cmd: `apisix version`):
- OS (cmd: `uname -a`):
- OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`):
- etcd version, if have (cmd: run `curl
http://127.0.0.1:9090/v1/server_info` to get the info from server-info API):
- apisix-dashboard version, if have:
- the plugin runner version, if the issue is about a plugin runner (cmd:
depended on the kind of runner):
- luarocks version, if the issue is about installation (cmd: `luarocks
--version`):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
