hailin0 edited a comment on issue #6261:
URL: https://github.com/apache/skywalking/issues/6261#issuecomment-817330492
# logback demo
### logback pattern
```text
[%tc] [%level] %date{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %logger{96}:%line -
%msg%n
```
### logback output to file
```text
[TC:N/A] [INFO] 2021-04-11 17:23:33.371 [thread-main] com.example.Main:100 -
log message 1
[TC:[service-a,[email protected],N/A,-1,N/A]]
[INFO] 2021-04-11 17:23:33.371 [thread-web-1]
com.example.controller.TestController:100 - log message 2
[TC:[service-a,[email protected],ddd92f52207c468e9cd03ddd107cd530.69.16181331190470001,0,ddd92f52207c468e9cd03ddd107cd530.69.16181331190470000]]
[INFO] 2021-04-11 17:25:19.062 [thread-web-2]
com.example.controller.TestController:150 - log message 3
```
### filebeat.yml config
```yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /tmp/filebeat/filebeat-in.log
processors:
- dissect:
tokenizer: '[TC:%{TC}] [%{logTags.level}] %{datetime}
[%{logTags.thread}] %{logTags.logger}:%{line} - %{body}'
field: "message"
target_prefix: ""
overwrite_keys: true
- dissect:
tokenizer:
'[%{service},%{serviceInstance},%{traceContext.traceId},%{traceContext.spanId},%{traceContext.traceSegmentId}]'
field: "TC"
target_prefix: ""
- script:
lang: javascript
id: filter_datetime_format
source: >
function process(event) {
var datetime = event.Get("datetime");
var datetimeArr = datetime.split(" ");
var dateArr = datetimeArr[0].split("-");
var timeArrs = datetimeArr[1].split(".");
var timeArr = timeArrs[0].split(":");
var mills = timeArrs[1];
var date = new Date();
date.setUTCFullYear(parseInt(dateArr[0]));
date.setUTCMonth(parseInt(dateArr[1]) - 1);
date.setUTCDate(parseInt(dateArr[2]));
date.setUTCHours(parseInt(timeArr[0]));
date.setUTCMinutes(parseInt(timeArr[1]));
date.setUTCSeconds(parseInt(timeArr[2]));
date.setUTCMilliseconds(parseInt(mills));
var timeMillis = date.getTime();
event.Put("timestamp", timeMillis);
}
- copy_fields:
fields:
- from: log.file.path
to: logTags.logfile
- from: agent.hostname
to: logTags.hostname
- from: agent.type
to: logTags.agent
- from: agent.version
to: logTags.agentVersion
- from: agent.id
to: logTags.agentId
- from: agent.name
to: logTags.agentName
fail_on_error: false
ignore_missing: true
- drop_fields:
fields: ["@timestamp", "@metadata", "ecs", "host", "input", "agent",
"log", "TC", "line", "message", "datetime"]
output.file:
path: "/tmp/filebeat"
filename: filebeat-out.log
```
### filebeat output json to kafka
```json
{
"@timestamp":"2021-04-11T15:40:49.025Z",
"@metadata":{
"beat":"filebeat",
"type":"_doc",
"version":"7.12.0"
},
"service":"service-a",
"serviceInstance":"[email protected]",
"traceContext":{
"traceId":"ddd92f52207c468e9cd03ddd107cd530.69.16181331190470001",
"spanId":"0",
"traceSegmentId":"ddd92f52207c468e9cd03ddd107cd530.69.16181331190470000"
},
"logTags":{
"thread":"thread-web-1",
"level":"INFO",
"logger":"com.example.controller.TestController",
"hostname":"5d960e25ed4c",
"filepath":"/tmp/filebeat/filebeat-in.log",
"agent":"filebeat",
"agentId":"9744570a-0e17-43c8-8958-c5f102ef7eb5",
"agentName":"5d960e25ed4c",
"agentVersion":"7.12.0"
},
"timestamp":1618161813371,
"body":"log message 3"
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
