kezhenxu94 commented on code in PR #9575:
URL: https://github.com/apache/skywalking/pull/9575#discussion_r965793485
##########
docs/en/concepts-and-designs/lal.md:
##########
@@ -240,6 +272,69 @@ metrics:
exp: http_response_time.sum(['le', 'service',
'instance']).increase('PT5M').histogram().histogram_percentile([50,70,90,99])
```
+- SlowSql
+
+SlowSql aims to convert LogData to DatabaseSlowStatement. It extracts data
from `parsed` result and save them as DatabaseSlowStatement. SlowSql will not
abort or edit logs, you can use other LAL for further processing.
+SlowSql will reuse `service`, `layer` and `timestamp` of extractor, so it is
necessary to use `SlowSQL` after setting these.
+We require a log tag `"LOG_KIND" = "SLOW_SQL"` to make OAP distinguish slow
SQL logs from other log reports.
+An example of JSON sent to OAP is as following:
+``` json
+[
+ {
+ "tags":{
+ "data":[
+ {
+ "key":"LOG_KIND",
+ "value":"SLOW_SQL"
+ }
+ ]
+ },
+ "body":{
+ "json":{
+
"json":"{\"time\":\"20220906153959\",\"id\":\"cb92c1a5b-2691e-fb2f-457a-9c72a392d9ed\",\"service\":\"root[root]@[localhost]\",\"statement\":\"select
sleep(2);\",\"layer\":\"MYSQL\",\"query_time\":2000}"
+ }
+ },
+ "service":"root[root]@[localhost]"
+ }
+]
+```
+
+- `statement`
+
+`statement` extracts the time bucket from the `parsed` result, and set it into
the `DatabaseSlowStatement`, which will be
Review Comment:
```suggestion
`statement` extracts the SQL statement from the `parsed` result, and set it
into the `DatabaseSlowStatement`, which will be
```
##########
oap-server/server-starter/src/main/resources/lal/mysql-slowsql.yaml:
##########
@@ -0,0 +1,34 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+rules:
+ - name: mysql-slowsql
+ dsl: |
+ filter {
+ json{
+ }
+ extractor{
+ layer parsed.layer as String
+ service parsed.service as String
+ timestamp parsed.time as String
+ if (tag("LOG_KIND") == "SLOW_SQL") {
+ slowSql {
+ id parsed.id as String
+ statement parsed.statement as String
+ latency parsed.query_time as Long
+ }
Review Comment:
```suggestion
slowSql {
id parsed.id as String
statement parsed.statement as String
latency parsed.query_time as Long
}
```
##########
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/dsl/spec/extractor/ExtractorSpec.java:
##########
@@ -215,6 +239,48 @@ public void metrics(@DelegatesTo(SampleBuilder.class)
final Closure<?> cl) {
}
}
+ @SuppressWarnings("unused")
+ public void slowSql(@DelegatesTo(SlowSqlSpec.class) final Closure<?> cl) {
+ if (BINDING.get().shouldAbort()) {
+ return;
+ }
+ LogData.Builder log = BINDING.get().log();
+ if (log.getLayer() == null
+ || log.getService() == null
+ || log.getTimestamp() < 1) {
+ LOGGER.warn("SlowSql extracts failed, maybe something is not
configured.");
+ return;
+ }
+ DatabaseSlowStatementBuilder builder = new
DatabaseSlowStatementBuilder(namingControl);
+ builder.setLayer(Layer.nameOf(log.getLayer()));
+ builder.setTimeBucket(log.getTimestamp());
Review Comment:
Hi, timeBucket and timestamp are different format, so you might need
conversion here
##########
docs/en/concepts-and-designs/lal.md:
##########
@@ -240,6 +272,69 @@ metrics:
exp: http_response_time.sum(['le', 'service',
'instance']).increase('PT5M').histogram().histogram_percentile([50,70,90,99])
```
+- SlowSql
+
+SlowSql aims to convert LogData to DatabaseSlowStatement. It extracts data
from `parsed` result and save them as DatabaseSlowStatement. SlowSql will not
abort or edit logs, you can use other LAL for further processing.
+SlowSql will reuse `service`, `layer` and `timestamp` of extractor, so it is
necessary to use `SlowSQL` after setting these.
+We require a log tag `"LOG_KIND" = "SLOW_SQL"` to make OAP distinguish slow
SQL logs from other log reports.
+An example of JSON sent to OAP is as following:
+``` json
+[
+ {
+ "tags":{
+ "data":[
+ {
+ "key":"LOG_KIND",
+ "value":"SLOW_SQL"
+ }
+ ]
+ },
+ "body":{
+ "json":{
+
"json":"{\"time\":\"20220906153959\",\"id\":\"cb92c1a5b-2691e-fb2f-457a-9c72a392d9ed\",\"service\":\"root[root]@[localhost]\",\"statement\":\"select
sleep(2);\",\"layer\":\"MYSQL\",\"query_time\":2000}"
+ }
+ },
+ "service":"root[root]@[localhost]"
+ }
+]
+```
+
+- `statement`
+
+`statement` extracts the time bucket from the `parsed` result, and set it into
the `DatabaseSlowStatement`, which will be
+persisted (if not dropped) and is used to associate with TopNDatabaseStatement.
+
+- `latency`
+
+`latency` extracts the latency from the `parsed` result, and set it into the
`DatabaseSlowStatement`, which will be
+persisted (if not dropped) and is used to associate with TopNDatabaseStatement.
+
+- `id`
+
+`id` extracts the id from the `parsed` result, and set it into the
`DatabaseSlowStatement`, which will be persisted (if not
+dropped) and is used to associate with TopNDatabaseStatement.
+
+A Example of LAL to distinguish slow logs:
+
+```groovy
+filter {
+ json{
+ }
+ extractor{
+ layer parsed.layer as String
+ service parsed.service as String
+ timestamp parsed.time as String
+ if (tag("LOG_KIND") == "SLOW_SQL") {
+ slowSql {
+ id parsed.id as String
+ statement parsed.statement as String
+ latency parsed.query_time as Long
+ }
+ }
+ }
+}
Review Comment:
Let's format these lines
```suggestion
filter {
json{
}
extractor{
layer parsed.layer as String
service parsed.service as String
timestamp parsed.time as String
if (tag("LOG_KIND") == "SLOW_SQL") {
slowSql {
id parsed.id as String
statement parsed.statement as String
latency parsed.query_time as Long
}
}
}
}
```
##########
oap-server/server-starter/src/main/resources/lal/mysql-slowsql.yaml:
##########
@@ -0,0 +1,34 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+rules:
+ - name: mysql-slowsql
+ dsl: |
+ filter {
+ json{
+ }
+ extractor{
+ layer parsed.layer as String
+ service parsed.service as String
+ timestamp parsed.time as String
+ if (tag("LOG_KIND") == "SLOW_SQL") {
Review Comment:
If `default` is not removed from
https://github.com/apache/skywalking/blob/e6b5f32abe35dbb7e900afe0bcfba06b957fa904/oap-server/server-starter/src/main/resources/application.yml#L242-L242
they should be persisted.
##########
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/dsl/spec/AbstractSpec.java:
##########
@@ -49,4 +52,13 @@ public void abort(final Closure<Void> cl) {
public Object propertyMissing(final String name) {
return BINDING.get().getVariable(name);
}
+
+ @SuppressWarnings("unused")
+ public String tag(String key) {
+ return BINDING.get().log().getTags().getDataList()
+ .stream()
+ .filter(data -> key.equals(data.getKey()))
+ .map(KeyStringValuePair::getValue)
+ .collect(Collectors.toList()).get(0);
+ }
Review Comment:
You need to make sure this doesn't throw
```suggestion
@SuppressWarnings("unused")
public String tag(String key) {
return BINDING.get().log().getTags().getDataList()
.stream()
.filter(data -> key.equals(data.getKey()))
.map(KeyStringValuePair::getValue)
.findFirst()
.orElse("");
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
