This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push: new 9dc4fd3b01 Support available layers of service in the topology. (#11844) 9dc4fd3b01 is described below commit 9dc4fd3b01e7e13d5a3cb83753e2fb74784b0ad8 Author: Wan Kai <wankai...@foxmail.com> AuthorDate: Tue Jan 30 12:51:15 2024 +0800 Support available layers of service in the topology. (#11844) --- docs/en/changes/changes.md | 7 ++- docs/en/swip/SWIP-4.md | 57 ++++++++++++++++++++++ docs/en/swip/readme.md | 1 + .../server/core/query/ServiceTopologyBuilder.java | 19 ++++++++ .../oap/server/core/query/type/Node.java | 5 ++ .../HierarchyQueryServiceTest.java | 2 +- .../core/query/ServiceTopologyBuilderTest.java | 51 ++++++++++++++++--- .../src/main/resources/query-protocol | 2 +- skywalking-ui | 2 +- 9 files changed, 134 insertions(+), 12 deletions(-) diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md index 6e7f129229..662bd179ed 100644 --- a/docs/en/changes/changes.md +++ b/docs/en/changes/changes.md @@ -38,13 +38,14 @@ - VIRTUAL_DATABASE -> MYSQL - VIRTUAL_DATABASE -> POSTGRESQL * Add Golang as a supported language for AMQP. +* Support available layers of service in the topology. #### UI * Fix the mismatch between the unit and calculation of the "Network Bandwidth Usage" widget in Linux-Service Dashboard. * Add theme change animation. * Implement the Service and Instance hierarchy topology. -* Support Tabs in the widget visiable when MQE expressions. +* Support Tabs in the widget visible when MQE expressions. * Support search on Marketplace. * Fix default route. * Fix layout on the Log widget. @@ -55,6 +56,10 @@ * Fix dashboard `K8S-Service-Root` metrics expression. * Add dashboards for Service/Instance Hierarchy. * Fix MQE in dashboards when using `Card widget`. +* Optimize tooltips style. +* Fix resizing window causes the trace graph to display incorrectly. +* Add the not found page(404). +* Enhance VNode logic and support multiple Trace IDs in span's ref. #### Documentation diff --git a/docs/en/swip/SWIP-4.md b/docs/en/swip/SWIP-4.md new file mode 100644 index 0000000000..6db61f6160 --- /dev/null +++ b/docs/en/swip/SWIP-4.md @@ -0,0 +1,57 @@ +# Support available layers of service in the topology. + +## Motivation + +UI could jump to the service dashboard and query service hierarchy from the topology node. +For now topology node includes name and ID but without layer, as the service could have multiple layers, +the limitation is that it is only works on the current layer which the topology represents: +1. UI could not jump into another layer's dashboard of the service. +2. UI could not query the service hierarchy from the topology node if the node is not in current layer. + +Here are typical use cases: +should have a chance to jump into another layer's dashboard of the service: +1. In the mesh topology, mesh(layer MESH) and mesh-dp(layer MESH_DP) share a similar topology, one node will have two layers. +2. In the mesh topology, agent(layer GENERAL) + virtual database(layer VIRTUAL_DATABASE), the node is in different layers. + +Both of these two cases have hybrid layer topology. If we could support that, we could have a better x-layer interaction. + +## Architecture Graph + +There is no significant architecture-level change. + +## Propose Changes + +Add the layers info into topology node: +1. When building the topology node fetch the layers info from the service according to the service id. +2. Return `layers` info in the `Node` when query the topology. + +## Imported Dependencies libs and their licenses. + +No new library is planned to be added to the codebase. + +## Compatibility + +About the **protocol**, there should be no breaking changes, but enhancements only. New field `layers` is going to be added to the +`Node` in the query protocol `topology.graphqls`. + +```graphql +type Node { + # The service ID of the node. + id: ID! + # The literal name of the #id. + name: String! + # The type name may be + # 1. The service provider/middleware tech, such as: Tomcat, SpringMVC + # 2. Conjectural Service, e.g. MySQL, Redis, Kafka + type: String + # It is a conjecture node or real node, to represent a service or endpoint. + isReal: Boolean! + # The layers of the service. + layers: [String!]! +} +``` + +## General usage docs + +This proposal doesn't impact the end user in any way of using SkyWalking. The remarkable change will be in the UI topology map, +users could jump into the proper layer's service dashboard and query the service hierarchy from the topology node. diff --git a/docs/en/swip/readme.md b/docs/en/swip/readme.md index 0e0c031a2d..3961e7d89c 100644 --- a/docs/en/swip/readme.md +++ b/docs/en/swip/readme.md @@ -71,6 +71,7 @@ All accepted and proposed SWIPs could be found in [here](https://github.com/apac Next SWIP Number: 4 ### Accepted SWIPs +- [SWIP-4 Support available layers of service in the topology](SWIP-4.md) - [SWIP-3 Support RocketMQ Monitoring](SWIP-3.md) - [SWIP-2 Collecting and Gathering Kubernetes Monitoring Data](SWIP-2.md) - [SWIP-1 Create and detect Service Hierarchy Relationship](SWIP-1.md) diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilder.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilder.java index 288534fb77..181e272e3b 100644 --- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilder.java +++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilder.java @@ -22,6 +22,7 @@ import java.util.HashMap; import java.util.LinkedList; import java.util.List; import java.util.Map; +import lombok.SneakyThrows; import lombok.extern.slf4j.Slf4j; import org.apache.skywalking.oap.server.core.Const; import org.apache.skywalking.oap.server.core.CoreModule; @@ -31,6 +32,7 @@ import org.apache.skywalking.oap.server.core.cache.NetworkAddressAliasCache; import org.apache.skywalking.oap.server.core.config.IComponentLibraryCatalogService; import org.apache.skywalking.oap.server.core.query.type.Call; import org.apache.skywalking.oap.server.core.query.type.Node; +import org.apache.skywalking.oap.server.core.query.type.Service; import org.apache.skywalking.oap.server.core.query.type.Topology; import org.apache.skywalking.oap.server.core.source.DetectPoint; import org.apache.skywalking.oap.server.library.module.ModuleManager; @@ -41,6 +43,8 @@ class ServiceTopologyBuilder { private final IComponentLibraryCatalogService componentLibraryCatalogService; private final NetworkAddressAliasCache networkAddressAliasCache; private final String userID; + private final ModuleManager moduleManager; + private MetadataQueryService metadataQueryService; ServiceTopologyBuilder(ModuleManager moduleManager) { this.componentLibraryCatalogService = moduleManager.find(CoreModule.NAME) @@ -50,6 +54,16 @@ class ServiceTopologyBuilder { .provider() .getService(NetworkAddressAliasCache.class); this.userID = IDManager.ServiceID.buildId(Const.USER_SERVICE_NAME, false); + this.moduleManager = moduleManager; + } + + private MetadataQueryService getMetadataQueryService() { + if (metadataQueryService == null) { + this.metadataQueryService = moduleManager.find(CoreModule.NAME) + .provider() + .getService(MetadataQueryService.class); + } + return metadataQueryService; } Topology build(List<Call.CallDetail> serviceRelationClientCalls, List<Call.CallDetail> serviceRelationServerCalls) { @@ -193,11 +207,16 @@ class ServiceTopologyBuilder { return topology; } + @SneakyThrows private Node buildNode(String sourceId, IDManager.ServiceID.ServiceIDDefinition sourceService) { Node serviceNode = new Node(); serviceNode.setId(sourceId); serviceNode.setName(sourceService.getName()); serviceNode.setReal(sourceService.isReal()); + Service service = getMetadataQueryService().getService(sourceId); + if (service != null) { + serviceNode.getLayers().addAll(service.getLayers()); + } return serviceNode; } } diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/type/Node.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/type/Node.java index cb224bd2b1..a9ef562dbd 100644 --- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/type/Node.java +++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/type/Node.java @@ -18,6 +18,8 @@ package org.apache.skywalking.oap.server.core.query.type; +import java.util.HashSet; +import java.util.Set; import lombok.Getter; import lombok.Setter; import org.apache.skywalking.oap.server.core.query.NotGraphQLField; @@ -35,6 +37,9 @@ public class Node { @Getter @Setter private boolean isReal; + @Getter + @Setter + private Set<String> layers = new HashSet<>(); /** * A flag indicate whether the {@link #type} has been set from the call detected from service side. diff --git a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/Hierarchy/HierarchyQueryServiceTest.java b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/hierarchy/HierarchyQueryServiceTest.java similarity index 99% rename from oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/Hierarchy/HierarchyQueryServiceTest.java rename to oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/hierarchy/HierarchyQueryServiceTest.java index 8049ca52c3..f0310eac0d 100644 --- a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/Hierarchy/HierarchyQueryServiceTest.java +++ b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/hierarchy/HierarchyQueryServiceTest.java @@ -16,7 +16,7 @@ * */ -package org.apache.skywalking.oap.server.core.Hierarchy; +package org.apache.skywalking.oap.server.core.hierarchy; import java.util.HashMap; import java.util.Map; diff --git a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilderTest.java b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilderTest.java index f5905fd743..a643a668c0 100644 --- a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilderTest.java +++ b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilderTest.java @@ -20,17 +20,21 @@ package org.apache.skywalking.oap.server.core.query; import java.util.ArrayList; import java.util.List; +import java.util.Set; +import lombok.SneakyThrows; import org.apache.skywalking.oap.server.core.CoreModule; import org.apache.skywalking.oap.server.core.CoreModuleConfig; import org.apache.skywalking.oap.server.core.CoreModuleProvider; import org.apache.skywalking.oap.server.core.MockModuleManager; import org.apache.skywalking.oap.server.core.MockModuleProvider; import org.apache.skywalking.oap.server.core.analysis.IDManager; +import org.apache.skywalking.oap.server.core.analysis.Layer; import org.apache.skywalking.oap.server.core.cache.NetworkAddressAliasCache; import org.apache.skywalking.oap.server.core.config.ComponentLibraryCatalogService; import org.apache.skywalking.oap.server.core.config.IComponentLibraryCatalogService; import org.apache.skywalking.oap.server.core.query.type.Call; import org.apache.skywalking.oap.server.core.query.type.Node; +import org.apache.skywalking.oap.server.core.query.type.Service; import org.apache.skywalking.oap.server.core.query.type.Topology; import org.apache.skywalking.oap.server.core.source.DetectPoint; import org.apache.skywalking.oap.server.library.module.ModuleManager; @@ -38,14 +42,19 @@ import org.junit.jupiter.api.Assertions; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.Mockito; +import org.powermock.reflect.Whitebox; + +import static org.mockito.Mockito.when; public class ServiceTopologyBuilderTest { private CoreModuleProvider moduleProvider; private ModuleManager moduleManager; + private MetadataQueryService metadataQueryService; @BeforeEach public void setupMetrics() throws Throwable { moduleProvider = Mockito.mock(CoreModuleProvider.class); + metadataQueryService = Mockito.mock(MetadataQueryService.class); moduleManager = new MockModuleManager() { @Override protected void init() { @@ -62,16 +71,22 @@ public class ServiceTopologyBuilderTest { }; } + @SneakyThrows @Test public void testServiceTopologyBuild() { + Service svrA = getSvrA(); + Service svrB = getSvrB(); final ServiceTopologyBuilder serviceTopologyBuilder = new ServiceTopologyBuilder(moduleManager); + Whitebox.setInternalState(serviceTopologyBuilder, "metadataQueryService", metadataQueryService); + when(metadataQueryService.getService(svrA.getId())).thenReturn(svrA); + when(metadataQueryService.getService(svrB.getId())).thenReturn(svrB); List<Call.CallDetail> serviceRelationClientCalls = new ArrayList<>(); Call.CallDetail call1 = new Call.CallDetail(); call1.buildFromServiceRelation( IDManager.ServiceID.buildRelationId( new IDManager.ServiceID.ServiceRelationDefine( - IDManager.ServiceID.buildId("SvrA", true), - IDManager.ServiceID.buildId("SvrB", true) + IDManager.ServiceID.buildId(svrA.getName(), true), + IDManager.ServiceID.buildId(svrB.getName(), true) ) ), // mtls @@ -83,8 +98,8 @@ public class ServiceTopologyBuilderTest { call2.buildFromServiceRelation( IDManager.ServiceID.buildRelationId( new IDManager.ServiceID.ServiceRelationDefine( - IDManager.ServiceID.buildId("SvrA", true), - IDManager.ServiceID.buildId("SvrB", true) + IDManager.ServiceID.buildId(svrA.getName(), true), + IDManager.ServiceID.buildId(svrB.getName(), true) ) ), // http @@ -98,8 +113,8 @@ public class ServiceTopologyBuilderTest { call3.buildFromServiceRelation( IDManager.ServiceID.buildRelationId( new IDManager.ServiceID.ServiceRelationDefine( - IDManager.ServiceID.buildId("SvrA", true), - IDManager.ServiceID.buildId("SvrB", true) + IDManager.ServiceID.buildId(svrA.getName(), true), + IDManager.ServiceID.buildId(svrB.getName(), true) ) ), // mtls @@ -111,8 +126,8 @@ public class ServiceTopologyBuilderTest { call4.buildFromServiceRelation( IDManager.ServiceID.buildRelationId( new IDManager.ServiceID.ServiceRelationDefine( - IDManager.ServiceID.buildId("SvrA", true), - IDManager.ServiceID.buildId("SvrB", true) + IDManager.ServiceID.buildId(svrA.getName(), true), + IDManager.ServiceID.buildId(svrB.getName(), true) ) ), // http @@ -125,8 +140,10 @@ public class ServiceTopologyBuilderTest { for (final Node node : topology.getNodes()) { if (node.getName().equals("SvrB")) { Assertions.assertEquals("http", node.getType()); + Assertions.assertEquals(Set.of(Layer.MESH.name(), Layer.MESH_DP.name()), node.getLayers()); } else if (node.getName().equals("SvrA")) { Assertions.assertEquals(null, node.getType()); + Assertions.assertEquals(Set.of(Layer.GENERAL.name()), node.getLayers()); } } for (final Call call : topology.getCalls()) { @@ -134,4 +151,22 @@ public class ServiceTopologyBuilderTest { Assertions.assertEquals(List.of("mtls", "http"), call.getTargetComponents()); } } + + private Service getSvrA() { + Service service = new Service(); + service.setId(IDManager.ServiceID.buildId("SvrA", true)); + service.setName("SvrA"); + service.setShortName("SvrA"); + service.setLayers(Set.of(Layer.GENERAL.name())); + return service; + } + + private Service getSvrB() { + Service service = new Service(); + service.setId(IDManager.ServiceID.buildId("SvrB", true)); + service.setName("SvrB"); + service.setShortName("SvrB"); + service.setLayers(Set.of(Layer.MESH.name(), Layer.MESH_DP.name())); + return service; + } } diff --git a/oap-server/server-query-plugin/query-graphql-plugin/src/main/resources/query-protocol b/oap-server/server-query-plugin/query-graphql-plugin/src/main/resources/query-protocol index 72fdfe5660..f0bdba688a 160000 --- a/oap-server/server-query-plugin/query-graphql-plugin/src/main/resources/query-protocol +++ b/oap-server/server-query-plugin/query-graphql-plugin/src/main/resources/query-protocol @@ -1 +1 @@ -Subproject commit 72fdfe56603c6a6135f932fb402c0890bb78b4e7 +Subproject commit f0bdba688afecff2167d018ecdaf509d2c8598f7 diff --git a/skywalking-ui b/skywalking-ui index f76500bb6e..860af150f7 160000 --- a/skywalking-ui +++ b/skywalking-ui @@ -1 +1 @@ -Subproject commit f76500bb6e5e796171f393994f88ede62a370b02 +Subproject commit 860af150f71f8dd0a2cbf555c8f7bd97954c5e53