Thanks for your reply!
Oh, yes, to avoid ID conflicts, we made some slight changes to OAP server, and
write a extra sync service. Details is as follows:
1. First of all, we add a config item to es storage, called "startIndex:
${SW_STORAGE_ES_START_INDEX:100000}", means all register ID is start from this
value(At the moment, we've written down a big enough number, it's not a good
idea, we are optimizing,figure out how to make the id unique). Therefore,
Skywalking in different cloud has a different sectional id.
2. The extra sync service in Public Cloudwill subscribe thePublic Cloud es data
changes(only metadata), and send the data to Public Cloud RocketMQ(Internet
HTTP Endpoint), meanwhile the extra sync service in Finance Cloud will receive
the message, and execute the normal register process.
3. The message is carrying the ID which from cross-cloud, we use a map to
storage the relationship of the normal register id and the cross-cloud id.
4. In the segment parse process, if the trace is across the cloud, we use the
id-relationship-map to exchange the cross-cloud id to normal id, make
Skywalking just like deal with the local trace.
5. Of course, also need to make a slight change to
server-query-plugin#TraceQuery, make sure the UI shows the cross-cloud trace
properly.
I don't know if it's explained clearly. Thanks again for your reply!
Lilong@iBOXCHAIN
Middleware Developer
Shenzhen iBOXCHAIN technology co. LTD
[email protected]
On 12/11/2019 20:31，Sheng Wu<[email protected]> wrote：
Hi Lilong
Thanks for sharing the details of your scenario.
I may need a little more about the detail.
I don't know how do you sync the metadata across the cloud? The register
could happen concurrently, how could you guarantee the ID is unique in the
global?
Sheng Wu 吴晟
Twitter, wusheng1108
成龙 <[email protected]> 于2019年12月11日周三 下午6:24写道：
Hello everyone,
We know that trace, topology and metrics across cloud was not expect, but
for a variety of reasons, we are now faced with the cross-cloud headache!
Our core payment service is in Alibaba Finance Cloud because of the
compliance audit, and most of the business service is in Alibaba Public
Cloud because of the cost considerations.
Here are some of our problems:
1. Each cloud has thier own Skywalking which responsible for the
service tracing in their respective cloud regions;
2. The network between the cloud is separated, we can't send the trace
segment to another directly;
3. Most of payment requests in Finance are came from Public Cloud, so
we can't just ignore these traces;
4. Besides if the trace is across cloud, it will throws
NullPointerException becuase of referenceId exchange
(ReferenceIdExchanger.java:60, baseed on 6.5.0), OAP server will print too
much exception stack.
To solve these problems, we simply sync the register metadata across the
cloud to make sure Skywalking in Finance Cloud can properly process the
cross-cloud segment. In this case, we can collect all segment whether the
trace is across cloud. Also we have the same traceId, we can find the same
trace in both side, even though it's just part of the whole trace.
I know maybe this is not a good idea, but it can solve our current
probolems. So I really look forward to a better solution, or if someone has
any good ideas or suggestions on this, please let me know!
Thanks.
| |
Lilong@iBOXCHAIN
Middleware Developer
|
|
Shenzhen iBOXCHAIN technology co. LTD
[email protected]
|
