Sure. Look forward for that. Could you share where your company is located? The city I mean. I may could have a chance to stop there to discuss f2f next year if there is some tech meeting happens there.
Sheng Wu 吴晟 Twitter, wusheng1108 立龙(Lilong) <[email protected]> 于2019年12月12日周四 下午4:27写道： > Yes, this solution bring in dependency of a new middleware, maybe a bit > heavy, but it seems that to solve the cross-network problem, middleware is > unavoidable. > > > I have not yet implemeted the idea of using Redis in the morning into > code. After I have finished one version of this solution and done enough > tests, I will show the details of the whole design. I will try my best to > optimize this implementation so that it can be adopted by the community. > > > Thanks. > _______________________________ > Lilong@iBOXCHAIN > Middleware Developer > > > On 12/12/2019 14:57，Sheng Wu<[email protected]> wrote： > 成龙 <[email protected]> 于2019年12月12日周四 上午11:33写道： > > > > OK, I will try to write a configurable register implementation. > Here are some of my thoughts: > 1. Choose a central middleware to generate global ID and provide a > channel to pub/sub the register message, Redis or Zookeeper should be > fine, but consider the need for an external network address, Redis would be > better. > 2. All the register global ID is from Redis, and the register request > will get to both clouds through pub/sub channel of Redis. > 3. Of course, this is a configurable feature, we can open the config > item when we truely faced with the cross-cloud problem. > > > I wonder if those thoughts will be accepted by the communitry as a > feature, we wanted to give all the changes back to the community, and > follow the version iteration of the community. > > > Thanks for considering this. I think the most challenge about this is, how > to make this works in the community. A new storage provider? Maybe? > In the community solution, we may need to keep the deployment solution as > simple as possible, but I don't know whether this solution is, look like it > includes Redis and some MQ tech. > Could you provide more detailed proposals? > > > > > > By the way, if the trace is from different Skywalking but in different > company, like we provide a payment service as a trading channel to another > company, and we both use Skywalking. Is there a problem with this scenario? > > > I think, technically, it is the same. Just make sure you are following your > company's security requirements. > > > > _______________________________ > Lilong@iBOXCHAIN > Middleware Developer > On 12/11/2019 21:48，Sheng Wu<[email protected]> wrote： > OK. I think I got the point. > > In my mind, one new storage provider is a better idea. > > I recommend you to read IRegisterLockDAO and IRegisterDAO implementations > in the storage provider(ES's, I think). > Then, you should try to write your own register implementation. Like > 1. Use one place to generate ID, such as using a central Redis to generate > ID. Across cloud accessible is the key point. > 2. Replicated write to both clouds for metadata. > 3. The query should keep your current change. > > You should be able to run all the things much more stable and easier than > what you do today. > > Sheng Wu 吴晟 > Twitter, wusheng1108 > > > 成龙 <[email protected]> 于2019年12月11日周三 下午9:40写道： > > > > Thanks for your reply! > > > Oh, yes, to avoid ID conflicts, we made some slight changes to OAP server, > and write a extra sync service. Details is as follows: > > > 1. First of all, we add a config item to es storage, called "startIndex: > ${SW_STORAGE_ES_START_INDEX:100000}", means all register ID is start from > this value(At the moment, we've written down a big enough number, it's not > a good idea, we are optimizing,figure out how to make the id unique). > Therefore, Skywalking in different cloud has a different sectional id. > 2. The extra sync service in Public Cloudwill subscribe thePublic Cloud es > data changes(only metadata), and send the data to Public Cloud > RocketMQ(Internet HTTP Endpoint), meanwhile the extra sync service in > Finance Cloud will receive the message, and execute the normal register > process. > 3. The message is carrying the ID which from cross-cloud, we use a map to > storage the relationship of the normal register id and the cross-cloud id. > 4. In the segment parse process, if the trace is across the cloud, we use > the id-relationship-map to exchange the cross-cloud id to normal id, make > Skywalking just like deal with the local trace. > 5. Of course, also need to make a slight change to > server-query-plugin#TraceQuery, make sure the UI shows the cross-cloud > trace properly. > > > I don't know if it's explained clearly. Thanks again for your reply! > > > > Lilong@iBOXCHAIN > Middleware Developer > Shenzhen iBOXCHAIN technology co. LTD > [email protected] > On 12/11/2019 20:31，Sheng Wu<[email protected]> wrote： > Hi Lilong > > Thanks for sharing the details of your scenario. > I may need a little more about the detail. > > I don't know how do you sync the metadata across the cloud? The register > could happen concurrently, how could you guarantee the ID is unique in the > global? > > > Sheng Wu 吴晟 > Twitter, wusheng1108 > > > 成龙 <[email protected]> 于2019年12月11日周三 下午6:24写道： > > > > Hello everyone, > > > We know that trace, topology and metrics across cloud was not expect, but > for a variety of reasons, we are now faced with the cross-cloud headache! > > > Our core payment service is in Alibaba Finance Cloud because of the > compliance audit, and most of the business service is in Alibaba Public > Cloud because of the cost considerations. > Here are some of our problems: > 1. Each cloud has thier own Skywalking which responsible for the > service tracing in their respective cloud regions; > 2. The network between the cloud is separated, we can't send the trace > segment to another directly; > 3. Most of payment requests in Finance are came from Public Cloud, so > we can't just ignore these traces; > 4. Besides if the trace is across cloud, it will throws > NullPointerException becuase of referenceId exchange > (ReferenceIdExchanger.java:60, baseed on 6.5.0), OAP server will print too > much exception stack. > > > To solve these problems, we simply sync the register metadata across the > cloud to make sure Skywalking in Finance Cloud can properly process the > cross-cloud segment. In this case, we can collect all segment whether the > trace is across cloud. Also we have the same traceId, we can find the same > trace in both side, even though it's just part of the whole trace. > > > I know maybe this is not a good idea, but it can solve our current > probolems. So I really look forward to a better solution, or if someone has > any good ideas or suggestions on this, please let me know! > > > Thanks. > > > | | > Lilong@iBOXCHAIN > Middleware Developer > | > | > Shenzhen iBOXCHAIN technology co. LTD > [email protected] > | > > >
