kezhenxu94@apache <[email protected]> 于2020年11月15日周日 下午9:16写道：
> > > I want to ask, how the docker-compose lands in your mind? I think CLI > can't > > control this part. > > > docker-compose just fits the requirements perfectly that we need, we have > been using it in the current E2E and turns out it has advantages in > starting many services correctly with health check and dependency > declaration. (One may say Kubernetes can do the same thing, but it’s kind > of heavy in E2E and GitHub Actions and complex). > > The SkyWalking-CLI won’t get involved with docker-compose related things, > it just provides query ability (the same as what it is doing now, just need > to check whether it covers all the needed query interfaces or not), we will > have a dedicated control program to > > (1) start docker-compose (existing standard mechanism, no new knowledge); > So, we would provide a script set to drive this? My question is more about, how to work with docker-compose, rather than challanging the choice. Sheng Wu 吴晟 Twitter, wusheng1108 > (2) query actual data (by SkyWalking-CLI, existing repository and codes); > (3) verify the actual data (by verification tool, a new tool, TODO); > (4) and determine the test result (success or failed); > > > ————————— > Zhenxu Ke (柯振旭) > GitHub @kezhenxu94 > > > > > Sheng Wu 吴晟 > > Twitter, wusheng1108 > > > > > > kezhenxu94@apache <[email protected]> 于2020年11月15日周日 下午5:27写道： > > > >> Hi the community, I want to raise a discussion to build the next > >> generation of E2E test framework for Apache SkyWalking, which may not > be a > >> high priority but a necessity. > >> > >> As we already know, tests in SkyWalking play a very important role in > the > >> contribution process, and the current E2E tests just work well to verify > >> every single pull request before merging, so why bother to build the > next > >> generation of E2E test framework? > >> > >> 1. In the SkyWalking's ecosystem, there are many sub-projects that vary > in > >> terms of language, almost all of them need E2E tests to help us verify > the > >> pull requests, but we can see that they reimplement the E2E test > framework > >> in their languages, (or even worse, some of them don’t have E2E tests at > >> all). Reimplementing the E2E test framework is unnecesarry and > introduces > >> more duplicated work when adding a new test case. The ultimate goal is > to > >> reuse all the test cases when needed. > >> > >> 2. The current E2E framework is built with Java, running Java-based > >> program is not an easy thing for non-Java developers (maintainers of > other > >> sub-projects such as PHP, C++, LUA, etc.), they need to set up the > >> environment correctly and then run the tests, neither is it an easy > skill > >> to debug the tests. > >> > >> 3. It’s a good opportunity to optimize the E2E tests because we added > many > >> cases and many duplicated codes that need to be removed. > >> > >> Therefore we're planning to build an unified, easy-to-use, and fast E2E > >> test framework to benifit all the sub-projects. > >> > >> Here are some rough ideas about this framework: > >> > >> 0. (Unchanged) We will continue to use docker-compose.yaml to > orchestrate > >> the services that are to be tested, it’s proved to be friendly and > >> debuggable because we can start it directly even if we are not writing > E2E > >> tests. > >> 1. This framework will take full advantages of the CLI project to query > >> the traces/metrics/logs, we don’t want to maintain two versions of query > >> codes anymore, (now we have a version in the test codes and one in the > CLI). > >> 2. We will build a CLI tool to compare the expected data and the actual > >> data, we now have a custom schema of the expected data yaml file, the > >> verification codes should be packaged into an executable command so > that it > >> can be executed standalone without Java and maven knowledge. I hope we > can > >> leverage existing standards to write the YAML files and do > verifications. > >> 3. Build an orchestrating tool to start the `docker-compose.yaml`, > health > >> check, query actual data, and verify the result. > >> 4. (Nice to have) wrap all the tools as a GitHub Actions to share > between > >> sub-projects. > >> > >> If you have any other better idea or want to complement to this > proposal, > >> please reply here. I will create an issue to track these tasks later. > >> > >> We have two students (who just finished the Summer Code 2020 Projects) > to > >> lead this part, Ke Zhang [1] and Huaxi Jiang [2], as I said this is a > not > >> high priority, you have adequate time to get yourselves familiar with > how > >> the components (OAP, WebUI, CLI, storages, etc.) of SkyWalking work as > well > >> as the ecosystem. Thanks for your interests and willingness to help. > >> > >> [1] https://github.com/Humbertzhang > >> [2] https://github.com/fgksgf/ > >> > >> ————————— > >> Zhenxu Ke (柯振旭) > >> GitHub @kezhenxu94 > >> > >> > > ————————— > Zhenxu Ke (柯振旭) > GitHub @kezhenxu94 > >
