Ali, Xiaolong, Instead of waiting for the whole pull request to pass, how about adding those go unit tests and code style verification as new Github Action?
So we can start moving towards Github Action. Thoughts? Thanks, Sijie On Tue, Dec 3, 2019 at 7:54 PM Ali Ahmed <alahmed...@gmail.com> wrote: > Absolutely we launch 30 jobs in parallel per pr so it should not be a > problem, once we finish migration of current jobs we can start the process > of adding new ones. > > -Ali > > On Tue, Dec 3, 2019 at 6:39 PM xiaolong ran <ranxiaolong...@gmail.com> > wrote: > > > Thanks Ali work for this, overall looks good to me. > > > > Currently, Jenkins does not check the unit tests and code specifications > > related > > to Go, as follows: > > > > - pulsar-client-go > > - pulsar-functions-go > > > > > > In GitHub action CI. Can we consider integrating them? > > https://github.com/apache/pulsar/issues/3774 < > > https://github.com/apache/pulsar/issues/3774> > > > > > > -- > > Thanks > > Xiaolong Ran > > > > > > > > > 在 2019年12月3日，上午8:52，Ali Ahmed <alahmed...@gmail.com> 写道： > > > > > > Here is PIP document > > > > > > > > > https://docs.google.com/document/d/13QKvfTj4-AMt3j7JTSE7LcyiXBK0Z0EGzp5HgMwpoj4/edit# > > > > > > On Sun, Dec 1, 2019 at 9:13 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > >> Without a PIP, the knowledge scatters out into many different places > and > > >> many different conversations. > > >> a PIP is better since other people in the community can participate in > > the > > >> development if they are interested in helping this. > > >> > > >> > > >> - Sijie > > >> > > >> On Sun, Dec 1, 2019 at 8:53 PM Ali Ahmed <alahmed...@gmail.com> > wrote: > > >> > > >>> Sure for unit test the plan is relatively simple. I have isolated a > > core > > >>> stable base which we can activate as a checkin condition. > > >>> > > >>> > > >>> > > >> > > > https://github.com/apache/pulsar/pull/5642/files#diff-00053b72be028281c6f9a446fbf91ef5 > > >>> > > >>> The unstable tests will be need to rewritten before being removed > from > > >>> exclusion we have benefits of dedicated machines for testing in > github > > so > > >>> we can give dedicated ports an an example to avoid contention. > > >>> This should take about a week a two or work. > > >>> > > >>> Integration tests are a bigger issue I don't think we can stabilize > the > > >>> current tests easily, we probably want to reorg things, move > > >>> external elements like spark, storm, flink and various io connectors > > in a > > >>> different module so we have lighter test load that we can harden > > against > > >>> failures. > > >>> > > >>> I can compile this into a PIP if community want's that. > > >>> > > >>> -Ali > > >>> > > >>> > > >>> > > >>> On Sun, Dec 1, 2019 at 8:37 PM Sijie Guo <guosi...@gmail.com> wrote: > > >>> > > >>>> Ali, > > >>>> > > >>>> Thank you for driving this. Overall looks good. > > >>>> > > >>>> I am fine with most of the proposal. However I would like to see > what > > >> is > > >>>> the plan to migrate the remaining java tests and integration tests. > > >>>> In your current email, there are a lot of unknowns regarding java > unit > > >>>> tests and integration tests. I would guess putting your proposal > into > > >>>> a PIP so that everyone can review the PIP before voting it. > > >>>> > > >>>> Thanks, > > >>>> Sijie > > >>>> > > >>>> > > >>>> On Sat, Nov 30, 2019 at 4:39 PM Ali Ahmed <alahmed...@gmail.com> > > >> wrote: > > >>>> > > >>>>> Hello everyone , > > >>>>> > > >>>>> I am doing experiments with github CI and I believe we are ready to > > >>> vote > > >>>> on > > >>>>> the transition plan. > > >>>>> Briefly the current jenkins env is problematic there is resources > > >>>>> contention, limited parallelism and lack of control to do > > >> experiments. > > >>>>> > > >>>>> The first version of the github actions ci is ready to merge. It > has > > >>>>> additional benefits that it won't trigger on documentation changes. > > >>>>> https://github.com/apache/pulsar/pull/5642 > > >>>>> > > >>>>> They were things tried but removed like caching artifacts, since > the > > >>>> cache > > >>>>> size is too small to use. > > >>>>> > > >>>>> From the current jobs in the workflow the cpp tests is stable > enough > > >> to > > >>>>> swap with the jenkins one we can merge this and request apache > infra > > >> to > > >>>>> disable the cpp jenkins job requirement. > > >>>>> I am experimenting with running unit tests and integration test in > > >>>> parallel > > >>>>> groups with subset of tests. It's not ready yet to be a checkin > > >>>> condition, > > >>>>> we need to iterate and improve it further. > > >>>>> > > >>>>> In summary the goal is to merge the current pr as is, swap the > > >> jenkins > > >>>> cpp > > >>>>> job and then further along move the unit test and integration test > > >>>> workflow > > >>>>> as they stabilize. > > >>>>> > > >>>> > > >>> > > >> > > > > >
