Re: [Suggestion] PR failing if test coverage went down
Cool. I'm busy recently and may not have time to make this change. So I'm going to raise a issue in github repo and put everything together, if anyone interested in this, he/she can help with this. On 2018/04/16 04:04:39, shang zonghai wrote: > +1 for considering small changes. > > We need resources to improve existing unit tests and increase unit coverage. > > Best Regards！ > yiji
Re: [Suggestion] PR failing if test coverage went down
+1 for considering small changes. We need resources to improve existing unit tests and increase unit coverage. Best Regards！ yiji
Re: [Suggestion] PR failing if test coverage went down
+1 for considering small changes. Pull request is welcome! On Mon, Apr 16, 2018 at 10:56 AM, hty...@gmail.com wrote: > If we just want to make sure we don't reject any document fix or tiny fix. X > should be 0.05 > If we want to accept some small change, we can consider 0.1 to 0.3 > > Regards, > Yunkun Huang > > On 2018/04/16 01:36:17, Huxing Zhang wrote: >> Hi, >> >> On Thu, Apr 12, 2018 at 11:07 AM, hty...@gmail.com wrote: >> > yeah. I understand your concern. >> > >> > This is a common requirement for coverage calc service. Codecov support >> > this feature. >> > In codecov.yml, we can config threshold value which allow the coverage to >> > drop by X%, and posting a success status. >> >> That is awesome! >> >> What is your suggestion of X? >> >> > >> > Regards, >> > Yunkun Huang >> > >> > On 2018/04/10 06:36:03, Huxing Zhang wrote: >> >> Hi, >> >> >> >> Thanks for the suggestion. >> >> >> >> I generally agree with the idea that we should prevent a pull request >> >> to be merged if the code coverage gets lower. >> >> >> >> But my concern is the code coverage might have a minor drop even when >> >> just fixing typo[1], which makes confused. >> >> >> >> Unless we can figure it out, or I think it will be better if we can >> >> set a threshold. >> >> >> >> For example, if the code coverage drops >=0.1%, prevent the PR to be >> >> merged. >> >> >> >> [1] https://github.com/apache/incubator-dubbo/pull/1562 >> >> >> >> On Tue, Apr 10, 2018 at 10:21 AM, htynkn wrote: >> >> > hey team, >> >> > >> >> > >> >> > In pull request >> >> > template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), >> >> > this is one item to ask all pr to >> >> > >> >> > >> >> >"Write necessary unit-test to verify your logic correction" >> >> > >> >> > >> >> > Codecov has feature to fail PR if test coverage went down. Which means >> >> > for any pull request, if it add some code without any unit test, the PR >> >> > will fail >> >> > This can help to increase test coverage. >> >> > >> >> > >> >> > Any thoughts? >> >> > >> >> > >> >> > Relate links: >> >> > 1.https://docs.codecov.io/docs/commit-status >> >> > 2.https://github.com/codecov/support/issues/351 >> >> > >> >> >> >> -- >> >> Best Regards！ >> >> Huxing >> >> >> >> >> >> -- >> Best Regards！ >> Huxing >> -- Best Regards！ Huxing
Re: [Suggestion] PR failing if test coverage went down
If we just want to make sure we don't reject any document fix or tiny fix. X should be 0.05 If we want to accept some small change, we can consider 0.1 to 0.3 Regards, Yunkun Huang On 2018/04/16 01:36:17, Huxing Zhang wrote: > Hi, > > On Thu, Apr 12, 2018 at 11:07 AM, hty...@gmail.com wrote: > > yeah. I understand your concern. > > > > This is a common requirement for coverage calc service. Codecov support > > this feature. > > In codecov.yml, we can config threshold value which allow the coverage to > > drop by X%, and posting a success status. > > That is awesome! > > What is your suggestion of X? > > > > > Regards, > > Yunkun Huang > > > > On 2018/04/10 06:36:03, Huxing Zhang wrote: > >> Hi, > >> > >> Thanks for the suggestion. > >> > >> I generally agree with the idea that we should prevent a pull request > >> to be merged if the code coverage gets lower. > >> > >> But my concern is the code coverage might have a minor drop even when > >> just fixing typo[1], which makes confused. > >> > >> Unless we can figure it out, or I think it will be better if we can > >> set a threshold. > >> > >> For example, if the code coverage drops >=0.1%, prevent the PR to be > >> merged. > >> > >> [1] https://github.com/apache/incubator-dubbo/pull/1562 > >> > >> On Tue, Apr 10, 2018 at 10:21 AM, htynkn wrote: > >> > hey team, > >> > > >> > > >> > In pull request > >> > template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), > >> > this is one item to ask all pr to > >> > > >> > > >> >"Write necessary unit-test to verify your logic correction" > >> > > >> > > >> > Codecov has feature to fail PR if test coverage went down. Which means > >> > for any pull request, if it add some code without any unit test, the PR > >> > will fail > >> > This can help to increase test coverage. > >> > > >> > > >> > Any thoughts? > >> > > >> > > >> > Relate links: > >> > 1.https://docs.codecov.io/docs/commit-status > >> > 2.https://github.com/codecov/support/issues/351 > >> > > >> > >> -- > >> Best Regards！ > >> Huxing > >> > > > > -- > Best Regards！ > Huxing >
Re: [Suggestion] PR failing if test coverage went down
Hi, On Thu, Apr 12, 2018 at 11:07 AM, hty...@gmail.com wrote: > yeah. I understand your concern. > > This is a common requirement for coverage calc service. Codecov support this > feature. > In codecov.yml, we can config threshold value which allow the coverage to > drop by X%, and posting a success status. That is awesome! What is your suggestion of X? > > Regards, > Yunkun Huang > > On 2018/04/10 06:36:03, Huxing Zhang wrote: >> Hi, >> >> Thanks for the suggestion. >> >> I generally agree with the idea that we should prevent a pull request >> to be merged if the code coverage gets lower. >> >> But my concern is the code coverage might have a minor drop even when >> just fixing typo[1], which makes confused. >> >> Unless we can figure it out, or I think it will be better if we can >> set a threshold. >> >> For example, if the code coverage drops >=0.1%, prevent the PR to be merged. >> >> [1] https://github.com/apache/incubator-dubbo/pull/1562 >> >> On Tue, Apr 10, 2018 at 10:21 AM, htynkn wrote: >> > hey team, >> > >> > >> > In pull request >> > template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), >> > this is one item to ask all pr to >> > >> > >> >"Write necessary unit-test to verify your logic correction" >> > >> > >> > Codecov has feature to fail PR if test coverage went down. Which means for >> > any pull request, if it add some code without any unit test, the PR will >> > fail >> > This can help to increase test coverage. >> > >> > >> > Any thoughts? >> > >> > >> > Relate links: >> > 1.https://docs.codecov.io/docs/commit-status >> > 2.https://github.com/codecov/support/issues/351 >> > >> >> -- >> Best Regards！ >> Huxing >> -- Best Regards！ Huxing
Re: [Suggestion] PR failing if test coverage went down
yeah. I understand your concern. This is a common requirement for coverage calc service. Codecov support this feature. In codecov.yml, we can config threshold value which allow the coverage to drop by X%, and posting a success status. Regards, Yunkun Huang On 2018/04/10 06:36:03, Huxing Zhang wrote: > Hi, > > Thanks for the suggestion. > > I generally agree with the idea that we should prevent a pull request > to be merged if the code coverage gets lower. > > But my concern is the code coverage might have a minor drop even when > just fixing typo[1], which makes confused. > > Unless we can figure it out, or I think it will be better if we can > set a threshold. > > For example, if the code coverage drops >=0.1%, prevent the PR to be merged. > > [1] https://github.com/apache/incubator-dubbo/pull/1562 > > On Tue, Apr 10, 2018 at 10:21 AM, htynkn wrote: > > hey team, > > > > > > In pull request > > template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), > > this is one item to ask all pr to > > > > > >"Write necessary unit-test to verify your logic correction" > > > > > > Codecov has feature to fail PR if test coverage went down. Which means for > > any pull request, if it add some code without any unit test, the PR will > > fail > > This can help to increase test coverage. > > > > > > Any thoughts? > > > > > > Relate links: > > 1.https://docs.codecov.io/docs/commit-status > > 2.https://github.com/codecov/support/issues/351 > > > > -- > Best Regards！ > Huxing >
Re: [Suggestion] PR failing if test coverage went down
Hi, Thanks for the suggestion. I generally agree with the idea that we should prevent a pull request to be merged if the code coverage gets lower. But my concern is the code coverage might have a minor drop even when just fixing typo[1], which makes confused. Unless we can figure it out, or I think it will be better if we can set a threshold. For example, if the code coverage drops >=0.1%, prevent the PR to be merged. [1] https://github.com/apache/incubator-dubbo/pull/1562 On Tue, Apr 10, 2018 at 10:21 AM, htynkn wrote: > hey team, > > > In pull request > template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), > this is one item to ask all pr to > > >"Write necessary unit-test to verify your logic correction" > > > Codecov has feature to fail PR if test coverage went down. Which means for > any pull request, if it add some code without any unit test, the PR will fail > This can help to increase test coverage. > > > Any thoughts? > > > Relate links: > 1.https://docs.codecov.io/docs/commit-status > 2.https://github.com/codecov/support/issues/351 > -- Best Regards！ Huxing
[Suggestion] PR failing if test coverage went down
hey team, In pull request template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), this is one item to ask all pr to "Write necessary unit-test to verify your logic correction" Codecov has feature to fail PR if test coverage went down. Which means for any pull request, if it add some code without any unit test, the PR will fail This can help to increase test coverage. Any thoughts? Relate links: 1.https://docs.codecov.io/docs/commit-status 2.https://github.com/codecov/support/issues/351
