Re: Makefile target names
On 22 January 2015 at 15:13, David Britton david.brit...@canonical.com wrote: lint: - make lint Could we also make[1] the charm linter lint the makefile for the presence of targets agreed in the outcome of this thread? [1] Pun fully intended :) -- Simon -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
On Thu, Jan 22, 2015 at 10:13 AM, David Britton david.brit...@canonical.com wrote: functional tests: - make functional-test We need to be careful about things like this - as bundletester is already looking in tests/ for the amulet suite and might end up running the integration tests multiple times according to the make target + bundletesters sniffer. This is kind of an overlap of concerns with how bundletester works and I feel warrants caution. Tim would know best on the behavior here, and can recommend / amend as required. -- All the best, Charles Butler charles.but...@canonical.com - Juju Charmer Come see the future of datacenter orchestration: http://jujucharms.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
Thanks for pointing out the yaml control file, that could be useful. But before we make any modifications to the OpenStack charms, I think it would be helpful to have an agreed-upon convention for the following in terms of Makefile target names: - nose / unit tests - make test - make unit_test - Both are in use. - 2 cents: I would reserve both of these for unit tests, never for amulet tests. - lint checks - make lint - Already unified on this afaict. - amulet tests - make test - make functional_test - Both are in use. - 2 cents: I think functional_test leaves no question as to usage. - charm-helpers sync - make sync - Already unified on this afaict. If there is not a documented convention, can we have the necessary discussions here to create one? Thanks again, Ryan On Wed, Jan 21, 2015 at 11:40 AM, Benjamin Saller benjamin.sal...@canonical.com wrote: While convention is great there is an additional path, you can if your project differs from the de facto standards, include an explicit list of targets in your tests/tests.yaml file makefile: - lint - unit_test - something_else That file allows customization of much of bundletesters policy. -Ben On Wed, Jan 21, 2015 at 9:05 AM, Ryan Beisner ryan.beis...@canonical.com wrote: Greetings, I'd like to invite discussion on Makefile target names. I've seen a few different takes on Makefile target naming conventions across charms. For example, in the OpenStack charms, `make test` runs amulet and `make unit_test` performs nose tests. In many/most other charms, `make test` infers unit/nose testing, and amulet target names can vary. As I understand bundletester: it expects `make test` to be unit tests. Amulet targets in the Makefile aren't processed if they exist. Instead, the executables in the test dir are fired off. And, I think that should all be quite fine as long as the charm's amulet make target isn't doing anything important. The net effect for OpenStack charms at the moment is that when they hit Juju QA, amulet fires off twice, and unit is not run. I'd like to make sure the OpenStack charms are in line with any established Makefile convention. Is there a reference or doc for such a convention? I've seen 'unit_test' and 'functional_test' target names in use as well, and I quite like those, as they leave no question as to purpose. To work around the variations we've seen across charms, server team's OSCI (OpenStack CI charm testing) ignores make target names, and instead parses the Makefile, looking for the right thing-to-do, then execs the target found. Bear in mind that OSCI isn't intended to be a replacement for general charm QA, rather it is an intense safety trigger for the OpenStack charm developers. We also want these charms to succeed in Juju QA / CI. Input and advice are much appreciated! Many thanks, Ryan Beisner -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
+1, but I would propose using hyphens for word separators, not underscores -- at least for the recommendation. I would also recommend *not* having multiple default names. As mentioned, the yaml control file I think can be used to override all this, so it still leaves room for individual preferences on the exact namings. unit tests: - make test unit tests dependencies: - make test-depends functional tests: - make functional-test lint: - make lint charm-helpers upstream sync: - make sync -- David Britton david.brit...@canonical.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
Same here, the OpenStack charms have charm proof in the lint target. I expect it would be run twice in that case. On Thu, Jan 22, 2015 at 10:36 AM, Simon Davy bloodearn...@gmail.com wrote: On 22 January 2015 at 16:29, David Britton david.brit...@canonical.com wrote: On Thu, Jan 22, 2015 at 04:17:26PM +, Simon Davy wrote: On 22 January 2015 at 15:13, David Britton david.brit...@canonical.com wrote: lint: - make lint Could we also make[1] the charm linter lint the makefile for the presence of targets agreed in the outcome of this thread? charm proof I like it. (bundle tester already runs this) Which is interesting, as my lint targets general runs charm proof too, so it'd be run twice in that case? Not a big issue, but if the charm store/review queue is automatically charm-proofing too, perhaps the make lint target should not be? -- Simon -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
On 22 January 2015 at 16:36, Simon Davy bloodearn...@gmail.com wrote: On 22 January 2015 at 16:29, David Britton david.brit...@canonical.com wrote: On Thu, Jan 22, 2015 at 04:17:26PM +, Simon Davy wrote: On 22 January 2015 at 15:13, David Britton david.brit...@canonical.com wrote: lint: - make lint Could we also make[1] the charm linter lint the makefile for the presence of targets agreed in the outcome of this thread? charm proof I like it. (bundle tester already runs this) Which is interesting, as my lint targets general runs charm proof too, so it'd be run twice in that case? Not a big issue, but if the charm store/review queue is automatically charm-proofing too, perhaps the make lint target should not be? -- Simon Whelp it's still nice to have as part of lint when developing the charm, and charm-proof isn't exactly the slowest process to run multiple times. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
Marco, I like your proposal with one change - we don't need the test.yaml changes. Instead I would suggest we add 'unit-test' to the list of default bundletester targets. So bundletester will run proof, lint, test, and unit-test (charm author should choose test or unit-test, not both). Bundletester will *not* run functional-test, since it runs everything +x in the tests/ dir, which almost always overlaps with functional-test. On Thu, Jan 22, 2015 at 11:57 AM, Marco Ceppi ma...@ondina.co wrote: We can also add Makefile checking to charm proof, for an even greater redundancy. To avoid multiple invocations of charm proof (not terrible, IMO) lint could be broken down further: lint: proof code_lint proof: charm proof code_lint: # Your code here Then have bundle tester sniff out code_lint, or use the test.yaml configuration to point lint to code_lint. Doesn't change UX for the author/contributor but does add a level of complexity. It seems like Makefile's are the overwhelming method for consolidating tasks for charms, I'd like to kick off the following proposal for Makefile format to be placed in charm create templates: ``` test: lint unit-test functional-test lint: proof code-lint sync: charm-helpers-sync code-lint: # FILL IN COMMANDS FOR PERFORMING CODE LINT unit-test: # COMMANDS REQUIRED TO UNIT TEST charm-helpers-sync: @scripts/sync.py functional-test: juju test proof: charm proof ``` With a test.yml file that contained the following: ``` makefile: - code-lint - unit-test ``` And where applicable, add a .venv target for python charms and recommend the use of having charm deps modeled in requirements.txt and pip installed to that virtualenv. Opinions, additions, concerns? On Thu Jan 22 2015 at 11:41:56 AM Wes Mason wesley.ma...@canonical.com wrote: On 22 January 2015 at 16:36, Simon Davy bloodearn...@gmail.com wrote: On 22 January 2015 at 16:29, David Britton david.brit...@canonical.com wrote: On Thu, Jan 22, 2015 at 04:17:26PM +, Simon Davy wrote: On 22 January 2015 at 15:13, David Britton david.brit...@canonical.com wrote: lint: - make lint Could we also make[1] the charm linter lint the makefile for the presence of targets agreed in the outcome of this thread? charm proof I like it. (bundle tester already runs this) Which is interesting, as my lint targets general runs charm proof too, so it'd be run twice in that case? Not a big issue, but if the charm store/review queue is automatically charm-proofing too, perhaps the make lint target should not be? -- Simon Whelp it's still nice to have as part of lint when developing the charm, and charm-proof isn't exactly the slowest process to run multiple times. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/ mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
On 22 January 2015 at 16:29, David Britton david.brit...@canonical.com wrote: On Thu, Jan 22, 2015 at 04:17:26PM +, Simon Davy wrote: On 22 January 2015 at 15:13, David Britton david.brit...@canonical.com wrote: lint: - make lint Could we also make[1] the charm linter lint the makefile for the presence of targets agreed in the outcome of this thread? charm proof I like it. (bundle tester already runs this) Which is interesting, as my lint targets general runs charm proof too, so it'd be run twice in that case? Not a big issue, but if the charm store/review queue is automatically charm-proofing too, perhaps the make lint target should not be? -- Simon -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
We can also add Makefile checking to charm proof, for an even greater redundancy. To avoid multiple invocations of charm proof (not terrible, IMO) lint could be broken down further: lint: proof code_lint proof: charm proof code_lint: # Your code here Then have bundle tester sniff out code_lint, or use the test.yaml configuration to point lint to code_lint. Doesn't change UX for the author/contributor but does add a level of complexity. It seems like Makefile's are the overwhelming method for consolidating tasks for charms, I'd like to kick off the following proposal for Makefile format to be placed in charm create templates: ``` test: lint unit-test functional-test lint: proof code-lint sync: charm-helpers-sync code-lint: # FILL IN COMMANDS FOR PERFORMING CODE LINT unit-test: # COMMANDS REQUIRED TO UNIT TEST charm-helpers-sync: @scripts/sync.py functional-test: juju test proof: charm proof ``` With a test.yml file that contained the following: ``` makefile: - code-lint - unit-test ``` And where applicable, add a .venv target for python charms and recommend the use of having charm deps modeled in requirements.txt and pip installed to that virtualenv. Opinions, additions, concerns? On Thu Jan 22 2015 at 11:41:56 AM Wes Mason wesley.ma...@canonical.com wrote: On 22 January 2015 at 16:36, Simon Davy bloodearn...@gmail.com wrote: On 22 January 2015 at 16:29, David Britton david.brit...@canonical.com wrote: On Thu, Jan 22, 2015 at 04:17:26PM +, Simon Davy wrote: On 22 January 2015 at 15:13, David Britton david.brit...@canonical.com wrote: lint: - make lint Could we also make[1] the charm linter lint the makefile for the presence of targets agreed in the outcome of this thread? charm proof I like it. (bundle tester already runs this) Which is interesting, as my lint targets general runs charm proof too, so it'd be run twice in that case? Not a big issue, but if the charm store/review queue is automatically charm-proofing too, perhaps the make lint target should not be? -- Simon Whelp it's still nice to have as part of lint when developing the charm, and charm-proof isn't exactly the slowest process to run multiple times. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/ mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
On Thu, Jan 22, 2015 at 04:57:36PM +, Marco Ceppi wrote: test: lint unit-test functional-test -1, I'd rather 'test' be unit testing only. Many charms have this already and it seems like unecessary busy work to change it. ``` makefile: - code-lint - unit-test ``` -1, vote for 'lint', 'test' (unit test only) at this level, and agree with Tim that it's redundnat to print these out, we should make the recommeneded defaults what bundletester supports. All else I agree with. -- David Britton david.brit...@canonical.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Makefile target names
While convention is great there is an additional path, you can if your project differs from the de facto standards, include an explicit list of targets in your tests/tests.yaml file makefile: - lint - unit_test - something_else That file allows customization of much of bundletesters policy. -Ben On Wed, Jan 21, 2015 at 9:05 AM, Ryan Beisner ryan.beis...@canonical.com wrote: Greetings, I'd like to invite discussion on Makefile target names. I've seen a few different takes on Makefile target naming conventions across charms. For example, in the OpenStack charms, `make test` runs amulet and `make unit_test` performs nose tests. In many/most other charms, `make test` infers unit/nose testing, and amulet target names can vary. As I understand bundletester: it expects `make test` to be unit tests. Amulet targets in the Makefile aren't processed if they exist. Instead, the executables in the test dir are fired off. And, I think that should all be quite fine as long as the charm's amulet make target isn't doing anything important. The net effect for OpenStack charms at the moment is that when they hit Juju QA, amulet fires off twice, and unit is not run. I'd like to make sure the OpenStack charms are in line with any established Makefile convention. Is there a reference or doc for such a convention? I've seen 'unit_test' and 'functional_test' target names in use as well, and I quite like those, as they leave no question as to purpose. To work around the variations we've seen across charms, server team's OSCI (OpenStack CI charm testing) ignores make target names, and instead parses the Makefile, looking for the right thing-to-do, then execs the target found. Bear in mind that OSCI isn't intended to be a replacement for general charm QA, rather it is an intense safety trigger for the OpenStack charm developers. We also want these charms to succeed in Juju QA / CI. Input and advice are much appreciated! Many thanks, Ryan Beisner -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Makefile target names
Greetings, I'd like to invite discussion on Makefile target names. I've seen a few different takes on Makefile target naming conventions across charms. For example, in the OpenStack charms, `make test` runs amulet and `make unit_test` performs nose tests. In many/most other charms, `make test` infers unit/nose testing, and amulet target names can vary. As I understand bundletester: it expects `make test` to be unit tests. Amulet targets in the Makefile aren't processed if they exist. Instead, the executables in the test dir are fired off. And, I think that should all be quite fine as long as the charm's amulet make target isn't doing anything important. The net effect for OpenStack charms at the moment is that when they hit Juju QA, amulet fires off twice, and unit is not run. I'd like to make sure the OpenStack charms are in line with any established Makefile convention. Is there a reference or doc for such a convention? I've seen 'unit_test' and 'functional_test' target names in use as well, and I quite like those, as they leave no question as to purpose. To work around the variations we've seen across charms, server team's OSCI (OpenStack CI charm testing) ignores make target names, and instead parses the Makefile, looking for the right thing-to-do, then execs the target found. Bear in mind that OSCI isn't intended to be a replacement for general charm QA, rather it is an intense safety trigger for the OpenStack charm developers. We also want these charms to succeed in Juju QA / CI. Input and advice are much appreciated! Many thanks, Ryan Beisner -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
