branch: elpa/buttercup commit 512b82b22ff3fe98baff23d93ef8733850558864 Author: Jorgen Schaefer <cont...@jorgenschaefer.de> Commit: Jorgen Schaefer <cont...@jorgenschaefer.de>
Rework documentation to make the readme more concise. Fixes #12 by adding an "why not ERT?" section. --- Makefile | 2 +- README.md | 684 ++++---------------------------------- docs/images/buttercup.jpg | Bin 0 -> 21461 bytes README.md => docs/introduction.md | 97 +----- docs/testrunners.md | 15 + docs/usage.md | 129 +++++++ 6 files changed, 221 insertions(+), 706 deletions(-) diff --git a/Makefile b/Makefile index 2192f61..8c82578 100644 --- a/Makefile +++ b/Makefile @@ -8,7 +8,7 @@ DISTFILES = buttercup.el buttercup-compat.el buttercup-pkg.el README.md all: test test: - $(EMACS) -batch -L . -l buttercup.el -f buttercup-run-markdown README.md + $(EMACS) -batch -L . -l buttercup.el -f buttercup-run-markdown docs/introduction.md ./bin/buttercup -L . compile: diff --git a/README.md b/README.md index 5eaa3b8..7c5a662 100644 --- a/README.md +++ b/README.md @@ -3,31 +3,22 @@ [](https://travis-ci.org/jorgenschaefer/emacs-buttercup) [](http://stable.melpa.org/#/buttercup) + + Buttercup is a behavior-driven development framework for testing Emacs -Lisp code. It is heavily inspired by -[Jasmine](https://jasmine.github.io/). So heavily inspired, in fact, -that half this page is more or less a verbatim copy of the -[Jasmine introduction](https://jasmine.github.io/edge/introduction.html). +Lisp code. It allows to group related tests so they can share common +set-up and tear-down code, and allows the programmer to “spy” on +functions to ensure they are called with the right arguments during +testing. -All code in this file can be run by Buttercup’s built-in markdown test -runner. Just use `make test` in the project directory to see the -output. +The framework is heavily inspired by +[Jasmine](https://jasmine.github.io/edge/introduction.html). -## Getting Started +## Example -You can install buttercup from -[MELPA Stable](http://stable.melpa.org/). Add the following to your -`init.el` or `.emacs` file: +*Full article: [Introduction](docs/introduction.md)* -``` -(require 'package) -(add-to-list 'package-archives - '("melpa-stable" . "http://stable.melpa.org/packages/") t) -``` - -This should allow you to `M-x package-install RET buttercup RET`. -Afterwards, in an otherwise empty directory, create a file called -`some-test.el` with the following contents: +A simple test looks like this. ```Lisp (describe "A suite" @@ -35,55 +26,23 @@ Afterwards, in an otherwise empty directory, create a file called (expect t :to-be t))) ``` -You can now use buttercup to run this test: - -``` -$ emacs -batch -f package-initialize -l buttercup -f buttercup-run-discover -A suite - contains a spec with an expectation -``` - -Congratulations, you just ran your first tests! - -## Full Project Example - -Tests should be run in an environment separate from your interactive -Emacs. To get such an environment and install packages into it, you -can use [Cask](https://github.com/cask/cask). For installation -instructions see there. - -In the directory from above, create a new file called `Cask` with the -following contents: - -``` -(source gnu) -(source melpa-stable) - -(development - (depends-on "buttercup")) -``` +## Installation and Usage -Now you can tell cask to download and install the depended-on packages -in the environment of the current directory: +*Full article: [Usage](docs/usage.md)* -``` -$ cask install -``` - -You only need to do this once. Subsequently, you can run buttercup -tests simply by invoking buttercup using cask: +You can install buttercup from +[MELPA Stable](http://stable.melpa.org/). Add the following to your +`init.el` or `.emacs` file: ``` -$ cask exec buttercup -A suite - contains a spec with an expectation +(require 'package) +(add-to-list 'package-archives + '("melpa-stable" . "http://stable.melpa.org/packages/") t) ``` -## Suites: `describe` Your Tests +This should allow you to `M-x package-install RET buttercup RET`. -A test suite begins with a call to the Buttercup macro `describe` with -the first parameter describing the suite and the rest being the body -of code that implements the suite. +Now create a file called `test-feature.el` with these contents: ```Lisp (describe "A suite" @@ -91,560 +50,55 @@ of code that implements the suite. (expect t :to-be t))) ``` -## Specs - -Specs are defined by calling the Buttercup macro `it`, which, like -`describe` takes a string and code. The string is the title of the -spec and the code is the spec, or test. A spec contains one or more -expectations that test the state of the code. An expectation in -Buttercup is an assertion that is either true or false. A spec with -all true expectations is a passing spec. A spec with one or more false -expectations is a failing spec. - -### It’s Just Functions - -The code arguments to `describe` and `it` is just turned into -functions internally, so they can contain any executable code -necessary to implement the rules. Emacs Lisp scoping rules apply, so -make sure to define your spec file to be lexically scoped. - -```Lisp -(describe "A suite is just a function" - (let ((a nil)) - (it "and so is a spec" - (setq a t) - (expect a :to-be t)))) -``` - -## Expectations - -Expectations are expressed with the `expect` function. Its first -argument is the actual value. The second argument is a test, followed -by expected values for the test to compare the actual value against. - -If there is no test, the argument is simply tested for being non-nil. -This can be used by people who dislike the matcher syntax. - -### Matchers - -Each matcher implements a boolean comparison between the actual value -and the expected value. It is responsible for reporting to Buttercup -if the expectation is true or false. Buttercup will then pass or fail -the spec. - -Any matcher can evaluate to a negative assertion by prepending it with -the `:not` matcher. - -```Lisp -(describe "The :to-be matcher compares with `eq'" - (it "and has a positive case" - (expect t :to-be t)) - (it "and can have a negative case" - (expect nil :not :to-be t))) -``` - -### Included Matchers - -Buttercup has a rich set of matchers included. Each is used here — all -expectations and specs pass. There is also the ability to write custom -matchers (see the `buttercup-define-matcher` macro for further -information) for when a project’s domain calls for specific assertions -that are not included below. - -```Lisp -(describe "Included matchers:" - (it "The :to-be matcher compares with `eq'" - (let* ((a 12) - (b a)) - (expect a :to-be b) - (expect a :not :to-be nil))) - - (describe "The :to-equal matcher" - (it "works for simple literals and variables" - (let ((a 12)) - (expect a :to-equal 12))) - - (it "should work for compound objects" - (let ((foo '((a . 12) (b . 34))) - (bar '((a . 12) (b . 34)))) - (expect foo :to-equal bar)))) - - (it "The :to-match matcher is for regular expressions" - (let ((message "foo bar baz")) - (expect message :to-match "bar") - (expect message :to-match (rx "bar")) - (expect message :not :to-match "quux"))) - - (it "The :to-be-truthy matcher is for boolean casting testing" - (let (a - (foo "foo")) - (expect foo :to-be-truthy) - (expect a :not :to-be-truthy))) - - (it "The :to-contain matcher is for finding an item in a list" - (let ((a '("foo" "bar" "baz"))) - (expect a :to-contain "bar") - (expect a :not :to-contain "quux"))) - - (it "The :to-be-less-than matcher is for mathematical comparisons" - (let ((pi 3.1415926) - (e 2.78)) - (expect e :to-be-less-than pi) - (expect pi :not :to-be-less-than e))) - - (it "The :to-be-greater-than matcher is for mathematical comparisons" - (let ((pi 3.1415926) - (e 2.78)) - (expect pi :to-be-greater-than e) - (expect e :not :to-be-greater-than pi))) - - (it "The :to-be-close-to matcher is for precision math comparison" - (let ((pi 3.1415926) - (e 2.78)) - (expect pi :not :to-be-close-to e 2) - (expect pi :to-be-close-to e 0))) - - (describe "The :to-throw matcher" - (it "is for testing if a function throws an exception" - (let ((foo (lambda () (+ 1 2))) - (bar (lambda () (+ a 1)))) - (expect foo :not :to-throw) - (expect bar :to-throw))) - (it "accepts a symbol to check for the signal thrown" - (let ((foo (lambda () (/ 1 0))) - (bar (lambda () (+ a 1)))) - (expect foo :not :to-throw 'void-variable) - (expect bar :to-throw 'void-variable))) - (it "optionally matches arguments to signals" - (let ((foo (lambda () (+ a 1))) - (bar (lambda () (+ a 1)))) - (expect foo :not :to-throw 'void-variable '(b)) - (expect bar :to-throw 'void-variable '(a)))))) -``` - -## Grouping Related Specs with `describe` - -The `describe` macro is for grouping related specs. The string -parameter is for naming the collection of specs, and will be -concatenated with specs to make a spec’s full name. This aids in -finding specs in a large suite. If you name them well, your specs read -as full sentences in traditional -[BDD](http://en.wikipedia.org/wiki/Behavior-driven_development) style. - -```Lisp -(describe "A spec" - (it "is just a function, so it can contain any code" - (let ((foo 0)) - (setq foo (1+ foo)) - - (expect foo :to-equal 1))) - - (it "can have more than one expectation" - (let ((foo 0)) - (setq foo (1+ foo)) - - (expect foo :to-equal 1) - (expect t :to-equal t)))) -``` - -### Setup and Teardown - -To help a test suite DRY up any duplicated setup and teardown code, -Buttercup provides the `before-each`, `after-each`, `before-all` and -`after-all` special forms. - -As the name implies, code blocks defined with `before-each` are called -once before each spec in the `describe` is run, and the `after-each` -code blocks are called once after each spec. - -Here is the same set of specs written a little differently. The -variable under test is defined at the top-level scope — the `describe` -block — and initialization code is moved into a `before-each` block. -The `after-each` block resets the variable before continuing. - -```Lisp -(describe "A spec using `before-each' and `after-each'" - (let ((foo 0)) - (before-each - (setq foo (1+ foo))) - - (after-each - (setq foo 0)) - - (it "is just a function, so it can contain any code" - (expect foo :to-equal 1)) - - (it "can have more than one expectation" - (expect foo :to-equal 1) - (expect t :to-equal t)))) -``` - -The `before-all` form is called only once before all the specs in -`describe` are run, and the `after-all` form is called after all specs -finish. These functions can be used to speed up test suites with -expensive setup and teardown. - -However, be careful using `before-all` and `after-all`! Since they are -not reset between specs, it is easy to accidentally leak state between -your specs so that they erroneously pass or fail. - -```Lisp -(describe "A spec using `before-all' and `after-all'" - (let (foo) - (before-all - (setq foo 1)) - - (after-all - (setq foo 0)) - - (it "sets the initial value of foo before specs run" - (expect foo :to-equal 1) - (setq foo (1+ foo))) - - (it "does not reset foo between specs" - (expect foo :to-equal 2)))) -``` - -### Nesting `describe` Blocks - -Calls to `describe` can be nested, with specs defined at any level. -This allows a suite to be composed as a tree of functions. Before a -spec is executed, Buttercup walks down the tree executing each -`before-each` function in order. After the spec is executed, Buttercup -walks through the `after-each` functions similarly. - -```Lisp -(describe "A spec" - (let (foo) - (before-each - (setq foo 0) - (setq foo (1+ foo))) - - (after-each - (setq foo 0)) - - (it "is just a function, so it can contain any code" - (expect foo :to-equal 1)) - - (it "can have more than one expectation" - (expect foo :to-equal 1) - (expect t :to-equal t)) - - (describe "nested inside a second describe" - (let (bar) - (before-each - (setq bar 1)) - - (it "can reference both scopes as needed" - (expect foo :to-equal bar)))))) -``` - -## Disabling Suites - -Suites and specs can be disabled with the `xdescribe` and `xit` -macros, respectively. These suites and any specs inside them are -skipped when run and thus their results will not appear in the -results. - -```Lisp -(xdescribe "A spec" - (let (foo) - (before-each - (setq foo 0) - (setq foo (1+ foo))) - - (it "is just a function, so it can contain any code" - (expect foo :to-equal 1)))) -``` - -## Pending Specs - -Pending specs do not run. - -Any spec declared with `xit` is marked as pending. - -Any spec declared without a function body will also be marked as -pending in results. - -```Lisp -(describe "Pending specs" - (xit "can be declared using `xit'" - (expect t :to-be nil)) - - (it "can be declared with `it' but without a body")) -``` - -## Spies - -Buttercup has test double functions called spies. While other -frameworks call these mocks and similar, we call them spies, because -their main job is to spy in on function calls. Also, Jasmine calls -them spies, and so do we. A spy can stub any function and tracks calls -to it and all arguments. A spy only exists in the `describe` or `it` -block it is defined in, and will be removed after each spec. There are -special matchers for interacting with spies. The -`:to-have-been-called` matcher will return true if the spy was called -at all. The `:to-have-been-called-with` matcher will return true if -the argument list matches any of the recorded calls to the spy. - -```Lisp -(describe "A spy" - (let (foo bar) - (before-each - (setf (symbol-function 'foo) - (lambda (value) - (setq bar value))) - - (spy-on 'foo) - - (foo 123) - (foo 456 "another param")) - - (it "tracks that the spy was called" - (expect 'foo :to-have-been-called)) - - (it "tracks all arguments of its calls" - (expect 'foo :to-have-been-called-with 123) - (expect 'foo :to-have-been-called-with 456 "another param")) - - (it "stops all execution on a function" - (expect bar :to-be nil)))) -``` - -### Spies: `:and-call-through` - -The keyword argument `:and-call-through` to `spy-on` will make the spy -call the original function instead of returning `nil`. - -```Lisp -(describe "A spy, when configured to call through" - (let (bar set-bar get-bar fetched-bar) - (before-each - (fset 'set-bar (lambda (val) - (setq bar val))) - (fset 'get-bar (lambda () - bar)) - - (spy-on 'get-bar :and-call-through) - - (set-bar 123) - (setq fetched-bar (get-bar))) - - (it "tracks that the spy was called" - (expect 'get-bar :to-have-been-called)) - - (it "should not affect other functions" - (expect bar :to-equal 123)) - - (it "when called returns the requested value" - (expect fetched-bar :to-equal 123)))) -``` - -### Spies: `:and-return-value` - -The keyword argument `:and-return-value` specifies the value the -spied-on function should return. - -```Lisp -(describe "A spy, when configured to fake a return value" - (let (bar set-bar get-bar fetched-bar) - (before-each - (fset 'set-bar (lambda (val) - (setq bar val))) - (fset 'get-bar (lambda () - bar)) - - (spy-on 'get-bar :and-return-value 745) - - (set-bar 123) - (setq fetched-bar (get-bar))) - - (it "tracks that the spy was called" - (expect 'get-bar :to-have-been-called)) - - (it "should not affect other functions" - (expect bar :to-equal 123)) - - (it "when called returns the requested value" - (expect fetched-bar :to-equal 745)))) -``` - -### Spies: `:and-call-fake` - -The keyword argument `:and-call-fake` delegates calls to a supplied -function. - -```Lisp -(describe "A spy, when configured with an alternate implementation" - (let (bar set-bar get-bar fetched-bar) - (before-each - (fset 'set-bar (lambda (val) - (setq bar val))) - (fset 'get-bar (lambda () - bar)) - - (spy-on 'get-bar :and-call-fake (lambda () 1001)) - - (set-bar 123) - (setq fetched-bar (get-bar))) - - (it "tracks that the spy was called" - (expect 'get-bar :to-have-been-called)) - - (it "should not affect other functions" - (expect bar :to-equal 123)) - - (it "when called returns the requested value" - (expect fetched-bar :to-equal 1001)))) -``` - -### Spies: `:and-throw-error` - -With the keyword argument `:and-throw-error`, all calls to the spy -will `signal` the specified value as an error. - -```Lisp -(describe "A spy, when configured to throw an error" - (let (bar set-bar get-bar fetched-bar) - (before-each - (fset 'set-bar (lambda (val) - (setq bar val))) - (fset 'get-bar (lambda () - bar)) - - (spy-on 'get-bar :and-throw-error 'error)) - - (it "throws the error" - (expect (lambda () (get-bar)) - :to-throw 'error)))) -``` - -### Other tracking properties - -Every call to a spy is tracked and exposed using the `spy-calls` -accessor. - -`spy-calls-any` returns `nil` if the spy has not been called at all, -and then `t` once at least one call happens. `spy-calls-count` returns -the number of times the spy was called. `spy-calls-args-for` returns -the arguments passed to a given call (by index). `spy-calls-all-args` -returns the arguments to all calls. `spy-calls-all` returns the -current buffer and arguments passed to all calls. -`spy-calls-most-recent` returns the current buffer and arguments for -the most recent call. `spy-calls-first` returns the current buffer and -arguments for the first call. - -Finally, `spy-calls-reset` clears all tracking for a spy. - -```Lisp -(describe "A spy" - (let (set-foo foo) - (before-each - (fset 'set-foo (lambda (val &rest ignored) - (setq foo val))) - (spy-on 'set-foo)) - - (it "tracks if it was called at all" - (expect (spy-calls-any 'set-foo) - :to-equal - nil) - - (set-foo 5) - - (expect (spy-calls-any 'set-foo) - :to-equal - t)) - - (it "tracks the number of times it was called" - (expect (spy-calls-count 'set-foo) - :to-equal - 0) - - (set-foo 2) - (set-foo 3) - - (expect (spy-calls-count 'set-foo) - :to-equal - 2)) - - (it "tracks the arguments of each call" - (set-foo 123) - (set-foo 456 "baz") - - (expect (spy-calls-args-for 'set-foo 0) - :to-equal - '(123)) - - (expect (spy-calls-args-for 'set-foo 1) - :to-equal - '(456 "baz"))) - - (it "tracks the arguments of all calls" - (set-foo 123) - (set-foo 456 "baz") - - (expect (spy-calls-all-args 'set-foo) - :to-equal - '((123) - (456 "baz")))) - - (it "can provide the context and arguments to all calls" - (set-foo 123) - - (expect (spy-calls-all 'set-foo) - :to-equal - `(,(make-spy-context :current-buffer (current-buffer) - :args '(123) - :return-value nil)))) - - (it "has a shortcut to the most recent call" - (set-foo 123) - (set-foo 456 "baz") - - (expect (spy-calls-most-recent 'set-foo) - :to-equal - (make-spy-context :current-buffer (current-buffer) - :args '(456 "baz") - :return-value nil))) - - (it "has a shortcut to the first call" - (set-foo 123) - (set-foo 456 "baz") - - (expect (spy-calls-first 'set-foo) - :to-equal - (make-spy-context :current-buffer (current-buffer) - :args '(123) - :return-value nil))) - - (it "can be reset" - (set-foo 123) - (set-foo 456 "baz") - - (expect (spy-calls-any 'set-foo) - :to-be - t) - - (spy-calls-reset 'set-foo) +You can now use buttercup to run this test: - (expect (spy-calls-any 'set-foo) - :to-be - nil)))) ``` +$ emacs -batch -f package-initialize -f buttercup-run-discover +Running 1 specs. +A suite + contains a spec with an expectation -## Test Runners - -Evaluating `describe` forms just stores the suites. You need to use a -test runner to actually evaluate them. Buttercup comes with two test -runners by default: - -- `buttercup-run-at-point` — Evaluate the topmost `describe` form at - point and run the suite it creates directly. Useful for interactive - development. But be careful, this uses your current environment, - which might not be clean (due to said interactive development). -- `buttercup-run-discover` — Find files in directories specified on - the command line, load them, and then run all suites defined - therein. Useful for being run in batch mode. -- `buttercup-run-markdown` — Run code in markdown files. Used to - run this file’s code. +Ran 1 specs, 0 failed, in 0.0 seconds. +``` + +Congratulations, you ran your first test! + +## Feature List + +- Shared set-up and tear-down sections to reduce code repetition and + share a common environment among tests. +- Easy to read and extensible `expect` macro to describe expected + behavior. +- Powerful mocking framework, called “spies,” to both cause them to + return expected values or throw errors as needed by the test, as + well as to ensure functions are called with expected arguments + during tests. +- Built to be run within a Continuous Integration environment, + including test runners to discover and execute tests in a directory + tree. + +### Why not ERT? + +Emacs comes with a testing framework, +[ERT](https://www.gnu.org/software/emacs/manual/html_mono/ert.html). +Buttercup was written to address some shortcomings of that framework. + +- ERT + [deliberately leaves it up to the programmer to define set-up and tear-down code](https://www.gnu.org/software/emacs/manual/html_mono/ert.html#Fixtures-and-Test-Suites), + which requires a lot of boiler-plate code for every set-up function. + Buttercup makes this easy and seamless. +- ERT has no good way of being run in a continuous integration + environment. There are + [external projects to make this less of a pain](https://github.com/rejeep/ert-runner.el) + instead. Once all is said and done, you installed six external + packages your project does not need just to run your own tests. And + that does not include a mocking library. +- ERT has no way of grouping related tests, requiring every test name + to share the same prefix, making names long and hard to read. + +Nonetheless, ERT is a great project. It introduced testing to Emacs, +and Buttercup learned a lot from its code to record a stack trace for +error display. Even though Buttercup tries to be a better testing +framework than ERT, we do wish ERT and the ERT maintainers all the +best and hope both frameworks can continue to benefit from each other. diff --git a/docs/images/buttercup.jpg b/docs/images/buttercup.jpg new file mode 100644 index 0000000..f6966b4 Binary files /dev/null and b/docs/images/buttercup.jpg differ diff --git a/README.md b/docs/introduction.md similarity index 85% copy from README.md copy to docs/introduction.md index 5eaa3b8..6e2938d 100644 --- a/README.md +++ b/docs/introduction.md @@ -1,84 +1,18 @@ -# Buttercup — Behavior-Driven Emacs Lisp Testing - -[](https://travis-ci.org/jorgenschaefer/emacs-buttercup) -[](http://stable.melpa.org/#/buttercup) +# Introduction Buttercup is a behavior-driven development framework for testing Emacs -Lisp code. It is heavily inspired by -[Jasmine](https://jasmine.github.io/). So heavily inspired, in fact, -that half this page is more or less a verbatim copy of the +Lisp code. It does not depend on any other Emacs Lisp libraries. It +has a clean, obvious syntax so that you can easily write tests. + +It is heavily inspired by [Jasmine](https://jasmine.github.io/). So +heavily inspired, in fact, that most of this page is more or less a +verbatim copy of the [Jasmine introduction](https://jasmine.github.io/edge/introduction.html). All code in this file can be run by Buttercup’s built-in markdown test runner. Just use `make test` in the project directory to see the output. -## Getting Started - -You can install buttercup from -[MELPA Stable](http://stable.melpa.org/). Add the following to your -`init.el` or `.emacs` file: - -``` -(require 'package) -(add-to-list 'package-archives - '("melpa-stable" . "http://stable.melpa.org/packages/") t) -``` - -This should allow you to `M-x package-install RET buttercup RET`. -Afterwards, in an otherwise empty directory, create a file called -`some-test.el` with the following contents: - -```Lisp -(describe "A suite" - (it "contains a spec with an expectation" - (expect t :to-be t))) -``` - -You can now use buttercup to run this test: - -``` -$ emacs -batch -f package-initialize -l buttercup -f buttercup-run-discover -A suite - contains a spec with an expectation -``` - -Congratulations, you just ran your first tests! - -## Full Project Example - -Tests should be run in an environment separate from your interactive -Emacs. To get such an environment and install packages into it, you -can use [Cask](https://github.com/cask/cask). For installation -instructions see there. - -In the directory from above, create a new file called `Cask` with the -following contents: - -``` -(source gnu) -(source melpa-stable) - -(development - (depends-on "buttercup")) -``` - -Now you can tell cask to download and install the depended-on packages -in the environment of the current directory: - -``` -$ cask install -``` - -You only need to do this once. Subsequently, you can run buttercup -tests simply by invoking buttercup using cask: - -``` -$ cask exec buttercup -A suite - contains a spec with an expectation -``` - ## Suites: `describe` Your Tests A test suite begins with a call to the Buttercup macro `describe` with @@ -631,20 +565,3 @@ Finally, `spy-calls-reset` clears all tracking for a spy. :to-be nil)))) ``` - - -## Test Runners - -Evaluating `describe` forms just stores the suites. You need to use a -test runner to actually evaluate them. Buttercup comes with two test -runners by default: - -- `buttercup-run-at-point` — Evaluate the topmost `describe` form at - point and run the suite it creates directly. Useful for interactive - development. But be careful, this uses your current environment, - which might not be clean (due to said interactive development). -- `buttercup-run-discover` — Find files in directories specified on - the command line, load them, and then run all suites defined - therein. Useful for being run in batch mode. -- `buttercup-run-markdown` — Run code in markdown files. Used to - run this file’s code. diff --git a/docs/testrunners.md b/docs/testrunners.md new file mode 100644 index 0000000..34aae59 --- /dev/null +++ b/docs/testrunners.md @@ -0,0 +1,15 @@ +# Test Runners + +Evaluating `describe` forms just stores the suites. You need to use a +test runner to actually evaluate them. Buttercup comes with three test +runners by default: + +- `buttercup-run-at-point` — Evaluate the topmost `describe` form at + point and run the suite it creates directly. Useful for interactive + development. But be careful, this uses your current environment, + which might not be clean (due to said interactive development). +- `buttercup-run-discover` — Find files in directories specified on + the command line, load them, and then run all suites defined + therein. Useful for being run in batch mode. +- `buttercup-run-markdown` — Run code in markdown files. Used to + run this file’s code. diff --git a/docs/usage.md b/docs/usage.md new file mode 100644 index 0000000..e6be162 --- /dev/null +++ b/docs/usage.md @@ -0,0 +1,129 @@ +# Usage + +Buttercup is primarily meant to be used non-interactively, to +automatically test a project independent of a user’s setup, before a +commit and on a continuous integration platform. Because of this, the +workflow for buttercup does not use interactive commands but instead +the command line. + +## Cask + +[Cask](https://github.com/cask/cask) is a project to create an +environment separate from your usual interactive Emacs environment. +This allows you to install the packages your project depends on, and +only those, and test your project in a well-defined environment. + +Buttercup works best in such environments, so the following best +practices rely on Cask to be installed. + +## Project Directory Layout + +A basic project layout requires a project file, called `feature.el` +here, a `Cask` file to define dependencies, and a `tests/` directory +for tests. It should look roughly like this: + +``` +feature/feature.el + Cask + tests/feature.el +``` + +**feature.el** + +```Lisp +(defun featurize (bug feature) + (format "It's not a %s, it's a %s" bug feature)) + +(provide 'feature) +``` + +**Cask** + +``` +(source gnu) +(source melpa-stable) + +(development + (depends-on "buttercup")) +``` + +**tests/test-feature.el** + +```Lisp +(require 'feature) + +(describe "The feature" + (it "can use bug and feature" + (expect (featurize "bug" "feature") + :to-equal + "It's not a bug, it's a feature"))) +``` + +## Running Tests + +You can now use Cask to run your tests. + +First, you have to install the dependencies. You only have to do this +once, or when the dependencies change: + +``` +$ cask +Extracting buttercup-1.1/ +Extracting buttercup-1.1/buttercup-compat.el +Extracting buttercup-1.1/buttercup.el +Extracting buttercup-1.1/bin/ +Extracting buttercup-1.1/bin/buttercup +Extracting buttercup-1.1/buttercup-pkg.el +Generating autoloads for buttercup-compat.el... +Generating autoloads for buttercup-compat.el...done +Generating autoloads for buttercup-pkg.el... +Generating autoloads for buttercup-pkg.el...done +Generating autoloads for buttercup.el... +Generating autoloads for buttercup.el...done +``` + +Now, you can run your tests: + +``` +$ cask exec buttercup -L . +Running 1 specs. + +The feature + can use bug and feature + +Ran 1 specs, 0 failed, in 0.0 seconds. +``` + +That’s it. + +You can run this command whichever way you like. Common choices +include a makefile or shell scripts. + +## Travis + +If your project is hosted on github, you can use +[Travis CI](https://travis-ci.org/) as your continuous integration +environment. Buttercup can easily be used in such a setup. Simply add +the following `.travis.yml` file: + +``` +language: emacs-lisp +env: + - EVM_EMACS=emacs-24.4-bin + - EVM_EMACS=emacs-24.5-bin +before_install: + - sudo mkdir /usr/local/evm + - sudo chown $(id -u):$(id -g) /usr/local/evm + - curl -fsSkL https://raw.github.com/rejeep/evm/master/go | bash + - export PATH="$HOME/.evm/bin:$PATH" + - evm install $EVM_EMACS --use + - curl -fsSkL https://raw.github.com/cask/cask/master/go | python + - export PATH="$HOME/.cask/bin:$PATH" + - cask +script: + - cask exec buttercup -L . +``` + +Most of the complexity here is from installing +[EVM](https://github.com/rejeep/evm) and Cask to be able to test your +project using different Emacs versions.