Mobrovac has submitted this change and it was merged. Change subject: Update to service-template-node v0.1.3 ......................................................................
Update to service-template-node v0.1.3 Change-Id: Icab18289fb220a3e5df033352faa3b3294d9d04f --- M .gitignore M doc/README.md M doc/commands.md A doc/deployment.md M doc/template.md M package.json A targets.yaml 7 files changed, 194 insertions(+), 16 deletions(-) Approvals: Mobrovac: Verified; Looks good to me, approved diff --git a/.gitignore b/.gitignore index a359b3f..978f2d1 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ .idea graphoid.iml +Dockerfile coverage config.yaml node_modules diff --git a/doc/README.md b/doc/README.md index 26adc44..d837f0a 100644 --- a/doc/README.md +++ b/doc/README.md @@ -8,6 +8,7 @@ 3. [Configuration](config.md) 4. [Useful Commands](commands.md) 5. [Coding Guide](coding.md) +6. [Deployment](deployment.md) Have fun while creating RESTful API services! diff --git a/doc/commands.md b/doc/commands.md index e1bc803..01b6c5a 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -95,3 +95,7 @@ After you log out completely and log back in, you should be able to run the above scripts without resorting to `sudo`. +## Deployment + +See [this document](deployment.md) for how to get ready to deploy your service. + diff --git a/doc/deployment.md b/doc/deployment.md new file mode 100644 index 0000000..bb93f7d --- /dev/null +++ b/doc/deployment.md @@ -0,0 +1,164 @@ +# Deployment + +Getting your service ready to be deployed on WMF production machines involves +several tasks. This document explains the steps needed to get started and how to +keep your deployable copy up-to-date. + +## Repositories + +Because Node.js services use npm dependencies which can be binary, these need to +be pre-built. Therefore, two repositories are needed; one for the source code of +your service, and the other, so-called *deploy* repository. Both should be +available as WM's Gerrit repositories with the paths +*mediawiki/services/your-service-name* and +*mediawiki/services/your-service-name/deploy*. When [requesting +them](https://www.mediawiki.org/wiki/Git/New_repositories/Requests) ask for the +former to be a clone of [the service +template](https://github.com/wikimedia/service-template-node) and the latter to +be empty. + +It is important to note that the deploy repository is only to be updated +directly before (re-)deploying the service, and not on each patch merge entering +the *master* branch of the regular repository. In other words, **the deploy +repository mirrors the code deployed in production at all times**. + +The remainder of the document assumes these two repositories have been created +and that you have cloned them using your Gerrit account, i.e. not anonymously, +with the following outline: + +``` +~/code/ + |- your-service + -- deploy +``` + +Furthermore, it is assumed that you have initialised the deploy repository: + +```bash +$ cd ~/code/deploy +$ git review -s +$ touch README.md +$ git add README.md +$ git commit -m "Initial commit" +$ git push -u origin master # or git review -R if this fails +# go to Gerrit and +2 your change, if needed and then: +$ git pull +``` + +Finally, if you haven't yet done so, do [basic service +configuration](config.md). + +The remainder of the document refers to these two repositories as the *source +repository* and the *deploy repository*, respectively. + +## Configuration + +The service template includes an automation script which updates the deploy +repository, but it needs to be configured properly in order to work. + +### package.json + +The first part of the configuration involves keeping your source repository's +`package.json` updated. Look for its [deploy stanza](../package.json#L49). +Depending on the exact machine on which your service will be deployed, you may +need to set `target` to either `ubuntu` or `debian`. + +The important thing is keeping the `dependencies` field up to date at all times. +There you should list all of the extra packages that are needed in order to +build the npm module dependencies. The `_all` field denotes packages which +should be installed regardless of the target distribution, but you can add +other, distribution-specific package lists, e.g.: + +```javascript +"deploy": { + "target": "ubuntu", + "dependencies": { + "ubuntu": ["pkg1", "pkg2"], + "debian": ["pkgA", "pkgB"], + "_all": ["pkgOne", "pkgTwo"] + } +} +``` + +In this example, with the current configuration, packages *pkg1*, *pkg2*, +*pkgOne* and *pkgTwo* are going to be installed before building the +dependencies. If, instead, the target is changed to `debian`, then *pkgA*, +*pkgB*, *pkgOne* and *pkgTwo* are selected. + +As a rule of thumb, **whenever you need to install extra packages into your +development environment for satisfying node module dependencies, add them to +*deploy.dependencies* to ensure the successful build and update of the deploy +repository**. + +### Local git + +The script needs to know where to find your local copy of the deploy repository. +To that end, when in your source repository, run: + +``` +git config deploy.dir /absolute/path/to/deploy/repo +``` + +Using the aforementioned local outline, you would type: + +``` +git config deploy.dir /home/YOU/code/deploy +``` + +The source repository is itself a submodule of the deploy repository. If its +name as specified in `package.json`'s `name` field does not match the actual +repository's name in Gerrit, run: + +``` +git config deploy.name name_in_gerrit +``` + +That will make the system look for the repository +`mediawiki/services/name_in_gerrit` when checking it out in the deploy +repository. + +## Testing + +Before updating the deploy repository you need to make sure your configuration +works as expected. To do that, in your source repository run: + +``` +./server.js docker-test +``` + +The script will build a new Docker image, install the needed packages and npm +dependencies and run the test suite. Tweak your code and configuration until +everything works as expected (and commit those changes). + +## Update + +The final step is updating the deploy repository. From the source repository +run: + +``` +./server.js build --deploy-repo +``` + +The script will: +- create the proper deploy repository outline +- fetch the updates +- ensure the submodule is present +- update the submodule +- build the npm dependencies +- commit the changes with a pretty-formatted message + +There is also a handy shortcut for sending the patch to Gerrit immediately. To +do so, add the `--review` argument to the call: + +``` +./server.js build --deploy-repo --review +``` + +Note that if no changes were made to the source repository, the script aborts +its execution. If, nevertheless, you need to rebuild the dependencies, you can +do so using: + +``` +./server.js build --deploy-repo --force +``` + diff --git a/doc/template.md b/doc/template.md index b8c9af9..f9d28ad 100644 --- a/doc/template.md +++ b/doc/template.md @@ -41,7 +41,7 @@ The WMF is in the process of switching its production servers to Debian Jessie. As people developing services might use different platforms, the template provides also a Dockerfile, with which one can execute their service inside a -container running Debian Jessie. +container running the production OS. ## Repository Outline @@ -62,6 +62,6 @@ client-side JS, etc.) - [`test`](../test/) - contains the test files for the example routes in the template; you should add your own here -- [`scripts/docker.js`](../scripts/docker.js) - a utility script building the - service's docker image and starting the container +- docker script - a utility script building the service's docker image and + starting the container, now part of [service-runner](wikimedia/service-runner) diff --git a/package.json b/package.json index e352bf2..48fcbfa 100644 --- a/package.json +++ b/package.json @@ -1,14 +1,13 @@ { "name": "graphoid", - "version": "0.1.0", + "version": "0.1.1", "description": "renders vega graphs from mediawiki pages", "main": "./app.js", "scripts": { "start": "service-runner", "test": "mocha", - "docker-start": "./scripts/docker.js", - "docker-test": "./scripts/docker.js --test", - "docker-cover": "./scripts/docker.js --cover", + "docker-start": "service-runner docker-start", + "docker-test": "service-runner docker-test", "coverage": "istanbul cover _mocha -- -R spec" }, "repository": { @@ -26,27 +25,33 @@ ], "license": "Apache2", "bugs": { - "url": "https://phabricator.wikimedia.org/tag/services/"; + "url": "https://phabricator.wikimedia.org/tag/service-template-node/"; }, "homepage": "https://www.mediawiki.org/wiki/Extension:Graph";, "dependencies": { "vega": "git+http://g...@github.com/nyurik/vega";, - "bluebird": "^2.9.14", - "body-parser": "^1.12.1", - "bunyan": "^1.3.4", + "bluebird": "^2.9.24", + "body-parser": "^1.12.3", + "bunyan": "^1.3.5", "compression": "^1.4.3", "domino": "^1.0.18", - "express": "^4.12.2", + "express": "^4.12.3", "js-yaml": "^3.2.7", "node-uuid": "^1.4.3", - "preq": "^0.3.12", - "service-runner": "^0.1.6" + "preq": "^0.3.13", + "service-runner": "^0.1.8" }, "devDependencies": { "assert": "^1.3.0", - "istanbul": "^0.3.8", - "mocha": "^2.2.1", + "istanbul": "^0.3.13", + "mocha": "^2.2.4", "mocha-jshint": "0.0.9", "mocha-lcov-reporter": "0.0.1" + }, + "deploy": { + "target": "ubuntu", + "dependencies": { + "_all": [] + } } } diff --git a/targets.yaml b/targets.yaml new file mode 100644 index 0000000..6a16a18 --- /dev/null +++ b/targets.yaml @@ -0,0 +1,3 @@ +debian: 'debian:jessie' +ubuntu: 'ubuntu:14.04' + -- To view, visit https://gerrit.wikimedia.org/r/205563 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Icab18289fb220a3e5df033352faa3b3294d9d04f Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/graphoid Gerrit-Branch: master Gerrit-Owner: Mobrovac <mobro...@wikimedia.org> Gerrit-Reviewer: Mobrovac <mobro...@wikimedia.org> Gerrit-Reviewer: Yurik <yu...@wikimedia.org> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
