Oops, hits end too early.
Okay -n 1 explains this > On 12 Oct 2015, at 13:32, Jan Lehnardt <[email protected]> wrote: > > >> On 12 Oct 2015, at 13:30, Dale Harvey <[email protected]> wrote: >> >> Started with >> >> python dev/run -n 1 --with-admin-party-please & >> >> and export COUCH_HOST='http://127.0.0.1:15984' >> >> >> On 12 October 2015 at 13:26, Jan Lehnardt <[email protected]> wrote: >> >>> >>>> On 12 Oct 2015, at 13:15, Dale Harvey <[email protected]> wrote: >>>> >>>>> - instead of fixed test db names like `test_suite_db` that are used in >>>> almost every test, >>>>> we now generate a random database name for each test. The JS tests >>>> expected db >>>>> deletion to be synchronous, and that’s no longer the case in 2.0. >>> Instead >>>> of >>>>> error-prone polling to wait for db deletion, we just use new db for >>> every >>>> test. >>>>> An added benefit now is that (some) tests can run in parallel. >>>> >>>> Little confused by this, PouchDB uses consistent names in its test suite >>>> (we had random >>>> ones for a while and moved away), we very much depend on the database >>> being >>>> deleted >>>> on the request returning and dont seem to have any problems with it >>> against >>>> master. >>> >>> Does the PouchDB test suite runs against the 5986 port, or with an n=1 >>> cluster? >>> This is with the default n=3 cluster against 5986. >>> >>> Best >>> Jan >>> -- >>> >>>> >>>> On 12 October 2015 at 12:21, Jan Lehnardt <[email protected]> wrote: >>>> >>>>> >>>>>> On 11 Oct 2015, at 18:53, Sebastian Rothbucher < >>>>> [email protected]> wrote: >>>>>> >>>>>> Hi Jan, >>>>>> >>>>>> I transferred some of my findings to your branch - pls. find a PR for >>> the >>>>>> first few tests submitted 2 your branch. I put in TODOs where either >>>>> there >>>>>> is most probably a bug (filed COUCHDB-2852 and COUCHDB-2853) or we have >>>>> to >>>>>> do some more work (like making config available). >>>>> >>>>> Excellent, thanks! :) >>>>> >>>>> >>>>>> Will from the Pouch team had the great idea of using -n 1 to mitigate >>>>>> Heisenbugs due to distribution. I adopted it and it did work quite >>> well. >>>>> It >>>>>> might make sense to put this into the dev/run call as a general rule, >>>>> esp. >>>>>> when checking for 201 (VS 202) status codes. Probably you know more on >>>>>> that. >>>>> >>>>> We should carefully evaluate this :) >>>>> >>>>> >>>>>> Anyways: I think we have to keep using _config on the local port as >>> many >>>>>> tests (like auth tests) depend on changed configurations. BTW: w/ -n 1 >>>>> it's >>>>>> only one call, but calling 3 _config(s) migt also work. >>>>>> Hopefully (I just arrived at some test w/ that recently) we can spare >>>>>> ourselves the _compact thing. I think we'd have to compact every single >>>>>> shard via local port, right? for the first tests, we can go w/out as it >>>>>> seems. Maybe we should stick to that for a while. >>>>> >>>>> My concern with this is that as far as I care, the local port is >>> irrelevant >>>>> to how CouchDB works. Of course it isn’t 100% irrelevant, but the >>> cluster >>>>> port is our publicised API for 2.0 and it’s the one that replaces the >>> 1.x >>>>> API for people upgrading. So having any tests that run on the local port >>>>> of 2.0 don’t help us in any way (that’s not to say it still useful to >>> have >>>>> tests against them for the moment). >>>>> >>>>> Many of the _config-dependent tests substitute out the _users or >>>>> _replicator >>>>> databases, maybe we can find a workaround for those by just using the >>> stock >>>>> databases. >>>>> >>>>> For the other tests, maybe we split them out into their own “test >>> suites” >>>>> that start their own test cluster in the configuration they need and >>> shut >>>>> it down afterwards. >>>>> >>>>> Best >>>>> Jan >>>>> -- >>>>> >>>>> >>>>>> >>>>>> Looking fw 2 your thoughts >>>>>> >>>>>> Best >>>>>> Sebastian >>>>>> >>>>>> P.S:: Maybe we can adopt some stuff from >>>>>> >>>>> >>> https://github.com/apache/couchdb/compare/master...sebastianrothbucher:clustertest >>>>>> >>>>>> P.P.S.: Will keep working on it, just not in the next few days, >>>>>> unfortunately ;-( >>>>>> >>>>>> On Sun, Oct 11, 2015 at 12:00 PM, Jan Lehnardt <[email protected]> wrote: >>>>>> >>>>>>> (I need your help deciding what to do with a bunch of tests, jump to >>>>>>> “Discussion” below if you don’t care about the details before.) >>>>>>> >>>>>>> * * * >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I spent some more time getting the 1.x JavaScript tests running on the >>>>>>> cluster port of 2.0. WIP here: >>>>> https://github.com/apache/couchdb/pull/354 >>>>>>> — I’ve gotten a lot further, but there is still a lot to do and >>> everyone >>>>>>> here can help, I’ve posted instructions on how to do so further down. >>> No >>>>>>> Erlang knowledge needed, this is ideal for folks who want to start >>> with >>>>>>> contributing to CouchDB. >>>>>>> >>>>>>> >>>>>>> Notable changes: >>>>>>> >>>>>>> - instead of fixed test db names like `test_suite_db` that are used in >>>>>>> almost every test, we now generate a random database name for each >>> test. >>>>>>> The JS tests expected db deletion to be synchronous, and that’s no >>>>> longer >>>>>>> the case in 2.0. Instead of error-prone polling to wait for db >>>>> deletion, we >>>>>>> just use new db for every test. An added benefit now is that (some) >>>>> tests >>>>>>> can run in parallel. >>>>>>> >>>>>>> - tests now clean up after themselves (incomplete): tests previously >>>>> only >>>>>>> collectively used three or four databases, and deleted and re-created >>>>> them >>>>>>> before each test was run. Now with the change of creating a new >>> database >>>>>>> per test, we also need to clean up databases at the end of a test, >>>>> because >>>>>>> otherwise we a) leave a lot of databases around and b) CouchDB runs >>> out >>>>> of >>>>>>> file descriptors during the test run. The cleanup isn’t 100% done yet, >>>>> so >>>>>>> we should check for stray databases after JS test runs until we’ve got >>>>> it >>>>>>> cleaned. One particular aspect here is when a test fails, it never >>>>> reaches >>>>>>> the cleanup code, so we might have to wrap everything into a try/catch >>>>>>> block. >>>>>>> >>>>>>> - I disabled all tests that still fail and made them so they print the >>>>>>> reason why they are failing, so that fixing them should be a lot >>> easier >>>>>>> now. Some of them print just “TODO”, their error cause is yet to be >>>>>>> determined. >>>>>>> >>>>>>> - couchdb.query() which previously executed temporary views, now under >>>>> the >>>>>>> hood creates a design doc with a random name and queries that view. >>> This >>>>>>> fixes all tests relying on temp views without having to change them >>> all >>>>>>> individually. This is because 2.0 doesn’t have temp views anymore >>>>> (which is >>>>>>> a good thing). >>>>>>> >>>>>>> - restartServer() is no longer used (mostly). >>>>>>> >>>>>>> I already filed a few bugs that showed things missing in the cluster >>> API >>>>>>> that we’d expect to be there: >>>>>>> >>>>>>> - https://issues.apache.org/jira/browse/COUCHDB-2849 >>>>>>> - https://issues.apache.org/jira/browse/COUCHDB-2850 >>>>>>> - https://issues.apache.org/jira/browse/COUCHDB-2851 >>>>>>> >>>>>>> Based on this I am ever more convinced we should get this test suite >>>>> work >>>>>>> against the 2.0 clustered API, because we’ll be finding some more >>>>>>> inconsistencies that we should fix before we invite beta testers. >>>>>>> >>>>>>> >>>>>>> ## Discussion: >>>>>>> >>>>>>> We have a whole number of tests that use the test-internal function >>>>>>> run_on_modified_server. They use the _config API to test CouchDB in >>>>>>> different configurations from the default one. All of these tests now >>>>> throw >>>>>>> an error that _config isn’t available on the clustered API. How should >>>>> we >>>>>>> handle them? >>>>>>> >>>>>>> The tests are: >>>>>>> >>>>>>> - erlang_views.js >>>>>>> - jsonp.js >>>>>>> - oauth_users_db.js >>>>>>> - oauth_users_db.js >>>>>>> - proxyauth.js >>>>>>> - reader_acl.js >>>>>>> - rewrite.js >>>>>>> - security_validation.js >>>>>>> - show_documents.js >>>>>>> - users_db.js >>>>>>> - view_errors.js >>>>>>> >>>>>>> >>>>>>> And a few tests aside from the proper compaction tests call _compact >>>>> which >>>>>>> is also not available on the cluster. What to do with them? >>>>>>> >>>>>>> - recreate_doc.js >>>>>>> - and a few more that are currently disabled for other reasons, sorry >>> I >>>>>>> don’t have a full list at this point. >>>>>>> >>>>>>> * * * >>>>>>> >>>>>>> >>>>>>> Bottom line: It’d be great if other folks here could join in the fun. >>> No >>>>>>> Erlang needed, this is purely JavaScript and CouchDB HTTP API-work :) >>>>>>> >>>>>>> >>>>>>> ## How to help: >>>>>>> >>>>>>> 1. Check out https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip >>>>>>> 2. run `make clean && ./configure -c --disable-docs --disable-fauxton >>> && >>>>>>> make` >>>>>>> 3. `make javascript` runs the whole test suite >>>>>>> >>>>>>> To run the whole test suite without waiting for rebar to confirm that >>> no >>>>>>> Erlang code needs to be compiled, run `./test/javascript/run` for the >>>>> whole >>>>>>> test suite, or `./test/javascript/run all_docs` to run an individual >>>>> test >>>>>>> (all_docs runs test/javascript/tests/all_docs.js, other file names >>> match >>>>>>> accordingly). >>>>>>> >>>>>>> Feel free to send me PRs against the branch on >>>>>>> https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip, I’ll merge >>>>>>> them, so the PR on apache/couchdb gets updated. >>>>>>> >>>>>>> I’m looking forward to your contributions! :) >>>>>>> >>>>>>> If you have any questions or have trouble getting started or just >>> would >>>>>>> like some hand-holding, let me know! >>>>>>> >>>>>>> >>>>>>> Best >>>>>>> Jan >>>>>>> -- >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> Professional Support for Apache CouchDB: >>>>> http://www.neighbourhood.ie/couchdb-support/ >>>>> >>>>> >>> >>> -- >>> Professional Support for Apache CouchDB: >>> http://www.neighbourhood.ie/couchdb-support/ >>> >>> > > -- > Professional Support for Apache CouchDB: > http://www.neighbourhood.ie/couchdb-support/ > -- Professional Support for Apache CouchDB: http://www.neighbourhood.ie/couchdb-support/
