Hi Marten,
I'd like to second Robert: it's great you're picking this up! Just to make
sure you avoid making old mistakes: here's a link to a previous discussion
on the topic:
http://mail-archives.apache.org/mod_mbox/couchdb-dev/201504.mbox/%3CCAGL_YkYQ7b%3DrTc_cxnSnJ4TZZavjsxKV-xa8XxukRaiDu7b%2BDg%40mail.gmail.com%3E
In essence, it's feasible to redirect the tests to the browser ports (code
is there) - but the cluster has just different characteristics: some tests
might work rather quickly, some tests will never work (because they make no
sense in the cluster like _config as it is today) and some tests will
require further adjustments (as ordering might be different, etc.). Will
had some idea of getting quicker but in the end I think it's some effort
left.
Anyway, hope you find this useful - and again: it's great you're getting
involved, keep going!
Best
Sebastian
On Mon, May 11, 2015 at 2:20 AM, Robert Kowalski <[email protected]> wrote:
> Hi Martin,
>
> > Whaw, I knew that dev/run was running 3 clusters at :
>
> In fact they are three nodes that are joined into a cluster
>
> Every node has still the "old" unclustered http interface from Couch
> 1.x, for dev/run we have
>
> 15984 - 16986
> 25984 - 26986
> 35984 - 36986
>
> The code for the old CouchDB http api (nodes) is located at [1] - I
> know that it is a bit confusing. The reason for it: historically
> CouchDB 2.x originates from BigCouch. BigCouch was a forked version of
> CouchDB with the Amazon Dynamo Paper applied _on top_ of the old plain
> CouchDB.
>
> In 2.x - on the clustered ports (15984, 25984, 35984) chttpd, rexi,
> fabric & co are taking care that requests on the clustered interface
> arrive at the node that is responsible for the requested part of the
> db.
>
> Changing the url is a good idea, but we will need to disable some
> tests (some features are not available on the clustered interface -
> e.g. compaction which just works on a per-node-level).
>
> We would also need to change the suite (explicitly pass a write quorum
> param of 3 to the requests (in case of the testsuite running against a
> three nodes cluster). We would need to do that in order to get stable
> tests as the cluster is just eventually consistent, meaning that per
> default it will return an "OK" after a document was written to two of
> our three nodes. That can lead to failing tests: Request A arrives at
> Node1, the Document is written to 2 nodes, we receive an OK with "202
> Accepted" instead of "201 Created".
>
> While the third Node(Node3) is still writing the document, our
> testclient got the OK and fires the next request to test if our doc
> was successfully written. We hit Node3 and get a "document not found",
> resulting in the testsuite to fail. We need to return every node in
> the cluster under test to return our expected result to get a stable,
> reliable testsuite, so we need to change the w-param (write quorum) to
> n (number of nodes). An excellent presentation reagrding this topic is
> [2]
>
>
> [1] https://github.com/apache/couchdb-couch
> [2]
> http://conf.couchdb.org/couchdb-conf-berlin-january-2013/slides/Introduction-to-BigCouch-CouchDB-Conf-Berlin-2013.pdf
>
> On Mon, May 11, 2015 at 12:02 AM, Martin Lagrange
> <[email protected]> wrote:
> > Hey, Robert, thanks for your reply !
> >
> > Whaw, I knew that dev/run was running 3 clusters at :
> >
> >
> > - http://127.0.0.1:25984/
> > - http://127.0.0.1:35984/
> > - http://127.0.0.1:15984/
> >
> > but I didn't know that it was running another cluster at :
> >
> > - http://localhost:15986/
> >
> > but I should have seen :
> > https://github.com/apache/couchdb/blob/master/dev/run#L419
> >
> > Do you mean that the cluster 15986 is not served by the compiled app ? By
> > what ? What do you mean exactly by
> > "are served by apache/couchdb-couch." ?
> >
> > There are some integration tests for chttpd in Erlang [3]
> >
> >
> > Yes, indeed but there are not so many and not as convenient as the
> > javascript tests :D
> >
> >
> >> - one idea
> >
> > for the future is to run the JS tests also against the cluster but it
> >
> > will need some further modifications of the JS test suite.
> >
> > As a trivial approach, wouldn't it be enough to just say to couchJS
> command
> > to run itself against the dev cluster ?
> > There is this -u option to specify a uri file to run the test against.
> >
> > https://github.com/apache/couchdb/blob/master/test/javascript/run#L74
> >
> >
> > Sorry in case I have misunderstood you.
> >
> >
> > 2015-05-10 21:13 GMT+02:00 Robert Kowalski <[email protected]>:
> >
> >> Hi Martin,
> >>
> >> see my responses inline
> >>
> >> On Sun, May 10, 2015 at 1:50 PM, Martin Lagrange
> >> <[email protected]> wrote:
> >> > Hi everyone,
> >> >
> >> > I am currently working on solving an issue in the rewriting module
> that
> >> > affect the rewrite process :
> >> >
> >>
> https://github.com/apache/couchdb-chttpd/blob/master/src/chttpd_rewrite.erl
> >> > and I have written some end-to-end regression tests :
> >> >
> >>
> https://github.com/apache/couchdb/blob/master/test/javascript/tests/rewrite.js
> >>
> >> Wow cool!
> >>
> >> > I modify the locally cloned couchdb-chttpd repository and compile the
> >> > program again, then run the server and the tests again.
> >> >
> >> > make && dev/run --with-admin-party-please
> >> >> test/javascript/run test/javascript/tests/rewrite.js
> >> >
> >> >
> >> > The thing is that I would like in someway to see what happen in the
> >> program
> >> > during the execution, especially when this tests are running.
> >> >
> >> > I have written :
> >> >
> >> > io:format('blabla'),
> >> >
> >> >
> >> > or
> >> >
> >> > couch_log:debug("blabla"),
> >> >
> >> >
> >> > in the erlang file but nothing seems to appear whether in the server
> >> > console or in any logs.
> >> >
> >>
> >>
> >> couchjs [1] runs against the old so-called backdoorports [2], which
> >> are served by apache/couchdb-couch. The cluster interface with the new
> >> "public" ports for the cluster are served by "couchdb-chttpd" - the
> >> part where you want t log.
> >>
> >> This means the JavaScript testsuite does not use code located in chttpd.
> >>
> >> There are some integration tests for chttpd in Erlang [3] - one idea
> >> for the future is to run the JS tests also against the cluster but it
> >> will need some further modifications of the JS testsuite.
> >>
> >>
> >> [1]
> https://github.com/apache/couchdb/blob/master/test/javascript/run#L27
> >> [2]
> >>
> https://github.com/apache/couchdb-couch/blob/master/priv/couch_js/http.c#L363
> >> [3]
> >>
> https://github.com/apache/couchdb-chttpd/blob/master/test/chttpd_db_test.erl
> >>
> >
> >
> >
> > --
> > Martin LAGRANGE
>
