On 21 Mar 2010, at 13:38, Robert Dionne wrote: > Ok Noah, This is only a test case issue, and not in the changes code as I > though. Jan found the issue and it works fine for me now in both FF and CLI. > -- Bob
As an added bonus, the test suite should now work 100% in WebKit (for me at least :) Cheers Jan -- > > > On Mar 21, 2010, at 1:30 PM, Jan Lehnardt wrote: > >> >> On 21 Mar 2010, at 12:24, Robert Dionne wrote: >> >>> >>> >>> >>> >>> On Mar 21, 2010, at 1:16 PM, Jan Lehnardt wrote: >>> >>>> >>>> On 21 Mar 2010, at 12:10, Noah Slater wrote: >>>> >>>>> What are the CLI tests, if not the etap tests? Are they integrated into >>>>> the build system? >>>> >>>> The CLI tests are the same as the browser tests, just run through our >>>> couchjs binary >>>> that has custom HTTP extensions to make the xhr work. At this point I >>>> don't think it >>>> is reliable enough to mimic browser behaviour and that we shouldn't use it >>>> for vetting >>>> the state of the code. >>> >>> This is likely true, but in this particular case I think there's a bug in >>> the changes code (that I'm trying to dig out). It's nice that it works on >>> your machine but on my machine, using FF, it fails often enough. Moreover >>> it's been around for a long time now so I figure it's worth figuring out. >>> >>> I don't have a dog in this fight (.ie. a paying customer) so this failure >>> doesn't bother me. With respect to policy, given the various bogocities of >>> browsers, I'd recommend something like these CLI tests plus the etaps ought >>> to be the "official" tests for vetting, and part of the build >> >> Not that I disagree, but part (most?) of the appeal of the browser based >> tests are that they run in a real-world client instead of the lab that is >> couchjs+http :) >> >> Cheers >> Jan >> -- >> >>> >>> >>>> >>>> It is very useful when developing new code to not have to switch to and >>>> reload the >>>> browser over and over again. >>>> >>>> Cheers >>>> Jan >>>> -- >>>> >>>> >>>> >>>> >>>>> >>>>> On 21 Mar 2010, at 17:05, Jan Lehnardt wrote: >>>>> >>>>>> >>>>>> On 21 Mar 2010, at 06:04, Robert Dionne wrote: >>>>>> >>>>>>> On Mar 21, 2010, at 4:00 AM, Jan Lehnardt wrote: >>>>>>> >>>>>>>> >>>>>>>> On 20 Mar 2010, at 20:06, Paul Davis wrote: >>>>>>>> >>>>>>>>> On Sat, Mar 20, 2010 at 2:31 PM, Noah Slater <[email protected]> wrote: >>>>>>>>>> I think faulty test case should block the release, if I am to have >>>>>>>>>> any >>>>>>>>>> future sanity preparing releases. I don't want to delay and longer, >>>>>>>>>> so if >>>>>>>>>> you guys are absolutely sure this is a test error and not code >>>>>>>>>> error, then I >>>>>>>>>> propose that the test be commented out. Our tests form a contract >>>>>>>>>> between >>>>>>>>>> us, internally, and our users. If that contract has a bug, it should >>>>>>>>>> be >>>>>>>>>> removed or fixed - or it simply dilutes the importance of contract. >>>>>>>>>> If some >>>>>>>>>> one comments out the test, and we agree it is not indicative of an >>>>>>>>>> important >>>>>>>>>> bug, I can call the vote within hours. >>>>>>>>>> >>>>>>>>> >>>>>>>>> I'd have to agree on this. From the point of view of a release, if a >>>>>>>>> test reports a failure then it should be made to not report a failure. >>>>>>>>> If that's accomplished by disabling it, then there will be a commit >>>>>>>>> with a message that explains why it was disabled and etc and such on >>>>>>>>> and so forth. >>>>>>>> >>>>>>>> I'd do that if the test was failing for me :) >>>>>>> >>>>>>> it's not failing for you when you run changes.js with the CLI ? Fails >>>>>>> for me every time. >>>>>> >>>>>> I don't consider the CLI tests as part of the official test suite just >>>>>> yet. >>>>>> >>>>>> Cheers >>>>>> Jan >>>>>> -- >>>>>> >>>>>>> >>>>>>> Anyway I poked at this a bit yesterday and am not 100% sure the issue >>>>>>> is in the test. I tried putting a sleep in with no luck. If my >>>>>>> understanding of the JS is correct, CouchDB is supposed to be >>>>>>> synchronous so it's not timing. >>>>>>> >>>>>>> If someone could comment on the test itself it would be helpful. The >>>>>>> section of the code that fails: >>>>>>> >>>>>>> // changes get all_docs style with deleted docs >>>>>>> var doc = {a:1}; >>>>>>> db.save(doc); >>>>>>> db.deleteDoc(doc); >>>>>>> var req = CouchDB.request("GET", >>>>>>> "/test_suite_db/_changes?filter=changes_filter/bop&style=all_docs"); >>>>>>> var resp = JSON.parse(req.responseText); >>>>>>> TEquals(3, resp.results.length, "should return matching rows"); >>>>>>> >>>>>>> >>>>>>> seems odd to me. all_docs as I read the code will return docs with >>>>>>> deletes and conflicts but in this call the filter bop will not apply to >>>>>>> the doc {a:1} so I'm not sure what this delete prior to the call is >>>>>>> about. Anyway I can make it fail in the debugger so perhaps I can find >>>>>>> the root cause. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Cheers >>>>>>>> Jan >>>>>>>> -- >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
