Hello, The old-tck completed, and the results are better than expected. The only failures were found here:
jsf/spec/ajax - 5 / 5 Fail (1 is excluded entirely) jsf/spec/render/commandlink - 3 /3 Fail jsf/spec/resource/packaging/classpath - 4 / 8 Fail jsf/spec/view/protectedview - 1 / 2 Fail 23 from the new TCK and 13 from the old TCK bring us to 36. The new TCK failures could be fixed via selenium updates as before. As for the old TCK, it would be best to move the whole application and run it on a selenium driver. The tests run from within a servlet, so all we would then need to do is look at the response (ie. check for"Test PASSED" ) I can start looking at the old tck (though it might be more complicated than what's suggested above). Werner (or anyone else), could you work on the new TCK with the list of tests sent in an earlier email? Thanks, Volodymyr On 2023/02/20 17:13:18 Paul Nicolucci wrote: > I did send a quick update to the Faces community: > https://www.eclipse.org/lists/faces-dev/msg00272.html > > Regards, > > Paul Nicolucci > > On Mon, Feb 20, 2023 at 11:50 AM Paul Nicolucci <[email protected]> > wrote: > > > Hi, > > > > If we're not reverting this, then @Werner can you let the Faces community > > know that you have more changes coming in? I sent a message hoping we could > > get a new TCK out: https://www.eclipse.org/lists/faces-dev/msg00264.html > > last week. > > > > I hate holding up a MyFaces 4.0.0 release when we were passing the TCK > > without these changes. I think if this can't be resolved quickly and with > > priority, we should really consider reverting. > > > > Vlad and I are working with our testing team to try and get a full list of > > failures that need to be fixed. > > > > Regards, > > > > Paul Nicolucci > > > > > > On Mon, Feb 20, 2023 at 11:34 AM Thomas Andraschko < > > [email protected]> wrote: > > > >> IMO the goal should be to NOT revert this changes > >> > >> @Werner Could you fix/refactor the tests maybe? I really dont like to > >> revert because TCK uses a technology which doesnt support long time > >> existing JS/HTML features.... > >> > >> Am Mo., 20. Feb. 2023 um 17:16 Uhr schrieb Volodymyr Siedlecki < > >> [email protected]>: > >> > >>> Hello, > >>> > >>> I regret bringing this topic up, but the new JavaScript code in RC5 is > >>> encountering more HTMLUnit trouble. > >>> > >>> I've pulled the RC5 jars into our application server and run our test > >>> suite against it, and we saw about 5-10% of tests fail. > >>> > >>> We have yet to run against the full TCK, but initial runs also show > >>> failures and errors (see list 1). The majority of the failures/errors are > >>> caused by either "missing formal parameter" or "syntax error". > >>> > >>> This issue can be traced back to Rhino not supporting "rest parameters" > >>> (Found https://github.com/mozilla/rhino/issues/652 via > >>> https://github.com/HtmlUnit/htmlunit/issues/232 ). > >>> > >>> > >>> [ERROR] > >>> ee.jakarta.tck.faces.test.javaee6.viewParamNullValueAjax.Issue4550IT.testViewParamNullValueAjax > >>> Time elapsed: 2.33 s <<< ERROR! ======= EXCEPTION START ======== Exception > >>> class=[net.sourceforge.htmlunit.corejs.javascript.EvaluatorException] > >>> com.gargoylesoftware.htmlunit.ScriptException: missing formal parameter ( > >>> http://localhost:9080/viewParamNullValueAjax/faces/jakarta.faces.resource/faces.js?ln=jakarta.faces#2) > >>> > >>> > >>> Back to the topic of the TCK, the ajax tests (faces22/ajax, faces23/ajax, > >>> etc) were ported over to use the selenium driver, but other tests, which > >>> also use ajax, were not ported (faces23/exactmapping, faces40/inputfile, > >>> etc). > >>> Another portion of the TCK that has yet to be tested is the old-tck > >>> (which has thousands of tests, some of which will be hit by these script > >>> exceptions). *We'd be losing testing coverage with the new scripts.* > >>> > >>> If we remove the rest parameters, then our problems might be resolved. > >>> It's not a guarantee and I'm not sure how feasible it would be. > >>> > >>> Unfortunately, though, a revert is the best course of action as I don't > >>> think challenging all of these previously working tests is a sound > >>> approach. > >>> > >>> My hope is to have a 4.0.0 release with the previous scripts. This would > >>> then give us time to automate MyFaces to run against the TCK and also > >>> move away from HTMLUnit. > >>> > >>> The new scripts could then be merged at a later time (with more > >>> confidence). > >>> > >>> 1) List of New TCK Failures: > >>> ee.jakarta.tck.faces.test.javaee6.viewParamNullValueAjax.Issue4550IT.testViewParamNullValueAjax > >>> ee.jakarta.tck.faces.test.javaee8.cdi.Spec1351IT.testInjectValidator > >>> ee.jakarta.tck.faces.test.javaee8.cdi.Spec1386IT.testInjectFlowMap > >>> ee.jakarta.tck.faces.test.javaee8.commandScript.Spec613IT.test > >>> ee.jakarta.tck.faces.test.servlet40.exactmapping.Spec1260IT.testLinkToNonExactMappedView > >>> ee.jakarta.tck.faces.test.servlet40.exactmapping.Spec1260IT.testAjaxFromExactMappedView > >>> ee.jakarta.tck.faces.test.servlet40.exactmapping.Spec1260IT.testExactMappedViewLoads > >>> ee.jakarta.tck.faces.test.servlet40.exactmapping.Spec1260IT.testPostBackOnLinkedNonExactMappedView > >>> ee.jakarta.tck.faces.test.servlet40.exactmapping.Spec1260IT.testPostBackToExactMappedView > >>> ee.jakarta.tck.faces.test.servlet40.exactmapping.Spec1260IT.testResourceReferenceFromExactMappedView > >>> ee.jakarta.tck.faces.test.servlet40.facelets.Issue4830IT.testUIRepeatResetValues > >>> ee.jakarta.tck.faces.test.servlet40.facelets.Issue5078IT.testUIRepeatVisitTreeDuringInvokeApplication > >>> ee.jakarta.tck.faces.test.javaee8.searchExpression.Issue4331IT.test > >>> ee.jakarta.tck.faces.test.javaee8.searchExpression.Spec1238IT.test > >>> ee.jakarta.tck.faces.test.javaee8.uiinput.Issue5081IT.testIssue4734 > >>> ee.jakarta.tck.faces.test.servlet50.inputfile.Spec1555IT.testMultipleSelectionNonAjax > >>> ee.jakarta.tck.faces.test.servlet50.inputfile.Spec1555IT.testSingleSelectionNonAjax > >>> ee.jakarta.tck.faces.test.servlet50.inputfile.Spec1555IT.testMultipleSelectionAjax > >>> ee.jakarta.tck.faces.test.servlet50.inputfile.Spec1555IT.testSingleSelectionAjax > >>> > >>> Thanks, > >>> > >>> Volodymyr > >>> > >>> >
