Re: [Dev] [Status update] Platform automation test - Cross products Single Sign-on with SAML 2.0 Auto test
Yes. We can aggregate multiple coverage reports and create a one report. But the problem is that We have to instrument the JAR file before starting the Distributed setup. I guess, Since this is platform test, Platform is already up and running. if so We have to instrument the JAR inside the server first before starting the servers. Thanks, Nuwanw On Wed, Feb 18, 2015 at 4:37 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Nuwan, On Wed, Feb 18, 2015 at 5:49 PM, Nuwan Dias nuw...@wso2.com wrote: Nice work. @Krishantha I have a question regarding getting code coverage for these. Since these are platform scenarios and don't execute at build time, is there a way we can find out the exact code coverage of all the test scenarios? We can merge the two reports and generate aggregated coverage report. This was already done for products like ESB with multiple test modules. However, in a distributed APIM setup merging reports will be bit difficult which we need to improve in future. For the time being we can merge the reports and generate coverage manually. Thanks, Krishantha. Currently the only comprehensive coverage comes from the tests that run at build time and we don't actually know the coverage as a whole (which includes the UI and platform scenarios). Thanks, NuwanD. On Wed, Feb 18, 2015 at 5:20 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: We have completed writing the above cross products (APIM IS) test case scenario and the test case covers the following sequence. 1. Publisher SSO login verifications 2. Creating a sample API 3. Change API state from created to published verify the API details 4. Verify avoidance of repeat authentication for APIM /store related verifications Regards On Tue, Feb 17, 2015 at 11:27 PM, Krishantha Samaraweera krishan...@wso2.com wrote: [adding dev] Thanks, Krishantha. On Tue, Feb 17, 2015 at 11:11 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi Krishantha, test scenario for [1] APIM wso2is-5.0.0 IS wso2am-1.9.0 [1] https://docs.wso2.com/display/AM180/Configuring+Single+Sign-on Basic cross products test scenario contains the following steps. 1. Invoke a http client call to publisher login. 2. Verify http response status codes and whether the SAML based SSO authentication was successful / unsuccessful ( Verify Login page / API Index page details.) 3. Verify avoidance of repeat authentication for APIM /store. Following steps were automated to-date. 1. HTTP call for /publisher 2. Invocation calls for samlsso endpoint 3. calls on commonauth endpoints 4. jaggery_acs jag related re-directions 5. /publisher login index page verifications I am currently working on automating the APIM .. /store related steps. Regards -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Nuwan Dias Associate Tech Lead - WSO2, Inc. http://wso2.com email : nuw...@wso2.com Phone : +94 777 775 729 -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Nuwan Wimalasekara Senior Software Engineer - Test Automation WSO2, Inc.: http://wso2.com lean. enterprise. middleware phone: +94 71 668 4620 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Status update] Platform automation test - Cross products Single Sign-on with SAML 2.0 Auto test
On Fri, Feb 20, 2015 at 12:03 PM, Dharshana Warusavitharana dharsha...@wso2.com wrote: I think its not practical to get code coverage in platform tests. Since there is always be an option to be all servers remote. We can do this, If proper deployment automation in place. Its a matter of instrumenting all packs and collecting coverage dump files back after test execution. Thanks, Krishantha. Thank you, Dharshana. On Fri, Feb 20, 2015 at 1:21 AM, Nuwan Wimalasekara nuw...@wso2.com wrote: Yes. We can aggregate multiple coverage reports and create a one report. But the problem is that We have to instrument the JAR file before starting the Distributed setup. I guess, Since this is platform test, Platform is already up and running. if so We have to instrument the JAR inside the server first before starting the servers. Thanks, Nuwanw On Wed, Feb 18, 2015 at 4:37 AM, Krishantha Samaraweera krishan...@wso2.com wrote: Hi Nuwan, On Wed, Feb 18, 2015 at 5:49 PM, Nuwan Dias nuw...@wso2.com wrote: Nice work. @Krishantha I have a question regarding getting code coverage for these. Since these are platform scenarios and don't execute at build time, is there a way we can find out the exact code coverage of all the test scenarios? We can merge the two reports and generate aggregated coverage report. This was already done for products like ESB with multiple test modules. However, in a distributed APIM setup merging reports will be bit difficult which we need to improve in future. For the time being we can merge the reports and generate coverage manually. Thanks, Krishantha. Currently the only comprehensive coverage comes from the tests that run at build time and we don't actually know the coverage as a whole (which includes the UI and platform scenarios). Thanks, NuwanD. On Wed, Feb 18, 2015 at 5:20 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: We have completed writing the above cross products (APIM IS) test case scenario and the test case covers the following sequence. 1. Publisher SSO login verifications 2. Creating a sample API 3. Change API state from created to published verify the API details 4. Verify avoidance of repeat authentication for APIM /store related verifications Regards On Tue, Feb 17, 2015 at 11:27 PM, Krishantha Samaraweera krishan...@wso2.com wrote: [adding dev] Thanks, Krishantha. On Tue, Feb 17, 2015 at 11:11 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi Krishantha, test scenario for [1] APIM wso2is-5.0.0 IS wso2am-1.9.0 [1] https://docs.wso2.com/display/AM180/Configuring+Single+Sign-on Basic cross products test scenario contains the following steps. 1. Invoke a http client call to publisher login. 2. Verify http response status codes and whether the SAML based SSO authentication was successful / unsuccessful ( Verify Login page / API Index page details.) 3. Verify avoidance of repeat authentication for APIM /store. Following steps were automated to-date. 1. HTTP call for /publisher 2. Invocation calls for samlsso endpoint 3. calls on commonauth endpoints 4. jaggery_acs jag related re-directions 5. /publisher login index page verifications I am currently working on automating the APIM .. /store related steps. Regards -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Nuwan Dias Associate Tech Lead - WSO2, Inc. http://wso2.com email : nuw...@wso2.com Phone : +94 777 775 729 -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Nuwan Wimalasekara Senior Software Engineer - Test Automation WSO2, Inc.: http://wso2.com lean. enterprise. middleware phone: +94 71 668 4620 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Dharshana Warusavitharana Senior Software Engineer , Test Automation WSO2 Inc. http://wso2.com email : dharsha...@wso2.com dharsha...@wso2.com Tel : +94 11 214 5345 Fax :+94 11 2145300 cell : +94770342233 blog : http://dharshanaw.blogspot.com lean . enterprise . middleware -- Krishantha Samaraweera
Re: [Dev] [Status update] Platform automation test - Cross products Single Sign-on with SAML 2.0 Auto test
Hi DimuthuD, On Wed, Feb 18, 2015 at 5:20 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: We have completed writing the above cross products (APIM IS) test case scenario and the test case covers the following sequence. 1. Publisher SSO login verifications 2. Creating a sample API 3. Change API state from created to published verify the API details 4. Verify avoidance of repeat authentication for APIM /store related verifications Did you cover the same scenario for tenant mode ? Thanks, Krishantha. Regards On Tue, Feb 17, 2015 at 11:27 PM, Krishantha Samaraweera krishan...@wso2.com wrote: [adding dev] Thanks, Krishantha. On Tue, Feb 17, 2015 at 11:11 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi Krishantha, test scenario for [1] APIM wso2is-5.0.0 IS wso2am-1.9.0 [1] https://docs.wso2.com/display/AM180/Configuring+Single+Sign-on Basic cross products test scenario contains the following steps. 1. Invoke a http client call to publisher login. 2. Verify http response status codes and whether the SAML based SSO authentication was successful / unsuccessful ( Verify Login page / API Index page details.) 3. Verify avoidance of repeat authentication for APIM /store. Following steps were automated to-date. 1. HTTP call for /publisher 2. Invocation calls for samlsso endpoint 3. calls on commonauth endpoints 4. jaggery_acs jag related re-directions 5. /publisher login index page verifications I am currently working on automating the APIM .. /store related steps. Regards -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Status update] Platform automation test - Cross products Single Sign-on with SAML 2.0 Auto test
Hi Nuwan, On Wed, Feb 18, 2015 at 5:49 PM, Nuwan Dias nuw...@wso2.com wrote: Nice work. @Krishantha I have a question regarding getting code coverage for these. Since these are platform scenarios and don't execute at build time, is there a way we can find out the exact code coverage of all the test scenarios? We can merge the two reports and generate aggregated coverage report. This was already done for products like ESB with multiple test modules. However, in a distributed APIM setup merging reports will be bit difficult which we need to improve in future. For the time being we can merge the reports and generate coverage manually. Thanks, Krishantha. Currently the only comprehensive coverage comes from the tests that run at build time and we don't actually know the coverage as a whole (which includes the UI and platform scenarios). Thanks, NuwanD. On Wed, Feb 18, 2015 at 5:20 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: We have completed writing the above cross products (APIM IS) test case scenario and the test case covers the following sequence. 1. Publisher SSO login verifications 2. Creating a sample API 3. Change API state from created to published verify the API details 4. Verify avoidance of repeat authentication for APIM /store related verifications Regards On Tue, Feb 17, 2015 at 11:27 PM, Krishantha Samaraweera krishan...@wso2.com wrote: [adding dev] Thanks, Krishantha. On Tue, Feb 17, 2015 at 11:11 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi Krishantha, test scenario for [1] APIM wso2is-5.0.0 IS wso2am-1.9.0 [1] https://docs.wso2.com/display/AM180/Configuring+Single+Sign-on Basic cross products test scenario contains the following steps. 1. Invoke a http client call to publisher login. 2. Verify http response status codes and whether the SAML based SSO authentication was successful / unsuccessful ( Verify Login page / API Index page details.) 3. Verify avoidance of repeat authentication for APIM /store. Following steps were automated to-date. 1. HTTP call for /publisher 2. Invocation calls for samlsso endpoint 3. calls on commonauth endpoints 4. jaggery_acs jag related re-directions 5. /publisher login index page verifications I am currently working on automating the APIM .. /store related steps. Regards -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Nuwan Dias Associate Tech Lead - WSO2, Inc. http://wso2.com email : nuw...@wso2.com Phone : +94 777 775 729 -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Status update] Platform automation test - Cross products Single Sign-on with SAML 2.0 Auto test
We have completed writing the above cross products (APIM IS) test case scenario and the test case covers the following sequence. 1. Publisher SSO login verifications 2. Creating a sample API 3. Change API state from created to published verify the API details 4. Verify avoidance of repeat authentication for APIM /store related verifications Regards On Tue, Feb 17, 2015 at 11:27 PM, Krishantha Samaraweera krishan...@wso2.com wrote: [adding dev] Thanks, Krishantha. On Tue, Feb 17, 2015 at 11:11 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi Krishantha, test scenario for [1] APIM wso2is-5.0.0 IS wso2am-1.9.0 [1] https://docs.wso2.com/display/AM180/Configuring+Single+Sign-on Basic cross products test scenario contains the following steps. 1. Invoke a http client call to publisher login. 2. Verify http response status codes and whether the SAML based SSO authentication was successful / unsuccessful ( Verify Login page / API Index page details.) 3. Verify avoidance of repeat authentication for APIM /store. Following steps were automated to-date. 1. HTTP call for /publisher 2. Invocation calls for samlsso endpoint 3. calls on commonauth endpoints 4. jaggery_acs jag related re-directions 5. /publisher login index page verifications I am currently working on automating the APIM .. /store related steps. Regards -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Status update] Platform automation test - Cross products Single Sign-on with SAML 2.0 Auto test
Nice work. @Krishantha I have a question regarding getting code coverage for these. Since these are platform scenarios and don't execute at build time, is there a way we can find out the exact code coverage of all the test scenarios? Currently the only comprehensive coverage comes from the tests that run at build time and we don't actually know the coverage as a whole (which includes the UI and platform scenarios). Thanks, NuwanD. On Wed, Feb 18, 2015 at 5:20 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: We have completed writing the above cross products (APIM IS) test case scenario and the test case covers the following sequence. 1. Publisher SSO login verifications 2. Creating a sample API 3. Change API state from created to published verify the API details 4. Verify avoidance of repeat authentication for APIM /store related verifications Regards On Tue, Feb 17, 2015 at 11:27 PM, Krishantha Samaraweera krishan...@wso2.com wrote: [adding dev] Thanks, Krishantha. On Tue, Feb 17, 2015 at 11:11 PM, Dimuthu De Lanerolle dimut...@wso2.com wrote: Hi Krishantha, test scenario for [1] APIM wso2is-5.0.0 IS wso2am-1.9.0 [1] https://docs.wso2.com/display/AM180/Configuring+Single+Sign-on Basic cross products test scenario contains the following steps. 1. Invoke a http client call to publisher login. 2. Verify http response status codes and whether the SAML based SSO authentication was successful / unsuccessful ( Verify Login page / API Index page details.) 3. Verify avoidance of repeat authentication for APIM /store. Following steps were automated to-date. 1. HTTP call for /publisher 2. Invocation calls for samlsso endpoint 3. calls on commonauth endpoints 4. jaggery_acs jag related re-directions 5. /publisher login index page verifications I am currently working on automating the APIM .. /store related steps. Regards -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- Dimuthu De Lanerolle Software Engineer WSO2 Inc. lean . enterprise . middlewear. http://wso2.com/ Tel. : +94 11 2145345 Fax : +94 11 2145300 email : dimut...@wso2.com -- Nuwan Dias Associate Tech Lead - WSO2, Inc. http://wso2.com email : nuw...@wso2.com Phone : +94 777 775 729 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
