Re: [Mailman-Developers] How can I run the REST tests against live servers?
I have managed to make the existing tests work through my proxy but it was challenging for a few reasons. The tests are hard coded to transmit their HTTP requests on port 9001. The assertEqual in the tests sometimes expect to see JSON data that includes, in some cases, absolute URLs with hard coded port numbers for example: "self_link": "http://localhost:9001/3.0/lists/bee.example.com"; To make the tests work I put my proxy at port 7001, changed the hard coded test requests to be transmitted to port 7001 instead of port 9001. I set testing.cfg to [webservice] port: 9001 Inside my proxy I configured all inbound queries to go to port 9001. So the server returns data including port 9001 in the self link URLs, and the tests successfully find the expected port number. Should “self links” to anything be hard coded to a host and port number? I did wonder whether the API should be returning self links as just relative URL (i.e. no http://hostname:port ) and it is up to something else to work out the host that the self link lives on. If self links include http://hostname:port then any proxy would need to transform the data on the way back top the client to remove or correct the host and port number. Andrew On 12 Dec 2014, at 11:34 am, Barry Warsaw wrote: On Dec 12, 2014, at 08:35 AM, Andrew Stuart wrote: > I’m writing an authenticating proxy for the Mailman REST API and want to make > sure everything works as expected against real infrastructure > (Postgres/Postfix/Sqlalchemy/Falcon). We definitely want such a proxy. It's always been our intent that the core's REST API is an adminipstrative API with full access. It should only be exposed on localhost or other highly secure interface. A proxy server such as you describe would be available on a public interface and would allow people to script the management of their list memberships and list administrations. > I was hoping that just pointing the REST tests request port to my proxy would > work but no dice - lots of failures - trying to diagnose the failures has led > me back to trying to get the test suite to work against the live servers. > > I’m wondering if maybe the shortest route here might not be for me to write a > new set of tests which purely talk REST and don’t touch the back end directly > at all. Except that’s not a short route. :-) > > Trying to bend the existing tests to meet my needs is looking harder than I > have time to deal with. Even in this case, I wouldn't expect the standard unittests to work. For example, if a normal user were to try to access the list admin API, you'd expect that to fail. I think one approach would be to continue to use the test servers, but have a backdoor connection which would let you set up the state before each test and reset the state after each test. Then the tests themselves would use the authenticated API to flex the things it can and shouldn't be able to do. It would be possible for example to expose some helper resources in the admin API which would not normally be exposed in production. Something under a top level /tests resource perhaps. Cheers, -Barry ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] How can I run the REST tests against live servers?
Andrew Stuart writes: > I’m writing an authenticating proxy for the Mailman REST API and > want to make sure everything works as expected against real > infrastructure (Postgres/Postfix/Sqlalchemy/Falcon). Determining that "things work as expected against real infrastructure" is not what the test suite does. AFAIK we don't have any such tests, and I don't really see how the tests we do have can be adapted to "tests against real infrastructure". The problem is specifying what to expect of real infrastructure. You could, for example, write tests that insert users, query for them, and then delete them, but this has several problems. Do you want to do these tests on "live" servers that might actually be in use? -- Probably not, but how do you prevent that? Attributes of the various objects created by the system may depend on context. -- The test needs to ignore any differences due merely to different context. Coverage is necessarily imperfect in the sense that some internal state changes can't be checked since the "before" state is indeterminate in a live system. And so on. ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] How can I run the REST tests against live servers?
On Dec 12, 2014, at 08:35 AM, Andrew Stuart wrote: >I’m writing an authenticating proxy for the Mailman REST API and want to make >sure everything works as expected against real infrastructure >(Postgres/Postfix/Sqlalchemy/Falcon). We definitely want such a proxy. It's always been our intent that the core's REST API is an adminipstrative API with full access. It should only be exposed on localhost or other highly secure interface. A proxy server such as you describe would be available on a public interface and would allow people to script the management of their list memberships and list administrations. >I was hoping that just pointing the REST tests request port to my proxy would >work but no dice - lots of failures - trying to diagnose the failures has led >me back to trying to get the test suite to work against the live servers. > >I’m wondering if maybe the shortest route here might not be for me to write a >new set of tests which purely talk REST and don’t touch the back end directly >at all. Except that’s not a short route. :-) > >Trying to bend the existing tests to meet my needs is looking harder than I >have time to deal with. Even in this case, I wouldn't expect the standard unittests to work. For example, if a normal user were to try to access the list admin API, you'd expect that to fail. I think one approach would be to continue to use the test servers, but have a backdoor connection which would let you set up the state before each test and reset the state after each test. Then the tests themselves would use the authenticated API to flex the things it can and shouldn't be able to do. It would be possible for example to expose some helper resources in the admin API which would not normally be exposed in production. Something under a top level /tests resource perhaps. Cheers, -Barry ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] Error installing mailman.client due to "Aurélien"
At first I thought I had introduced a regression but it's actually just my name :-) Don't blame me, I didn't choose it! (Florian, can you fix that? And by the way my last name is "Bompard", without the "A") Thanks ! A. 2014-12-11 21:08 GMT+01:00 Andrew Stuart : > Hello, > > Referring to transcript below, the word Aurélien seems to be causing a > unicode error when setting up mailman client. > > When I cut it out, it works. > > thanks > > > (venv2.7)ubuntu@server01:~$ bzr branch lp:mailman.client > You have not informed bzr of your Launchpad ID, and you must do this to > write to Launchpad or access private data. See "bzr help launchpad-login". > Branched 58 revisions. > (venv2.7)ubuntu@server01:~$ cd mailman.client/ > (venv2.7)ubuntu@server01:~/mailman.client$ ls > bin COPYING.LESSER distribute_setup.py Makefile MANIFEST.in > README.txt setup.cfg setup_helpers.py setup.py src _static template.py > (venv2.7)ubuntu@server01:~/mailman.client$ python setup.py develop > Traceback (most recent call last): > File "setup.py", line 40, in > 'src/mailmanclient/NEWS.txt'), > File "/home/ubuntu/mailman.client/setup_helpers.py", line 138, in > long_description > if not value.endswith(NL): > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 162: > ordinal not in range(128) > (venv2.7)ubuntu@server01:~/mailman.client$ vi src/mailmanclient/NEWS.txt > (venv2.7)ubuntu@server01:~/mailman.client$ cat src/mailmanclient/NEWS.txt > === > NEWS for mailman.client > === > > 1.0.0b1 (-xx-xx) > > * Addedd an improved test harness using WebTest. Contributed by Aurélien > Abompard. > > > 1.0.0a1 (2014-03-15) > > > * Initial release. > (venv2.7)ubuntu@server01:~/mailman.client$ vi src/mailmanclient/NEWS.txt > (venv2.7)ubuntu@server01:~/mailman.client$ cat src/mailmanclient/NEWS.txt > === > NEWS for mailman.client > === > > 1.0.0b1 (-xx-xx) > > > > 1.0.0a1 (2014-03-15) > > > * Initial release. > (venv2.7)ubuntu@server01:~/mailman.client$ > ___ > Mailman-Developers mailing list > Mailman-Developers@python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > https://mail.python.org/mailman/options/mailman-developers/aurelien%40bompard.org > > Security Policy: http://wiki.list.org/x/QIA9 ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] How can I run the REST tests against live servers?
Hi Barry I’m writing an authenticating proxy for the Mailman REST API and want to make sure everything works as expected against real infrastructure (Postgres/Postfix/Sqlalchemy/Falcon). I was hoping that just pointing the REST tests request port to my proxy would work but no dice - lots of failures - trying to diagnose the failures has led me back to trying to get the test suite to work against the live servers. I’m wondering if maybe the shortest route here might not be for me to write a new set of tests which purely talk REST and don’t touch the back end directly at all. Except that’s not a short route. :-) Trying to bend the existing tests to meet my needs is looking harder than I have time to deal with. Andrew On 12 Dec 2014, at 8:27 am, Barry Warsaw wrote: On Dec 12, 2014, at 07:52 AM, Andrew Stuart wrote: > I want to run the REST tests against my live Mailman server with its Postfix > mail server and Postgres database, via: > > nose2 -v -P rest > > After many (many) hours of digging deep and figuring out how it works, I > suspect at this stage that maybe the tests set up their own mock servers? If > so, I’m guessing these mock servers are getting in the way of using the live > servers and thus preventing the tests from running? > > Assuming it is correct that the tests are setting up their own mock http smtp > ltmp responders, is there a way to turn this behaviour off so the tests will > run against my live REST/Postgres/Postfix? Hi Andrew, You're right that the test suite starts its own http and lmtp servers. Take a look at src/mailman/testing/layers.py for how this is controlled, and review src/mailman/testing/nose.py and this documentation on nose2 support for Zope testrunner-style layers: https://nose2.readthedocs.org/en/latest/plugins/layers.html I think even if you could somehow substitute real servers for the testing ones, you'll still have a problem running the test suite against them, to any semblance of success. The tests are dependent on specific arrangements of sample data in order to run successfully. E.g. you'll see that the ConfigLayer, which is the "base class"[1] creates an example.com domain, so that all a test needs to do is create a mailing list in that domain without first having to create the domain. The test suite is also very careful to restore pristine state between individual tests; look for reset_the_world() which is run after every test, and does exactly what it describes. This ensures that no state from one test can affect the state of another test. In the real world testing you propose above, you would have quite a bit of bleed-through I suspect. Can you describe why you want to run the test suite against a live Mailman server? I do think it would be interesting to build an integration test suite that would talk to real MTAs and databases, but that would take a fair bit of work and probably be separate from the unittests, which aim to gather 100% code coverage. Cheers, -Barry [1] Although layers don't really support normal Python class hierarchies. ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au Security Policy: http://wiki.list.org/x/QIA9 ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] How can I run the REST tests against live servers?
On Dec 12, 2014, at 07:52 AM, Andrew Stuart wrote: >I want to run the REST tests against my live Mailman server with its Postfix >mail server and Postgres database, via: > >nose2 -v -P rest > >After many (many) hours of digging deep and figuring out how it works, I >suspect at this stage that maybe the tests set up their own mock servers? If >so, I’m guessing these mock servers are getting in the way of using the live >servers and thus preventing the tests from running? > >Assuming it is correct that the tests are setting up their own mock http smtp >ltmp responders, is there a way to turn this behaviour off so the tests will >run against my live REST/Postgres/Postfix? Hi Andrew, You're right that the test suite starts its own http and lmtp servers. Take a look at src/mailman/testing/layers.py for how this is controlled, and review src/mailman/testing/nose.py and this documentation on nose2 support for Zope testrunner-style layers: https://nose2.readthedocs.org/en/latest/plugins/layers.html I think even if you could somehow substitute real servers for the testing ones, you'll still have a problem running the test suite against them, to any semblance of success. The tests are dependent on specific arrangements of sample data in order to run successfully. E.g. you'll see that the ConfigLayer, which is the "base class"[1] creates an example.com domain, so that all a test needs to do is create a mailing list in that domain without first having to create the domain. The test suite is also very careful to restore pristine state between individual tests; look for reset_the_world() which is run after every test, and does exactly what it describes. This ensures that no state from one test can affect the state of another test. In the real world testing you propose above, you would have quite a bit of bleed-through I suspect. Can you describe why you want to run the test suite against a live Mailman server? I do think it would be interesting to build an integration test suite that would talk to real MTAs and databases, but that would take a fair bit of work and probably be separate from the unittests, which aim to gather 100% code coverage. Cheers, -Barry [1] Although layers don't really support normal Python class hierarchies. ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
[Mailman-Developers] How can I run the REST tests against live servers?
I want to run the REST tests against my live Mailman server with its Postfix mail server and Postgres database, via: nose2 -v -P rest After many (many) hours of digging deep and figuring out how it works, I suspect at this stage that maybe the tests set up their own mock servers? If so, I’m guessing these mock servers are getting in the way of using the live servers and thus preventing the tests from running? Assuming it is correct that the tests are setting up their own mock http smtp ltmp responders, is there a way to turn this behaviour off so the tests will run against my live REST/Postgres/Postfix? thanks Andrew ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
[Mailman-Developers] Error installing mailman.client due to "Aurélien"
Hello, Referring to transcript below, the word Aurélien seems to be causing a unicode error when setting up mailman client. When I cut it out, it works. thanks (venv2.7)ubuntu@server01:~$ bzr branch lp:mailman.client You have not informed bzr of your Launchpad ID, and you must do this to write to Launchpad or access private data. See "bzr help launchpad-login". Branched 58 revisions. (venv2.7)ubuntu@server01:~$ cd mailman.client/ (venv2.7)ubuntu@server01:~/mailman.client$ ls bin COPYING.LESSER distribute_setup.py Makefile MANIFEST.in README.txt setup.cfg setup_helpers.py setup.py src _static template.py (venv2.7)ubuntu@server01:~/mailman.client$ python setup.py develop Traceback (most recent call last): File "setup.py", line 40, in 'src/mailmanclient/NEWS.txt'), File "/home/ubuntu/mailman.client/setup_helpers.py", line 138, in long_description if not value.endswith(NL): UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 162: ordinal not in range(128) (venv2.7)ubuntu@server01:~/mailman.client$ vi src/mailmanclient/NEWS.txt (venv2.7)ubuntu@server01:~/mailman.client$ cat src/mailmanclient/NEWS.txt === NEWS for mailman.client === 1.0.0b1 (-xx-xx) * Addedd an improved test harness using WebTest. Contributed by Aurélien Abompard. 1.0.0a1 (2014-03-15) * Initial release. (venv2.7)ubuntu@server01:~/mailman.client$ vi src/mailmanclient/NEWS.txt (venv2.7)ubuntu@server01:~/mailman.client$ cat src/mailmanclient/NEWS.txt === NEWS for mailman.client === 1.0.0b1 (-xx-xx) 1.0.0a1 (2014-03-15) * Initial release. (venv2.7)ubuntu@server01:~/mailman.client$ ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
