Welcome dcs3spp!
One thing I noted in your snippet was you where using pdb (or ipdb). That looks like you’re not using an IDE that is able to debug ;) I would really recommend using PyCharm for such tinkering, cause even the community version has a well integrated debugger and supports py.test (once you configured your project to use it – one click in the configuration). Best, Andi From: 'dcs3spp' via pylons-discuss <[email protected]> Reply-To: <[email protected]> Date: Tuesday, 6. November 2018 at 10:23 To: pylons-discuss <[email protected]> Subject: Re: [pylons-discuss] Understanding how to perform functional testing using transactions with pyramid + SQLAlchemy + pytest + webtest for views that use a postgreSQL database. Hi Andi, Thankyou for sharing the fixtures and approach, much appreciated. So the SQLAlchemy test session is shared in the registry which triggers a bypass of the pyramid_tm session for the request when testing. That would solve the problem I have been encountering since, with this approach the session from testing is shared with web request thread and rolled back per test. I think that the difficulties I have been facing were as a result of me using two separate SQLAlchemy sessions, one for the test and one per pyramid web request. My approach in earlier code posting was an attempt to try to bind the sessions to an outer transaction for the test and an inner nested transaction for the web requested. However, I think they were separate transaction managers. I quickly became confused and lost concerning the effects of nested transactions. Presumably, once the inner transaction web request commits it would be difficult to rollback to the outer test transaction anyway. In summary, like your idea will give it a try.... Thanks again for sharing the code and concept. Kind Regards dcs3spp On Tuesday, 6 November 2018 06:54:52 UTC, Andi Blake wrote: Hi dcs3spp, I remember it was pretty complicated for me to setup something as you describe: one sqla session per test, which will be automatically rolled back. You are right, that in the TestApp there is a separate transaction handling, which destroys your approach. To solve that problem for me (no idea if this was elegant or not, but it does what you describe) I did the following. Created a TestApp with a custom registry field: @pytest.fixture(scope="function") def _test_app(router: Router) -> TestApp: return TestApp(router) @pytest.fixture(scope="function") def test_app(_test_app: TestApp, _sqla_session) -> TestApp: """wrapper to ensure the fixture-created `sqla_session` will be picked up in `test_app`""" _test_app.app.registry.settings['paste.testing'] = True _test_app.app.registry['paste.testing.session'] = _sqla_session return _test_app along with the usual sqla fixure @pytest.fixture(scope="session") def _sqla_session(pyramid_config, sqla_engine) -> Session: """ Depending on this fixture is comparable to a integration test, which has nothing more than the orm properly defined. Which is helpful, but not the full application configured. """ session_factory = get_session_factory(sqla_engine, db_session) return session_factory() @pytest.fixture(scope="function") def sqla_session(_sqla_session: Session, test_app): """ wrap a transaction """ # TODO andi: magically there is a transaction active here. why. # t = _sqla_session.begin() yield _sqla_session # this is the important `rollback` _sqla_session.transaction.rollback() Then I have something, that probably `pyramid_tm` does under the hood: a request listener, which handles a “per request session”. def add_tm_session(req): # this property is set in `webtest.app.TestApp#do_request` if 'paste.testing' in req.environ and req.environ['paste.testing'] is True: from pyramid.threadlocal import get_current_registry registry = get_current_registry() # in case of integration testing, we set this registry key with the `sqla_session` from # around. this allows us to operate in the same session, so we don't need any commits. return registry.get('paste.testing.session', None) # request.tm is the transaction manager used by pyramid_tm return get_tm_session(session_factory, req.tm) # make request.dbsession available for use in Pyramid config.add_request_method(add_tm_session, 'dbsession', reify=True) You can see the “magic” in there. What I found in my documentation was this link (another was dead already), which I found helpful: https://gist.github.com/inklesspen/4504383 The good thing about this pattern for me was, that I could apply this to Celery jobs in tests or Spyne integration - aka pretty flexible. Hope this helps. Andi On 5. Nov 2018, at 19:37, 'dcs3spp' via pylons-discuss <[email protected]> wrote: Hi, I am a newbie having difficulty understanding and getting functional testing working with pyramid, SQLAlchemy, pytest and webtest. I am using pyramid 1.10. Hoping that someone is able to advise a way forward or direct me to any useful resources. I have written the fixture below that creates a SQL Alchemy session for each test and initialises data within a transaction, based upon documentation for functional testing at the wiki that uses unittest. When the fixture completes the transaction aborts and closes the session. When the next test runs the fixture will create a new transaction and reinitialise the data. @pytest.fixture def session(request, testapp): factory = testapp.app.registry['dbsession_factory'] engine = factory.kw['bind'] # create all the tables in db Base.metadata.create_all(engine) log.info ("Creating root transaction for the test session") with transaction.manager as tx: from plantoeducate_data.models import get_tm_session session=get_tm_session(factory, transaction.manager) brief=DocumentTemplateModel () brief.DocumentDescription='brief' brief.DocumentTypeID=1 brief.DocumentFilePath='brief.ott' feedback=DocumentTemplateModel () feedback.DocumentDescription='feedback' feedback.DocumentTypeID=2 feedback.DocumentFilePath='feedback.ott' session.add_all([brief, feedback]) #session.flush() yield session log.info("Rolling back root transaction") transaction.abort() session.close() I have two tests that use the fixture, listed below: def test_delete_document(self, testapp, session): doc=session.query(DocumentTemplateModel).first() import pdb; pdb.set_trace() # delete the document res = testapp.delete('/documents/templates/{}'.format(doc.DocumentID), status=204) def test_filter_documents(self, testapp, session): res = testapp.get('/documents/templates/1', status=200) expectedTemplatesCount = 2 import pdb; pdb.set_trace() I think that pyramid_tm creates a session from a SQLAlchemy session factory for each request and hooks up the current active transaction. When the first test is run I can see data in the session, however the request session in the view does not see the data. When the second test runs there is no data at all in the session that was created by the test fixture. How do I make the data initialised in the test visible to the view being tested? Is it possible to perform testing by initialising data in SQLAlchemy, making it visible to the request session in the view and then rolling back state in preparation for subsequent test? Kind Regards dcs3spp -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/bf247ef1-1a48-4934-9bce-205d166a5c64%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/8daf90a7-3485-4a1c-8cbf-caa9153a054f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/1DBB4446-969E-4BF8-976B-AA633B182EA0%40googlemail.com. For more options, visit https://groups.google.com/d/optout.
