Hi Andi, Thankyou for sharing the fixtures and approach, much appreciated. So the SQLAlchemy test session is shared in the registry which triggers a bypass of the pyramid_tm session for the request when testing.
That would solve the problem I have been encountering since, with this approach the session from testing is shared with web request thread and rolled back per test. I think that the difficulties I have been facing were as a result of me using two separate SQLAlchemy sessions, one for the test and one per pyramid web request. My approach in earlier code posting was an attempt to try to bind the sessions to an outer transaction for the test and an inner nested transaction for the web requested. However, I think they were separate transaction managers. I quickly became confused and lost concerning the effects of nested transactions. Presumably, once the inner transaction web request commits it would be difficult to rollback to the outer test transaction anyway. In summary, like your idea will give it a try.... Thanks again for sharing the code and concept. Kind Regards dcs3spp On Tuesday, 6 November 2018 06:54:52 UTC, Andi Blake wrote: > > Hi dcs3spp, > > > > I remember it was pretty complicated for me to setup something as you > describe: one sqla session per test, which will be automatically rolled > back. > > > > You are right, that in the TestApp there is a separate transaction > handling, which destroys your approach. To solve that problem for me (no > idea if this was elegant or not, but it does what you describe) I did the > following. > > > > Created a TestApp with a custom registry field: > > @pytest.fixture(scope=*"function"*) > *def *_test_app(router: Router) -> TestApp: > *return *TestApp(router) > > > @pytest.fixture(scope=*"function"*) > *def *test_app(_test_app: TestApp, _sqla_session) -> TestApp: > > *"""wrapper to ensure the fixture-created `sqla_session` will be picked up in > `test_app`""" *_test_app.app.registry.settings[*'paste.testing'*] = > *True *_test_app.app.registry[*'paste.testing.session'*] = _sqla_session > *return *_test_app > > along with the usual sqla fixure > > > > @pytest.fixture(scope=*"session"*) > *def *_sqla_session(pyramid_config, sqla_engine) -> Session: > > > > > *""" Depending on this fixture is comparable to a integration test, which > has nothing more than the orm properly defined. Which is helpful, but not > the full application configured. """ *session_factory = > get_session_factory(sqla_engine, db_session) > *return *session_factory() > > > @pytest.fixture(scope=*"function"*) > *def *sqla_session(_sqla_session: Session, test_app): > > > > > > *""" wrap a transaction """ # TODO andi: magically there is a > transaction active here. why. # t = _sqla_session.begin() **yield > *_sqla_session > > > *# this is the important `rollback` *_sqla_session.transaction.rollback() > > > > Then I have something, that probably `pyramid_tm` does under the hood: a > request listener, which handles a “per request session”. > > > > *def *add_tm_session(req): > > *# this property is set in `webtest.app.TestApp#do_request` **if > 'paste.testing' in *req.environ *and *req.environ[*'paste.testing'*] *is > True*: > *from *pyramid.threadlocal *import *get_current_registry > > registry = get_current_registry() > > > *# in case of integration testing, we set this registry key with the > `sqla_session` from # around. this allows us to operate in the same > session, so we don't need any commits. **return > *registry.get(*'paste.testing.session'*, *None*) > > *# request.tm <http://request.tm> is the transaction manager used by > pyramid_tm **return *get_tm_session(session_factory, req.tm) > > > *# make request.dbsession available for use in > Pyramid*config.add_request_method(add_tm_session, *'dbsession'*, reify=*True*) > > You can see the “magic” in there. > > > > What I found in my documentation was this link (another was dead already), > which I found helpful: https://gist.github.com/inklesspen/4504383 > > > > The good thing about this pattern for me was, that I could apply this to > Celery jobs in tests or Spyne integration - aka pretty flexible. > > > > Hope this helps. > > > > Andi > > > > > > On 5. Nov 2018, at 19:37, 'dcs3spp' via pylons-discuss < > [email protected] <javascript:>> wrote: > > > > Hi, > > > > I am a newbie having difficulty understanding and getting functional > testing working with pyramid, SQLAlchemy, pytest and webtest. I am using > pyramid 1.10. Hoping that someone is able to advise a way forward or direct > me to any useful resources. > > > > I have written the fixture below that creates a SQL Alchemy session for > each test and initialises data within a transaction, based upon > documentation for functional testing at the wiki > <https://docs.pylonsproject.org/projects/pyramid/en/latest/tutorials/wiki2/tests.html> > that > uses unittest. > > When the fixture completes the transaction aborts and closes the session. > When the next test runs the fixture will create a new transaction and > reinitialise the data. > > > > @pytest.fixture > > def session(request, testapp): > > > > factory = testapp.app.registry['dbsession_factory'] > > engine = factory.kw['bind'] > > > > # create all the tables in db > > Base.metadata.create_all(engine) > > > > log.info ("Creating root transaction for the test session") > > with transaction.manager as tx: > > from plantoeducate_data.models import get_tm_session > > session=get_tm_session(factory, transaction.manager) > > > > brief=DocumentTemplateModel () > > brief.DocumentDescription='brief' > > brief.DocumentTypeID=1 > > brief.DocumentFilePath='brief.ott' > > feedback=DocumentTemplateModel () > > feedback.DocumentDescription='feedback' > > feedback.DocumentTypeID=2 > > feedback.DocumentFilePath='feedback.ott' > > > > session.add_all([brief, feedback]) > > #session.flush() > > > > yield session > > > > log.info("Rolling back root transaction") > > transaction.abort() > > session.close() > > > > > > I have two tests that use the fixture, listed below: > > def test_delete_document(self, testapp, session): > > doc=session.query(DocumentTemplateModel).first() > > import pdb; pdb.set_trace() > > # delete the document > > res = > testapp.delete('/documents/templates/{}'.format(doc.DocumentID), status=204) > > > > > > def test_filter_documents(self, testapp, session): > > res = testapp.get('/documents/templates/1', status=200) > > > > expectedTemplatesCount = 2 > > import pdb; pdb.set_trace() > > > I think that *pyramid_tm* creates a session from a SQLAlchemy session > factory for each request and hooks up the current active transaction. > > > > When the first test is run I can see data in the session, however the > request session in the view does not see the data. > > When the second test runs there is no data at all in the session that was > created by the test fixture. > > > > How do I make the data initialised in the test visible to the view being > tested? Is it possible to perform testing by initialising data in > SQLAlchemy, making it visible to the request session in the view and then > rolling back state in preparation for subsequent test? > > > > Kind Regards > > > > > > dcs3spp > > > > -- > You received this message because you are subscribed to the Google Groups > "pylons-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] > <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/pylons-discuss/bf247ef1-1a48-4934-9bce-205d166a5c64%40googlegroups.com > > <https://groups.google.com/d/msgid/pylons-discuss/bf247ef1-1a48-4934-9bce-205d166a5c64%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/8daf90a7-3485-4a1c-8cbf-caa9153a054f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
