Thanks Michael,
I’d be glad to have this appear in a cookiecutter – let me know how I can help.
For the record, what I missed to mention (but is actually the bigger selling
point for me):
Apart from the automatic rollback functionality, that approach allows putting
anything to the session, no matter where. I.e. you can `add` and `flush`
something in a test, before calling the `TestApp` in the same test, which will
have access to what was `add`ed to that session. Still without anything being
committed. You can imagine how easy writing integration tests is in the end.
Best, Andi
From: <[email protected]> on behalf of Michael Merickel
<[email protected]>
Reply-To: <[email protected]>
Date: Tuesday, 6. November 2018 at 16:58
To: <[email protected]>
Subject: Re: [pylons-discuss] Understanding how to perform functional testing
using transactions with pyramid + SQLAlchemy + pytest + webtest for views that
use a postgreSQL database.
Andi, I think this is a fantastic approach. The key here is to override
request.dbsession to a mocked out version which is not connected to pyramid_tm
at all, giving you full control of commit/rollback from the outside. One extra
step would be to disable pyramid_tm entirely by setting environ['tm.active'] =
True which would let you mock out request.tm if you were using it for any other
purposes. This can be done from the TestApp constructor (you could also use
this for the dbsession to avoid using the get_current_request threadlocal) via
TestApp(app, extra_environ={'tm.active': True, 'dbsession': dbsession}).
We should look at adding this pattern to the cookiecutter to make it more
accessible for people.
On Tue, Nov 6, 2018 at 12:54 AM 'andi' via pylons-discuss
<[email protected]> wrote:
Hi dcs3spp,
I remember it was pretty complicated for me to setup something as you describe:
one sqla session per test, which will be automatically rolled back.
You are right, that in the TestApp there is a separate transaction handling,
which destroys your approach. To solve that problem for me (no idea if this was
elegant or not, but it does what you describe) I did the following.
Created a TestApp with a custom registry field:
@pytest.fixture(scope="function")
def _test_app(router: Router) -> TestApp:
return TestApp(router)
@pytest.fixture(scope="function")
def test_app(_test_app: TestApp, _sqla_session) -> TestApp:
"""wrapper to ensure the fixture-created `sqla_session` will be picked up
in `test_app`"""
_test_app.app.registry.settings['paste.testing'] = True
_test_app.app.registry['paste.testing.session'] = _sqla_session
return _test_app
along with the usual sqla fixure
@pytest.fixture(scope="session")
def _sqla_session(pyramid_config, sqla_engine) -> Session:
"""
Depending on this fixture is comparable to a integration test, which has
nothing more than
the orm properly defined. Which is helpful, but not the full application
configured.
"""
session_factory = get_session_factory(sqla_engine, db_session)
return session_factory()
@pytest.fixture(scope="function")
def sqla_session(_sqla_session: Session, test_app):
"""
wrap a transaction
"""
# TODO andi: magically there is a transaction active here. why.
# t = _sqla_session.begin()
yield _sqla_session
# this is the important `rollback`
_sqla_session.transaction.rollback()
Then I have something, that probably `pyramid_tm` does under the hood: a
request listener, which handles a “per request session”.
def add_tm_session(req):
# this property is set in `webtest.app.TestApp#do_request`
if 'paste.testing' in req.environ and req.environ['paste.testing'] is True:
from pyramid.threadlocal import get_current_registry
registry = get_current_registry()
# in case of integration testing, we set this registry key with the
`sqla_session` from
# around. this allows us to operate in the same session, so we don't
need any commits.
return registry.get('paste.testing.session', None)
# request.tm is the transaction manager used by pyramid_tm
return get_tm_session(session_factory, req.tm)
# make request.dbsession available for use in Pyramid
config.add_request_method(add_tm_session, 'dbsession', reify=True)
You can see the “magic” in there.
What I found in my documentation was this link (another was dead already),
which I found helpful: https://gist.github.com/inklesspen/4504383
The good thing about this pattern for me was, that I could apply this to Celery
jobs in tests or Spyne integration - aka pretty flexible.
Hope this helps.
Andi
On 5. Nov 2018, at 19:37, 'dcs3spp' via pylons-discuss
<[email protected]> wrote:
Hi,
I am a newbie having difficulty understanding and getting functional testing
working with pyramid, SQLAlchemy, pytest and webtest. I am using pyramid 1.10.
Hoping that someone is able to advise a way forward or direct me to any useful
resources.
I have written the fixture below that creates a SQL Alchemy session for each
test and initialises data within a transaction, based upon documentation for
functional testing at the wiki that uses unittest.
When the fixture completes the transaction aborts and closes the session. When
the next test runs the fixture will create a new transaction and reinitialise
the data.
@pytest.fixture
def session(request, testapp):
factory = testapp.app.registry['dbsession_factory']
engine = factory.kw['bind']
# create all the tables in db
Base.metadata.create_all(engine)
log.info ("Creating root transaction for the test session")
with transaction.manager as tx:
from plantoeducate_data.models import get_tm_session
session=get_tm_session(factory, transaction.manager)
brief=DocumentTemplateModel ()
brief.DocumentDescription='brief'
brief.DocumentTypeID=1
brief.DocumentFilePath='brief.ott'
feedback=DocumentTemplateModel ()
feedback.DocumentDescription='feedback'
feedback.DocumentTypeID=2
feedback.DocumentFilePath='feedback.ott'
session.add_all([brief, feedback])
#session.flush()
yield session
log.info("Rolling back root transaction")
transaction.abort()
session.close()
I have two tests that use the fixture, listed below:
def test_delete_document(self, testapp, session):
doc=session.query(DocumentTemplateModel).first()
import pdb; pdb.set_trace()
# delete the document
res = testapp.delete('/documents/templates/{}'.format(doc.DocumentID),
status=204)
def test_filter_documents(self, testapp, session):
res = testapp.get('/documents/templates/1', status=200)
expectedTemplatesCount = 2
import pdb; pdb.set_trace()
I think that pyramid_tm creates a session from a SQLAlchemy session factory for
each request and hooks up the current active transaction.
When the first test is run I can see data in the session, however the request
session in the view does not see the data.
When the second test runs there is no data at all in the session that was
created by the test fixture.
How do I make the data initialised in the test visible to the view being
tested? Is it possible to perform testing by initialising data in SQLAlchemy,
making it visible to the request session in the view and then rolling back
state in preparation for subsequent test?
Kind Regards
dcs3spp
--
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/pylons-discuss/bf247ef1-1a48-4934-9bce-205d166a5c64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/pylons-discuss/A1460691-D307-4B5C-846C-E8CD20BE8FA3%40googlemail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwFcagH3GzLMdJZ2qCa%3D9csv5nEzcRR47%2Bg5k%2BYFMH1kRA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/pylons-discuss/5929CFD4-982F-44E0-A02F-71E048E1D6A6%40googlemail.com.
For more options, visit https://groups.google.com/d/optout.
