Re: [openstack-dev] [fuel] oddness with sqlalchemy db().refresh(object)
Hello Andrew, here is the problem: https://review.openstack.org/#/c/78406/5/nailgun/nailgun/test/base.py line 268. If you remove this this line then in https://review.openstack.org/#/c/78406/5/nailgun/nailgun/network/manager.py`if` block from line 131 will be run, especially db.commit() which will save all pending changes, including new task_provision.weight value. Regards On Sat, Apr 12, 2014 at 12:33 AM, Andrew Woodward wrote: > Recently in one of my changes [1] I was fighting with one of the unit > tests showing a failure for a test which should have been outside the > sphere of influence. > > Traceback (most recent call last): > File > "/home/andreww/.virtualenvs/fuel/local/lib/python2.7/site-packages/mock.py", > line 1190, in patched > return func(*args, **keywargs) > File > "/home/andreww/git/fuel-web/nailgun/nailgun/test/integration/test_task_managers.py", > line 65, in test_deployment_task_managers > self.assertEquals(provision_task.weight, 0.4) > AssertionError: 1.0 != 0.4 > > After walking through a number of times and finally playing with it we > where able to find that the db().refresh(task_provision) call appeared > to be reseting the object. This caused the loss of the weight being > set to 0.4 (1.0 is the model default). db().commit(), db().flush() and > no call to db all caused the test to pass again. > > Does anyone have any input on why this would occur? The oddly odd part > is that this test doesn't fail outside of the change set in [1] > > [1] > https://review.openstack.org/#/c/78406/8/nailgun/nailgun/task/manager.py > > -- > Andrew > Mirantis > Ceph community > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Łukasz Oleś ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [fuel] oddness with sqlalchemy db().refresh(object)
Hello Andrew, I've reproduced the issue on the patch set #5 too. But at the same time, I have no issue on the latest patch set (#9). Thanks, Igor On Mon, Apr 14, 2014 at 9:00 AM, Andrew Woodward wrote: > Igor, > > Ryan was able to repeat the issue in his env by checking out the patch set > (5) with db().refresh(task_provision) enabled. > > > On Sun, Apr 13, 2014 at 3:28 AM, Igor Kalnitsky > wrote: > >> Hi guys, >> >> *@Roman*, we have >> >> db().refresh(task_provision) >> >> since the line above (self._call_silently) may change task status >> directly in db, not in SQLAlchemy session. >> >> *@Andrew*, it's really strange. I just checkout your request, return >> refreshing task and run tests. >> As a result, I have no failed tests in the test_task_manager.py. >> >> Are you sure your environment is ok? >> >> Thanks, >> Igor >> >> >> On Sun, Apr 13, 2014 at 12:57 PM, Roman Podoliaka < >> rpodoly...@mirantis.com> wrote: >> >>> Hi Andrew, >>> >>> I believe, it's just the way SQLAlchemy Session works: all the changes >>> you've made within a session aren't propagated to the db (read: no >>> actual queries are issued) until you explicitly do: >>> >>> - flush(), or >>> - commit() (as commit() calls flush() first), or >>> - Query - one(), first(), all(), update(), delete() - as these are >>> actions that can only be performed by contacting the db. >>> >>> >>> db().refresh(task_provision) call appeared to be reseting the object >>> >>> Yes, and this is exactly what it is supposed to do: get the current >>> state of the model instance from the db (basically: select * from >>> model_table where id = instance_primary_key_value). This means, that >>> all the changes you've made, but haven't flushed yet, will be lost. >>> I've made a small snippet to see this in action: >>> http://xsnippet.org/359888/ (with logging of SQL queries enabled) >>> >>> I hope this helps. I'm just wondering, why would you want to call >>> refresh() there? >>> >>> Thanks, >>> Roman >>> >>> On Sat, Apr 12, 2014 at 1:33 AM, Andrew Woodward >>> wrote: >>> > Recently in one of my changes [1] I was fighting with one of the unit >>> > tests showing a failure for a test which should have been outside the >>> > sphere of influence. >>> > >>> > Traceback (most recent call last): >>> > File >>> "/home/andreww/.virtualenvs/fuel/local/lib/python2.7/site-packages/mock.py", >>> > line 1190, in patched >>> > return func(*args, **keywargs) >>> > File >>> "/home/andreww/git/fuel-web/nailgun/nailgun/test/integration/test_task_managers.py", >>> > line 65, in test_deployment_task_managers >>> > self.assertEquals(provision_task.weight, 0.4) >>> > AssertionError: 1.0 != 0.4 >>> > >>> > After walking through a number of times and finally playing with it we >>> > where able to find that the db().refresh(task_provision) call appeared >>> > to be reseting the object. This caused the loss of the weight being >>> > set to 0.4 (1.0 is the model default). db().commit(), db().flush() and >>> > no call to db all caused the test to pass again. >>> > >>> > Does anyone have any input on why this would occur? The oddly odd part >>> > is that this test doesn't fail outside of the change set in [1] >>> > >>> > [1] >>> https://review.openstack.org/#/c/78406/8/nailgun/nailgun/task/manager.py >>> > >>> > -- >>> > Andrew >>> > Mirantis >>> > Ceph community >>> > >>> > ___ >>> > OpenStack-dev mailing list >>> > OpenStack-dev@lists.openstack.org >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > -- > Andrew > Mirantis > Ceph community > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [fuel] oddness with sqlalchemy db().refresh(object)
Igor, Ryan was able to repeat the issue in his env by checking out the patch set (5) with db().refresh(task_provision) enabled. On Sun, Apr 13, 2014 at 3:28 AM, Igor Kalnitsky wrote: > Hi guys, > > *@Roman*, we have > > db().refresh(task_provision) > > since the line above (self._call_silently) may change task status > directly in db, not in SQLAlchemy session. > > *@Andrew*, it's really strange. I just checkout your request, return > refreshing task and run tests. > As a result, I have no failed tests in the test_task_manager.py. > > Are you sure your environment is ok? > > Thanks, > Igor > > > On Sun, Apr 13, 2014 at 12:57 PM, Roman Podoliaka > wrote: > >> Hi Andrew, >> >> I believe, it's just the way SQLAlchemy Session works: all the changes >> you've made within a session aren't propagated to the db (read: no >> actual queries are issued) until you explicitly do: >> >> - flush(), or >> - commit() (as commit() calls flush() first), or >> - Query - one(), first(), all(), update(), delete() - as these are >> actions that can only be performed by contacting the db. >> >> >>> db().refresh(task_provision) call appeared to be reseting the object >> >> Yes, and this is exactly what it is supposed to do: get the current >> state of the model instance from the db (basically: select * from >> model_table where id = instance_primary_key_value). This means, that >> all the changes you've made, but haven't flushed yet, will be lost. >> I've made a small snippet to see this in action: >> http://xsnippet.org/359888/ (with logging of SQL queries enabled) >> >> I hope this helps. I'm just wondering, why would you want to call >> refresh() there? >> >> Thanks, >> Roman >> >> On Sat, Apr 12, 2014 at 1:33 AM, Andrew Woodward >> wrote: >> > Recently in one of my changes [1] I was fighting with one of the unit >> > tests showing a failure for a test which should have been outside the >> > sphere of influence. >> > >> > Traceback (most recent call last): >> > File >> "/home/andreww/.virtualenvs/fuel/local/lib/python2.7/site-packages/mock.py", >> > line 1190, in patched >> > return func(*args, **keywargs) >> > File >> "/home/andreww/git/fuel-web/nailgun/nailgun/test/integration/test_task_managers.py", >> > line 65, in test_deployment_task_managers >> > self.assertEquals(provision_task.weight, 0.4) >> > AssertionError: 1.0 != 0.4 >> > >> > After walking through a number of times and finally playing with it we >> > where able to find that the db().refresh(task_provision) call appeared >> > to be reseting the object. This caused the loss of the weight being >> > set to 0.4 (1.0 is the model default). db().commit(), db().flush() and >> > no call to db all caused the test to pass again. >> > >> > Does anyone have any input on why this would occur? The oddly odd part >> > is that this test doesn't fail outside of the change set in [1] >> > >> > [1] >> https://review.openstack.org/#/c/78406/8/nailgun/nailgun/task/manager.py >> > >> > -- >> > Andrew >> > Mirantis >> > Ceph community >> > >> > ___ >> > OpenStack-dev mailing list >> > OpenStack-dev@lists.openstack.org >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Andrew Mirantis Ceph community ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [fuel] oddness with sqlalchemy db().refresh(object)
Hi guys, *@Roman*, we have db().refresh(task_provision) since the line above (self._call_silently) may change task status directly in db, not in SQLAlchemy session. *@Andrew*, it's really strange. I just checkout your request, return refreshing task and run tests. As a result, I have no failed tests in the test_task_manager.py. Are you sure your environment is ok? Thanks, Igor On Sun, Apr 13, 2014 at 12:57 PM, Roman Podoliaka wrote: > Hi Andrew, > > I believe, it's just the way SQLAlchemy Session works: all the changes > you've made within a session aren't propagated to the db (read: no > actual queries are issued) until you explicitly do: > > - flush(), or > - commit() (as commit() calls flush() first), or > - Query - one(), first(), all(), update(), delete() - as these are > actions that can only be performed by contacting the db. > > >>> db().refresh(task_provision) call appeared to be reseting the object > > Yes, and this is exactly what it is supposed to do: get the current > state of the model instance from the db (basically: select * from > model_table where id = instance_primary_key_value). This means, that > all the changes you've made, but haven't flushed yet, will be lost. > I've made a small snippet to see this in action: > http://xsnippet.org/359888/ (with logging of SQL queries enabled) > > I hope this helps. I'm just wondering, why would you want to call > refresh() there? > > Thanks, > Roman > > On Sat, Apr 12, 2014 at 1:33 AM, Andrew Woodward wrote: > > Recently in one of my changes [1] I was fighting with one of the unit > > tests showing a failure for a test which should have been outside the > > sphere of influence. > > > > Traceback (most recent call last): > > File > "/home/andreww/.virtualenvs/fuel/local/lib/python2.7/site-packages/mock.py", > > line 1190, in patched > > return func(*args, **keywargs) > > File > "/home/andreww/git/fuel-web/nailgun/nailgun/test/integration/test_task_managers.py", > > line 65, in test_deployment_task_managers > > self.assertEquals(provision_task.weight, 0.4) > > AssertionError: 1.0 != 0.4 > > > > After walking through a number of times and finally playing with it we > > where able to find that the db().refresh(task_provision) call appeared > > to be reseting the object. This caused the loss of the weight being > > set to 0.4 (1.0 is the model default). db().commit(), db().flush() and > > no call to db all caused the test to pass again. > > > > Does anyone have any input on why this would occur? The oddly odd part > > is that this test doesn't fail outside of the change set in [1] > > > > [1] > https://review.openstack.org/#/c/78406/8/nailgun/nailgun/task/manager.py > > > > -- > > Andrew > > Mirantis > > Ceph community > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [fuel] oddness with sqlalchemy db().refresh(object)
Hi Andrew, I believe, it's just the way SQLAlchemy Session works: all the changes you've made within a session aren't propagated to the db (read: no actual queries are issued) until you explicitly do: - flush(), or - commit() (as commit() calls flush() first), or - Query - one(), first(), all(), update(), delete() - as these are actions that can only be performed by contacting the db. >>> db().refresh(task_provision) call appeared to be reseting the object Yes, and this is exactly what it is supposed to do: get the current state of the model instance from the db (basically: select * from model_table where id = instance_primary_key_value). This means, that all the changes you've made, but haven't flushed yet, will be lost. I've made a small snippet to see this in action: http://xsnippet.org/359888/ (with logging of SQL queries enabled) I hope this helps. I'm just wondering, why would you want to call refresh() there? Thanks, Roman On Sat, Apr 12, 2014 at 1:33 AM, Andrew Woodward wrote: > Recently in one of my changes [1] I was fighting with one of the unit > tests showing a failure for a test which should have been outside the > sphere of influence. > > Traceback (most recent call last): > File > "/home/andreww/.virtualenvs/fuel/local/lib/python2.7/site-packages/mock.py", > line 1190, in patched > return func(*args, **keywargs) > File > "/home/andreww/git/fuel-web/nailgun/nailgun/test/integration/test_task_managers.py", > line 65, in test_deployment_task_managers > self.assertEquals(provision_task.weight, 0.4) > AssertionError: 1.0 != 0.4 > > After walking through a number of times and finally playing with it we > where able to find that the db().refresh(task_provision) call appeared > to be reseting the object. This caused the loss of the weight being > set to 0.4 (1.0 is the model default). db().commit(), db().flush() and > no call to db all caused the test to pass again. > > Does anyone have any input on why this would occur? The oddly odd part > is that this test doesn't fail outside of the change set in [1] > > [1] https://review.openstack.org/#/c/78406/8/nailgun/nailgun/task/manager.py > > -- > Andrew > Mirantis > Ceph community > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [fuel] oddness with sqlalchemy db().refresh(object)
Recently in one of my changes [1] I was fighting with one of the unit tests showing a failure for a test which should have been outside the sphere of influence. Traceback (most recent call last): File "/home/andreww/.virtualenvs/fuel/local/lib/python2.7/site-packages/mock.py", line 1190, in patched return func(*args, **keywargs) File "/home/andreww/git/fuel-web/nailgun/nailgun/test/integration/test_task_managers.py", line 65, in test_deployment_task_managers self.assertEquals(provision_task.weight, 0.4) AssertionError: 1.0 != 0.4 After walking through a number of times and finally playing with it we where able to find that the db().refresh(task_provision) call appeared to be reseting the object. This caused the loss of the weight being set to 0.4 (1.0 is the model default). db().commit(), db().flush() and no call to db all caused the test to pass again. Does anyone have any input on why this would occur? The oddly odd part is that this test doesn't fail outside of the change set in [1] [1] https://review.openstack.org/#/c/78406/8/nailgun/nailgun/task/manager.py -- Andrew Mirantis Ceph community ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
