The session executes in one of many web-server threads, but there is no
multi-threading with respect to the session or the objects. The session
that was closed is within an initializer and upon return a method on the
object is executed that creates a new session and tries to attach objects
retrieved from the first session. This method does spawn threads but the
exception happens before any threads are spawned. The thread target does
not take objects, only python strings. only identifiers used in new
sessions to query the objects again. The milliseconds I referred to are
between the close() of the first session and the add() that fails in the
second session.
The code I say appears to expect the session_id to remain on detached
states in certain situations is that the check in Session._attach checks
not only the session_id but that the referenced session still exists in
_sessions:
if state.session_id and \
state.session_id is not self.hash_key and \
state.session_id in _sessions:
raise sa_exc.InvalidRequestError(
"Object '%s' is already attached to session '%s' "
"(this is '%s')" % (orm_util.state_str(state),
state.session_id, self.hash_key))
I am interested in this in the hopes that sheds light on the source of the
intermittent failures.
In debugging this I have noticed that after the first session has been
closed() and the initializer has returned the session remains in _sessions.
However, if I call gc.collect() the session is removed, suggesting it just
hasn't been fully cleaned up yet. Since it takes both the state referencing
the session and the session still existing in _sessions, I can't help but
wonder if this is a gc issue. Unfortunately it is not feasible to add
gc.collect() calls in our production application as it introduces too much
overhead.
On Thu, Sep 4, 2014 at 3:20 PM, Michael Bayer <[email protected]>
wrote:
>
> On Sep 4, 2014, at 5:00 PM, Lonnie Hutchinson <[email protected]> wrote:
>
> >
> >
> > I am using sqlalchemy 0.8.5 with mysql 5.5 and think an intermittent
> failure I am seeing may be due to an issue in sqlalchemy. Very infrequently
> I receive an error from Session.add() stating the instance "is already
> attached to session 'XXX' (this is 'YYY')" (see below for stack trace). I
> understand the typical reason this error is raised, but I do not believe
> the instance should still be attached to XXX, even though it was shortly
> before (milliseconds) since Session.close() was called on the session.
>
> OK I'm seeing some red flags here. I see the word "intermittent", which
> almost always means, "multiple threads". Then I see the word,
> "milliseconds". Are we using threads? Are we trying to time things?
> Because looking at 0.8.5, when you call close(), the objects in that
> Session are absolutely de-assocaited with that Session, including that
> session_id is set to None.
>
> This is very simple to confirm:
>
> from sqlalchemy import *
> from sqlalchemy.orm import *
> from sqlalchemy.ext.declarative import declarative_base
>
> Base = declarative_base()
>
> class A(Base):
> __tablename__ = 'a'
>
> id = Column(Integer, primary_key=True)
>
> e = create_engine("sqlite://", echo=True)
> Base.metadata.create_all(e)
>
> sess = Session(e)
> a1 = A()
> sess.add(a1)
> sess.commit()
>
> assert a1._sa_instance_state.session_id == sess.hash_key
> sess.close()
> assert a1._sa_instance_state.session_id is None
>
> there's no "milliseconds" here. close() completes, and that id is gone.
>
> If you are playing with threads, the only way to guarantee one thread
> finishes something before the other starts is to either join() that thread
> or use mutexes.
>
>
> > My understanding of Session.close() is that upon return all instances
> should be detached from the session and their instance state session_id
> attribute should no longer reference the session. This does not appear to
> be happening since adding the instance to a different session fails since
> the instance is still attached to the previous session.
>
> Above I illustrate that this is not the case. The code path is clear:
>
> session.py -> line 942 -> close() calls:
> session.py -> line 952 -> self.expunge_all()
> session.py -> line 965 calls state._detach() on everything in identity +
> _new. (question. are you trying to move an object that was deleted?
> that might have issues, but that's not a valid use case).
> state.py -> line 166 -> _detach() calls:
> state.py -> line 167: self.session_id = self._strong_obj = None
>
> > In reading the code it appears as though it is expected in certain
> situations for the session_id to continue to refer to the session after
> close since the check for already attached checks session_id as well as
> whether the session remains in the sessions registry.
>
> I'm not seeing that in 0.8.5 at all. what line of code are you
> referring to ?
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
