I've isolated what's happening here, but please send me scripts that generate data next time so that I don't need to take the time to reproduce all of this.

the issue is not related to eager loading, it has to do with a many-to-one load of TaxonRelationship.referenced_taxon should pull from the identity map without emitting SQL, but in this polymorphic case it does not occur.

Here is the workaround:

        mapper.polymorphic_identity = identity
        mapper.polymorphic_map[identity] = mapper
        mapper._identity_class = mapper.inherits._identity_class


see attached.

# -*- coding: utf-8 -*-
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base

DYNAMIC = True
#DYNAMIC = False


Base = declarative_base()

_rank_table = Table('rank', Base.metadata,
        Column('id', Integer, primary_key=True),
        Column('label', String)
    )

_taxon_table = Table('taxon', Base.metadata,
        Column('id', Integer, primary_key=True),
        Column('refno', String),
        Column('rank_id', Integer),
        ForeignKeyConstraint(['rank_id'], ['rank.id'])
    )

_taxon_relationship_table = Table('taxon_relationship', Base.metadata,
        Column('taxon_id', Integer, primary_key=True),
        Column('referenced_taxon_id', Integer, primary_key=True),
        ForeignKeyConstraint(['taxon_id'], ['taxon.id']),
        ForeignKeyConstraint(['referenced_taxon_id'], ['taxon.id']),
    )

class Rank(Base):
    __table__ = _rank_table

class Taxon(Base):
    __table__ = _taxon_table

    __mapper_args__ = {
        'polymorphic_on': 'rank_id',
    }

    rank = relationship('Rank',
                innerjoin=True,
                backref=backref('taxa'))

class Family(Taxon):
    if not DYNAMIC:
        __mapper_args__ = {
            'polymorphic_identity':-2147483640,
        }

class Genus(Taxon):
    if not DYNAMIC:
        __mapper_args__ = {
            'polymorphic_identity':-2147483636,
        }

class Species(Taxon):
    if not DYNAMIC:
        __mapper_args__ = {
            'polymorphic_identity':-2147483630,
        }

class Subspecies(Taxon):

    if not DYNAMIC:
        __mapper_args__ = {
            'polymorphic_identity':-2147483629,
        }

class Variety(Taxon):

    if not DYNAMIC:
        __mapper_args__ = {
            'polymorphic_identity':-2147483628,
        }
#    else:
#        __mapper_args__ = {
#            'polymorphic_identity':"BOGUS",
#        }

class Form(Taxon):

    if not DYNAMIC:
        __mapper_args__ = {
            'polymorphic_identity':-2147483626,
        }

class Cultivar(Taxon):

    if not DYNAMIC:
        __mapper_args__ = {
            'polymorphic_identity':-2147483624,
        }

class Group(Taxon):

    if not DYNAMIC:
        __mapper_args__ = {
            'polymorphic_identity':-2147483623,
        }
    #else:
    #    __mapper_args__ = {
    #        'polymorphic_identity':"BOGUS",
    #    }


class TaxonRelationship(Base):
    __table__ = _taxon_relationship_table

    taxon = relationship('Taxon',
                innerjoin=True,
                primaryjoin='Taxon.id==TaxonRelationship.taxon_id',
                backref=backref('relationships',
                    cascade='all, delete-orphan',
                    passive_deletes=True))

    referenced_taxon = relationship('Taxon',
                innerjoin=True,
                primaryjoin='Taxon.id==TaxonRelationship.referenced_taxon_id',
                backref=backref('referenced_relationships'))


def _set_polymorphic_identity(subclass, identity):
    # http://stackoverflow.com/questions/15112340/how-can-i-set-polymorphic-identity-after-a-class-is-defined
    mapper = class_mapper(subclass)
    if not DYNAMIC:
        assert mapper.polymorphic_identity == identity
        assert mapper.polymorphic_map is mapper.base_mapper.polymorphic_map
        assert mapper.polymorphic_map[identity] is mapper
    else:
        mapper.polymorphic_identity = identity
        mapper.polymorphic_map[identity] = mapper
        #mapper._identity_class = mapper.inherits._identity_class
def initialize_polymorphic_identity(session):
    data = {"family": -2147483640,
        "genus": -2147483636,
        "species": -2147483630,
        "subspecies": -2147483629,
        "variety": -2147483628,
        "form": -2147483626,
        "cultivar": -2147483624,
        "group": -2147483623}

    session.add_all([
        Rank(id=v, label=k) for k, v in data.items()
    ])
    ranks = session.query(Rank).all()

    def get_rank_id(label):
        found_rank = None
        for rank in ranks:
            if rank.label == label:
                found_rank = rank
                break
        if not found_rank:
            raise Exception("No rank with label '" + label + "' has been found!")
        return found_rank.id

    def set_rank_identity(subclass, label):
        _set_polymorphic_identity(subclass, get_rank_id(label))

    set_rank_identity(Family, 'family')
    set_rank_identity(Genus, 'genus')
    set_rank_identity(Species, 'species')
    set_rank_identity(Subspecies, 'subspecies')
    set_rank_identity(Variety, 'variety')
    set_rank_identity(Form, 'form')
    set_rank_identity(Cultivar, 'cultivar')
    set_rank_identity(Group, 'group')


# Initialization
engine = create_engine('postgresql://scott:tiger@localhost/test', echo=True)
Base.metadata.drop_all(engine)
Base.metadata.create_all(engine)

Session = sessionmaker(bind=engine)
session = Session()

# Initialization polymorphic_identity
initialize_polymorphic_identity(session)

t1, t2 = Family(refno='t1'), Cultivar(refno='t2')

tr = TaxonRelationship(taxon=t1, referenced_taxon=t2)

session.add(tr)
session.commit()
t1.refno, t2.refno, tr.taxon_id

print "----------------"
print tr.referenced_taxon


On Jul 31, 2013, at 4:09 AM, Etienne Rouxel <[email protected]> wrote:

Hello

Better than a long speech, here are two files (eagerloading1.py, 
eagerloading2.py) which, I though, were supposed to do the same thing. Indeed, the difference is that in the first file, the polymorphic_identity is hard coded and in the second file, it is set dynamically afterward (like explained in http://stackoverflow.com/questions/15112340/how-can-i-set-polymorphic-identity-after-a-class-is-defined).


The result is that both files trigger the query to get all the taxa. However, the second file also triggers a new query each time there is an access to the referenced_taxon, which is supposed to be loaded already and of course, that query in a loop ruins the performances.


So my question is, how can set the polymorphic_identity dynamically and still avoid the new queries in the loop?


I also provided some the SLQ commands to quickly reproduce the results (test-data.sql).

Tests have been performed with SQLAlchemy==0.8.2


--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
<eagerloading1.py><eagerloading2.py><test-data.sql>

Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail

Reply via email to