I implemented the changes you suggested, but I am running into errors with
some polymorphic relations. Using a different theme, here is an example:
class AbstractVehicle(object):
__tablename__ = "vehicles"
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False)
type = Column(String)
__mapper_args__ = {"polymorphic_on": type}
class AbstractCar(AbstractVehicle):
__tablename__ = "cars"
id = Column("id", Integer, primary_key=True)
__mapper_args__ = {"polymorphic_identity": "car"}
__table_args__ = (
ForeignKeyConstraint(["id"], ["vehicles.id"]),
)
class AbstractCoupe(AbstractCar):
__mapper_args__ = {"polymorphic_identity": "coupe"}
Base = declarative_base()
class Vehicle(AbstractVehicle, Base):
pass
class Car(AbstractCar, Vehicle):
pass
class Coupe(AbstractCoupe, Car):
pass
This resulted in the following error:
File ".../model/vehicle.py", line 255, in <module>
class Coupe(AbstractCoupe, Car):
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/api.py",
line 55, in __init__
_as_declarative(cls, classname, cls.__dict__)
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 88, in _as_declarative
_MapperConfig.setup_mapping(cls, classname, dict_)
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 103, in setup_mapping
cfg_cls(cls_, classname, dict_)
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 131, in __init__
self._setup_table()
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 395, in _setup_table
**table_kw)
File ".../env/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line
398, in __new__
"existing Table object." % key)
InvalidRequestError: Table 'cars' is already defined for this MetaData
instance. Specify 'extend_existing=True' to redefine options and columns
on an existing Table object.
After adding "extend_existing=True" to the AbstractCoupe model, I received
this error:
File ".../model/vehicle.py", line 255, in <module>
class Coupe(AbstractCoupe, Car):
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/api.py",
line 55, in __init__
_as_declarative(cls, classname, cls.__dict__)
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 88, in _as_declarative
_MapperConfig.setup_mapping(cls, classname, dict_)
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 103, in setup_mapping
cfg_cls(cls_, classname, dict_)
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 135, in __init__
self._early_mapping()
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 138, in _early_mapping
self.map()
File
".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py",
line 530, in map
**self.mapper_args
File "<string>", line 2, in mapper
File ".../env/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py", line
627, in __init__
self._configure_properties()
File ".../env/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py", line
1292, in _configure_properties
self._adapt_inherited_property(key, prop, False)
File ".../env/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py", line
1514, in _adapt_inherited_property
self._configure_property(key, prop, init=False, setparent=False)
File ".../env/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py", line
1541, in _configure_property
prop.columns[0])
File ".../env/lib/python2.7/site-packages/sqlalchemy/sql/selectable.py",
line 476, in corresponding_column
if self.c.contains_column(column):
File
".../env/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line
754, in __get__
obj.__dict__[self.__name__] = result = self.fget(obj)
File ".../env/lib/python2.7/site-packages/sqlalchemy/sql/selectable.py",
line 553, in columns
return self._columns.as_immutable()
AttributeError: 'Table' object has no attribute '_columns'
Is there a way to fix or work around this?
Thanks,
Angie
On Wed, Jun 22, 2016 at 7:27 PM, Mike Bayer <[email protected]>
wrote:
>
>
> On 06/22/2016 04:53 PM, Angie Ellis wrote:
>
>> Mike,
>>
>> Thank you for the response! I do have control over the external library.
>> I like your idea having both the abstract classes and concrete classes
>> available in the library because I have use cases for both in my
>> applications. However, I'm not quite sure how to configure the
>> inheritance and declarative base. What am I doing wrong here?
>>
>>
>> class AbstractFoo(object):
>> __abstract__ = True
>> __tablename__ = "foo"
>> id = Column(Integer, primary_key=True)
>> type = Column(String)
>> foo_bar_id = Column(Integer, ForeignKey("foo_bar.id
>> <http://foo_bar.id>"))
>> foo_bar = relationship("FooBar", backref=backref("foo"))
>>
>> __mapper_args__ = {"polymorphic_on": type}
>>
>>
>> class AbstractBar(object):
>> __abstract__ = True
>> __mapper_args__ = {"polymorphic_identity": "bar"}
>>
>>
>> class AbstractFooBar(object):
>> __abstract__ = True
>> __tablename__ = "foo_bar"
>> id = Column(Integer, primary_key=True)
>>
>>
>>
>> Base = declarative_base()
>>
>>
>> class Foo(AbstractFoo, Base):
>> pass
>>
>>
>> class Bar(AbstractBar, Foo):
>> pass
>>
>>
>> class FooBar(AbstractFooBar, Base):
>> pass
>>
>>
>> Does AbstractBar need to inherit from AbstractFoo? Do the abstract
>> classes need to inherit their own declarative base? Do I need to change
>> the inheritance order? The classes Foo, Bar, and FooBar are not getting
>> mapped.
>>
>
> So the way you have it is that those are mixins, which is fine, and I
> guess that's how we need it because you want to keep multiple Base's in
> play at the same time. You don't need the __abstract__ keyword in that
> case. So to map that looks like the following POC:
>
> from sqlalchemy import *
> from sqlalchemy.orm import *
> from sqlalchemy.ext.declarative import declarative_base
> from sqlalchemy.ext.declarative import declared_attr
>
>
> class AbstractFoo(object):
> __tablename__ = "foo"
>
> id = Column(Integer, primary_key=True)
> type = Column(String)
>
> @declared_attr
> def foo_bar_id(cls):
> return Column(Integer, ForeignKey("foo_bar.id"))
>
> @declared_attr
> def foo_bar(cls):
> return relationship("FooBar", backref=backref("foo"))
>
> __mapper_args__ = {"polymorphic_on": type}
>
>
> class AbstractBar(AbstractFoo):
> __mapper_args__ = {"polymorphic_identity": "bar"}
>
>
> class AbstractFooBar(object):
> __tablename__ = "foo_bar"
> id = Column(Integer, primary_key=True)
>
>
> Base = declarative_base()
>
>
> class Foo(AbstractFoo, Base):
> pass
>
>
> class Bar(AbstractBar, Foo):
> pass
>
>
> class FooBar(AbstractFooBar, Base):
> pass
>
> e = create_engine("sqlite://", echo=True)
> Base.metadata.create_all(e)
> s = Session(e)
>
> s.add(Bar(foo_bar=FooBar()))
>
> s.commit()
> s.close()
>
> b1 = s.query(Foo).first()
> print b1
> print b1.foo_bar
>
>
>
>
>
>
>> Thanks,
>> Angie
>>
>> On Fri, Jun 3, 2016 at 12:38 PM, Mike Bayer <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>>
>>
>> On 06/03/2016 02:44 PM, Angie E wrote:
>>
>> Rather than creating mixin classes that models inherit from, I
>> have a
>> use case that requires me to configure classes the other way
>> around. The
>> classes that would normally be mixin classes need to be the
>> classes that
>> inherit from the models as well as the class that model objects
>> are
>> created from. This is because the models and the mapper
>> configurations
>> are in an external library from the main repository. I need to
>> pass in
>> the host for the engine from the main repository to the models
>> library
>> before any of the models are loaded so they can load with the
>> declarative base already configured.
>>
>>
>> I hope you are using reflection for these models, otherwise there's
>> no reason to have a depedency on the Engine for the model declaration.
>>
>>
>> After the engine information is
>>
>> passed in, the session, Base class, and everything is created
>> within a
>> sort of base class that the models inherit from. Here is a
>> simplified
>> example:
>>
>>
>> class SQLAlchemyBase(object):
>> metadata = None
>> Session = None
>> Base = object
>> sessionfactory = sessionmaker()
>>
>> def initialize(self, host):
>> engine = create_engine(host)
>> self.metadata = MetaData(bind=engine)
>> self.Session = scoped_session(self.sessionfactory)
>> self.Base = declarative_base(metadata=self.metadata)
>>
>> models = SQLAlchemyBase()
>>
>> (The models inherit from models.Base)
>>
>>
>> So the SQLAlchemyBase will be imported into the main repository,
>> the
>> initialize method will be called, passing in the host for the
>> engine,
>> and the models can then be imported.
>>
>>
>> Looks like no reflection taking place. I'd do away with the
>> MetaData(bind=engine), and just have the engine as part of the
>> sessionmaker(), and the sessionmaker() here also doesn't need to
>> have anything to do with the "library", if the "library" is just
>> defining model classes.
>>
>> The "bound metadata" pattern is highly discouraged in modern
>> SQLAlchemy because it leads to this kind of confusion, e.g. that one
>> needs an Engine in order to declare models. You don't.
>>
>>
>>
>>
>>
>>
>> The main repository has its own
>>
>> classes with the same names as the models and have additional
>> methods
>> that a normal mixin class would have to extend functionality.
>> However, I
>> am unable to create model objects using the classes in the main
>> repository because I can't get the mappers to play nice with this
>> unusual inheritance that extends from the external models library.
>> Additionally, in the models library, there are models that have
>> multiple
>> levels of inherited polymorphic relationships. Here is an
>> example that
>> is similar one of the more basic inherited polymorphic
>> relationships:
>>
>>
>> **Models Library**
>>
>> class Foo(models.Base):
>>
>> __tablename__ = "foo"
>> id = Column(Integer, primary_key=True)
>> type = Column(String)
>> foo_bar_id = Column(Integer, ForeignKey("foo_bar.id
>> <http://foo_bar.id>"))
>>
>> foo_bar = relationship(Foo, backref=backref("foos"))
>>
>> __mapper_args__ = {"polymorphic_on": type}
>>
>>
>> class Bar(Foo):
>>
>> __mapper_args__ = {"polymorphic_identity": "bar"}
>>
>>
>> class FooBar(models.Base):
>>
>> __tablename__ = "foo_bar"
>> id = Column(Integer, primary_key=True)
>>
>>
>> **Main Repository**
>>
>> from separate_library.models import models, Foo as BaseFoo, Bar as
>> BaseBar, FooBar as BaseFooBar
>>
>>
>> class Foo(BaseFoo):
>>
>> @classmethod
>> def custom_create_method(cls, **kw):
>> foo_obj = cls(**kw)
>> models.session.add(foo_obj)
>> models.session.flush()
>>
>>
>> class Bar(BaseBar):
>> pass
>>
>>
>> class FooBar(BaseFooBar):
>> pass
>>
>>
>> The original error I was getting was something like this:
>>
>> "InvalidRequestError: One or more mappers failed to initialize -
>> can't
>> proceed with initialization of other mappers. Original
>> exception was:
>> Multiple classes found for path "Foo" in the registry of this
>> declarative base. Please use a fully module-qualified path."
>>
>>
>> that only happens if someone is using a string name inside of
>> relationship(), like this:
>>
>> foobars = relationship("Foo")
>>
>> If the "library" is doing that, and you can't change it, easy
>> solution, use a different name for your "Foo" that's also in the 3rd
>> party library. Or declare your classes against a different base:
>>
>> Base = declarative_base()
>>
>> class Foo(other_package.Foo, Base):
>> # ...
>>
>>
>> class Bar(other_package.Bar, Base):
>> # ...
>>
>> I'm not 100% sure that will work but there's probably a way to make
>> that work.
>>
>>
>>
>>
>> So I tried putting the full path in the relationships. Then it
>> started
>> giving me an error like this:
>>
>>
>> oh, OK, then you're fine there.
>>
>>
>>
>> "FlushError: Attempting to flush an item of type <class
>> 'main_module.models.Foo'> as a member of collection "FooBar.foos".
>> Expected an object of type <class 'separate_library.models.Foo'>
>> or a
>> polymorphic subclass of this type. If <class
>> 'main_module.models.Foo'>
>> is a subclass of <class 'separate_library.models.Foo'>,
>> configure mapper
>> "Mapper|Foo|foos" to load this subtype polymorphically, or set
>> enable_typechecks=False to allow any subtype to be accepted for
>> flush.
>>
>>
>> Well, when this relationship loads, you'd like it to return a list
>> of main_module.models.Foo objects and not
>> separate_library.models.Foo. Your model here needs to have a
>> polymorphic identity assigned so that it can do this. I don't see
>> those set up in your mappings, you need to put those in explcitly.
>> If you're trying to re-use the same names that the superclass has,
>> you might need to manipulate Foo.__mapper__.polymorphic_map
>> directly, not sure.
>>
>>
>>
>>
>> Essentially, the main problem is getting the classes in the main
>> module
>> to point to and act like the model classes. For example, when I
>> try to
>> create relationships, it says it expected an object of type
>> 'separate_library.models.Foo' instead of 'main_module.models.Foo'.
>>
>>
>> other than using a different name, there are ways to manipulate the
>> registry that is doing this but they aren't publicly supported
>> patterns, unless the easy "use a different Base" trick works out here.
>>
>> Additionally, in the polymorphic relationships, I can't get the
>> polymorphic_identity to populate for the polymorphic_on column.
>> For
>> example, Bar in the main repository will have the 'type' column
>> empty
>> when the object is initially created.
>>
>>
>> when you make a subclass like you are doing, you need to give it its
>> own identity, so it at least needs to have "polymorphic_identity"
>> set up locally.
>>
>> Overall there would be a question here, which is: do you have
>> control over this external library? If so, I would design it to be
>> used for subclassing by another declarative system. Assign names
>> like "AbstractFoo" to the classes and additionally use the
>> __abstract__ = True declarative flag to mark them as such, then use
>> @declared_attr around __mapper_args__ to assign mapper arguments
>> like polymorphic identity to subclasses properly.
>>
>> If you do *not* have control over this external library, then it is
>> not intended to be used in this way; the most preferable way to go
>> about this would be to get the maintainers of it to change it in
>> order to allow it to be used in this way. An easy way for the
>> external library to work like this would be that it supplies both
>> the __abstract__ classes with nice names like AbstractFoo, then for
>> its own purposes it supplies its own Foo / Bar concrete classes on
>> top of a separate declarative Base, and you'd skip using that part.
>>
>>
>>
>>
>> One idea I tried was to add a metaclass to the declarative base
>> in the
>> models library and modify the mappers in the __init__ method
>> during
>> their initialization. I made progress this way, but haven't
>> gotten it to
>> work completely.
>>
>>
>> There's proabably ways to make this work with more tricks and
>> metaclasses but they aren't supported on the SQLAlchemy end and they
>> will be brittle, they will break when declarative changes how it
>> does things internally.
>>
>>
>>
>> Sorry for the complex explanation, but this is a complex
>> problem. I am
>> not able to change anything about the models or the use case,
>> unfortunately. I have to work within these constraints.
>>
>>
>> If these are two different systems within your company, it would be
>> a better investment to get the upstream system to work more
>> flexibly, rather than invest efforts working around the system's
>> limitations downstream. But, if the overlapping names and the
>> polymorphic identities are the only problem you should be able to
>> work around those.
>>
>>
>>
>> If anyone can
>>
>> offer ideas on how to configure the mappers for the classes in
>> the main
>> repository to act like the models in the model library, I would
>> be very
>> grateful.
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from
>> it, send
>> an email to [email protected]
>> <mailto:sqlalchemy%[email protected]>
>> <mailto:[email protected]
>> <mailto:sqlalchemy%[email protected]>>.
>> To post to this group, send email to [email protected]
>> <mailto:[email protected]>
>> <mailto:[email protected]
>> <mailto:[email protected]>>.
>> Visit this group at https://groups.google.com/group/sqlalchemy.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to a topic in
>> the Google Groups "sqlalchemy" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/sqlalchemy/u_CiTJIK9Gs/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected]
>> <mailto:sqlalchemy%[email protected]>.
>> To post to this group, send email to [email protected]
>> <mailto:[email protected]>.
>> Visit this group at https://groups.google.com/group/sqlalchemy.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to [email protected]
>> <mailto:[email protected]>.
>> To post to this group, send email to [email protected]
>> <mailto:[email protected]>.
>> Visit this group at https://groups.google.com/group/sqlalchemy.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "sqlalchemy" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sqlalchemy/u_CiTJIK9Gs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> To post to this group, send email to [email protected].
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
