Re: [sqlalchemy] How correct sort items by @hibrid_property

[Simon King]

The error you're getting doesn't have anything to do with using the
property in an order_by. It's being triggered just by accessing
"Product.Rating". WIth hybrid properties, when you access them via the
class as you've done here, the "self" parameter is set to the Product class
itself. So on the second line of the function, you are actually saying
"reviewsCount = len(Product.Reviews)", which doesn't make any sense.

You need to define a separate implementation of the hybrid property that
returns an SQL construct so that it can be embedded in an SQL query, using
the ".expression" decorator. Since you need to access a
separate table to calculate your Rating, you probably need a correlated
subquery, something like this:

https://docs.sqlalchemy.org/en/20/orm/extensions/hybrid.html#correlated-subquery-relationship-hybrid

That example uses "func.sum" to calculate a sum over the related rows. For
your purposes, you probably want "func.avg" to calculate the mean.

(Note that the performance of this query is going to get worse as the
number of products and ratings increases)

Hope that helps,

Simon

On Sun, Oct 29, 2023 at 9:51 AM Nik  wrote:

> Hello, guys!
> I have a simple code, which not correct work
> He throw error if i try sort items by @hybrid_property
>
> *My version of entity class:*
> class Product(Base) :
>
> __tablename__ = 'products'
>
> Id = Column('id', Integer, primary_key=True, autoincrement=True)
> TitleTranslit = Column('title_translit', String)
> Title = Column('title', String)
> Description = Column('description', String)
> CurrentPrice = Column('current_price', Integer)
> OldPrice = Column('old_price', Integer)
> CategoryId = Column('category_id', Integer, ForeignKey('categories.id
> '))
> Characteristics:List[ProductCharacteristic] =
> relationship('ProductCharacteristic',
> primaryjoin="and_(ProductCharacteristic.ProductId==Product.Id)",
> lazy='joined')
> Reviews:List[ProductReview] = relationship('ProductReview',
> primaryjoin="and_(ProductReview.ProductId==Product.Id)", lazy='joined')
> Images:List[ProductImage] = relationship('ProductImage',
> primaryjoin="and_(ProductImage.ProductId==Product.Id)", lazy='joined')
> NumberOfViews = Column('number_of_views', Integer)
>
> @hybrid_property
> def Rating(self) :
> result = 0
> reviewsCount = len(self.Reviews)
> if (reviewsCount > 0) :
> for review in self.Reviews :
> result += review.Rating
> result = result / reviewsCount
> return result
>
> *My version of call sort event by @hybrid_property:*
> filteredProductsQuery = self._sessionService.DBContext.query(Product)\
>
> .filter(Product.CategoryId.in_(categoriesIds)).outerjoin(Product.Characteristics)
>
> if sortType == SortType.HighestRate.value :
> filteredProductsQuery =
> filteredProductsQuery.order_by(Product.Rating.desc())
>
> *Throwed error if i try execute this query:*
> TypeError: object of type 'InstrumentedAttribute' has no len()
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/d6ccf7c1-fd12-4fb8-b16d-5280f54d506an%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdKp5z%3D9CE5WdJMw7ntmSwGxRxCigO0EiqVuwr-YO9R8A%40mail.gmail.com.

Re: [sqlalchemy] Difference b/w creating a DeclarativeBase class vs assigning DeclarativeBase()

[Simon King]

I don't think this code was ever correct:

Base = DeclarativeBase()

Before SQLAlchemy 2.0, there was a declarative_base() function that was
used in the same way:

from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

...but in SQLAlchemy 2.0, the preferred form is:

from sqlalchemy.orm import DeclarativeBase

class Base(DeclarativeBase):
pass

The new approach works better with type-analysis tools like mypy.

https://docs.sqlalchemy.org/en/20/changelog/whatsnew_20.html#whatsnew-20-orm-declarative-typing

Hope that helps,

Simon

On Tue, Oct 10, 2023 at 8:12 AM satya dev  wrote:

> What is the difference between
> class Base(DeclarativeBase):
> pass
> vs
> Base = DeclarativeBase()
> I am following the SQLAlchemy Tutorial for declaring mapped classes
>  when I inherit
> the Base class I can access the metadata and create my tables but when I
> assign it to Base and try to create the tables it throws an error saying
> class User(Base):
> TypeError: DeclarativeBase() takes no arguments
>
>
> Code for reference:
> from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column,
> relationship
> from sqlalchemy import Integer, String, ForeignKey, MetaData, create_engine
> from typing import List, Optional
>
>
> engine = create_engine("postgresql://db_user:db_pw@localhost
> :5432/alembic_learning")
> # class Base(DeclarativeBase):
> # pass
>
> Base = DeclarativeBase()
>
> class User(Base):
> __tablename__ = "users"
>
> id: Mapped[int] = mapped_column(primary_key=True)
> name:Mapped[str] = mapped_column(String(30))
> fullname:Mapped[Optional[str]]
>
> addresses:Mapped[List["Address"]] = relationship(back_populates="user")
>
> def __repr__(self) -> str:
> return f"User(id={self.id!r}, name={self.name!r},
> fullname={self.fullname!r})"
> class Address(Base):
> __tablename__ = "address"
>
> id:Mapped[int] = mapped_column(primary_key=True)
> email_address:Mapped[str]
> user_id = mapped_column(ForeignKey("users.id"))
>
> user:Mapped[User] = relationship(back_populates="addresses")
>
> def __repr__(self)-> str:
> return f"Address(id={self.id!r}, email_address={self.email_address})"
>
> Base.metadata.create_all(bind=engine)
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/5b7821d7-ef3e-4b49-ae5c-880851b5ab43n%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeGMdwBm%3DgxzsH8jjBGX%2By-EHeawFUPC31OvcsK5ki9UA%40mail.gmail.com.

Re: [sqlalchemy] Issuing Raw SQL and Returning a List of Objects

[Simon King]

My perspective: the SQLAlchemy ORM really comes into its own when you are
making use of its Unit of Work system to load a batch of objects from the
database, manipulate those objects, and then flush your changes back to the
database. If you are only *loading* data then you don't need a lot of the
functionality of the ORM, and you might consider using SQLAlchemy Core
instead.

Using SQLAlchemy Core to execute SQL strings is very simple:

https://docs.sqlalchemy.org/en/20/core/connections.html#basic-usage

You can use the objects that come back from those calls directly (they have
attributes named after the columns from the query), or you could trivially
convert them into instances of some class that you've defined.

It sounds like the sort of work you do involves writing code to access
pre-existing databases, in which case writing SQL directly makes a lot of
sense, and you have no need for the schema-definition parts of SQLAlchemy.
But there are other classes of application for which the schema-definition
tools are very useful. I have written many applications for which the
database didn't already exist, so allowing SQLAlchemy to create the tables
was the obvious way to go (with Alembic for migrations as the schema
changed over time). SQLAlchemy also gives a certain amount of independence
from the underlying database, meaning that I can run most of my tests using
SQLite despite using Postgres or MySQL in production.

In summary: use the right tool for the job :-)

Simon


On Mon, Aug 21, 2023 at 6:48 PM Mike Graziano  wrote:

> Hi Mike,
>
>
>
> Thanks for that info.  It was just what I needed. I also want to thank you
> for your YouTube tutorials on SQLAlchemy. They are fantastic.
>
>
>
> I don’t want to make this a huge post, but I have a real pet peeve
> concerning ORMs.  I come from a Java background where I used MyBatis as my
> ORM.  What I love about MyBatis was
>
>
>
> -   I could use raw SQL which I personally feel is superior.  My argument
> here is simple: Why learn another “language” for issuing SQL statements
> when we have already spent a fair amount of time learning SQL.  Also, raw
> SQL is easily testable with either command line or GUI tools?
>
> -   The ORM should just use the mapped models in order to execute SQL
> using mapping that in and of themselves doesn’t/shouldn’t care about the
> tables.  Unless you are creating a table with the ORM which I have found to
> be rare, the ORM shouldn’t care about the table structure other than field
> names with the possibility of aliases and data types.  Why define more than
> what we need in order to populate a plain old object (POO – language
> agnostic).  Why include characteristics like primary key, nullability,
> etc?  Some Pydantic-like validation is handy, but can be table agnostic.
> Let’s extract the data via SQL and return POOs.  In that regard, I liken
> the ORM to a Data Transfer Object (DTO).
>
> -   As I have already mentioned, how often do you really use an
> application to create tables.  Often, they already exist.  Furthermore, it
> is just more natural to use command‑line SQL or a GUI to create the
> tables.  In fact, it is not uncommon to use a GUI like PgAdmin or DBeaver
> to create the database elements that you need and then use that tool to
> derive all sorts of scripts to perform common activities such as backup,
> restore, etc. that can be scheduled.
>
>
>
> There is a very handy Java framework call BeanIO ( http://beanio.org/)
> that I feel exemplifies the points I am trying to make.  With BeanIO, it is
> possible to extract data from a variety of file formats and populate
> POJOs.  BeanIO is only interested in the layout of the data. It is a
> convenience framework that allows for OOP design of an application.  I feel
> that MyBatis does this also.  It has substantial DB integration, but
> strives to connect the POJO to the database without enforcing design.
> Using so-called Entity’s enforces a design that ORMs should not be forced
> to obey if all you are looking for is a translation from SQL to a POO.
>
>
>
> Once again, thanks for you help and sorry for my ranting, but as I’ve said
> I have a pet peeve with ORMs that are enforcing more than I think is
> necessary to translate SQL to a POO.
>
> On Thursday, August 17, 2023 at 8:04:58 PM UTC-4 Mike Bayer wrote:
>
>> the raw SQL to ORM mapping pattern has a lot of limitations but it is
>> documented at
>> https://docs.sqlalchemy.org/en/20/orm/queryguide/select.html#getting-orm-results-from-textual-statements
>> .
>>
>>
>>
>> On Thu, Aug 17, 2023, at 4:26 PM, Mike Graziano wrote:
>>
>> To all,
>>
>> I am new to Python and SQLAlchemy.  I was a Java developer who used the
>> MyBatis ORM.  I was using PONY ORM for a while, but was concerned that
>> SQLAlchemy is the gold standard for ORMs and Python, but there is something
>> about MyBatis that I can't seem to find in SQLAlchemy, but maybe I have not
>> googled enough.
>>
>> In short, I don't want to use SQLAlchemy's select.  

Re: [sqlalchemy] Change a select clause + add a join automatically

[Simon King]

I think this is the intended use for the do_orm_execute event and the
with_loader_criteria query option:

https://docs.sqlalchemy.org/en/14/orm/session_events.html#do-orm-execute-global-criteria
https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.with_loader_criteria

You ought to be able to use the event hook to add a loader criteria option
targeting the appropriate translation.

Simon


On Tue, Apr 11, 2023 at 5:13 PM Nishant Varma 
wrote:

> Hello,
>
> I have this schema:
>
> class Question(Base):
> __tablename__ = "question"
> idn = Column(Integer, primary_key=True)
> text = Column("text", String)
>
> translations = relationship("Translation", backref="question")
>
> class Translation(Base):
> __tablename__ = "translation"
> idn = Column(Integer, primary_key=True)
> qid = Column(Integer, ForeignKey("question.idn"))
> lang = Column(String)
> text = Column(String)
>
> I want to automatically join Question with a *single* Translation (lang
> filter) when the real query happens. Language will be supplied runtime. I
> tried to think of this as a *property*, but that doesn't seem to work.
>
> Currently, I am using a  simple function to achieve this:
>
> def translate(query, lang):
> cols = list(query.selectable.selected_columns)
> index = next(
> (
> i
> for i, c in enumerate(cols)
> if c.name == "text" and c.table.name == "question"
> ),
> None,
> )
> text = case(
> [(Translation.text.is_not(None), Translation.text)],
> else_=Question.text,
> )
> cols[index] = text.label("text")
> query._set_entities(cols)  # noqa:W0212
> query = query.outerjoin(
> Translation,
> and_(Question.idn == Translation.qid, Translation.lang == lang),
> )
> return query
>
> Usage: query = translate(query, "it")
>
> 1. Is there a good place to hook this so that it's baked into the Question
> table? The tricky part is when and where to pass "it". I guess that can be
> only done during session.query. If so, how to do that? (I know about
> bind-params, so I am thinking that will be an option here.)
>
> 2. Is there a better approach to solve this problem? I tried looking up
> examples, but didn't find one that fits my purpose (should be simple,
> should be at a single place etc.).
>
> SQLAlchemy 1.3 or 1.4
>
> Thank you for help in advance,
> Nishant
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/CAPy-swN5KF1tUj29VBY6Dtyjq%3DtV%3D7Uzv71VkpcuSvB8a5Dz9g%40mail.gmail.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfb1OQ09OEUqhRUB8vW0v57OS1n_M%2B7ESRh2RFB8gFSTw%40mail.gmail.com.

Re: [sqlalchemy] Queries Across 13 Models - How to improve efficiency?

[Simon King]

It looks like all your models share the same "id" value - is that right? If
so, you ought to be able to load all of them in a single query, something
like this (untested):

def getmodels(dbsession, id):
models = [M1, M2, M3]
conditions = [(M.id == id) for M in models]
instances = dbsession.query(*models).filter(*conditions).first()
instancedict = {i.__table__.name: i for i in instances}

Hope that helps,

Simon

On Thu, Jan 26, 2023 at 3:05 PM Shuba  wrote:

> Hi!
> I have a web application whose main feature is to query 13 different
> database tables when the user performs a search. I'm performing the query
> like this: https://gist.github.com/bmare/8c11ba6efcb97ba14dc30bc260aebc6c
> , and I'm wondering if there is a better method. I like having all the
> instances because it makes it pretty when going on to render the web page
> with all this information. At the same time, every time I render the page
> there are 13 queries being performed. I've indexed the columns I'm
> searching so the queries run quickly, but I'm wondering if I can make this
> more efficient somehow. Any ideas would be appreciated.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/f427181d-655b-4806-9f28-8cd0afb513efn%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexf%2B-S7GjK3Q18YGnDEC4SuyuTODJEB1G02Rz4ROVNo6EQ%40mail.gmail.com.

Re: [sqlalchemy] Breaking Integration with Locust Tests? Having a hard time debugging

[Simon King]

The stack trace shows that the exception is being raised in
sqlalchemy_utils/types/uuid.py. Looking at sqlalchemy_utils on Github, this
is a bug that has been reported and fixed, but the fix hasn't been released
yet:

https://github.com/kvesteri/sqlalchemy-utils/pull/643

You could either install the unreleased version of sqlalchemy-utils, or
downgrade SQLAlchemy to a non-beta version.

Hope that helps,

Simon

On Fri, Nov 4, 2022 at 11:22 PM 'Theo Chitayat' via sqlalchemy <
sqlalchemy@googlegroups.com> wrote:

> I have a model that uses ChoiceType like an enum for my FastAPI app. As
> soon as I install *locust* from pip, i get errors like this:
> *from sqlalchemy_utils import ChoiceType File
> "/Users/theo/.local/share/virtualenvs/optis2-data-service-iqC41Vso/lib/python3.9/site-packages/sqlalchemy_utils/__init__.py",
> line 59, in  from .types import ( # noqa File
> "/Users/theo/.local/share/virtualenvs/optis2-data-service-iqC41Vso/lib/python3.9/site-packages/sqlalchemy_utils/types/__init__.py",
> line 42, in  from .uuid import UUIDType # noqa File
> "/Users/theo/.local/share/virtualenvs/optis2-data-service-iqC41Vso/lib/python3.9/site-packages/sqlalchemy_utils/types/uuid.py",
> line 8, in  sqlalchemy_version = tuple([int(v) for v in
> __version__.split(".")]) File
> "/Users/theo/.local/share/virtualenvs/optis2-data-service-iqC41Vso/lib/python3.9/site-packages/sqlalchemy_utils/types/uuid.py",
> line 8, in  sqlalchemy_version = tuple([int(v) for v in
> __version__.split(".")]) ValueError: invalid literal for int() with base
> 10: '0b2'*
>
> I've tried a number of trial and error changes - but I have no idea where
> this error stems from. Is there anything I can do to support this?
>
> Thanks in advance
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/203e2aaf-0daa-4b97-bed5-beaf59c3b469n%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdtCZYgWKwYT5uxVZVOKZz%3DLjwbJd7cME2jZP_H5X-11Q%40mail.gmail.com.

Re: [sqlalchemy] iterate sqlalchemy query over for loop in my template python-flask

[Simon King]

I can't see anything obviously wrong with your code. If you turn on debug
logs, you'll be able to see the SQL that is being executed, and the rows
that are coming back. Paste those logs here and maybe we'll be able to help
more.

Simon

On Tue, Oct 25, 2022 at 2:38 PM Abdellah ALAOUI ISMAILI <
my.alaoui...@gmail.com> wrote:

> yes, all tags have a color...
> Can you suggest me another way to do this, please?
> thank you .
>
> Le mardi 25 octobre 2022 à 10:52:17 UTC+1, Simon King a écrit :
>
>> Turn on debug logs (add echo="debug" to your db connection string) to see
>> the SQL you are executing and the results you are getting back:
>>
>>
>> https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.echo
>>
>> Are you sure that the database actually contains colours for those tags?
>>
>> Simon
>>
>> On Sat, Oct 22, 2022 at 3:37 PM Abdellah ALAOUI ISMAILI <
>> my.ala...@gmail.com> wrote:
>>
>>> wheel ... in my template, I get just the first tag color, returned from
>>> the function.
>>>
>>> this is the result of my HTML file source code :
>>>
>>> 
>>>  LOG
>>> 
>>>   EXADATA
>>> 
>>>   DMZ_PRIVE
>>>  
>>>  
>>>
>>> I hope it's clear,
>>> thank you .
>>>
>>> Le vendredi 21 octobre 2022 à 09:23:48 UTC+1, Simon King a écrit :
>>>
>>>> I don't understand the question. Are you saying that only one tag is
>>>> displayed? If so, that's not a problem with SQLAlchemy, it's a problem with
>>>> your template logic.
>>>>
>>>> If that's not what you mean, you need to give us more information. What
>>>> is the value of "server.tags", and what is the output from the template?
>>>>
>>>> Simon
>>>>
>>>> On Thu, Oct 20, 2022 at 9:05 AM Abdellah ALAOUI ISMAILI <
>>>> my.ala...@gmail.com> wrote:
>>>>
>>>>> Hello,
>>>>> I call a function in my template that returns sqlalchemy query result,
>>>>> (color value from the name of the tag). this is the query function :
>>>>>
>>>>> *def get_tag_color(name): *
>>>>> *return db.session.query(Tag.tag_color).filter(Tag.tag_name ==
>>>>> name).scalar() *
>>>>>
>>>>> and I call it in my template file:
>>>>> * {% if server.tags %}*
>>>>> * {% for tag in server.tags.split(",") %} *
>>>>> * >>>> class="label"> {{tag}} *
>>>>> * {% endfor %} {% endif %} *
>>>>>
>>>>> the problem is that I get just one result of the first tag. do you
>>>>> have any idea what I miss?
>>>>>
>>>>> Thank you
>>>>>
>>>>> --
>>>>> SQLAlchemy -
>>>>> The Python SQL Toolkit and Object Relational Mapper
>>>>>
>>>>> http://www.sqlalchemy.org/
>>>>>
>>>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>>>> description.
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "sqlalchemy" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to sqlalchemy+...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/sqlalchemy/4f9d1bab-479d-47d5-89cf-c305026ec3d7n%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/sqlalchemy/4f9d1bab-479d-47d5-89cf-c305026ec3d7n%40googlegroups.com?utm_medium=email_source=footer>
>>>>> .
>>>>>
>>>> --
>>> SQLAlchemy -
>>> The Python SQL Toolkit and Object Relational Mapper
>>>
>>> http://www.sqlalchemy.org/
>>>
>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>> description.
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "sqlalchemy" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to sqlalchemy+...@googlegroups.com.
>>>
>> To view this discussion

Re: [sqlalchemy] iterate sqlalchemy query over for loop in my template python-flask

[Simon King]

Turn on debug logs (add echo="debug" to your db connection string) to see
the SQL you are executing and the results you are getting back:

https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.echo

Are you sure that the database actually contains colours for those tags?

Simon

On Sat, Oct 22, 2022 at 3:37 PM Abdellah ALAOUI ISMAILI <
my.alaoui...@gmail.com> wrote:

> wheel ... in my template, I get just the first tag color, returned from
> the function.
>
> this is the result of my HTML file source code :
>
> 
>  LOG
> 
>   EXADATA
> 
>   DMZ_PRIVE
> 
>  
>
> I hope it's clear,
> thank you .
>
> Le vendredi 21 octobre 2022 à 09:23:48 UTC+1, Simon King a écrit :
>
>> I don't understand the question. Are you saying that only one tag is
>> displayed? If so, that's not a problem with SQLAlchemy, it's a problem with
>> your template logic.
>>
>> If that's not what you mean, you need to give us more information. What
>> is the value of "server.tags", and what is the output from the template?
>>
>> Simon
>>
>> On Thu, Oct 20, 2022 at 9:05 AM Abdellah ALAOUI ISMAILI <
>> my.ala...@gmail.com> wrote:
>>
>>> Hello,
>>> I call a function in my template that returns sqlalchemy query result,
>>> (color value from the name of the tag). this is the query function :
>>>
>>> *def get_tag_color(name): *
>>> *return db.session.query(Tag.tag_color).filter(Tag.tag_name ==
>>> name).scalar() *
>>>
>>> and I call it in my template file:
>>> * {% if server.tags %}*
>>> * {% for tag in server.tags.split(",") %} *
>>> * >> class="label"> {{tag}} *
>>> * {% endfor %} {% endif %} *
>>>
>>> the problem is that I get just one result of the first tag. do you have
>>> any idea what I miss?
>>>
>>> Thank you
>>>
>>> --
>>> SQLAlchemy -
>>> The Python SQL Toolkit and Object Relational Mapper
>>>
>>> http://www.sqlalchemy.org/
>>>
>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>> description.
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "sqlalchemy" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to sqlalchemy+...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/sqlalchemy/4f9d1bab-479d-47d5-89cf-c305026ec3d7n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/sqlalchemy/4f9d1bab-479d-47d5-89cf-c305026ec3d7n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/28c544b5-5445-45f9-ac8e-1dc786984d82n%40googlegroups.com
> <https://groups.google.com/d/msgid/sqlalchemy/28c544b5-5445-45f9-ac8e-1dc786984d82n%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexexdt23MLpk%3DSjAWSufgDzrjfgJxVwiAX%2B5JwbUsYisPQ%40mail.gmail.com.

Re: [sqlalchemy] iterate sqlalchemy query over for loop in my template python-flask

[Simon King]

I don't understand the question. Are you saying that only one tag is
displayed? If so, that's not a problem with SQLAlchemy, it's a problem with
your template logic.

If that's not what you mean, you need to give us more information. What is
the value of "server.tags", and what is the output from the template?

Simon

On Thu, Oct 20, 2022 at 9:05 AM Abdellah ALAOUI ISMAILI <
my.alaoui...@gmail.com> wrote:

> Hello,
> I call a function in my template that returns sqlalchemy query result,
> (color value from the name of the tag). this is the query function :
>
> *def get_tag_color(name): *
> *return db.session.query(Tag.tag_color).filter(Tag.tag_name ==
> name).scalar() *
>
> and I call it in my template file:
> * {% if server.tags %}*
> * {% for tag in server.tags.split(",") %} *
> *  class="label"> {{tag}} *
> * {% endfor %} {% endif %} *
>
> the problem is that I get just one result of the first tag. do you have
> any idea what I miss?
>
> Thank you
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/4f9d1bab-479d-47d5-89cf-c305026ec3d7n%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfk_Y_OeYFMxwyCS0n5xCQaSBQo_iShsT%3DLJm0cQX5b7w%40mail.gmail.com.

Re: [sqlalchemy] Can’t make the Composite Comparator work…

[Simon King]

(I haven't used any of these features, so the following is just a guess)

In your assertion, you are comparing two *Point instances*. The SQLAlchemy
comparator_factory mechanism has not made any changes to the Point class
itself, and Point doesn't define __gt__, hence your TypeError.

The point of the comparator_factory is to add class-level behaviour when
you are building SQL expressions. In this case, you could write a query
like this:

query = dbsession.query(Vertex).filter(Vertex.start > Point(2, 2))

Hope that helps,

Simon

On Fri, Sep 16, 2022 at 9:38 AM jens.t...@gmail.com 
wrote:

> Hello,
>
> I’m noodling through the Composite Column Types
>  examples, and
> can’t make it work. Based on the code on that page I put together a minimal,
> reproducible example
>  (attached)
> which fails with
>
>
>
>
>
> *Traceback (most recent call last):  File "/path/to/test.py", line 75, in
> assert v1.start > v2.startTypeError: '>' not supported between
> instances of 'Point' and 'Point'*
>
> Why is that? What am I missing? I expected to see SQL generated that
> implements the “greater than” between two *Point* instances (or perhaps
> *Vertex* instance, not sure, probably not).
>
> Much thanks!
> Jens
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/149c1ae1-94af-4a4e-884c-171f72eb010bn%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeqBnAUrWJbrOyKS3aL5FsZvnCGOoU-VuOGeSOHh41aEQ%40mail.gmail.com.

Re: [sqlalchemy] Just starting with sqlacodegen: wrong command?

[Simon King]

Here's the command you ran:

(venv) jgarcia@Javier-PC:/var/www$ sudo pip install sqlacodegen==3.0.0b3

The "(venv)" at the beginning of your prompt suggests that you are in an
activated virtualenv. But when you run "sudo pip", you are installing the
package globally. You now have 2 different versions of sqlacodegen
installed in different locations.

You should probably have run "pip install" *without* "sudo". If you *need*
sudo (because your virtualenv is owned by root), you must ensure that you
run the "pip" that is installed in the virtualenv, by using an absolute
path.

Hope that helps,

Simon

On Tue, Jun 14, 2022 at 5:32 PM Javier Garcia  wrote:

> This is what I'm getting when I try to upgrade to 3.0.0b3.
>
> (venv) jgarcia@Javier-PC:/var/www$ sudo pip install sqlacodegen==3.0
> ERROR: Could not find a version that satisfies the requirement
> sqlacodegen==3.0 (from versions: 1.0.0, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4,
> 1.1.5, 1.1.6, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.3.0, 3.0.0b1, 3.0.0b2, 3.0.0b3,
> 3.0.0rc1)
> ERROR: No matching distribution found for sqlacodegen==3.0
> (venv) jgarcia@Javier-PC:/var/www$ sudo pip install sqlacodegen==3.0.0b3
> Collecting sqlacodegen==3.0.0b3
>   Downloading sqlacodegen-3.0.0b3-py3-none-any.whl (19 kB)
> Requirement already satisfied: inflect>=4.0.0 in
> /usr/local/lib/python3.8/dist-packages (from sqlacodegen==3.0.0b3) (5.6.0)
> Requirement already satisfied: SQLAlchemy>=1.4.0 in
> /usr/local/lib/python3.8/dist-packages (from sqlacodegen==3.0.0b3) (1.4.37)
> Collecting importlib-metadata; python_version < "3.10"
>   Downloading importlib_metadata-4.11.4-py3-none-any.whl (18 kB)
> Requirement already satisfied: greenlet!=0.4.17; python_version >= "3" and
> (platform_machine == "aarch64" or (platform_machine == "ppc64le" or
> (platform_machine == "x86_64" or (platform_machine == "amd64" or
> (platform_machine == "AMD64" or (platform_machine == "win32" or
> platform_machine == "WIN32")) in /usr/local/lib/python3.8/dist-packages
> (from SQLAlchemy>=1.4.0->sqlacodegen==3.0.0b3) (1.1.2)
> Collecting zipp>=0.5
>   Downloading zipp-3.8.0-py3-none-any.whl (5.4 kB)
> Installing collected packages: zipp, importlib-metadata, sqlacodegen
>   Attempting uninstall: sqlacodegen
> Found existing installation: sqlacodegen 2.3.0
> Uninstalling sqlacodegen-2.3.0:
>   Successfully uninstalled sqlacodegen-2.3.0
> Successfully installed importlib-metadata-4.11.4 sqlacodegen-3.0.0b3
> zipp-3.8.0
> (venv) jgarcia@Javier-PC:/var/www$ sqlscodegen --version
>
> Command 'sqlscodegen' not found, did you mean:
>
>   command 'sqlacodegen' from deb sqlacodegen (1.1.6-2build1)
>
> Try: sudo apt install 
>
> (venv) jgarcia@Javier-PC:/var/www$ sqlacodegen --version
> 2.3.0
>
> On Tue, Jun 14, 2022 at 5:25 PM Javier Garcia 
> wrote:
>
>> Thanks Simon, do you know how could I install that version as when I
>> upgrade the sqlacodegen I still get the version 2.3.0?
>>
>> Javier
>>
>> El martes, 14 de junio de 2022 a las 17:21:31 UTC+1, Simon King escribió:
>>
>>> Based on the CHANGES file, it looks like --generator is a new option in
>>> v3.0.0:
>>>
>>> https://github.com/agronholm/sqlacodegen/blob/master/CHANGES.rst
>>>
>>> Simon
>>>
>>> On Tue, Jun 14, 2022 at 5:06 PM Javier Garcia 
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have tried to run something like this:
>>>>
>>>> sqlacodegen --generator tables mysql+pymysql://user:password@localhost
>>>> /dbname
>>>>
>>>> but I get:
>>>>
>>>> usage: sqlacodegen [-h] [--version] [--schema SCHEMA] [--tables TABLES]
>>>> [--noviews] [--noindexes] [--noconstraints] [--nojoined] [--noinflect]
>>>> [--noclasses] [--nocomments] [--outfile OUTFILE] [url]
>>>> sqlacodegen: error: unrecognized arguments: --generator
>>>> mysql+pymysql://root:***@localhost/test_user
>>>>
>>>> My sqlacodegen version is 2.3.0.
>>>>
>>>> Any help?
>>>>
>>>> --
>>>> SQLAlchemy -
>>>> The Python SQL Toolkit and Object Relational Mapper
>>>>
>>>> http://www.sqlalchemy.org/
>>>>
>>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>>> description.
>>>> ---
>>>> You received this message because you are subscri

Re: [sqlalchemy] Just starting with sqlacodegen: wrong command?

[Simon King]

Based on the CHANGES file, it looks like --generator is a new option in
v3.0.0:

https://github.com/agronholm/sqlacodegen/blob/master/CHANGES.rst

Simon

On Tue, Jun 14, 2022 at 5:06 PM Javier Garcia  wrote:

> Hi,
>
> I have tried to run something like this:
>
> sqlacodegen --generator tables mysql+pymysql://user:password@localhost
> /dbname
>
> but I get:
>
> usage: sqlacodegen [-h] [--version] [--schema SCHEMA] [--tables TABLES]
> [--noviews] [--noindexes] [--noconstraints] [--nojoined] [--noinflect]
> [--noclasses] [--nocomments] [--outfile OUTFILE] [url]
> sqlacodegen: error: unrecognized arguments: --generator
> mysql+pymysql://root:***@localhost/test_user
>
> My sqlacodegen version is 2.3.0.
>
> Any help?
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/6bea9449-3c19-491f-ace7-c17a8440a03an%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexejJ5tB6Ww7kkiZv%3D6jkwVOMf68gHdJ5APvK7VwzOWMTA%40mail.gmail.com.

Re: [sqlalchemy] simple query takes to long

[Simon King]

How many rows are you fetching, and how many columns in each row?

On Thu, Jun 9, 2022 at 8:37 AM Trainer Go  wrote:

> Hello Jonathan,
>
> i already executed the query without using pandas in my programm
>
> query = "SELECT"
> for row in conn.execute(query).fetchall():
> pass
>
> the result was the same runtime with pandas.
>
> So this cant be the problem. I think so.
>
> Greetings Manuel
>
> Jonathan Vanasco schrieb am Mittwoch, 8. Juni 2022 um 17:28:01 UTC+2:
>
>> When you select in the database ui tool, you are just displaying raw data.
>>
>> When you select within your code snippets above, Python is creating
>> pandas' DataFrame objects for the results.
>>
>> These two concepts are not comparable at all.  Converting the SQL data to
>> Python data structures in Pandas (and SQLAlchemy's ORM) is a lot of
>> overhead - and that grows with the result size.
>>
>> You can use memory and code profiling tools to explore this and see where
>> the issues are. The best approach is what Philip suggested above though,
>> and not use pandas, so you can see how Python/SqlAlchemy handles the raw
>> data.
>>
>>
>>
>>
>>
>> On Wednesday, June 8, 2022 at 9:28:38 AM UTC-4 Trainer Go wrote:
>>
>>> Hello Phil,
>>>
>>> i tested both and without printing the result.
>>>
>>> table_df = pd.read_sql_query(''SELECT, engine)
>>> #print(table_df)
>>> #query = "SELECT"
>>> #for row in conn.execute(query).fetchall():
>>> #pass
>>>
>>>
>>> both have nearly the same runtime. So this is not my problem. And yes,
>>> they are the same queries cause i copy pasted the select from my DBUI where
>>> is tested first the results and the runtime and i expected the same runtime
>>> in my program but no ;)
>>>
>>> Greeting Manuel
>>>
>>> Philip Semanchuk schrieb am Mittwoch, 8. Juni 2022 um 15:04:08 UTC+2:
>>>


 > On Jun 8, 2022, at 8:29 AM, Trainer Go  wrote:
 >
 > When im using pandas with pd.read_sql_query()
 > with chunksize to minimiza the memory usage there is no difference
 between both runtimes..

 Do you know that, or is that speculation?

 >
 > table_df = pd.read_sql_query('''select , engine, chunksize = 3)
 >
 > for df in table_df:
 > print(df)
 >
 > the runtime is nearly the same like 5 minutes

 Printing to the screen also takes time, and your terminal probably
 buffers the results, which requires memory allocation. I’m not saying this
 is your problem (it probably isn’t), but your test still involves pandas
 and your terminal, both of which cloud the issue. You would benefit from
 simplifying your tests.

 Did you try this suggestion from my previous email?


 > for row in conn.execute(my_query).fetchall():
 > pass

 Also, are you 100% sure you’re executing the same query from SQLAlchemy
 that you’re pasting into your DB UI?

 Cheers
 Philip



 >
 >
 >
 > #print(table_df) result: #generator object
 SQLDatabase._query_iterator at 0x0DC69C30>
 > I dont know if the query will be triggered by using print(table_df)
 the result is generator object SQLDatabase._query_iterator at 0x0DC69C30>
 >
 > but the runtime is 6 seconds like in the DBUI im using.
 >
 > I have no clue what to do.
 >
 > Greetings Manuel
 >
 > Trainer Go schrieb am Mittwoch, 8. Juni 2022 um 09:27:04 UTC+2:
 > thank you Philip,
 >
 > I will test it today.
 >
 >
 > Greetings Manuel
 >
 > Philip Semanchuk schrieb am Dienstag, 7. Juni 2022 um 17:13:28 UTC+2:
 >
 >
 > > On Jun 7, 2022, at 5:46 AM, Trainer Go  wrote:
 > >
 > > Hello guys,
 > >
 > > Im executing 2 queries in my python program with sqlalchemy using
 the pyodbc driver.
 > > The database is a Adaptive SQL Anywhere Version 7 32 Bit.
 > >
 > > When im executing the queries in a DB UI it takes 5-6 seconds for
 both together and when im using the same queries in my python programm it
 takes 5-6 minutes instead of 6 seconds. What im doing wrong? Im new at
 this.
 >
 > To start, debug one query at a time, not two.
 >
 > Second, when you test a query in your DB UI, you’re probably already
 connected to the database. Your Python program has to make the connection —
 that’s an extra step, and it might be slow. If you step through the Python
 program in the debugger, you can execute one statement at a time (the
 connection and the query) to understand how long each step takes. That will
 help to isolate the problem.
 >
 > Third, keep in mind that receiving results takes time too. If your DB
 UI is written in C or some other language that allocates memory very
 efficiently, it might be a lot faster than building a Pandas dataframe.
 >
 > You might want to eliminate Pandas entirely so you don’t have to
 question whether or not that’s the source of your 

Re: [sqlalchemy] how i can avoid the existing database table while generating the initial revision using alembic

[Simon King]

Did you regenerate your migration script after adding the hooks?

I would start by putting some print statements in the include_name hook to
see how it is being called. You should see it called for every object in
the database. You can then decide which names to return True for, and which
ones to return False for. The example in the docs is probably a reasonable
default:

def include_name(name, type_, parent_names):
print("Checking %s %s" % (type_, name))
if type_ == "table":
result = name in target_metadata.tables
else:
result = True
print("Returning %s" % result)
return result

Use that hook in your env.py and regenerate your migration script. You
should see output for each object in the database.

Simon


On Wed, Jun 8, 2022 at 5:59 PM Vishal Shimpi 
wrote:

> Thank you Simon for your response. yes i am using autogenerate feature. i
> tried with include_object and include_name hooks. but it won't work for me.
> after adding hook also alembic touches to existing tables..
>
> if you send the code snipet for env.py file.. that will really help me..
>
> Thank you.
>
> On Wed, 8 Jun 2022, 10:00 pm Simon King,  wrote:
>
>> If I understand correctly, you used Alembic's "autogenerate" feature to
>> create your migration script. This feature compares the table definitions
>> in your application with the table definitions in the database and then
>> generates a script to alter the database to match your application.
>>
>> You can instruct alembic to ignore certain objects in the database using
>> either the "include_name" or "include_object" hooks:
>>
>>
>> https://alembic.sqlalchemy.org/en/latest/autogenerate.html#controlling-what-to-be-autogenerated
>>
>> The first example there ("Omitting Table Names from the Autogenerate
>> Process") is probably what you want.
>>
>> Hope that helps,
>>
>> Simon
>>
>> On Wed, Jun 8, 2022 at 3:15 PM Vishal Shimpi 
>> wrote:
>>
>>> I am working on fastapi, in which i have created models and i am inteded
>>> to create the table in sql server database, however when i am runing my
>>> first migration, alembic detected removal of existing table which are not
>>> belongs to my work. Can somebody help how i can create my tables and avoid
>>> others tables to be removed and recreated again
>>>
>>> --
>>> SQLAlchemy -
>>> The Python SQL Toolkit and Object Relational Mapper
>>>
>>> http://www.sqlalchemy.org/
>>>
>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>> description.
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "sqlalchemy" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to sqlalchemy+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/sqlalchemy/3a89be66-24a0-4215-8b00-b4c11c52bbf9n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/sqlalchemy/3a89be66-24a0-4215-8b00-b4c11c52bbf9n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>> description.
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to sqlalchemy+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfFR%2Basd7oT8Lskimf-NqvDTs7cY--PLXf_5HQfrunwbw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfFR%2Basd7oT8Lskimf-NqvDTs7cY--PLXf_5HQfrunwbw%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this

Re: [sqlalchemy] how i can avoid the existing database table while generating the initial revision using alembic

[Simon King]

If I understand correctly, you used Alembic's "autogenerate" feature to
create your migration script. This feature compares the table definitions
in your application with the table definitions in the database and then
generates a script to alter the database to match your application.

You can instruct alembic to ignore certain objects in the database using
either the "include_name" or "include_object" hooks:

https://alembic.sqlalchemy.org/en/latest/autogenerate.html#controlling-what-to-be-autogenerated

The first example there ("Omitting Table Names from the Autogenerate
Process") is probably what you want.

Hope that helps,

Simon

On Wed, Jun 8, 2022 at 3:15 PM Vishal Shimpi 
wrote:

> I am working on fastapi, in which i have created models and i am inteded
> to create the table in sql server database, however when i am runing my
> first migration, alembic detected removal of existing table which are not
> belongs to my work. Can somebody help how i can create my tables and avoid
> others tables to be removed and recreated again
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/3a89be66-24a0-4215-8b00-b4c11c52bbf9n%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfFR%2Basd7oT8Lskimf-NqvDTs7cY--PLXf_5HQfrunwbw%40mail.gmail.com.

Re: [sqlalchemy] Does alembic support multiple databases?

[Simon King]

There are a few possibilities. You could have separate configuration files
for each database (eg. alembic-dev.ini and alembic-prod.ini), and choose
between them with the "--config" command line option.

If you want to stick to a single configuration file, you could put both
connection strings in your config file with different prefixes. For example:

sqlalchemy.dev.url = driver://user:pass@localhost/dbname
sqlalchemy.prod.url = driver://user:pass@localhost/dbname

Then, in your env.py file, locate the lines that say:

connectable = engine_from_config(
config.get_section(config.config_ini_section),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)

...and change it to use either "sqlalchemy.dev." or "sqlalchemy.prod." as
the prefix based on some condition. For example, you could use
"context.get_x_argument" and pass the environment name on the command line:

https://alembic.sqlalchemy.org/en/latest/api/runtime.html#alembic.runtime.environment.EnvironmentContext.get_x_argument

Hope that helps,

Simon


On Mon, May 9, 2022 at 8:57 PM thavha tsiwana 
wrote:

> wondering if there is more clearer information or a code snippet for this,
> I am also struggling to use 2 databases (1 dev, 1 prod) in my alembic
> project, I have successfully ran the migrations on the dev database, now I
> want to run my migrations to the prod database, I have no idea on how to
> modify alembic.ini and env.py file,,, please help
>
> On Tuesday, 01 February 2022 at 15:08:20 UTC+2 skira@gmail.com wrote:
>
>> If you have git example, share, please.
>> On Monday, 30 April 2012 at 04:10:14 UTC+3 limodou wrote:
>>
>>> On Sun, Apr 29, 2012 at 11:13 PM, Michael Bayer
>>>  wrote:
>>> >
>>> > On Apr 29, 2012, at 10:56 AM, limodou wrote:
>>> >
>>> >> On Sun, Apr 29, 2012 at 10:42 PM, Michael Bayer
>>> >>  wrote:
>>> >>> You would assemble a multi-database scheme of your choosing in
>>> env.py.   If you do "alembic init multidb" you'll see an example of one.
>>>   How env.py is organized depends greatly on the relationship of the
>>> databases to each other, that is, to what degree they are mirrors of each
>>> other versus storing different schemas.
>>> >>>
>>> >>
>>> >> If I ran the command:
>>> >>
>>> >>alembic init multidb
>>> >>
>>> >> It'll create multidb folder and copy files in it. But I saw the
>>> >> alembic.ini will be the same one. So if I should change it myself?
>>> And
>>> >> how to let alembic know different database when executing commands
>>> >> like: revision, upgrade, etc. It seems that no database parameter
>>> >> existed.
>>> >>
>>> >> And if I can manage different databases in one directory or in one
>>> ini file?
>>> >
>>> > multidb has a different alembic.ini as an example.  If you already had
>>> an alembic.ini there it wouldn't overwrite it.
>>> >
>>> > if you really wanted two completely independent sets of migration
>>> scripts, then you'd run two migration environments.
>>> >
>>> > They can share the same alembic.ini like this:
>>> >
>>> > [my_db_one]
>>> > sqlalchemy.url =
>>> >
>>> > [my_db_two]
>>> > sqlalchemy.url =
>>> >
>>> > you then run alembic with "alembic -n my_db_one" or "alembic -n
>>> my_db_two".The "default" config area is set by -n.
>>> >
>>> > A single env.py script can get multiple database URLs in any way it
>>> wants, as it determines how config is accessed.   If you look in the
>>> multidb/env.py script, you'll see it's pulling multiple database urls from
>>> one section using config.get_section(name) - config file:
>>> >
>>> > [alembic]
>>> > # path to migration scripts
>>> > script_location = ${script_location}
>>> >
>>> > # template used to generate migration files
>>> > # file_template = %%(rev)s_%%(slug)s
>>> >
>>> > databases = engine1, engine2
>>> >
>>> > [engine1]
>>> > sqlalchemy.url = driver://user:pass@localhost/dbname
>>> >
>>> > [engine2]
>>> > sqlalchemy.url = driver://user:pass@localhost/dbname2
>>> >
>>> > usage:
>>> >
>>> >config = context.config
>>> >
>>> >db_names = config.get_main_option('databases')
>>> >
>>> >for name in re.split(r',\s*', db_names):
>>> >engines[name] = rec = {}
>>> >rec['engine'] = engine_from_config(
>>> >config.get_section(name),
>>> >prefix='sqlalchemy.',
>>> >poolclass=pool.NullPool)
>>> >
>>> > Over here I have both forms of multi db at the same time.   There's
>>> two migration environments, and one migration environment does two
>>> databases that are largely mirrored, so three databases total.   All three
>>> make use of a common env.py script that's in my application as a library,
>>> they then implement an env.py in the migration environment that draws upon
>>> the myapp/lib/env.py script for common features.
>>> >
>>> > You can pass instructions to a single env.py that may be controlling
>>> multiple databases using --tag:
>>> >
>>> > "alembic --tag my_tag"
>>> >
>>> > 

Re: [sqlalchemy] Session management for general functions within a class

[Simon King]

It's difficult to debug this without a script that we can run to reproduce
the problem. What kind of object is self.db_session? You use it as a
context manager without calling it, so I don't think it can be a
sessionmaker or a session.

You're nesting calls to the context manager:

# in load_new_data
with self.db_session as outersession:
# add new_obj to outersession
# call move_to_error
with self.db_session as innersession:
# add new_obj to innersession

Are innersession and outersession supposed to be the same object? If they
are different sessions, you're trying to add new_obj to both of them, which
is going to be a problem.

If it were me, I would explicitly pass the session to the move_to_error
method. If you don't like that, you can also use
sqlalchemy.orm.object_session to get the session that new_obj already
belongs to.

Hope that helps,

Simon

On Fri, Apr 29, 2022 at 5:10 AM Andrew Martin  wrote:

> Hi all, I'm struggling a bit with best practices for my ETL application.
>
> Each part of the ETL app is completely separate from the others, but I
> have a MixIn for some common functions that each of them need to do, like
> move this record to error if there's a data integrity problem. Or move this
> record to manual review if there's insufficient data to move it along to
> the next stage of the ETL.
>
> The problem I'm having is that I don't understand the correct way to pass
> an object to a function, update it, and eventually commit it.
>
> I have for example:
>
> class DataMoverMixin:
> def __init__(self) -> None:
> self.db_session = get_db_session()
> 
>
> self.move_to_error(obj: Any, error_stage: str, traceback: Exception)
> -> bool:
> logger.info("Moving object to error.")
> json_data = json.dumps(obj, cls=AlchemyEncoder)
> e = Error(
> id=obj.id,
> error_stage=error_stage,
> error_message=repr(traceback),
> error_data=json_data,
> )
> obj.status = "error"
> with self.db_session as session:
> session.add(e)
> session.add(obj)
> session.commit()
> logger.info("Successfully moved object to error.")
> return True
>
> class IngestDataManager(DataMoverMixin):
> def __init__(self):
> super().__init__()
> 
>
>
> def load_new_data(self, accounts: List[Dict]) -> bool:
> for acc in accounts:
> new_obj = NewObj(**acc)
> with self.db_session as session:
> session.add(new_obj)
> session.commit()
> # now the raw data is loaded, I need to check if it
> conforms and do some stuff  with the newly created id.
> session.refresh(new_obj)
> if not new_obj.important_stuff:
>  self.move_to_error(new_obj, "ingest_integrity_error",
> f"missing {important stuff} for account_id: {new_obj.id}
>
>
> This is the simplest example of what does and doesn't work. And I can tell
> from the errors that I must be doing something very anti pattern, but I
> can't quite figure out what.
>
> This pattern gives me a DetachedInstanceError.
>
> So if I change Mixin.move_to_error like so:
>
> . . .
> with self.db_session as session:
> session.refresh(obj)
> obj.status = "error"
> session.add(e)
> session.add(obj)
> session.commit()
> . . .
>
> I get no error. But also the changes to the obj are not actually committed
> to the DB.
> The new record for error is committed.
>
> My expectation was that by attaching the session to the class that any
> method on the class would reference the same session, and that using the
> context manager was just a good practice to open and close it. But that
> doesn't seem to be the case.
>
> I might certainly be wrong, but it appears that when you pass an
> SQLAlchemy object to a function inside of a session context manager, it
> does not carry the session with it?
>
> And also reopening what I think is the session in a context manager fixes
> that but also then doesn't allow me to update the object?
>
> I guess I'm just kinda confused, and I'm sure there's a better way to do
> this.
>
> I've searched around a lot to try and understand this problem, but for
> whatever reason, nothing has clicked for me about what I'm doing wrong.
>
> Appreciate any help from people.
>
> -andrew
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> 

Re: [sqlalchemy] query many-many with asssociation table

[Simon King]

I think it should work if you join to the *relationship* explicitly

ie.

session.query(User).join(User.user_groups).filter(...)

Hope that helps,

Simon


On Tue, Apr 5, 2022 at 9:48 PM Jason Hoppes 
wrote:

> I want to select all users in a particular group. I have a users table,
> user_groups table, and a users_group_users_asc table to associate the two.
> Note this is not a self referencing relationship there are three different
> tables involved not two like the example in the documentation. I have the
> following configuration:
>
> user_groups_users = Table('user_groups_users_asc', Base.metadata,
>   Column('user_group_id', ForeignKey('
> user_groups.id', ondelete='CASCADE')),
>   Column('user_id', ForeignKey('users.id',
> ondelete='CASCADE'))
> )
>
> class User(Base):
> __tablename__ = 'users'
> id_ = Column('id', BigInteger, primary_key=True)
> username = Column('username', String(255))
> user_groups = relationship('UserGroup',
>secondary=user_groups_users,
>back_populates='users)
>
> class UserGroup(Base):
> __tablename__ = 'user_groups'
> id_ = Column('id', BigInteger, primary_key=True)
> group_name = Column('group_name', String(255), nullable=False)
> description = Column('description', Text)
> users = relationship('User',
>  secondary=user_groups_users,
>  back_populates='user_groups',
>  passive_deletes=True)
>
> As I suspected the following query gives me an error:
>
> session.query(User).join(UserGroup).filter(UserGroup.group_name ==
> grp_name).all()
>
> Don't know how to join to . Please
> use the .select_from() method to establish an explicit left side, as well
> as providing an explicit ON clause if not present already to help resolve
> the ambiguity.
>
> Thank you in advance for your help.
>
> - Jason
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/43edd22c-7eca-427b-907e-57e20d665f6en%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdfe3cz%2BFxiLLzAHhAswb%2BY1WEKf8ACxqYoDYU5qExK5g%40mail.gmail.com.

Re: [sqlalchemy] do_connect listener called couple of times

[Simon King]

I don't have Oracle, but here's a complete runnable example using MySQL
(hopefully the formatting will survive this time):

import sqlalchemy as sa
import MySQLdb

# start with the wrong password to force a connection error
engine = sa.create_engine("mysql://user:wrong@db/db")

@sa.event.listens_for(engine, "do_connect")
def receive_do_connect(dialect, conn_rec, cargs, cparams):
print(f" do_connect ")
print(f"cargs: {cargs}")
print(f"cparams: {cparams}")
try:
return MySQLdb.connect(**cparams)
except Exception as e:
print(f"EXCEPTION: {e}")
# Store the correct password in cparams for future connection
attempts
cparams["passwd"] = "correct"
return MySQLdb.connect(**cparams)

conn1 = engine.connect()
print(f"conn1: {conn1}")

conn2 = engine.connect()
print(f"conn2: {conn2}")


And here's the output:

 do_connect 
cargs: []
cparams: {'host': 'db', 'db': 'db', 'user': 'user', 'passwd': 'wrong',
'client_flag': 2}
EXCEPTION: (1045, "Access denied for user 'user'@'172.18.0.9' (using
password: YES)")
conn1: 
 do_connect 
cargs: []
cparams: {'host': 'db', 'db': 'db', 'user': 'user', 'passwd': 'correct',
'client_flag': 2}
conn2: 


As you can hopefully see, the first connection attempt triggered an
exception. The correct password was then stored in cparams, and the next
time we called engine.connect(), the correct password was already passed in
and no exception was raised.

Simon


On Wed, Mar 23, 2022 at 4:36 AM Srinu Chp  wrote:

> Hello Simon,
>
> I tried your suggestion as POC:
>
> def setup_event_handlers(engine):
> @event.listens_for(engine, 'do_connect')
> def receive_do_connect(dialect, conn_rec, cargs, cparams):
> print("inside do_connect")
> print('password %s' % cparams['password'])
> try:
> print("inside try")
> return cx_Oracle.connect(*cargs, **cparams)
> except Exception as e:
> print("inside catch")
> cparams['password'] = "NewPassword"
> return cx_Oracle.connect(*cargs, **cparams)
>
> Every time except block is triggered even I set correct password in Except
> block. As per document once cparams password set it should pass new
> password for new request. Can you please suggest if I miss anything here?
>
> Regards,
> Pydi
>
> On Tuesday, March 22, 2022 at 9:33:57 AM UTC-7 Srinu Chp wrote:
>
>> Hello Simon,
>>
>> Thank you very much for detail information.
>>
>> Regards,
>> Pydi
>>
>> On Tuesday, March 22, 2022 at 3:11:34 AM UTC-7 Simon King wrote:
>>
>>> I don't know anything about Airflow. Are you sure that each of these
>>> tasks is running inside the same Python interpreter/process? I see
>>> Airflow can distribute tasks among workers:
>>>
>>> https://airflow.apache.org/docs/apache-airflow/stable/executor/index.html
>>>
>>> This sounds like a problem that is going to be very specific to your
>>> deployment environment. If you have multiple worker processes, you're
>>> going to need some way to distribute the new password to each of the
>>> workers (eg. a shared cache)
>>>
>>> But regardless of that, you're still not following the pattern from
>>> the documentation. I don't understand why you are creating a new
>>> engine inside your do_connect handler. You should be creating a
>>> cx-Oracle connection and returning it. The parameters passed to the
>>> do_connect handler have already been parsed out of the connection
>>> string. So for example, if your connection string includes a
>>> "password=some_password" parameter, then cparams will have a
>>> "password" key with the value "some_password". The same cparams
>>> dictionary will be passed to the do_connect handler each time, so if
>>> you mutate the dictionary (eg. by updating the "password" key), the
>>> next call to the handler will contain the new value.
>>>
>>> If each task invocation is creating a new engine using a connection
>>> string that is out of date, then none of that will help you, but that
>>> would be an Airflow problem, not an SQLAlchemy problem.
>>>
>>> Simon
>>>
>>> On Mon, Mar 21, 2022 at 8:27 PM Srinu Chp  wrote:
>>> >
>>> > Hello Simon,
>>> >
>>> > I tried POC approach in my project where we are using Airflow using
>>> sqlalchemy to connect to db. Here is the event code:
>>> >
>>> > @event.listens_for(engine, "do_connect")
>>> > def receive_do_connect(dialect, 

Re: [sqlalchemy] do_connect listener called couple of times

[Simon King]

I don't know anything about Airflow. Are you sure that each of these
tasks is running inside the same Python interpreter/process? I see
Airflow can distribute tasks among workers:

https://airflow.apache.org/docs/apache-airflow/stable/executor/index.html

This sounds like a problem that is going to be very specific to your
deployment environment. If you have multiple worker processes, you're
going to need some way to distribute the new password to each of the
workers (eg. a shared cache)

But regardless of that, you're still not following the pattern from
the documentation. I don't understand why you are creating a new
engine inside your do_connect handler. You should be creating a
cx-Oracle connection and returning it. The parameters passed to the
do_connect handler have already been parsed out of the connection
string. So for example, if your connection string includes a
"password=some_password" parameter, then cparams will have a
"password" key with the value "some_password". The same cparams
dictionary will be passed to the do_connect handler each time, so if
you mutate the dictionary (eg. by updating the "password" key), the
next call to the handler will contain the new value.

If each task invocation is creating a new engine using a connection
string that is out of date, then none of that will help you, but that
would be an Airflow problem, not an SQLAlchemy problem.

Simon

On Mon, Mar 21, 2022 at 8:27 PM Srinu Chp  wrote:
>
> Hello Simon,
>
> I tried POC approach in my project where we are using Airflow using 
> sqlalchemy to connect to db. Here is the event code:
>
> @event.listens_for(engine, "do_connect")
> def receive_do_connect(dialect, conn_rec, cargs, cparams):
> global SQL_ALCHEMY_CONN
> log.info("receive_do_connect called for user AIRFLOW.")
> log.info("user details DB: {}".format(SQL_ALCHEMY_CONN))
> # creating new engine to valide using cx_oracle driver
> engine_new = 
> create_engine(f'{SQL_ALCHEMY_CONN}/?encoding=UTF-8=UTF-8', 
> max_identifier_length=128)
> try:
> with engine_new.connect() as conn:
> log.info(conn.scalar("select sysdate from dual"))
> SQL_ALCHEMY_CONN = "testNew_try"
> except Exception as e:
> # check for invalid user/pwd error
> if search('ORA-01017', str(e)):
> log.info("receive_do_connect exception occurred during engine connection e: 
> {}".format(e))
> ...
> //update with new password
> SQL_ALCHEMY_CONN = "testNew_except"
> # this log print new value with updated password
> log.info("user details DB after update in except block: 
> {}".format(SQL_ALCHEMY_CONN))
> cparams['New password']
>
> global SQL_ALCHEMY_CONN value is set during initialization. Once password is 
> rotated I am trying to update the SQL_ALCHEMY_CONN so that next request will 
> not go in except block. Every time logs print old SQL_ALCHEMY_CONN value even 
> value is update in except block.
>
> second approach:
> I tried to set env variable in except block:
> os.environ['AIRFLOW__CORE__SQL_ALCHEMY_CONN']
> env variable also refer to old value even after updating in except block.
>
> Can you please suggestion?
> Regards,
> Pydi
> On Monday, March 21, 2022 at 10:55:09 AM UTC-7 Srinu Chp wrote:
>>
>> Hello Simon,
>>
>> Perfect, working as expected in standalone POC. Thank you quick help
>>
>> Regards,
>> Pydi
>>
>> On Monday, March 21, 2022 at 9:52:04 AM UTC-7 Simon King wrote:
>>>
>>> As suggested here:
>>>
>>> https://docs.sqlalchemy.org/en/14/core/engines.html#fully-replacing-the-dbapi-connect-function
>>>
>>> In your do_connect handler, rather than calling engine.connect(), you
>>> need to call cx_Oracle.connect(), and return the result. You can wrap
>>> this in an exception handler that detects the "incorrect password"
>>> error to fetch new credentials. Something like this perhaps:
>>>
>>> @event.listens_for(engine, 'do_connect')
>>> def receive_do_connect(dialect, conn_rec, cargs, cparams):
>>> try:
>>> return cx_Oracle.connect(*cargs, **cparams)
>>> except :
>>> cparams["password"] = get_new_password()
>>> return cx_Oracle.connect(*args, **cparams)
>>>
>>> Hope that helps,
>>>
>>> Simon
>>>
>>> On Mon, Mar 21, 2022 at 4:26 PM Srinu Chp  wrote:
>>> >
>>> > Hello Simon,
>>> >
>>> > Thank you for prompt response. I really appreciate your help. I am trying 
>>> > to achieve password rotation and we are using secret client to fetch new 
>>> > password. I tried do_connect event and fetch new pa

Re: [sqlalchemy] do_connect listener called couple of times

[Simon King]

As suggested here:

https://docs.sqlalchemy.org/en/14/core/engines.html#fully-replacing-the-dbapi-connect-function

In your do_connect handler, rather than calling engine.connect(), you
need to call cx_Oracle.connect(), and return the result. You can wrap
this in an exception handler that detects the "incorrect password"
error to fetch new credentials. Something like this perhaps:

@event.listens_for(engine, 'do_connect')
def receive_do_connect(dialect, conn_rec, cargs, cparams):
try:
return cx_Oracle.connect(*cargs, **cparams)
except :
cparams["password"] = get_new_password()
return cx_Oracle.connect(*args, **cparams)

Hope that helps,

Simon

On Mon, Mar 21, 2022 at 4:26 PM Srinu Chp  wrote:
>
> Hello Simon,
>
> Thank you for prompt response. I really appreciate your help. I am trying to 
> achieve password rotation and we are using secret client to fetch new 
> password. I tried do_connect event and fetch new password from secret client, 
> working as expected but we are facing performance issue as we are every time 
> connecting to secret client(3~5sec for each request). Instead I am trying to 
> achieve if connect fails then fetch from secret client.
>
> I tried with handle_error event, when i get error check for invalid user/pwd 
> and update session with latest engine. This approach also did not help
>
> Any insights are highly appreciated. Please suggest best approach.
>
> Regards,
> Pydi
> On Monday, March 21, 2022 at 2:22:04 AM UTC-7 Simon King wrote:
>>
>> I don't really understand what's going on in your code, but you seem
>> to be calling engine.connect() inside your "do_connect" event handler.
>> I would expect that to trigger another "do_connect" event, which in
>> turn will call engine.connect() again, which will trigger another
>> "do_connect" event, and so on. I'm surprised the application gets as
>> far as it does. Maybe the exception handler inside receive_do_connect
>> is allowing it to stumble on.
>>
>> Simon
>>
>> On Mon, Mar 21, 2022 at 4:51 AM Srinu Chp  wrote:
>> >
>> > Hello Team,
>> >
>> > I tried to create a standalone application POC for sqlalchemy to db 
>> > connection. When I registered do_connect event, I see event is triggered 
>> > couple of times for one call:
>> > sqlalchemy_connection.py
>> >
>> > import os
>> > import cx_Oracle
>> > from sqlalchemy import create_engine
>> > from sqlalchemy import event
>> >
>> > cx_Oracle.init_oracle_client(lib_dir=os.environ.get("HOME") + 
>> > "/Downloads/instantclient_19_8")
>> > SQLALCHEMY_CONN = "test"
>> > count = 0
>> > engine = None
>> > def connect_db(pwd):
>> > global count
>> > global engine
>> > print(SQLALCHEMY_CONN)
>> > username = "ADMIN"
>> > password = pwd
>> > dsn = "pydidb_high"
>> > engine = create_engine(
>> > f'oracle://{username}:{password}@{dsn}/?encoding=UTF-8=UTF-8', 
>> > max_identifier_length=128)
>> > setup_event_handlers(engine)
>> >
>> > def setup_event_handlers(engine):
>> > @event.listens_for(engine, 'do_connect')
>> > def receive_do_connect(dialect, conn_rec, cargs, cparams):
>> > print("inside do_connect")
>> > global count
>> > try:
>> > with engine.connect() as conn:
>> > print("inside do_connect try block")
>> > print(conn.scalar("select sysdate from dual"))
>> > count += 2
>> > except Exception as e:
>> > print("inside do_connect except block")
>> > count += 1
>> >
>> > def db_connect_test():
>> > print(engine)
>> > with engine.connect() as conn:
>> > print(conn.scalar("select sysdate from dual"))
>> >
>> > gevent_sync.py
>> >
>> > import gevent
>> > import random
>> > import sqlalchemy_connection
>> >
>> > def task(pid):
>> > gevent.sleep(random.randint(0,2)*0.001)
>> > print('Task %s done' % pid)
>> > sqlalchemy_connection.connect_db(**)
>> > sqlalchemy_connection.db_connect_test()
>> >
>> > def synchronous():
>> > for i in range(1,2):
>> > task(i)
>> > # sqlalchemy_connection.connect_db(**)
>> >
>> > def asynchronous():
>> > threads = [gevent.spawn(task, i) for i in range(2)]
>> > gevent.joinall(threads)
>> >
>> > print('Synchronous:')
>> &

Re: [sqlalchemy] do_connect listener called couple of times

[Simon King]

I don't really understand what's going on in your code, but you seem
to be calling engine.connect() inside your "do_connect" event handler.
I would expect that to trigger another "do_connect" event, which in
turn will call engine.connect() again, which will trigger another
"do_connect" event, and so on. I'm surprised the application gets as
far as it does. Maybe the exception handler inside receive_do_connect
is allowing it to stumble on.

Simon

On Mon, Mar 21, 2022 at 4:51 AM Srinu Chp  wrote:
>
> Hello Team,
>
> I tried to create a standalone application POC for sqlalchemy to db 
> connection. When I registered do_connect event, I see event is triggered 
> couple of times for one call:
> sqlalchemy_connection.py
>
> import os
> import cx_Oracle
> from sqlalchemy import create_engine
> from sqlalchemy import event
>
> cx_Oracle.init_oracle_client(lib_dir=os.environ.get("HOME") + 
> "/Downloads/instantclient_19_8")
> SQLALCHEMY_CONN = "test"
> count = 0
> engine = None
> def connect_db(pwd):
> global count
> global engine
> print(SQLALCHEMY_CONN)
> username = "ADMIN"
> password = pwd
> dsn = "pydidb_high"
> engine = create_engine(
> f'oracle://{username}:{password}@{dsn}/?encoding=UTF-8=UTF-8', 
> max_identifier_length=128)
> setup_event_handlers(engine)
>
> def setup_event_handlers(engine):
> @event.listens_for(engine, 'do_connect')
> def receive_do_connect(dialect, conn_rec, cargs, cparams):
> print("inside do_connect")
> global count
> try:
> with engine.connect() as conn:
> print("inside do_connect try block")
> print(conn.scalar("select sysdate from dual"))
> count += 2
> except Exception as e:
> print("inside do_connect except block")
> count += 1
>
> def db_connect_test():
> print(engine)
> with engine.connect() as conn:
> print(conn.scalar("select sysdate from dual"))
>
> gevent_sync.py
>
> import gevent
> import random
> import sqlalchemy_connection
>
> def task(pid):
> gevent.sleep(random.randint(0,2)*0.001)
> print('Task %s done' % pid)
> sqlalchemy_connection.connect_db(**)
> sqlalchemy_connection.db_connect_test()
>
> def synchronous():
> for i in range(1,2):
> task(i)
> # sqlalchemy_connection.connect_db(**)
>
> def asynchronous():
> threads = [gevent.spawn(task, i) for i in range(2)]
> gevent.joinall(threads)
>
> print('Synchronous:')
> synchronous()
> print('count %s ' % sqlalchemy_connection.count)
>
> # print('Asynchronous:')
> # asynchronous()
> # print('count %s' % sqlalchemy_connection.count)
>
> Output:
>
> Synchronous:
> Task 1 done
> test
> Engine(oracle://ADMIN:***@pydidb_high/?encoding=UTF-8=UTF-8)
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect
> inside do_connect except block
> inside do_connect try block
> 2022-03-21 04:39:47
> inside do_connect try block
> 2022-03-21 04:39:49
> inside do_connect try block
> 2022-03-21 04:39:51
> inside do_connect try block
> 2022-03-21 04:39:54
> inside do_connect try block
> 2022-03-21 04:39:56
> inside do_connect try block
> 2022-03-21 04:39:59
> inside do_connect try block
> 2022-03-21 04:40:01
> inside do_connect try block
> 2022-03-21 04:40:04
> inside do_connect try block
> 2022-03-21 04:40:09
> inside do_connect try block
> 2022-03-21 04:40:15
> inside do_connect try block
> 2022-03-21 04:40:17
> inside do_connect try block
> 2022-03-21 04:40:19
> inside do_connect try block
> 2022-03-21 04:40:21
> inside do_connect try block
> 2022-03-21 04:40:24
> 2022-03-21 04:40:26
> count 29
>
> highly appreciate any inputs.
> Regards,
> Pydi
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/08096636-06c4-478f-a54d-0bc8f71db414n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: [sqlalchemy] None! Can't pickle : it's not the same object as sqlalchemy.orm.session.Session

[Simon King]

I haven't used the multiprocessing library, but if it uses pickle to
transfer SQLAlchemy objects, it's going to be difficult to make it
work. Objects loaded via the SQLAlchemy ORM hold a references to the
Session that was used to load them, which in turn holds a reference to
a database connection. Pickling the instance therefore tries to pickle
the Session as well, which almost certainly isn't what you want.

Rather than passing ORM instances to worker processes, it would
probably be better to either:

a) Pass a unique identifier for the instance, and have the worker
process reload the instance from the database itself, or
b) Send simpler (non-SQLAlchemy) objects to the worker,

Hope that helps,

Simon

On Wed, Mar 16, 2022 at 2:34 AM Haris Damara  wrote:
>
> Hi,
>
> Please help.
>
> Find issue with error message "None! Can't pickle  'sqlalchemy.orm.session.Session'>: it's not the same object as 
> sqlalchemy.orm.session.Session"
> while run the sqlalchemy orm query and/or execute syntax in Process 
> (multiprocessing)
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/f98608ed-1eb6-49d3-90c2-642973c990f0n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexcAEJNvyVve_3SBMZi9WL2H_3v18qKLALz3QEC8nMaRZA%40mail.gmail.com.

Re: [sqlalchemy] Issue "translating" raw SQL to SQLAlchemy ORM query

[Simon King]

By default, relationship loading is deliberately not affected by your
join conditions. If you want a relationship property to be restricted
to the rows you've selected in your query, you need to use the
"contains_eager()" query option:

https://docs.sqlalchemy.org/en/14/orm/loading_relationships.html#using-contains-eager-to-load-a-custom-filtered-collection-result

Hope that helps,

Simon

On Wed, Feb 23, 2022 at 5:19 PM shuhari2020  wrote:
>
> FROM: 
> https://stackoverflow.com/questions/71225408/issue-translating-raw-sql-to-sqlalchemy-orm-query
>
> I have the following raw SQL statement that I am having trouble "translating" 
> into a SQLAlchemy query:
>
> (the hardcoded value 38 is just for testing)
>
> SELECT * FROM public.data_appquestion AS question
>
> /* ANSWER JOIN */
> LEFT JOIN (SELECT * FROM public.data_appanswer) AS answer
> ON ( answer.separation_app_question_id = question.id AND answer.is_active = 
> true
> AND answer.separation_app_session_id = 38 )
>
> /* OPTION XREFS JOIN */
> LEFT JOIN (SELECT * FROM public.data_appansweroptionxref) AS options_xref
> ON ( options_xref.separation_app_answer_id = answer.id )
>
> /* OPTIONs JOIN */
> LEFT JOIN (SELECT * FROM public.data_appoption) AS answered_option
> ON ( options_xref.separation_app_option_id = answered_option.id )
>
> /* UPLOAD JOIN */
> LEFT JOIN (SELECT * FROM public.data_appfileupload) AS uploads
> ON ( uploads.separation_app_answer_id = answer.id )
>
> WHERE question.is_active = true
> AND answer.is_active = true OR answer.is_active = NULL
> AND options_xref.is_active = true OR options_xref.is_active = NULL
> AND uploads.to_delete = false OR uploads.to_delete = NULL
> ORDER BY question.id;
>
> I have tried something like this, but the "filter" statement already does not 
> seem to work as I need it to:
>
> db_questions = db.query(models.AppQuestion).\
> filter(models.AppQuestion.is_active == True).\
> outerjoin(models.AppAnswer, and_( models.AppAnswer.app_question_id == 
> models.AppQuestion.id, models.AppAnswer.app_session_id == 38 ) ).\
> outerjoin(models.AppAnswer.app_options).\
> outerjoin(models.AppAnswerOptionXref.app_option).\
> outerjoin(models.AppFileUpload.app_question).\
> order_by(asc(models.AppQuestion.order_number)).all()
>
> The models all have the relevant "relationship" entries, so there is no issue 
> for the query to find the relevant models via their foreign keys. The issue 
> is that they are not filtered as they are in the raw SQL.
>
> My result includes a joined "AppAnswer", but it does not filter it according 
> to ```app_session_id == 38
>
> I'm not very familiar with joining SQL queries and usually working with the 
> Django ORM, which never had me run into an issue like this.
>
> Let me know if I need to add more info and thanks a lot for any replies!
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/3b1eb2d6-1736-41f3-9cd3-29f0cd9af737n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexcQQAUt184-NvdZLv1468okgkWRnuBMaSOeMx1o_8DYmA%40mail.gmail.com.

Re: [sqlalchemy] What is the best way to declare a table with 260 columns and add rows on that table

[Simon King]

tion(packet)
> mysql.connector.errors.ProgrammingError: 1054 (42S22): Unknown column 
> 'mls_number' in 'field list'
> The above exception was the direct cause of the following exception:
> Traceback (most recent call last):
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/sql.py",
>  line 1419, in to_sql
> raise err
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/sql.py",
>  line 1411, in to_sql
> table.insert(chunksize, method=method)
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/sql.py",
>  line 845, in insert
> exec_insert(conn, keys, chunk_iter)
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/sql.py",
>  line 762, in _execute_insert
> conn.execute(self.table.insert(), data)
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
>  line 1289, in execute
> return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sqlalchemy/sql/elements.py",
>  line 325, in _execute_on_connection
> return connection._execute_clauseelement(
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
>  line 1481, in _execute_clauseelement
> ret = self._execute_context(
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
>  line 1845, in _execute_context
> self._handle_dbapi_exception(
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
>  line 2026, in _handle_dbapi_exception
> util.raise_(
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sqlalchemy/util/compat.py",
>  line 207, in raise_
> raise exception
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
>  line 1782, in _execute_context
> self.dialect.do_executemany(
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sqlalchemy/engine/default.py",
>  line 729, in do_executemany
> cursor.executemany(statement, parameters)
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mysql/connector/cursor.py",
>  line 670, in executemany
> return self.execute(stmt)
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mysql/connector/cursor.py",
>  line 568, in execute
> self._handle_result(self._connection.cmd_query(stmt))
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mysql/connector/connection.py",
>  line 854, in cmd_query
> result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
>   File 
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mysql/connector/connection.py",
>  line 664, in _handle_result
> raise errors.get_exception(packet)
> sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 
> 1054 (42S22): Unknown column 'mls_number' in 'field list'
> [SQL: INSERT INTO mlstreb (mls_number, `0`) VALUES (%(mls_number)s, %(0)s)]
> [parameters: ({'mls_number': 'mls_number', '0': 'C5405199'}, {'mls_number': 
> 'active', '0': True}, {'mls_number': 'class_name', '0': 'RES'}, 
> {'mls_number': 'active_date', '0': datetime.date(2022, 1, 3)})]
> (Background on this error at: https://sqlalche.me/e/14/f405)
>
> On Thu, Feb 24, 2022 at 4:01 AM Simon King  wrote:
>>
>> Before we do that, you said that you tried pandas dataframe.to_sql but
>> it didn't work - can you explain what you mean? Did it raise an error,
>> or produce the wrong result, or something else?
>>
>> Simon
>>
>> On Wed, Feb 23, 2022 at 9:13 PM janio mendonca junior
>>  wrote:
>> >
>> > Hi Simon,
>> >
>> > Thank you for your help. I am brand new working with SQLalchemy, really 
>> > appreciate if you explain how to generate the metadata with the list of 
>> > column names from the .CSV to create the tables?
>> >
>> > On Wed, Feb 23, 2022, 3:52 PM Simon King  wrote:
>> >>
>> >> Build a list of Column objects from the columns in the CSV file, and
>>

Re: [sqlalchemy] What is the best way to declare a table with 260 columns and add rows on that table

[Simon King]

Before we do that, you said that you tried pandas dataframe.to_sql but
it didn't work - can you explain what you mean? Did it raise an error,
or produce the wrong result, or something else?

Simon

On Wed, Feb 23, 2022 at 9:13 PM janio mendonca junior
 wrote:
>
> Hi Simon,
>
> Thank you for your help. I am brand new working with SQLalchemy, really 
> appreciate if you explain how to generate the metadata with the list of 
> column names from the .CSV to create the tables?
>
> On Wed, Feb 23, 2022, 3:52 PM Simon King  wrote:
>>
>> Build a list of Column objects from the columns in the CSV file, and
>> use that list to create a Table:
>>
>> https://docs.sqlalchemy.org/en/14/core/metadata.html
>>
>> Once you've created the Table, you can insert data into it using the
>> table.insert() method:
>>
>> https://docs.sqlalchemy.org/en/14/core/tutorial.html#executing-multiple-statements
>>
>> Hope that helps,
>>
>> Simon
>>
>> On Wed, Feb 23, 2022 at 8:42 PM janio mendonca junior
>>  wrote:
>> >
>> > Hi all,
>> >
>> > I have a inquiry from my job to create 2 tables related one-to-one and 
>> > insert some rows on the table. I have a .CSV with the data dictionary from 
>> > the table and I am wondering to know how to declare the tables columns 
>> > automatically without write one by one column (there are 260 columns). 
>> > Same thing for the insert, how to add rows to the multiple columns table 
>> > without write column by column?
>> >
>> > I have the data in a Data frame but I was not able to insert it using 
>> > df.to_sql from pandas. Do you guys have any similar example?
>> >
>> > Thank you all
>> >
>> > --
>> > SQLAlchemy -
>> > The Python SQL Toolkit and Object Relational Mapper
>> >
>> > http://www.sqlalchemy.org/
>> >
>> > To post example code, please provide an MCVE: Minimal, Complete, and 
>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> > description.
>> > ---
>> > You received this message because you are subscribed to the Google Groups 
>> > "sqlalchemy" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to sqlalchemy+unsubscr...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/sqlalchemy/CADF7wwb0_ncRuU_CadqFegE9583W-xWWD4x%3DGy8V%3DgW0jKtcyg%40mail.gmail.com.
>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example.  See  http://stackoverflow.com/help/mcve for a full 
>> description.
>> ---
>> You received this message because you are subscribed to the Google Groups 
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to sqlalchemy+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sqlalchemy/CAFHwexexvrpmr%3DEA4w%3DmncyiKyxj0yk%3Dcr9_%2Br%3Db3MCyOiHg%3DA%40mail.gmail.com.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/CADF7wwYLD-OrxU9-Eq5mnSMoCBJ-EcFEcD%3DVXUEJJNLCx_dOjw%40mail.gmail.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfbN8Wt6MPp1PY6-gptjL5zdMmjy5z5YAFsczdgES%2B8Uw%40mail.gmail.com.

Re: [sqlalchemy] What is the best way to declare a table with 260 columns and add rows on that table

[Simon King]

Build a list of Column objects from the columns in the CSV file, and
use that list to create a Table:

https://docs.sqlalchemy.org/en/14/core/metadata.html

Once you've created the Table, you can insert data into it using the
table.insert() method:

https://docs.sqlalchemy.org/en/14/core/tutorial.html#executing-multiple-statements

Hope that helps,

Simon

On Wed, Feb 23, 2022 at 8:42 PM janio mendonca junior
 wrote:
>
> Hi all,
>
> I have a inquiry from my job to create 2 tables related one-to-one and insert 
> some rows on the table. I have a .CSV with the data dictionary from the table 
> and I am wondering to know how to declare the tables columns automatically 
> without write one by one column (there are 260 columns). Same thing for the 
> insert, how to add rows to the multiple columns table without write column by 
> column?
>
> I have the data in a Data frame but I was not able to insert it using 
> df.to_sql from pandas. Do you guys have any similar example?
>
> Thank you all
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/CADF7wwb0_ncRuU_CadqFegE9583W-xWWD4x%3DGy8V%3DgW0jKtcyg%40mail.gmail.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexexvrpmr%3DEA4w%3DmncyiKyxj0yk%3Dcr9_%2Br%3Db3MCyOiHg%3DA%40mail.gmail.com.

Re: [sqlalchemy] Remove/Filter a query.all() results, add a 'virtual' column

[Simon King]

Hi, welcome to Python and SQLAlchemy :-)

If you want to do some extra filtering on the results, you can iterate
over the results, decide whether each item matches your filter
conditions, and if it does, append it to a new list, something like
this:

filtered_results = []
for part in query.all():
if :
filtered_results.append(part)

You can add new properties and methods to your BLPart class. The exact
syntax would depend on how you want to use them. Here's the simplest
example:

class BLPart(db.Model):
# column definitions etc.
ITEMTYPE = Column(...)

# Non-db attributes
foreground = "#000"
background = "#fff"

Now every BLPart instance will have "foreground" and "background"
attributes with those values.

If you need something more complicated than that, let us know how you
would want to use them.

Hope that helps,

Simon

On Sun, Feb 20, 2022 at 9:37 PM Cp Divers  wrote:
>
> Hello Guys, this is my very first post here. I'm not sure this the the right 
> place. I'm a week old with Python and SQLAlchemy. And I believe I'm missing a 
> couple concept, hopefully you can help|
>
> I do have this class
>
> class BLPart(db.Model):
> __tablename__ = 'BL_parts'
>
> ITEMTYPE = Column(String(1, 'utf8mb4_unicode_ci'), nullable=False)
> ITEMID = Column(String(20, 'utf8mb4_unicode_ci'), primary_key=True)
> ITEMDESC = Column(Text(collation='utf8mb4_unicode_ci'), nullable=False, 
> index=True)
> CATEGORY = Column(ForeignKey('BL_categories.category_id'), 
> nullable=False, index=True, server_default=text("0"))
> ITEMWEIGHT = Column(String(10, 'utf8mb4_unicode_ci'))
> ITEMDIMX = Column(Float, server_default=text("0"))
> ITEMDIMY = Column(Float, server_default=text("0"))
> ITEMDIMZ = Column(Float, server_default=text("0"))
> In my code python code I got something like this:
>
> if form.validate_onsubmit():
> search = form.search.data.strip()
> query = db.session.query(BLPart).filter(
> or(
> BLPart.ITEMDESC.contains(search, autoescape=True),
> BLPart.ITEMID.contains(search, autoescape=True)
> )
> )
>
> results= query.all()
> print(type(results)) #
> print(results) #[, , ,  3005f1>, , , , 
> Here are my 2 main questions,
>
> 1) I do have some 'smart' filtering that I would like to do after I get the 
> results
> Based on that filter I'd like to remove some rows from the results variables
>
> 2) In my class, I have 8 columns, based on my filtering I want to create a 
> new 'virtual' column which does not exist in the DB
> Let's say I want to create a Foreground color and a background color 'column' 
> in the results variable
> How can I achieve this ?
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/01812940-c02f-4ad4-9a2d-adfdb736a13cn%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexe_%3DMzCEpRWO%2BOQotOucJNqMb9zDCOPKXdJWh2vA-qHHg%40mail.gmail.com.

Re: [sqlalchemy] SQL expression object expected, got object of type instead

[Simon King]

I'm sorry, I still don't understand - what do you expect
query(Genome.attributes) to do? Can you give an example?

Thanks,

Simon

On Wed, Jan 26, 2022 at 11:30 PM Simon  wrote:
>
> Sorry for that the question is not clear. The question is how can we query a 
> database's property.
>
> Given the above example, if query(Genome.id) or query(Genome.created_date), 
> it works fine. But if I query the property, query(Genome.attributes): it 
> raises the exception.
>
> Thanks.
> Simon
>
> On Wednesday, January 26, 2022 at 12:53:52 AM UTC+13 Simon King wrote:
>>
>> Can you show the part of *your* code that is triggering the error, and
>> explain what you are trying to do? Plain python properties aren't
>> normally very useful when accessed via a class. "Genome.attributes"
>> returns a property object, not the return value from the function, and
>> I don't understand what you are trying to do with it.
>>
>> Thanks,
>>
>> Simon
>>
>> On Mon, Jan 24, 2022 at 2:03 AM Simon  wrote:
>> >
>> > Hi there,
>> >
>> > I got a problem about 'sqlalchemy.exc.ArgumentError: SQL expression object 
>> > expected, got object of type  instead'
>> >
>> > My SQLAlchemy version is 1.3.22. I have a database like
>> >
>> > Class Genome:
>> > id = Column(Integer, primary_key=True)
>> > created_date = Column(Datetime, nullable=False)
>> >
>> > @property
>> > def attributes(self):
>> > return "something"
>> >
>> >
>> > If using the query, it reports an error through the elements.py in the 
>> > sqlAlchemy
>> >
>> > def _literal_as(element, text_fallback):
>> > if isinstance(element, Visitable):
>> > return element
>> > elif hasattr(element, "__clause_element__"):
>> > return element.__clause_element__()
>> > elif isinstance(element, util.string_types):
>> > return text_fallback(element)
>> > elif isinstance(element, (util.NoneType, bool)):
>> > return _const_expr(element)
>> > else:
>> > raise exc.ArgumentError(
>> > "SQL expression object expected, got object of type %r "
>> > "instead" % type(element)
>> > )
>> >
>> > This exception is not raised if I directly query genome's column name such 
>> > as created_date or id.
>> >
>> > I am wondering 1) could the column name and property be used 
>> > interchangeably in some way? Or say how could to query a table's property 
>> > in the way of querying a table's column? 2) what are some significant 
>> > differences between table's column name and property?
>> >
>> > Thanks.
>> >
>> >
>> >
>> >
>> > --
>> > SQLAlchemy -
>> > The Python SQL Toolkit and Object Relational Mapper
>> >
>> > http://www.sqlalchemy.org/
>> >
>> > To post example code, please provide an MCVE: Minimal, Complete, and 
>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> > description.
>> > ---
>> > You received this message because you are subscribed to the Google Groups 
>> > "sqlalchemy" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to sqlalchemy+...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/sqlalchemy/7f03c9bb-9324-4e3e-8aea-cd0d46f9021bn%40googlegroups.com.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/2fbf2d83-6865-4b9b-8db6-cb5f60e0eaffn%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeT_AcTbbJ561Rx%3D3jDdAFxshTMHrJ7jFSzKiGMZC6Y9Q%40mail.gmail.com.

Re: [sqlalchemy] SQL expression object expected, got object of type instead

[Simon King]

Can you show the part of *your* code that is triggering the error, and
explain what you are trying to do? Plain python properties aren't
normally very useful when accessed via a class. "Genome.attributes"
returns a property object, not the return value from the function, and
I don't understand what you are trying to do with it.

Thanks,

Simon

On Mon, Jan 24, 2022 at 2:03 AM Simon  wrote:
>
> Hi there,
>
> I got a problem about 'sqlalchemy.exc.ArgumentError: SQL expression object 
> expected, got object of type  instead'
>
> My SQLAlchemy version is 1.3.22. I have a database like
>
> Class Genome:
> id = Column(Integer, primary_key=True)
> created_date = Column(Datetime, nullable=False)
>
> @property
> def attributes(self):
>  return "something"
>
>
> If using the query, it reports an error through the elements.py in the 
> sqlAlchemy
>
> def _literal_as(element, text_fallback):
> if isinstance(element, Visitable):
> return element
> elif hasattr(element, "__clause_element__"):
> return element.__clause_element__()
> elif isinstance(element, util.string_types):
> return text_fallback(element)
> elif isinstance(element, (util.NoneType, bool)):
> return _const_expr(element)
> else:
> raise exc.ArgumentError(
> "SQL expression object expected, got object of type %r "
> "instead" % type(element)
> )
>
> This exception is not raised if I directly query genome's column name such as 
> created_date or id.
>
> I am wondering 1) could the column name and property be used interchangeably 
> in some way? Or say how could to query a table's property in the way of 
> querying a table's column? 2) what are some significant differences between 
> table's column name and property?
>
> Thanks.
>
>
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/7f03c9bb-9324-4e3e-8aea-cd0d46f9021bn%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfKm%2Bx8AMg8de%2BXx5CQ6AM62%2BWfth3v%2BgGB8LzE9SbK%3Dw%40mail.gmail.com.

Re: [sqlalchemy] Re: How to add the index_elements to the on_conflict_do_update() method

[Simon King]

Does the table definition in postgres match your SQLAlchemy
definition? Adding "unique=True" to the SQLAlchemy table definition
will not automatically add an index to an existing table in the
database.

If you connect to the database using "psql" and run "\d
message_symbol", does it show the primary key and/or unique index?

Simon

On Tue, Dec 7, 2021 at 10:27 PM Chaozy Z  wrote:
>
> I also tried to add unique=True to the column message_id but still fail with 
> the same error
>
> On Tuesday, 7 December 2021 at 22:21:49 UTC Chaozy Z wrote:
>>
>> Hi there, I just started to learn SQLAlchemy. 0
>>
>> I have a `on_conflict_do_update` command as follows:
>>
>> ```
>>
>> insert_stmt = insert(MessageSymbol).values(message_id=12345, symbol_id=1)
>>
>> do_update_stmt = insert_stmt.on_conflict_do_update( 
>> index_elements=['message_id'], set_=dict( symbol_id=123 ) )
>>
>> ```
>>
>> and my MessageSymbol is defined as follow:
>>
>> ```
>>
>> class MessageSymbol(Base):
>>
>>  __tablename__ = "message_symbol"
>>
>>
>> message_id = Column(BigInteger, primary_key=True, nullable=False)
>>
>>
>> symbol_id = Column(BigInteger, nullable=False)
>>
>> ```
>>
>> When the command is executed it throws the error:
>>
>> ```
>>
>> sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidColumnReference) 
>> there is no unique or exclusion constraint matching the ON CONFLICT 
>> specification
>>
>> [SQL: INSERT INTO message_symbol (message_id, symbol_id) VALUES 
>> (%(message_id)s, %(symbol_id)s) ON CONFLICT (message_id) DO UPDATE SET 
>> symbol_id = %(param_1)s]
>> [parameters: {'message_id': 12345, 'symbol_id': 1, 'param_1': 123}]
>>
>> ```
>>
>> Since I have defined the `message_id` as the primary key I assume it should 
>> be a unique constraint. I am wondering what else is being missing?
>>
>> Chaozy
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/734ae718-fde8-4b54-9de9-b2d81698a0dfn%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexem_JXPsNS_1_rqCPb1FS8MFLViMt0C3q_1gctD7SFrOw%40mail.gmail.com.

Re: [sqlalchemy] raise error on insert/update PK?

[Simon King]

You ought to be able to use the "sqlalchemy.func" system:

https://docs.sqlalchemy.org/en/14/core/tutorial.html#functions

server_default=sa.func.gen_random_uuid()

Hope that helps,

Simon

On Fri, Nov 19, 2021 at 6:21 AM jens.t...@gmail.com
 wrote:
>
> Tim,
>
> I wanted to offload the UUID generation for the PK to the server 
> (server_default instead of just default argument). But I wasn’t able to find 
> gen_random_uuid() documented in the PostgreSQL dialect, should probably be 
> under Column Valued Functions?
>
> So I ended up using server_default=text("gen_random_uuid()")
>
> Is that the (currently) recommended way? It seems to work…
>
> Thanks!
> Jens
>
> On Thursday, January 4, 2018 at 7:50:21 AM UTC+10 timc...@gmail.com wrote:
>>
>> Thanks so much! Really appreciate the example.
>>
>>
>>
>>
>> On Wednesday, January 3, 2018 at 3:46:47 PM UTC-5, Mike Bayer wrote:
>>>
>>> On Wed, Jan 3, 2018 at 1:18 PM, Tim Chen  wrote:
>>> > Let's say I'm using a uuid PK for my models with a ` server_default` set 
>>> > to
>>> > `gen_random_uuid()` (in PostgreSQL).  Is there a way to ensure the
>>> > server_default value?  I would like to catch any INSERT or UPDATE 
>>> > statements
>>> > that set the PK value and raise an error if possible.
>>>
>>> Sure, I would use a before_cursor_execute() handler that does this.
>>> You can see the parameters and the statement coming in and raise an
>>> error if they have column values you don't want to see.
>>>
>>> http://docs.sqlalchemy.org/en/latest/core/events.html#sqlalchemy.events.ConnectionEvents.before_cursor_execute
>>>
>>> this is cleanest using some of the internal state of the context so
>>> here is a demo:
>>>
>>> from sqlalchemy import *
>>> from sqlalchemy.orm import *
>>> from sqlalchemy import event
>>> from sqlalchemy.ext.declarative import declarative_base
>>>
>>> Base = declarative_base()
>>>
>>>
>>> class A(Base):
>>> __tablename__ = 'a'
>>> id = Column(Integer, primary_key=True)
>>> x = Column(Integer)
>>>
>>> e = create_engine("sqlite://", echo=True)
>>>
>>>
>>> @event.listens_for(e, "before_cursor_execute")
>>> def receive_before_cursor_execute(conn, cursor, statement, parameters,
>>> context, executemany):
>>> if context.isinsert:
>>> table = context.compiled.statement.table
>>> for col in table.primary_key:
>>> if col.key in context.compiled.binds:
>>> raise TypeError("no pk allowed!")
>>>
>>>
>>> Base.metadata.create_all(e)
>>>
>>> s = Session(e)
>>> s.add(A(x=5))
>>> s.commit()  # OK
>>>
>>> s.add(A(id=2, x=7))
>>> s.commit() # not OK
>>>
>>>
>>>
>>>
>>>
>>>
>>> >
>>> > --
>>> > SQLAlchemy -
>>> > The Python SQL Toolkit and Object Relational Mapper
>>> >
>>> > http://www.sqlalchemy.org/
>>> >
>>> > To post example code, please provide an MCVE: Minimal, Complete, and
>>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>> > description.
>>> > ---
>>> > You received this message because you are subscribed to the Google Groups
>>> > "sqlalchemy" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send an
>>> > email to sqlalchemy+...@googlegroups.com.
>>> > To post to this group, send email to sqlal...@googlegroups.com.
>>> > Visit this group at https://groups.google.com/group/sqlalchemy.
>>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/96f5ba79-1dd5-487f-b493-32f0144289f4n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfO7aCbih4%2BWOPdScCsm5t99gVR1yXUJ-iCz65yurdCCg%40mail.gmail.com.

Re: [sqlalchemy] Re: SqlAlchemy with Postgres: How can I make a query that checks if a string is in a list inside a json column

[Simon King]

For what it's worth, I think the "?" operator would work for this with
JSONB, but not with JSON:

postgres=# select '["user1", "user2"]'::jsonb ? 'user1';
?column?
--
 t
(1 row)

postgres=# select '["user1", "user2"]'::jsonb ? 'user2';
?column?
--
 t
(1 row)

postgres=# select '["user1", "user2"]'::jsonb ? 'user3';
?column?
--
 f
(1 row)


SQLAlchemy surfaces the "?" operator as .has_key

https://docs.sqlalchemy.org/en/14/dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSONB.Comparator.has_key

https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/dialects/postgresql/json.py#L33

Simon


On Mon, Oct 18, 2021 at 10:35 PM Jonathan Vanasco  wrote:
>
> I'm not sure, but AFAIK, this type of search isn't *easily* doable in 
> PostgreSQL. The json and jsonb operators and functions are really targeting 
> "object literals" style data, not lists.
>
> https://www.postgresql.org/docs/current/functions-json.html
>
> In the past, I think one could search against the column like text and 
> match/regex out a list value like `"user1"` - but that didn't work right.
>
> This type of search is possible with advanced PostgreSQL queries, by using 
> the functions like json_array_elements on a field and joining against that. 
> That's really not within the scope of SQLAlchemy or this list though, and 
> you'll have better luck search (or asking) on Stack Overflow.  There are a 
> handful of questions and solutions there on this topic.
>
> Once you can figure out the PostgreSQL queries to accomplish what you want, 
> this list can help you convert it to SQLAlchemy if you have trouble.
>
> On Wednesday, October 13, 2021 at 9:50:16 AM UTC-4 chat...@gmail.com wrote:
>>
>> Imagine a Postgres JSON column with values like below:
>>
>> "["user1", "user2"]"
>>
>> Is there any way to query a postgres JSON (not JSONB) column with 
>> SqlAlchemy,like above that checks if the value "user1" is contained in this 
>> column?
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/8f1986f6-4a39-4cad-93f2-a8d1c392b4b2n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfnFazzHYEgM%2BKt9QqcHM-HQFWkpjofvih%3Dm5GNL5G4dQ%40mail.gmail.com.

Re: [sqlalchemy] Using SQLAlchemy to check if column is in numeric ranges

[Simon King]

You want a combination of the "between" function/method:

https://docs.sqlalchemy.org/en/14/core/sqlelement.html#sqlalchemy.sql.expression.between

...and the "or_" function:

https://docs.sqlalchemy.org/en/14/core/sqlelement.html#sqlalchemy.sql.expression.or_

Something like this:

ranges = [(18, 25), (40, 55), (60, 70)]
conditions = [User.age.between(lower, upper) for (lower, upper) in ranges]
condition = sqlalchemy.or_(*conditions)
users = session.query(User).filter(condition).all()

Hope that helps,

Simon

On Tue, Aug 31, 2021 at 10:07 AM chat...@gmail.com  wrote:
>
> Hello All!! , I have a list of age ranges i.e 18-25, 40-55 and more how can I 
> make make a query using SQLAlchemy that will check if a Column is in ranges 
> [18,25] OR [40-55] OR [60-70]
> Regards,Christos
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/e6a8019c-beb4-482a-9262-b752391a16f4n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexd1zOhzkGiavjzPODr9wWr8KxhY4q6mSOVFN1UT1LE%3Daw%40mail.gmail.com.

Re: [sqlalchemy] Join multiple tables with association tables

[Simon King]

It's difficult to tell from your code what your intention is. Is the
relationship between Fact and Info meant to be many-to-many? And likewise
the relationship between Text and Info?

Forgetting SQLAlchemy for a moment, what is the SQL that you want to
produce?


Does the script below do what you want?


import sqlalchemy as sa
import sqlalchemy.orm as saorm
from sqlalchemy.ext.declarative import declarative_base


Base = declarative_base()


facts_info = sa.Table(
"facts_info",
Base.metadata,
sa.Column(
"fact_id", sa.Integer, sa.ForeignKey("fact.id"), primary_key=True
),
sa.Column(
"info_id", sa.Integer, sa.ForeignKey("info.id"), primary_key=True
),
)


info_text = sa.Table(
"info_text",
Base.metadata,
sa.Column(
"info_id", sa.Integer, sa.ForeignKey("info.id"), primary_key=True
),
sa.Column(
"text_id", sa.Integer, sa.ForeignKey("text.id"), primary_key=True
),
)


class Fact(Base):
__tablename__ = "fact"

id = sa.Column(sa.Integer, primary_key=True)
fact = sa.Column(sa.String(500), nullable=False, unique=True)
created_at = sa.Column(sa.DateTime)
updated_at = sa.Column(sa.DateTime)

info = saorm.relationship(
"Info", secondary=facts_info, back_populates="facts"
)


class Info(Base):
__tablename__ = "info"

id = sa.Column(sa.Integer, primary_key=True)
filename = sa.Column(sa.String(50))
format = sa.Column(sa.String(10))

facts = saorm.relationship(
"Fact", secondary=facts_info, back_populates="info"
)
text = saorm.relationship(
"Text", secondary=info_text, back_populates="info"
)


class Text(Base):
__tablename__ = "text"

id = sa.Column(sa.Integer, primary_key=True)
text = sa.Column(sa.String(1000))

# Relationships
info = saorm.relationship(
"Info", secondary=info_text, back_populates="text"
)


if __name__ == "__main__":
engine = sa.create_engine("sqlite://", echo=True)
Base.metadata.create_all(engine)

Session = saorm.sessionmaker(bind=engine)

session = Session()

# two facts
facts = [Fact(fact="factone"), Fact(fact="facttwo")]
# three infos, first two are associated with both facts, third is
# only linked to second fact
infos = [
Info(filename="infoone", facts=facts),
Info(filename="infotwo", facts=facts),
Info(filename="infothree", facts=facts[1:]),
]
# three texts, first two linked to first info instance, third
# linked to third info instance
texts = [
Text(text="textone", info=[infos[0]]),
Text(text="texttwo", info=[infos[0]]),
Text(text="textthree", info=[infos[2]]),
]
session.add_all(facts + infos + texts)
session.flush()

# Joining to both facts_info and info_text in the same query
# doesn't really make sense, because it would end up producing a
# cartesian product between those tables. Instead we'll use a
# subquery against facts_info to select the info ids we are
# interested in.
info_ids = (
session.query(facts_info.c.info_id)
.filter(facts_info.c.fact_id == 1)
)
query = (
session.query(Info, Text)
.filter(Info.id.in_(info_ids))
.join(Info.text)
)

# Note that this only outputs Info objects that have at least one
# text object associated with them. If you want to include Info
# objects without a related Text object, change the
# ".join(Info.text)" to ".outerjoin(Info.text)"
for (info, text) in query.all():
print("Info(filename=%r) Text(text=%r)" % (info.filename,
text.text))



Hope that helps,

Simon


On Mon, Aug 9, 2021 at 10:48 PM 'timbecks' via sqlalchemy <
sqlalchemy@googlegroups.com> wrote:

> I am trying to figure out the correct join query setup within SQLAlchemy,
> but I can't seem to get my head around it.
>
> I have the following table setup (simplified, I left out the non-essential
> fields):
>
> [image: Unbenannt.png]
>
> [image: Unbenannt2.png]
>
> The facts are associated to info, info is associated to text. Text and
> facts aren't directly associated.
>
> I would like to join them all together but can't figure out to do so.
>
> In this example I would like to get all instaces of "Info" that are
> associated to Fact.id = 1 and all "Text" instances that are associated to
> that "Info" instance. I came up with
>  select(Info, Text)
>   .join(facts_info)
>   .join(Facts)
>   .join(info_text)
>   .join(Text)
>   here(Facts.id ==1)
>
> But it obviously gives me an error.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.

Re: [sqlalchemy] prevent (raise exceptions) on bytestring values for non-byte types

[Simon King]

I can think of a couple of options:

1. Create a TypeDecorator for String and Text columns that raises an
error if it sees a bytestring. This will only flag the error when the
session is flushed.
2. Listen for mapper_configured events, iterate over the mapper
properties and add an "AttributeEvents.set" listener for each one.
This should flag the error when a bytestring is assigned to a mapped
attribute.

Hope that helps,

Simon

On Fri, Jul 30, 2021 at 5:10 PM 'Jonathan Vanasco' via sqlalchemy
 wrote:
>
> Mike, thanks for replying but go back to vacation.
>
> Anyone else: I am thinking more about an event that can be used to catch, 
> perhaps log, all bytes that go in.  I only use a few column classes that 
> expect bytestrings, but many that do not.  I've gotten every known bug so 
> far, but I'd like to make sure I'm not just lucky.
>
> On Thursday, July 29, 2021 at 6:05:03 PM UTC-4 Mike Bayer wrote:
>>
>> The Unicode datatype will emit a warning if you pass it a bytestring.  you 
>> can use that instead of String, or use a datatype with your own assertions 
>> based on 
>> https://docs.sqlalchemy.org/en/14/core/custom_types.html#coercing-encoded-strings-to-unicode
>>
>>
>>
>> On Thu, Jul 29, 2021, at 5:17 PM, 'Jonathan Vanasco' via sqlalchemy wrote:
>>
>> I am finally at the tail end of migrating my largest (and hopefully last) 
>> Python2 application to Python3.
>>
>> An issue that has popped up a lot during this transition, is when a py3 
>> bytestring gets submitted into SqlAlchemy.
>>
>> When that happens, it looks like SqlAlchemy just passes the value into 
>> psycopg2, which wraps it in an object, and I get a psycopg exception that 
>> bubbles up to SqlAlchemy:
>>
>> >sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) 
>> > operator does not exist: character varying = bytea
>> >LINE 3: WHERE foo = '\x626337323133...
>> >HINT: No operator matches the given name and argument type(s). You 
>> > might need to add explicit type casts.
>> >
>> >WHERE foo = %(foo)s
>> >LIMIT %(param_1)s]
>> >[parameters: {'foo': > > 0x10fe99060>, 'param_1': 1}]
>> >(Background on this error at: http://sqlalche.me/e/13/f405)
>>
>> Is there an easy way to catch this in SQLAlchemy *before* sending this to 
>> the driver and executing it on the server?  I'd like to ensure I'm catching 
>> everything I should, and nothing is working just by-chance.
>>
>>
>>
>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> description.
>> ---
>> You received this message because you are subscribed to the Google Groups 
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to sqlalchemy+...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sqlalchemy/f70bf020-d120-46fb-96d1-d5509ff9b3c3n%40googlegroups.com.
>>
>>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/d6f8d50c-9465-41bc-a293-d8295c35ecc1n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexcivJM8RxyX6cJKQRBd%3D4cB%2BZ9QtiiwmrKw0MEM%3DFLhuQ%40mail.gmail.com.

Re: [sqlalchemy] In the onupdate function, how to get the value of the row of records to be updated

[Simon King]

I don't think you'll be able to get what you want in an onupdate
function. You'd probably be better off with the before_insert and
before_update mapper events:

https://docs.sqlalchemy.org/en/14/orm/events.html#sqlalchemy.orm.MapperEvents.before_insert

https://docs.sqlalchemy.org/en/14/orm/events.html#sqlalchemy.orm.MapperEvents.before_update

Hope that helps,

Simon

On Sun, Jun 20, 2021 at 5:37 PM peizhi qin  wrote:
>
> I have a table ,the definition is as follows：
> class TEnterprise(Base):
> __tablename__ = 't_enterprise'
> __table_args__ = (
> {'mysql_engine': 'InnoDB', 'mysql_charset': 'latin1'}
> )
>
> Fent_id = Column(String(32), primary_key=True)
> Fent_type = Column(INTEGER(11), nullable=False)
> Fcomb_cert_state = Column(INTEGER(11), nullable=False)
> Fstate = Column(INTEGER(11), nullable=False)
> Fbusiness_license_copy = Column(String(256), nullable=False)
> Fbusiness_license_number = Column(String(32), nullable=False)
> Fmerchant_name = Column(String(128), nullable=False)
> Fsign = Column(String(64), nullable=False, server_default=text("''"))
>
> the Fsign=md5(Fcomb_cert_state=${Fcomb_cert_state}_id=${Fent_id})
>
> i can use  context.current_parameters get the value of the field to be 
> updated, but how to get the value of the field that has not been updated, 
> thanks for help!
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/bc9a6c71-90dd-4c7a-a5a8-9712f43d24f6n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfwaB5PUdvYj9sGukhtM36iWTc%2B41pG1vMdmV5dYLuVJA%40mail.gmail.com.

Re: [sqlalchemy] Revert multiple commits using a savepoint

[Simon King]

Which version of SQLAlchemy are you using, and how are you creating
your engine? I believe savepoints are handled differently in SA 1.4 if
you are using the "future-style" engine.

Do these doc links help you at all?

https://docs.sqlalchemy.org/en/14/orm/session_transaction.html#nested-transaction

"""
Calling the Session.commit() or Connection.commit() methods will
always commit the outermost transaction; this is a SQLAlchemy 2.0
specific behavior that is reversed from the 1.x series.
"""

https://docs.sqlalchemy.org/en/14/orm/session_transaction.html#session-subtransactions

"""
The “subtransaction” pattern that was often used with autocommit mode
is also deprecated in 1.4. This pattern allowed the use of the
Session.begin() method when a transaction were already begun,
resulting in a construct called a “subtransaction”, which was
essentially a block that would prevent the Session.commit() method
from actually committing.
"""

Simon

On Sat, Jun 19, 2021 at 4:27 PM kaboo HD  wrote:
>
>
> I am trying to test an endpoint in flask and I need to "refresh" the DB after 
> some commits.
>
> The idea I had was something like :
>
> db=SQLAlchemy()
>
> db.session.begin_nested()
>
> db.session.add(Obj)
> db.session.commit()
>
> # some code where last commit is important
>
> db.session.add(Obj2)
> db.session.commit()
>
> # some other code where last commit is important
>
> db.session.rollback()
>
> But it doesn't work unless I have a single commit.
>
> Can you help me please :)
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/d77d4621-8f0a-47fd-8e79-ca3de3a4b5edn%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexf7%2Bw7aiSHgcDR5_xenff_m08LTPp%3DyHgGvO-CZu7DREg%40mail.gmail.com.

Re: [sqlalchemy] Common datetime call

[Simon King]

On Fri, Jun 18, 2021 at 12:21 AM jca...@gmail.com  wrote:
>
> Hi,
> Does a means exist to generically call a local datetime func such that it 
> renders as SYSDATE in Oracle and GETDATE() in SQL Server?
>
> Thanks,
> jlc
>

Do you need those functions explicitly? I think both databases support
the CURRENT_TIMESTAMP function (which you'd access as
sqlalchemy.func.current_timestamp()).

If you really do want to use different functions based on the
database, the compiler extension is what you want:

https://docs.sqlalchemy.org/en/14/core/compiler.html#utc-timestamp-function

Hope that helps,

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexduw0SbGTZw09sMH457Eqf1zWQ8hs967HoBrTV_0R6g0Q%40mail.gmail.com.

Re: [sqlalchemy] versioned_history example uses deprecated Column.copy() method

[Simon King]

Thanks Mike, I'll see what I can do.

Simon

On Wed, Jun 16, 2021 at 1:26 PM Mike Bayer  wrote:
>
> HI Simon -
>
> I believe that example for now should vendor its own "copy()" function that 
> does what's needed.   the function that's there is already un-doing some of 
> the work of the old copy() method in any case.  I think for history table we 
> need column name, datatype, and maybe nullable constraint.
>
> We can accept PRs for this if this is something you would be interested in, 
> although your contributions to the SQLA mailing list are already many :)
>
>
>
> On Wed, Jun 16, 2021, at 7:15 AM, Simon King wrote:
>
> Hi all,
>
> I'm updating an app from SA 1.3 to 1.4 and getting a SADeprecationWarning:
>
> The Column.copy() method is deprecated and will be removed in a
> future release. (deprecated since: 1.4)
>
> The code triggering the warning is based on the versioned_history example:
>
> https://docs.sqlalchemy.org/en/14/_modules/examples/versioned_history/history_meta.html
>
> ...and here's the offending function:
>
> def _col_copy(col):
> orig = col
> col = col.copy()
> orig.info["history_copy"] = col
> col.unique = False
> col.default = col.server_default = None
> col.autoincrement = False
> return col
>
> For the moment I've switched to calling the private _copy() method
> instead, but is there any recommendation of a better approach?
>
> (I did see the issue at
> https://github.com/sqlalchemy/sqlalchemy/issues/5953 and understand
> why the copy() method was deprecated)
>
> Thanks a lot,
>
> Simon
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example.  See  http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdUVb11FTq%3DzxuzNsp-FszVHrJ8KT-yDjhyNkWqhKoJnQ%40mail.gmail.com.
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/46661334-a2b9-4548-9003-b487b725615d%40www.fastmail.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexf42bqjXmG9TQPd%2B0h%2BZYZAoFHnxk-ZvN2OH0MiyLRJrw%40mail.gmail.com.

[sqlalchemy] versioned_history example uses deprecated Column.copy() method

[Simon King]

Hi all,

I'm updating an app from SA 1.3 to 1.4 and getting a SADeprecationWarning:

The Column.copy() method is deprecated and will be removed in a
future release. (deprecated since: 1.4)

The code triggering the warning is based on the versioned_history example:

https://docs.sqlalchemy.org/en/14/_modules/examples/versioned_history/history_meta.html

...and here's the offending function:

def _col_copy(col):
orig = col
col = col.copy()
orig.info["history_copy"] = col
col.unique = False
col.default = col.server_default = None
col.autoincrement = False
return col

For the moment I've switched to calling the private _copy() method
instead, but is there any recommendation of a better approach?

(I did see the issue at
https://github.com/sqlalchemy/sqlalchemy/issues/5953 and understand
why the copy() method was deprecated)

Thanks a lot,

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdUVb11FTq%3DzxuzNsp-FszVHrJ8KT-yDjhyNkWqhKoJnQ%40mail.gmail.com.

Re: [sqlalchemy] checking in

[Simon King]

You can see the archives at https://groups.google.com/g/sqlalchemy to
get an idea of the traffic.

Simon

On Mon, Jun 14, 2021 at 10:25 PM Rich Shepard  wrote:
>
> I've not worked with SQLAlchemy for several years but now want to use it in
> a couple of applications. I've not seen messages on this maillist for a very
> long time so I tried subscribing and learned that I'm still subscribed.
>
> Am I the only one on this list now?
>
> If not I wonder why messages aren't arriving in my INBOX.
>
> Rich
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example.  See  http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/alpine.LNX.2.20.2106141423470.11603%40salmo.appl-ecosys.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexc2WRWhb3LZEymfeHypSYv%3DeOQOmC5%2BtNdm%2B1x3Xw%2BBFA%40mail.gmail.com.

Re: [sqlalchemy] Postgresql JSON object update.

[Simon King]

You can use the "op" method:

https://docs.sqlalchemy.org/en/14/core/sqlelement.html#sqlalchemy.sql.expression.Operators.op

some_column.op("||")(other_column)

Hope that helps,

Simon

On Mon, May 24, 2021 at 4:12 PM Massimiliano della Rovere
 wrote:
>
> In postgresql the || operator is the only way (no, the concat() function 
> doesn't work) to concat 2 JSONB dicts; note that this works only with JSONB 
> and not JSON.
>
> Example:
>
> suppose column "t.c" contains '{"a": 1}'::jsonb
>
> SELECT t.c || jsonb_build_object('b', 2);
>
> gives
>
>  {"a": 1, "b": 2}
>
> Question: How can I obtain the || operator in sqlalchemy (core, 1.3) to merge 
> 2 JSONB dicts?
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/CADKhPGRmKtTvZa4jpWby67v7fO-xbO31oJm%3D8jpb%2BrKS5_t2WQ%40mail.gmail.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdnftgwEef9pOZ5UK3VTgLLziFTY%2B%2BRTbz1bgk56-3SyQ%40mail.gmail.com.

Re: [sqlalchemy] How to use user defined python function inside a sqlalchemy filter?

[Simon King]

The point of hybrid attributes is to allow you to construct a property
that can be evaluated against an instance (in which case it is "normal
python"), or against a class (in which case it needs to return an SQL
expression). *Sometimes* the same python code can work in both
contexts, but only if it is a very simple expression.

Here's your example:

@hybrid_property
def is_id_valid(self):
# some logic here
if self.id % 3 == 0:
return True
else:
return False

If you had already retrieved an instance of GeneralBeqReq from the
database, you could access its "is_id_valid property" and it would
work as expected, because "self.id" would be a normal python integer.
But if you try to access it from the class (ie.
GeneralBeqReq.is_id_valid), it won't do what you expect. "self" will
actually be the class, GeneralBeqReq. "self.id" will be a special
SQLAlchemy "InstrumentedAttribute" object. It happens that
InstrumentedAttributes do support the % operator, returning an SQL
expression object, so "self.id % 3 == 0" will actually return an SQL
expression. But using it inside an "if" statement doesn't make any
sense - you're checking the boolean value of the SQL expression
object, not the result of evaluating it against any particular row. So
GeneralBeqReq.is_id_valid is just going to return a constant (either
True or False, not sure which).

You'd need to add an alternative definition of the is_id_valid
property which returns an SQL expression:

https://docs.sqlalchemy.org/en/14/orm/extensions/hybrid.html#defining-expression-behavior-distinct-from-attribute-behavior

If you can't express your validation function as an SQL expression,
you can't pass it to Query. You'd have to postprocess the query
results in python instead, like this:

a = session.query(GeneralBeqReq)\
.filter(GeneralBeqReq.c.BeqReqStatus == 1)\
.all()
a = [obj for obj in a if obj.is_id_valid]

Simon

On Wed, May 5, 2021 at 4:28 PM Yaakov Bressler  wrote:
>
> Query would be modified to the following:
>
> a = session.query(GeneralBeqReq)\
> .filter(
> GeneralBeqReq.c.BeqReqStatus == 1, # this doesn't look right, but 
> whatevz
> GeneralBeqReq.is_id_valid() == True, # a bit redundant, but explicit 
> is better than implicit
> )\
> .all()
>
> On Wednesday, May 5, 2021 at 11:26:01 AM UTC-4 Yaakov Bressler wrote:
>>
>> Would it be wrong to assume that the desired function could be added as a 
>> hybrid attribute, then queried through the class obj?
>>
>> Example:
>>
>> from sqlalchemy.ext.hybrid import hybrid_property
>>
>> class GeneralBeqReq(Base):
>> ...
>> @hybrid_property
>> def is_id_valid(self):
>> # some logic here
>> if self.id % 3 == 0:
>> return True
>> else:
>> return False
>>
>> On Wednesday, April 28, 2021 at 5:01:32 AM UTC-4 Simon King wrote:
>>>
>>> Parameters that you pass to the Query.filter function are eventually
>>> going to be rendered into an SQL statement, so your is_id_valid
>>> function probably needs to return something built from SQLAlchemy's
>>> SQL expression language:
>>> https://docs.sqlalchemy.org/en/14/core/tutorial.html
>>>
>>> If you can explain the sort of validation that is_id_valid needs to
>>> do, we might be able to help more.
>>>
>>> Simon
>>>
>>>
>>> On Wed, Apr 28, 2021 at 6:33 AM Gyanaranjan Nayak  wrote:
>>> >
>>> > I have a function with name is_id_valid(id) which returns either True or 
>>> > False.
>>> >
>>> > I want to pass this function inside a sqlalchemy query inside the filter 
>>> > condition.
>>> > My query example is :
>>> >
>>> > a = session.query(GeneralBeqReq).filter(GeneralBeqReq.c.BeqReqStatus == 1,
>>> > is_id_valid (GeneralBeqReq.c.id) == True).all()
>>> >
>>> > When I run the above query It is throwing the following error.
>>> >
>>> > AttributeError: Neither 'Column' object nor 'Comparator' object has an 
>>> > attribute 'strip'
>>> >
>>> >
>>> > Can you please guide me how to use this function inside my query ?
>>> >
>>> >
>>> > --
>>> > SQLAlchemy -
>>> > The Python SQL Toolkit and Object Relational Mapper
>>> >
>>> > http://www.sqlalchemy.org/
>>> >
>>> > To post example code, please provide an MCVE: Minimal, Complete, and 
>>> > Verifiable Example. See http://stackoverflow.com/h

Re: [sqlalchemy] How to use user defined python function inside a sqlalchemy filter?

[Simon King]

Parameters that you pass to the Query.filter function are eventually
going to be rendered into an SQL statement, so your is_id_valid
function probably needs to return something built from SQLAlchemy's
SQL expression language:
https://docs.sqlalchemy.org/en/14/core/tutorial.html

If you can explain the sort of validation that is_id_valid needs to
do, we might be able to help more.

Simon


On Wed, Apr 28, 2021 at 6:33 AM Gyanaranjan Nayak  wrote:
>
> I have a function with name is_id_valid(id) which returns either True or 
> False.
>
> I want to pass this function inside a sqlalchemy query inside the filter 
> condition.
> My query example is :
>
> a = session.query(GeneralBeqReq).filter(GeneralBeqReq.c.BeqReqStatus == 1,
>is_id_valid (GeneralBeqReq.c.id) == True).all()
>
> When I run the above query It is throwing the following error.
>
> AttributeError: Neither 'Column' object nor 'Comparator' object has an 
> attribute 'strip'
>
>
> Can you please guide me how to use this function inside my query ?
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/f388cfb8-0b0d-4ab3-8e26-84c4eb91b4a9n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexctFbqaWWaSFG7QZ8kkX-Fr7N71NtBfb4FXYv8nkjfPwQ%40mail.gmail.com.

Re: Re[4]: [sqlalchemy] Invertinace mapped type_id to fix value for each child class

[Simon King]

I probably wouldn't use this:

if test_type == ChildClass1().typ_id:

...simply because creating an instance of the object just to get
access to the typ_id seems like a waste of effort. If you really need
to check integer typ_id values, the staticmethod approach seems fine.

Simon

On Mon, Apr 12, 2021 at 8:58 PM 'Sören Textor' via sqlalchemy
 wrote:
>
> Hi Simon
> Again you really helped me out. I don't know what point I missed, but
> now it works. As usual it's not as simpe le or lets say there are a lot
> more code pieces to change before I can really test it in my code. But I
> got it.
>
> just one more thing:
> I often have to check if a given tpye t the class type. Therefore I
> usally use the statci method.
>
> Thus what would you do:
>
>  if test_type == ChildClass1().typ_id:
> or
>  if test_type==ChildClass.TypID():
>
> and to ensure only TypId exists fpr that type:
>  __mapper_args__ = {
>  "polymorphic_identity": ChildClass.TypID(),
>  }
>
> And as I said: Thanks a lot!
>
> SirAnn
>
>
> -- Originalnachricht --
> Von: "Simon King" 
> An: sqlalchemy@googlegroups.com
> Gesendet: 12.04.2021 20:26:48
> Betreff: Re: Re[2]: [sqlalchemy] Invertinace mapped type_id to fix value
> for each child class
>
> >Here's a standalone working example:
> >
> >import sqlalchemy as sa
> >import sqlalchemy.orm as saorm
> >from sqlalchemy.ext.declarative import declarative_base
> >
> >
> >Base = declarative_base()
> >
> >
> >class Objekt(Base):
> > __tablename__ = "objekt"
> > id = sa.Column(sa.Integer, primary_key=True)
> > typ_id = sa.Column(sa.Integer, sa.ForeignKey("objekt_typ.id"))
> > typ = saorm.relationship("ObjektTyp")
> > name = sa.Column(sa.String(100))
> >
> > __mapper_args__ = {
> > "polymorphic_on": typ_id,
> > }
> >
> >
> >class ObjektTyp(Base):
> > __tablename__ = "objekt_typ"
> > id = sa.Column(sa.Integer, primary_key=True)
> > name = sa.Column(sa.String(100))
> >
> >
> >class ChildObjekt1(Objekt):
> > __tablename__ = "child_objekt1"
> > id = sa.Column(sa.Integer, sa.ForeignKey(Objekt.id), primary_key=True)
> > text = sa.Column(sa.String(255))
> >
> > __mapper_args__ = {
> > "polymorphic_identity": 1,
> > }
> >
> >
> >class ChildObjekt2(Objekt):
> > __tablename__ = "child_objekt2"
> > id = sa.Column(sa.Integer, sa.ForeignKey(Objekt.id), primary_key=True)
> > text = sa.Column(sa.String(255))
> >
> > __mapper_args__ = {
> > "polymorphic_identity": 2,
> > }
> >
> >
> >if __name__ == "__main__":
> > engine = sa.create_engine("sqlite://")
> > Base.metadata.create_all(bind=engine)
> > Session = saorm.sessionmaker(bind=engine)
> >
> > session = Session()
> > child1type = ObjektTyp(id=1, name="child1")
> > child2type = ObjektTyp(id=2, name="child1")
> >
> > child1 = ChildObjekt1(text="child 1 text")
> > child2 = ChildObjekt2(text="child 2 text")
> >
> > session.add_all([child1type, child2type, child1, child2])
> > session.flush()
> >
> > for obj in session.query(Objekt):
> > print(obj)
> >
> >
> >Simon
> >
> >On Mon, Apr 12, 2021 at 6:40 PM 'Sören Textor' via sqlalchemy
> > wrote:
> >>
> >>  class Objekt(db.Model):
> >>   __tablename__ = 'objekt'
> >>
> >>   def __init__(self,**kwargs):
> >>   super().__init__(**kwargs)
> >>
> >>   id = db.Column(db.Integer, primary_key=True)
> >>   typ_id = db.Column(db.Integer, db.ForeignKey('objekt_typ.id'))
> >>   typ= db.relationship("ObjektTyp")
> >>   name   = db.Column(db.String(100))
> >>
> >>   __mapper_args__ = {
> >>   'polymorphic_on': typ_id
> >>   }
> >>
> >>  class ChildObjekt1(Objekt):
> >>   __versioned__ = {}
> >>   __tablename__ = 'child_objekt1'
> >>
> >>   @staticmethod
> >>   def TypId():
> >>   return 7
> >>
> >>   # User fields
> >>   def __init__(self,**kwargs):
> >>   super().__init__(**kwargs)
> >>   #su

Re: Re[2]: [sqlalchemy] Invertinace mapped type_id to fix value for each child class

[Simon King]

Here's a standalone working example:

import sqlalchemy as sa
import sqlalchemy.orm as saorm
from sqlalchemy.ext.declarative import declarative_base


Base = declarative_base()


class Objekt(Base):
__tablename__ = "objekt"
id = sa.Column(sa.Integer, primary_key=True)
typ_id = sa.Column(sa.Integer, sa.ForeignKey("objekt_typ.id"))
typ = saorm.relationship("ObjektTyp")
name = sa.Column(sa.String(100))

__mapper_args__ = {
"polymorphic_on": typ_id,
}


class ObjektTyp(Base):
__tablename__ = "objekt_typ"
id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(100))


class ChildObjekt1(Objekt):
__tablename__ = "child_objekt1"
id = sa.Column(sa.Integer, sa.ForeignKey(Objekt.id), primary_key=True)
text = sa.Column(sa.String(255))

__mapper_args__ = {
"polymorphic_identity": 1,
}


class ChildObjekt2(Objekt):
__tablename__ = "child_objekt2"
id = sa.Column(sa.Integer, sa.ForeignKey(Objekt.id), primary_key=True)
text = sa.Column(sa.String(255))

__mapper_args__ = {
"polymorphic_identity": 2,
}


if __name__ == "__main__":
engine = sa.create_engine("sqlite://")
Base.metadata.create_all(bind=engine)
Session = saorm.sessionmaker(bind=engine)

session = Session()
child1type = ObjektTyp(id=1, name="child1")
child2type = ObjektTyp(id=2, name="child1")

child1 = ChildObjekt1(text="child 1 text")
child2 = ChildObjekt2(text="child 2 text")

session.add_all([child1type, child2type, child1, child2])
session.flush()

for obj in session.query(Objekt):
print(obj)


Simon

On Mon, Apr 12, 2021 at 6:40 PM 'Sören Textor' via sqlalchemy
 wrote:
>
> class Objekt(db.Model):
>  __tablename__ = 'objekt'
>
>  def __init__(self,**kwargs):
>  super().__init__(**kwargs)
>
>  id = db.Column(db.Integer, primary_key=True)
>  typ_id = db.Column(db.Integer, db.ForeignKey('objekt_typ.id'))
>  typ= db.relationship("ObjektTyp")
>  name   = db.Column(db.String(100))
>
>  __mapper_args__ = {
>  'polymorphic_on': typ_id
>  }
>
> class ChildObjekt1(Objekt):
>  __versioned__ = {}
>  __tablename__ = 'child_objekt1'
>
>  @staticmethod
>  def TypId():
>  return 7
>
>  # User fields
>  def __init__(self,**kwargs):
>  super().__init__(**kwargs)
>  #super().__init__(typ_id=ChildObjekt1.TypId(), **kwargs)
>
>  ###
>  id db.Column(db.Integer, db.ForeignKey('objekt.id'),
> primary_key=True)
>  text = db.Column(db.String(255 ), default='')
>
>  __mapper_args__ = {
>  'polymorphic_identity': 7,
>  }
>
>
> leads to:
> venv\lib\site-packages\sqlalchemy\orm\mapper.py", line 1542, in
> _configure_polymorphic_setter
>  self.polymorphic_on = self._props[self.polymorphic_on]
> KeyError: 'typ_id'
>
> raise exception
> sqlalchemy.exc.ArgumentError: Can't determine polymorphic_on value
> 'typ_id' - no attribute is mapped to this name.
>
> maybe i do something totally worg.. I am also using sql continuum
>
> -- Originalnachricht --
> Von: "Simon King" 
> An: sqlalchemy@googlegroups.com
> Gesendet: 12.04.2021 19:06:11
> Betreff: Re: [sqlalchemy] Invertinace mapped type_id to fix value for
> each child class
>
> >I don't understand this comment:
> >
> >>  I though on polymorphic_on, but I think that does not work because of the 
> >> fact that type_id ha a foreign key ...
> >
> >As far as I can tell, you ought to have this in the base class:
> >
> > __mapper_args__ = {
> > 'polymorphic_on': typ_id
> > }
> >
> >And this in the subclass:
> >
> > __mapper_args__ = {
> > 'polymorphic_identity': 7,
> > }
> >
> >...and you should get rid of the typ_id function and the
> >"Objekt.typ_id = ChildClass.typ_id" line.
> >
> >Does that work for you?
> >
> >Simon
> >
> >On Mon, Apr 12, 2021 at 5:18 PM 'Sören Textor' via sqlalchemy
> > wrote:
> >>
> >>  I run into a problem and don't know how to solve it.
> >>  The theory is very simple: I habe one base class table with name, id and 
> >> type column
> >>  The child class shall have a unique type_id (all child_class1 objekt 
> >> shall get type_id 7, all child_class2 objekts type_id = 8, ...)
> >>
> >>  How can I map the base class typ_id to an hard coded value for eahc class 
> >> typ

Re: [sqlalchemy] Invertinace mapped type_id to fix value for each child class

[Simon King]

I don't understand this comment:

> I though on polymorphic_on, but I think that does not work because of the 
> fact that type_id ha a foreign key ...

As far as I can tell, you ought to have this in the base class:

__mapper_args__ = {
'polymorphic_on': typ_id
}

And this in the subclass:

__mapper_args__ = {
'polymorphic_identity': 7,
}

...and you should get rid of the typ_id function and the
"Objekt.typ_id = ChildClass.typ_id" line.

Does that work for you?

Simon

On Mon, Apr 12, 2021 at 5:18 PM 'Sören Textor' via sqlalchemy
 wrote:
>
> I run into a problem and don't know how to solve it.
> The theory is very simple: I habe one base class table with name, id and type 
> column
> The child class shall have a unique type_id (all child_class1 objekt shall 
> get type_id 7, all child_class2 objekts type_id = 8, ...)
>
> How can I map the base class typ_id to an hard coded value for eahc class 
> type.
> My actual approach does not change the type_id-columns of Objekt and after 
> saving the objekt the column Objekt.type_id entry is always empty for all 
> entries :-(
>
> class Objekt(db.Model):
> __tablename__ = 'objekt'
>
> def __init__(self,**kwargs):
> super().__init__(**kwargs)
>
> id = db.Column(db.Integer, primary_key=True)
> typ_id = db.Column(db.Integer, db.ForeignKey('objekt_typ.id'))
> typ= db.relationship("ObjektTyp")
> name   = db.Column(db.String(100))
>
> class ChildClass1(Objekt):
> __tablename__ = 'child_class1'
>
> @staticmethod
> def typ_id():
> return 7
>
> def __init__(self,**kwargs):
> super().__init__(**kwargs)
> Objekt.typ_id = ChildClass1.typ_id() ### fix type
>
> id   = db.Column(db.Integer, db.ForeignKey('objekt.id'), primary_key=True)
> text = db.Column(db.String(255 ), default='')
>
> __mapper_args__ = {
> 'polymorphic_identity':'child_class1',
> }
>
>
> any ideas where to look? I though on polymorphic_on, but I think that does 
> not work because of the fact that type_id ha a foreign key ...
>
> SirAnn
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/ema56ad245-cad9-4096-8c55-9d75e8d52ea2%40textors-01.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeCQd6%2B5-O%3D1H8J8Zmfrg8vDsPmHjLg4PFY9RTySNdJ3w%40mail.gmail.com.

Re: [sqlalchemy] Injecting User info into _history table to track who performed the change

[Simon King]

I'm not 100% confident here (the history_meta code does some pretty
complicated stuff to deal with inheritance), but I'll give it a go:

1. You've adapted history_meta to add an extra column to history
tables, called "accountable".
2. You've got an inheritance hierarchy (CompoundAdministration
inherits from Process)
3. Both CompoundAdministration and Process use the Versioned mixin
4. This results in 2 new mapped classes, ProcessHistory and
CompoundAdministrationHistory.
5. CompoundAdministrationHistory inherits from ProcessHistory

Step 5 is the problem: CompoundAdministrationHistory inherits from
ProcessHistory, but both tables have an "accountable" column. Normally
that's a problem - when you set the
CompoundAdministrationHistory.accountable attribute, should SQLAlchemy
update the column in the compound_administration_history table, or
process_history, or both? SQLAlchemy defaults to updating both, but
warns you about the ambiguity.

In this case, you really do want that property to target both columns,
so your fix is correct:

properties["accountable"] = (table.c.accountable,) + tuple(
super_history_mapper.attrs.accountable.columns
)

This says that the "accountable" property should target the column on
the local (inheriting) table as well as whatever columns the parent
class was targeting.

You should test that when you modify a CompoundAdministration object,
you get a new row in both the compound_administration_history and the
process_history tables, and that the "accountable" column is set
correctly.

I hope that makes sense,

Simon

On Wed, Mar 24, 2021 at 2:25 PM JPLaverdure  wrote:
>
> Hi Simon,
>
> Thanks for pointing out the collision, it kinda flew under the radar !
> I renamed the column from "user" to "accountable" and but still got
>
> SAWarning:
> Implicitly combining column process_history.accountable with column 
> compound_administration_history.accountable under attribute 'accountable'.
> Please configure one or more attributes for these same-named columns 
> explicitly.
>
> As mentioned, these tables both also have a "changed" attribute, but did not 
> throw the warnings...
> Looking a bit further, I spotted this piece of code in history_meta:
> properties["changed"] = (table.c.changed,) + tuple(
> super_history_mapper.attrs.changed.columns
> )
> So I added:
> properties["accountable"] = (table.c.accountable,) + tuple(
> super_history_mapper.attrs.accountable.columns
> )
>
> And the warnings have disappeared.
>
> Could you explain what these instructions actually do ?
>
> Thanks
> On Wednesday, March 24, 2021 at 5:18:02 a.m. UTC-4 Simon King wrote:
>>
>> I think the warning message is slightly misleading, probably because
>> of the inheritance, but the fundamental problem is likely that your
>> "process" table already has a "user" column, and you're trying to add
>> a second "user" column in the process_history table, which can't work.
>>
>> If you use a different column name to store the user in the history
>> table, does the warning go away?
>>
>> Simon
>>
>> On Tue, Mar 23, 2021 at 7:17 PM JPLaverdure  wrote:
>> >
>> > It seems I lost my previous email.. Here it is again:
>> >
>> > Sure !
>> > Here are 2 classes for which the generated _history sister tables 
>> > (generated by history_meta.py) throw the warnings:
>> >
>> > The Parent class:
>> >
>> > class Process(Versioned, Base, UtilityMixin):
>> > __tablename__ = 'process'
>> > __table_args__ = {}
>> >
>> > id = Column(Integer, primary_key=True)
>> > discriminator = Column('type', String(64))
>> > timestamp = Column(DateTime, nullable=False)
>> > start_date = Column(Date, nullable=False)
>> > am_pm = Column(String(8))
>> > probable_end_date = Column(Date)
>> > end_date = Column(Date)
>> > user = Column(String(128), nullable=False)
>> > comments = Column(Text)
>> >
>> > __mapper_args__ = {'polymorphic_on': discriminator, 
>> > 'polymorphic_identity': 'process',
>> > 'order_by': [start_date.desc(), id.desc()]}
>> >
>> > protocol_id = Column(Integer, ForeignKey('protocol.id', 
>> > onupdate='cascade', ondelete='restrict'))
>> > process_type_id = Column(Integer, ForeignKey('process_type.id', 
>> > onupdate='cascade', ondelete='restrict'))
>> >
>> > protocol = relationship(Protocol, backref='processes', uselist=False)
>> > process_type = relationship(ProcessType, backref='processes', 
>> > uselist=False)
>> >
>

Re: [sqlalchemy] Injecting User info into _history table to track who performed the change

[Simon King]

I think the warning message is slightly misleading, probably because
of the inheritance, but the fundamental problem is likely that your
"process" table already has a "user" column, and you're trying to add
a second "user" column in the process_history table, which can't work.

If you use a different column name to store the user in the history
table, does the warning go away?

Simon

On Tue, Mar 23, 2021 at 7:17 PM JPLaverdure  wrote:
>
> It seems I lost my previous email.. Here it is again:
>
> Sure !
> Here are 2 classes for which the generated _history sister tables (generated 
> by history_meta.py) throw the warnings:
>
> The Parent class:
>
> class Process(Versioned, Base, UtilityMixin):
> __tablename__ = 'process'
> __table_args__ = {}
>
> id = Column(Integer, primary_key=True)
> discriminator = Column('type', String(64))
> timestamp = Column(DateTime, nullable=False)
> start_date = Column(Date, nullable=False)
> am_pm = Column(String(8))
> probable_end_date = Column(Date)
> end_date = Column(Date)
> user = Column(String(128), nullable=False)
> comments = Column(Text)
>
> __mapper_args__ = {'polymorphic_on': discriminator, 'polymorphic_identity': 
> 'process',
> 'order_by': [start_date.desc(), id.desc()]}
>
> protocol_id = Column(Integer, ForeignKey('protocol.id', onupdate='cascade', 
> ondelete='restrict'))
> process_type_id = Column(Integer, ForeignKey('process_type.id', 
> onupdate='cascade', ondelete='restrict'))
>
> protocol = relationship(Protocol, backref='processes', uselist=False)
> process_type = relationship(ProcessType, backref='processes', uselist=False)
>
> The Child class:
>
> class CompoundAdministration(Process):
> __tablename__ = 'compound_administration'
> __table_args__ = {}
> __mapper_args__ = {'polymorphic_identity': 'compound_admin'}
>
> id = Column(Integer, ForeignKey('process.id', onupdate='cascade', 
> ondelete='cascade'), primary_key=True)
> dose = Column(String(64))
> substance = Column(String(128))
> frequency = Column(String(64))
> duration = Column(String(64))
>
> route_id = Column(Integer, ForeignKey('administration_route.id', 
> onupdate='cascade', ondelete='restrict'))
> route = relationship(AdministrationRoute, uselist=False)
>
>
> As reminder, versioning was implemented using this recipe/example from SQLA:
> https://docs.sqlalchemy.org/en/13/_modules/examples/versioned_history/history_meta.html
>
> And here is the associated warning:
>
> SAWarning:
> Implicitly combining column process_history.user with column 
> compound_administration_history.user under attribute 'user'.
> Please configure one or more attributes for these same-named columns 
> explicitly.
>
> Thanks for your help resolving this,
>
> JP
> On Tuesday, March 23, 2021 at 6:24:03 a.m. UTC-4 Simon King wrote:
>>
>> Can you show us the mapping definitions that are triggering these warnings?
>>
>> On Mon, Mar 22, 2021 at 6:29 PM JPLaverdure  wrote:
>> >
>> > Hi,
>> >
>> > Thanks for your support guys.
>> >
>> > I've implemented logging the user's email in the _history tables by adding 
>> > this Column definition inside my history_meta.py file:
>> > cols.append(
>> > Column(
>> > "user",
>> > String,
>> > info=version_meta,
>> > )
>> > )
>> > but i'm running into a good load of SAWarnings stating that there is an 
>> > implicit combining of the "user" column taking place
>> > (I have multi-table inheritance setup for some entities, those are the 
>> > ones throwing the warning)
>> > I don't get why the column "changed" (which holds the timestamp of the 
>> > change) and is defined in exactly the same way does not generate these 
>> > warnings ?
>> > What configuration setting am I missing here ?
>> >
>> > I found this
>> > https://docs.sqlalchemy.org/en/13/faq/ormconfiguration.html#i-m-getting-a-warning-or-error-about-implicitly-combining-column-x-under-attribute-y
>> > But it doesn't seem to fit 100% with what I'm seeing inside history_meta.py
>> >
>> > Thanks !!
>> > On Monday, March 15, 2021 at 4:33:40 p.m. UTC-4 Jonathan Vanasco wrote:
>> >>
>> >> Going beyond what Simon did..
>> >>
>> >> I typically make make a table like `user_transaction`, which has all of 
>> >> the relevant information for the transaction:
>> >>
>> >> * User ID
>> >> * Timestamp
>> >> * Remote IP
>> >>
>> >> Using the sqlalchemy hooks, I'll the

Re: [sqlalchemy] Injecting User info into _history table to track who performed the change

[Simon King]

Can you show us the mapping definitions that are triggering these warnings?

On Mon, Mar 22, 2021 at 6:29 PM JPLaverdure  wrote:
>
> Hi,
>
> Thanks for your support guys.
>
> I've implemented logging the user's email in the _history tables by adding 
> this Column definition inside my history_meta.py file:
> cols.append(
>   Column(
> "user",
> String,
> info=version_meta,
>   )
> )
> but i'm running into a good load of SAWarnings stating that there is an 
> implicit combining of the "user" column taking place
> (I have multi-table inheritance setup for some entities, those are the ones 
> throwing the warning)
> I don't get why the column "changed" (which holds the timestamp of the 
> change) and is defined in exactly the same way does not generate these 
> warnings ?
> What configuration setting am I missing here ?
>
> I found this
> https://docs.sqlalchemy.org/en/13/faq/ormconfiguration.html#i-m-getting-a-warning-or-error-about-implicitly-combining-column-x-under-attribute-y
> But it doesn't seem to fit 100% with what I'm seeing inside history_meta.py
>
> Thanks !!
> On Monday, March 15, 2021 at 4:33:40 p.m. UTC-4 Jonathan Vanasco wrote:
>>
>> Going beyond what Simon did..
>>
>> I typically make make a table like `user_transaction`, which has all of the 
>> relevant information for the transaction:
>>
>> * User ID
>> * Timestamp
>> * Remote IP
>>
>> Using the sqlalchemy hooks, I'll then do something like:
>>
>> * update the object table with the user_transaction id
>> or
>> * use an association table that tracks a user_transaction_id to an object id 
>> and version
>>
>> FYI, Simon -- as of a few weeks ago, that pattern is now part of the pyramid 
>> sqlalchemy starter template!
>>
>> On Monday, March 15, 2021 at 6:46:02 AM UTC-4 Simon King wrote:
>>>
>>> I use pyramid as a web framework, and when I create the DB session for
>>> each request, I add a reference to the current request object to the
>>> DB session. The session object has an "info" attribute which is
>>> intended for application-specific things like this:
>>>
>>> https://docs.sqlalchemy.org/en/13/orm/session_api.html#sqlalchemy.orm.session.Session.info
>>>
>>> Then, in the before_flush event handler, I retrieve the request object
>>> from session.info, and then I can add whatever request-specific info I
>>> want to the DB.
>>>
>>> Simon
>>>
>>> On Sun, Mar 14, 2021 at 4:05 PM JPLaverdure  wrote:
>>> >
>>> > Hi Elmer,
>>> >
>>> > Thanks for your reply !
>>> > My issue is not with obtaining the info I want to inject (the logged in 
>>> > users's email), I already have that all ready to go :)
>>> >
>>> > My whole database is versioned using the history_meta.py example from 
>>> > SQLAlchemy
>>> > https://docs.sqlalchemy.org/en/13/_modules/examples/versioned_history/history_meta.html
>>> >
>>> > I was hoping for a simple way to inject the user info into the _history 
>>> > row creation steps.
>>> >
>>> > The SQLAlchemy example makes use of this event listener:
>>> >
>>> > def versioned_session(session):
>>> >
>>> > @event.listens_for(session, "before_flush")
>>> > def before_flush(session, flush_context, instances):
>>> > for obj in versioned_objects(session.dirty):
>>> > create_version(obj, session)
>>> > for obj in versioned_objects(session.deleted):
>>> > create_version(obj, session, deleted=True)
>>> >
>>> > So I'm tempted to follow the same strategy and just override this 
>>> > listener to supplement it with the user info but I'm wondering how to 
>>> > pass in non SQLAlchemy info into its execution context...
>>> >
>>> > So basically, I have the info I want to inject, I'm just not sure how to 
>>> > pass it to SQLAlchemy
>>> >
>>> > Thanks,
>>> >
>>> > JP
>>> >
>>> > On Friday, March 12, 2021 at 6:55:19 p.m. UTC-5 elmer@gmail.com wrote:
>>> >>
>>> >> Hi JP,
>>> >>
>>> >> Depending on how you've implemented your history tracking, that routine 
>>> >> is quite far removed from your web framework and getting a neat, clean 
>>> >> way of dealing with that might not be within reach.
>>> >>
>

Re: [sqlalchemy] sqlalchemy get table with a string

[Simon King]

OK, I see. You're creating tables dynamically, and you want to be able
to insert data into those tables.

I think it'll be easier to use SQLAlchemy Core for this, rather than
the ORM. You can use reflection to load table definitions from the
database.

https://docs.sqlalchemy.org/en/14/core/reflection.html#reflecting-database-objects
https://docs.sqlalchemy.org/en/14/core/tutorial.html#coretutorial-insert-expressions

That would look something like this (completely untested):

# if there's a chance multiple people can call this
# at the same time, you'll need to add a lock around it
def get_table(table_name):
   if table_name in meta.tables:
   return meta.tables[table_name]
   return Table(table_name, meta, autoload_with=db.session.connection())

table_name = request.form.get("varsayilanlar")
table = get_table(table_name)
insert = table.insert().values(**tablo)
db.session.execute(insert)

Simon

On Wed, Mar 17, 2021 at 4:19 PM FURKAN bilgin  wrote:
>
> I updated sqlalchemy and now I get an error when accessing the database. And 
> their codes need to be coded:
> What I really wanted to do was add data to a table with json data in the / 
> main / write path. but now I get an error when I send / main / new-app post 
> request (I updated sqlalchemy and it happened) related codes are below
> Sorry for taking your time, there may not be a solution. I am thinking of 
> rewriting the codes.
>
>
> from sqlalchemy import MetaData, Table, Column, Integer, String, 
> create_engine, DateTime,VARCHAR
> from flask import Flask, abort, render_template, request,session,jsonify
> from flask_sqlalchemy import SQLAlchemy
> from datetime import datetime
> from sqlalchemy import text
> import json
>
> meta = MetaData()
> engine = create_engine('sqlite:Users/dell/Desktop/2.2/db.db')
> conn = engine.connect()
>
> app = Flask(__name__)
>
> app.config["SQLALCHEMY_DATABASE_URI"] = 
> 'sqlite:Users/dell/Desktop/2.2/db.db'
> db = SQLAlchemy(app)
>
> app.secret_key = "Lbgsa,pdsa_ıda)6Kyw%61"
>
> @app.route("/main/new-app",methods = ["GET","POST"])
> def new_app():
> if request.method == "GET":
> return render_template("gg.html")
> if application.query.filter_by(name = 
> request.form.get("username")).first():
>return "UserName..."
> isim = request.form.get("username")
> varsayilanlar   = request.form.get("varsayilanlar")
> d_varsayilanlar = json.loads(varsayilanlar).keys()
> lis = [Column('id', Integer, primary_key = True),Column('date', DateTime, 
> nullable=False, default=datetime.now)]
> for i in d_varsayilanlar:
> lis.append(Column(i,VARCHAR(80)))
> Table(isim, meta, *lis)
> meta.create_all(engine)
> new_app = application(name = isim,
> password = request.form.get("password"),
> anaSayfa = request.form.get("anaSayfa"),
> manuelSayfa  = 
> request.form.get("manuelSayfa"),
> dinamikSayfa = 
> request.form.get("dinamikSayfa"),
> varsayilanlar= 
> request.form.get("varsayilanlar"),
> dinamik  = 
> bool(request.form.get("dinamik")),
> kitap= 
> bool(request.form.get("kitap")),
> kalem= 
> bool(request.form.get("kalem")))
> db.session.add(new_app)
> db.session.commit()
> return "200OK"
>
> @app.route("/main/write",methods = ["GET","POST"])
> def write():
> if request.method == "GET":
> return "POST!"
> app = application.query.filter_by(name = 
> request.form.get("username")).first()
> if not (app):
>return "Uygulama Bulunamadı"
> elif not(app.kalem) and (app.password != request.form.get("password")):
> return "TABLE NOT FOUND"
> tablo = json.loads(request.form.get("varsayilanlar"))
> #i want to get table here
> new_row = meta.tables[request.form.get("username")]
> employee = new_row(**tablo)
> db.session.add(employee)
> db.session.commit()
> return "200ok"
> class application(db.Model):
> id   = db.Column(db.Integer,primary_key = True)
> name = db.Column(db.String(80))
> password = db.Column(db.String(80))
> anaSayfa = db.Column(db.String(80))
&g

Re: [sqlalchemy] sqlalchemy get table with a string

[Simon King]

I assumed you were defining classes corresponding to your database
tables, as shown here:

https://docs.sqlalchemy.org/en/14/orm/tutorial.html#declare-a-mapping

If that's not how you're using SQLAlchemy, you'll have to show your code.

Simon

On Wed, Mar 17, 2021 at 2:07 PM FURKAN bilgin  wrote:
>
> I think we keep it in RAM in the first method, so it may be a problem if the 
> program is restarted. and I guess I don't understand what you mean by Base 
> class.
> 17 Mart 2021 Çarşamba tarihinde saat 14:27:31 UTC+3 itibarıyla Simon King 
> şunları yazdı:
>>
>> There are lots of ways of doing this. One option is to provide a
>> dictionary when creating your declarative_base:
>>
>> classes = {}
>> Base = declarative_base(class_registry=classes)
>>
>> Now you can look up classes by name in that classes dictionary:
>>
>> def get_table_by_name(name):
>> return classes[name]
>>
>> Another option could be to iterate over Base.__subclasses__:
>>
>> def get_table_by_name(name):
>> for cls in Base.__subclasses__():
>> if cls.__name__ == name:
>> return cls
>>
>> Hope that helps,
>>
>> Simon
>>
>> On Tue, Mar 16, 2021 at 7:14 PM FURKAN bilgin  wrote:
>> >
>> > table_name = "table_name"
>> >
>> > #get table as table
>> >
>> > new = table(**tablo)
>> > db.session.add(table)
>> > db.session.commit()
>> >
>> > --
>> > SQLAlchemy -
>> > The Python SQL Toolkit and Object Relational Mapper
>> >
>> > http://www.sqlalchemy.org/
>> >
>> > To post example code, please provide an MCVE: Minimal, Complete, and 
>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> > description.
>> > ---
>> > You received this message because you are subscribed to the Google Groups 
>> > "sqlalchemy" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to sqlalchemy+...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/sqlalchemy/c3c7c369-7d7f-41b0-b6f3-273b6c76314dn%40googlegroups.com.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/1f98d725-f15f-4fde-9fe2-4205f71eb1d8n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeUNOP-dUkpxem2dpkJNPs5XQTpmx9rgbbJ6-M45%2BvLRQ%40mail.gmail.com.

Re: [sqlalchemy] sqlalchemy get table with a string

[Simon King]

There are lots of ways of doing this. One option is to provide a
dictionary when creating your declarative_base:

classes = {}
Base = declarative_base(class_registry=classes)

Now you can look up classes by name in that classes dictionary:

def get_table_by_name(name):
return classes[name]

Another option could be to iterate over Base.__subclasses__:

def get_table_by_name(name):
for cls in Base.__subclasses__():
if cls.__name__ == name:
return cls

Hope that helps,

Simon

On Tue, Mar 16, 2021 at 7:14 PM FURKAN bilgin  wrote:
>
> table_name = "table_name"
>
> #get table as table
>
> new = table(**tablo)
> db.session.add(table)
> db.session.commit()
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/c3c7c369-7d7f-41b0-b6f3-273b6c76314dn%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfPY_VqEeD0Rurc_HDyw%2BnRFsOSGq_Oeo66ZgjBSnEkpg%40mail.gmail.com.

Re: [sqlalchemy] Injecting User info into _history table to track who performed the change

[Simon King]

I use pyramid as a web framework, and when I create the DB session for
each request, I add a reference to the current request object to the
DB session. The session object has an "info" attribute which is
intended for application-specific things like this:

https://docs.sqlalchemy.org/en/13/orm/session_api.html#sqlalchemy.orm.session.Session.info

Then, in the before_flush event handler, I retrieve the request object
from session.info, and then I can add whatever request-specific info I
want to the DB.

Simon

On Sun, Mar 14, 2021 at 4:05 PM JPLaverdure  wrote:
>
> Hi Elmer,
>
> Thanks for your reply !
> My issue is not with obtaining the info I want to inject (the logged in 
> users's email), I already have that all ready to go :)
>
> My whole database is versioned using the history_meta.py example from 
> SQLAlchemy
> https://docs.sqlalchemy.org/en/13/_modules/examples/versioned_history/history_meta.html
>
> I was hoping for a simple way to inject the user info into the _history row 
> creation steps.
>
> The SQLAlchemy example makes use of this event listener:
>
> def versioned_session(session):
>
> @event.listens_for(session, "before_flush")
> def before_flush(session, flush_context, instances):
> for obj in versioned_objects(session.dirty):
> create_version(obj, session)
>for obj in versioned_objects(session.deleted):
> create_version(obj, session, deleted=True)
>
> So I'm tempted to follow the same strategy and just override this listener to 
> supplement it with the user info but I'm wondering how to pass in non 
> SQLAlchemy info into its execution context...
>
> So basically, I have the info I want to inject, I'm just not sure how to pass 
> it to SQLAlchemy
>
> Thanks,
>
> JP
>
> On Friday, March 12, 2021 at 6:55:19 p.m. UTC-5 elmer@gmail.com wrote:
>>
>> Hi JP,
>>
>> Depending on how you've implemented your history tracking, that routine is 
>> quite far removed from your web framework and getting a neat, clean way of 
>> dealing with that might not be within reach.
>>
>> However, most web frameworks have some concept of a threadlocal request (or 
>> function to retrieve it), which you could invoke and if such a request 
>> exists, you could use that to load whatever user identity you have available 
>> on there (again, the details differ, but this tends to be a shared feature). 
>> From there you can store the user either as a foreign key, or a unique 
>> identifier like email. Which one you pick would depend on how you want the 
>> history to be affected when you delete a user record for example.
>>
>>
>>
>> On Fri, Mar 12, 2021 at 11:58 PM JPLaverdure  wrote:
>>>
>>> Hello everyone,
>>>
>>> We already have the ability to timestamp the creation of the history row, 
>>> but it would also be interesting to be able to track the user responsible 
>>> for the content update.
>>> I would like to get suggestions on the best way to achieve this.
>>>
>>> I realize this is somewhat outside the scope of sqlalchemy as the notion of 
>>> a "logged in user" is more closely related to the context of the app/webapp 
>>> using SQLAlchemy as its ORM but maybe other people would benefit from 
>>> having a way to inject arbitrary data in the history table.
>>>
>>> Ideally, I would like the insert in the _history table to be atomic, so I 
>>> feel like hooking an update statement to an event might not be the way to 
>>> go.
>>> I'm tempted to modify the signature of before_flush but I'm not sure where 
>>> it gets called.
>>>
>>> Any help is welcome !
>>> Thanks
>>>
>>> JP
>>>
>>> --
>>> SQLAlchemy -
>>> The Python SQL Toolkit and Object Relational Mapper
>>>
>>> http://www.sqlalchemy.org/
>>>
>>> To post example code, please provide an MCVE: Minimal, Complete, and 
>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>>> description.
>>> ---
>>> You received this message because you are subscribed to the Google Groups 
>>> "sqlalchemy" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to sqlalchemy+...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/sqlalchemy/82a24998-14e1-4ff4-a725-dd25c20a8bf2n%40googlegroups.com.
>>
>>
>>
>> --
>>
>> Elmer
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/58bb6713-18f4-4d69-8d7b-a27772711bd5n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object 

Re: [sqlalchemy] Create Sqlalchemy ORM class from regular class gets "has no attribute ''_sa_instance_state''"

[Simon King]

I haven't followed your code in detail, but I think the problem might be here:

clazz = school.Class('12', 'A')

students = [
Student("Name1", "Sname1", clazz=clazz, code='aa7'),
Student("Name2", "Sname2", clazz=clazz, code='bb7'),
Student("Name3", "Sname3", clazz=clazz, code='cc7')
]

You are creating an instance of "school.Class", which is the
non-sqlalchemy base class. You probably meant to create an instance of
"Class", which is the SQLAlchemy-mapped subclass, didn't you?

Simon

On Fri, Mar 12, 2021 at 11:10 AM ScottBot  wrote:
>
> I have a game that I am coding for school (not a project or homework) and I 
> am trying to use a SQLite database with sqlalchemy to store the game info. I 
> don't know what the problem is or how I can fix it. Any help is appreciated.
> https://stackoverflow.com/questions/66591466/sqlalchemy-orm-from-regular-class
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/b3e6fb67-6cab-4484-8c39-a01999640e67n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexcFYWK7yu9hW42LgUDB3LL%2BFAuCJqogpvYDjBe1MHigNQ%40mail.gmail.com.

Re: [sqlalchemy] SQLAlchemy database record created on import of module

[Simon King]

I suggest you set up an event listener for the "after_attach" event on
your session:

https://docs.sqlalchemy.org/en/13/orm/events.html#sqlalchemy.orm.events.SessionEvents.after_attach

Then you can set a breakpoint in the listener (or raise an exception,
or use the traceback module to print a stack trace) to find out where
in your code you are adding this object to the session.

Hope that helps,

Simon

On Fri, Mar 12, 2021 at 4:17 AM Advanced Spectrum
 wrote:
>
> I have a Base class like this:
>
> class FlagTable(Base):
>
> __tablename__ = "FLAG_TABLE"
>
> FLAG_TABLE_ID = Column(Integer, primary_key=True, autoincrement="auto")
> COLUMN_1 = Column(Boolean)
> COLUMN_2 = Column(Boolean)
> COLUMN_3 = Column(Boolean)
>
> I have a Python function in 'FlagTable.py' that would create a record with 
> the ORM FlagTable object:
>
> from FlagTable import FlagTable
>
> def addFlagsToFlagTable(arg1, arg2, arg3):
>
> myFlags= FlagTable(COLUMN_1=arg1, COLUMN_2=arg1, COLUMN_3=arg1)
> session.add(myFlags)
> session.commit()
>
> #main code starts here
> callFunc = addFlagsToFlagTable(true, false, false)
>
>
> When I step through the code in debugger mode I see that the record I want to 
> add to my postgres database table gets added at the time of the import. It 
> does not even wait for when the addFlagsToFlagTable function gets called in 
> my main code. My *'Base = declarative_base()'* statement is in another class 
> which is related to FlagTable class. Can someone educate me a bit on what is 
> going on?
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/7306a853-4017-4018-bc13-fa01aa8bb5adn%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexc6Wqeyf5w8CPsrm_jK_YJgV6au7UUQ-2fxxzHja6QQSg%40mail.gmail.com.

Re: [sqlalchemy] Usage instructions for citext support not working

[Simon King]

Here's the error message:

zsh: no matches found: sqlacodegen[citext]

ie. this message is coming from your shell, not pip. Zsh treats the
[...] part of the command as a filename pattern and tries to expand
it:

http://zsh.sourceforge.net/Doc/Release/Expansion.html#Filename-Generation

To get zsh to not treat it as a pattern, enclose it in quotes:

pip install 'sqlalcodegen[citext]'

Hope that helps,

Simon

On Mon, Feb 15, 2021 at 12:38 PM Wilfried L. Bounsi  wrote:
>
> Hello,
>
> I've been trying to enable citext support by following the installation 
> instructions in the README.md file, however it is not working, as you can see 
> in the screenshot below.
>
> I'm on Ubuntu 18.04
>
> Can you help me please?
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/47609864-49d1-4438-852d-9913d329c4fen%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexf0BTdBXtRd4Fb84TG9f6GhL2Kin5UBahky_SqF_jjdGQ%40mail.gmail.com.

Re: [sqlalchemy] SQLAlchemy (v.1.3.22) can not find Teradata engine inside Amazon Glue Job's script in Amazon environment

[Simon King]

Ah, OK, so the real problem is that the teradata package is trying to
load a .so file from site-packages.zip and failing. This presumably
happens when the module is imported, and Python is catching the
underlying exception and raising an ImportError instead.

It sounds like the teradata package is not expecting to be loaded from
a zip file at all, so you might have to find a different way of
packaging it for the Amazon environment. As you suspected, it's not an
SQLAlchemy question any more.

Good luck!

Simon

On Thu, Feb 4, 2021 at 10:43 AM Anhelina Rudkovska
 wrote:
>
> Thanks a lot for your answer and for explanations, Simon!
> According to your suggestions I checked yesterday site-packages.zip and yes, 
> directory teradatasqlalchemy-17.0.0.0.dist-info contains entry_point.txt. I 
> also compared all content of site-packages.zip with the same libs' 
> directories in python site-packages folder and everything the same. Also I 
> was trying to use site-packages.zip created in different systems: windows 
> through git bash, macos and inside linux-based docker container with mounted 
> volume to save .zip I got in container - so in every try it was created using 
> command line interfaces.
> Some screenshots of .zip file content and corresponding .dist-info content 
> were attached.
>
> Also I found out the way to resolve problem mentioned in this topic 
> yesterday, there were such steps:
> > from sqalchemy.dialects import registry
> > registry.register("teradatasql", "teradatasqlalchemy.dialect", 
> > "TeradataDialect")
>
> Engine was finally successfully created inside AWS Glue job's env, but then 
> OSError with teradatasql.so file was occurred (also there is attached 
> screenshot). In local env there is no such problem.
>
> I believe this is not related to SQAlchemy itself, but to Spark context and 
> Amazon environment. 
> https://stackoverflow.com/questions/61931878/running-teradatasql-driver-for-python-code-using-spark
>  here is the same topic, but everything in my site-packages.zip identical to 
> ordinary site-packages in python dir.
>
> I 99% sure that it's not the question to SQAlchemy team, but to summarize the 
> state of an issue I mentioned it. And probably I also hope that someone 
> accidentally know something about it. A big sorry for wasting your time and 
> thank you for an effort you did!
>
> BR, Anhelina
> вторник, 2 февраля 2021 г. в 19:04:00 UTC+2, Simon King:
>>
>> SQLAlchemy uses setuptools entry points to load database drivers.
>> Here's the definition for the teradata dialect:
>>
>> https://github.com/Teradata/sqlalchemy-teradata/blob/master/setup.py#L25
>>
>> For that to work, you would normally have a directory called something
>> like "sqlalchemy_teradata-0.1.0.dist-info" (or possible .egg-info) in
>> your site-packages. The directory would contain an entry_points.txt
>> file that points to the dialect class.
>>
>> Does your site-packages.zip contain that dist info directory with the
>> entry_points file inside?
>>
>> Simon
>>
>> On Mon, Feb 1, 2021 at 4:50 PM Anhelina Rudkovska
>>  wrote:
>> >
>> >
>> > Everything works as expected on local machine.
>> >
>> > Code fails where SQLAlchemy db engine initializes. We use latest 
>> > (17.0.0.8) release of https://pypi.org/project/teradatasqlalchemy/ library 
>> > to provide DB engine for SQLAlchemy. SQLAlchemy reports that it can not 
>> > load plugin teradatasql. I attached screenshot with error and piece of 
>> > code which is used to establish connection.
>> > Seems like library pkg_resources which is called inside SQLAlchemy can't 
>> > resolve teradatasql inside Amazon environment from .zip. Site-packages 
>> > shipped for Amazon as site-packages.zip placed on AWS s3.
>> > Direct imports of teradatasql or pkg_resources work fine (or 
>> > teradatasqlalchemy which is located in .zip with site-packages on s3 too). 
>> > Site-packages in archive look same as
>> > site-packages in their directory on local machine (i.e. where python 
>> > located or in virtual env, or inside filesystem of docker container).
>> >
>> > To develop script and run ETL job locally we use container (as described 
>> > here 
>> > https://aws.amazon.com/blogs/big-data/developing-aws-glue-etl-jobs-locally-using-a-container/)
>> > created from our image (installation of python libraries for script from 
>> > requirements.txt in Dockerfile was added) which inherits from 
>> > amazon/aws-glue-libs.
>> >
>> > I also notified Amazon Support.
>> >
>

Re: [sqlalchemy] Creating column SQLAlchemy property on parent class based on child column property

[Simon King]

I don't think you're going to find a way to do that built in to SQLAlchemy.

When you write "session.query(Parent)", SQLAlchemy constructs a query
against the "parent" table. But to filter by your "is_done" property,
it would suddenly need to join every child table into the query and
construct a complicated WHERE clause along the lines of "(type_ =
'child_one' AND child_one.requested_type_one =
child_one.delivered_type_one) OR (type_ = 'child_two' AND
child_two.requested_type_two = child_one.delivered_type_two)".

In theory, I think you *could* implement this as a hybrid property on
Parent, where the "expression" part of the hybrid property constructs
a subquery with a union or a join of all the child tables. It'll be
pretty messy, and might not perform particularly well.

Simon

On Wed, Feb 3, 2021 at 10:08 AM Mehrdad Pedramfar
 wrote:
>
> The structure I have created is like below:
>
>
> class Parent(Base):
> __tablename__ = 'parent'
>
> id = Field(
> Integer,
> primary_key=True
> )
>
> type_ = Field(
> String(50),
> readonly=True
> )
>
> __mapper_args__ = {
> 'polymorphic_on': type_,
> }
>
> class ChildOne(Parent):
> __tablename__ = 'child_one'
> __mapper_args__ = {
> 'polymorphic_identity': 'child_one'
> }
>
> id = Field(
> Integer,
> ForeignKey('parent.id'),
> primary_key=True
> )
>
> requested_type_one = Column(
> Integer,
> nullable=False,
> )
> delivered_type_one = Column(
> Integer,
> nullable=False,
> )
> is_done = column_property(
> requested_type_one == delivered_type_one
> )
>
> class ChildTwo(Parent):
> __tablename__ = 'child_two'
> __mapper_args__ = {
> 'polymorphic_identity': 'child_two'
> }
>
> id = Field(
> Integer,
> ForeignKey('parent.id'),
> primary_key=True
> )
>
> requested_type_two = Column(
> Integer,
> nullable=False,
> )
> delivered_type_two = Column(
> Integer,
> nullable=False,
> )
> is_done = column_property(
> requested_type_two == delivered_type_two
> )
>
>
> What I am looking for is to execute ORM query like this:
>
> session.query(Parent).filter(Parent.is_done.is_(True)).all()
>
> which raises `Parent class does not have is_done` error.
>
> I want  that parent class gets Child class's `is_done` based on different 
> types of child classes, I have tried to created `is_done` as 
> `column_property` on parent but I couldn't make it work. Also I tried using 
> `hybrid_property` and neither it is.
>
> What should I do to make Parent class get `is_done` from its children?
>
> stackoverflow link: 
> https://stackoverflow.com/questions/66024558/creating-column-sqlalchemy-property-on-parent-class-based-on-child-column-proper
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/1310342569.2406235.1612346870964%40mail.yahoo.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdFiYC-AotO5ZNcNFLGfnYXEHmJpHB%3DWg3hQ0mCXHKu5g%40mail.gmail.com.

Re: [sqlalchemy] SQLAlchemy (v.1.3.22) can not find Teradata engine inside Amazon Glue Job's script in Amazon environment

[Simon King]

SQLAlchemy uses setuptools entry points to load database drivers.
Here's the definition for the teradata dialect:

https://github.com/Teradata/sqlalchemy-teradata/blob/master/setup.py#L25

For that to work, you would normally have a directory called something
like "sqlalchemy_teradata-0.1.0.dist-info" (or possible .egg-info) in
your site-packages. The directory would contain an entry_points.txt
file that points to the dialect class.

Does your site-packages.zip contain that dist info directory with the
entry_points file inside?

Simon

On Mon, Feb 1, 2021 at 4:50 PM Anhelina Rudkovska
 wrote:
>
>
> Everything works as expected on local machine.
>
> Code fails where SQLAlchemy db engine initializes. We use latest (17.0.0.8) 
> release of https://pypi.org/project/teradatasqlalchemy/ library to provide DB 
> engine for SQLAlchemy. SQLAlchemy reports that it can not load plugin 
> teradatasql. I attached screenshot with error and piece of code which is used 
> to establish connection.
> Seems like library pkg_resources which is called inside SQLAlchemy can't 
> resolve teradatasql inside Amazon environment from .zip. Site-packages 
> shipped for Amazon as site-packages.zip placed on AWS s3.
> Direct imports of teradatasql or pkg_resources work fine (or 
> teradatasqlalchemy which is located in .zip with site-packages on s3 too). 
> Site-packages in archive look same as
> site-packages in their directory on local machine (i.e. where python located 
> or in virtual env, or inside filesystem of docker container).
>
> To develop script and run ETL job locally we use container (as described here 
> https://aws.amazon.com/blogs/big-data/developing-aws-glue-etl-jobs-locally-using-a-container/)
> created from our image (installation of python libraries for script from 
> requirements.txt in Dockerfile was added) which inherits from 
> amazon/aws-glue-libs.
>
> I also notified Amazon Support.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/87ce8dc9-b015-4ef0-a92a-5c5c10b528fan%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfk13QA3YOtjbnt%3D4oYh2AS93mt3eH3y9SnjpbsWjKMOg%40mail.gmail.com.

Re: [sqlalchemy] Behaviour when setting a foreign key column.

[Simon King]

You have missed something important, but I don't know if it will clear
up all your questions :-)

In your example, c.company_id doesn't get populated until the first
flush. Until then, c.company_id is None. So when you wrote:

# Case 1: update the _id doesn't seem to reflect
p.company_id = c.company_id

You were actually writing:

p.company_id = None

This explains why p.company and p.company_id are None all the way
through Case 1.

However, even if you added a session.flush() after adding the objects
to the session, setting a foreign key attribute directly does not
cause the associated relationship to be updated immediately:

https://docs.sqlalchemy.org/en/13/faq/sessions.html#i-set-the-foo-id-attribute-on-my-instance-to-7-but-the-foo-attribute-is-still-none-shouldn-t-it-have-loaded-foo-with-id-7

Hope that helps,

Simon

On Mon, Dec 14, 2020 at 12:50 PM João Miguel Neves
 wrote:
>
> Hi all,
>
> I'm sorry, as I feel like this is kind of a beginner question. I'd expect 
> that when I set a column attribute the respective relationship field would 
> change or at least be marked as changed (so a posterior access can be loaded 
> later on another access), but I've found a case where it doesn't. I've tried 
> to find a bit of code to represent what I'm struggling with, hopefully it 
> will be clear enough.
>
> # --- START ---
> from sqlalchemy import Column, create_engine, ForeignKey, Integer, String
>
> from sqlalchemy.orm import sessionmaker, relationship
> from sqlalchemy.ext.declarative import declarative_base
>
>
> engine = create_engine("sqlite://", echo=True)
> Session = sessionmaker(engine)
>
> Base = declarative_base()
>
>
> class Company(Base):
> __tablename__ = "company"
>
> company_id = Column(Integer, primary_key=True)
> name = Column(String(50))
>
>
> class Project(Base):
> __tablename__ = "project"
>
> project_id = Column(Integer, primary_key=True)
> name = Column(String(50))
> company_id = Column(Integer, ForeignKey("company.company_id"), index=True)
> company = relationship("Company")
>
>
> Base.metadata.create_all(engine)
>
> session = Session()
>
> c = Company(name="First Company")
>
> p = Project(name="First Project")
>
> session.add(c)
> session.add(p)
>
> assert p.company is None
> assert p.company_id is None
>
> # Case 1: update the _id doesn't seem to reflect
> p.company_id = c.company_id
>
> assert p.company is None
> assert p.company_id == c.company_id
>
> session.flush()
>
> # Wasn't expecting the None here
> assert p.company is None
> assert p.company_id is None
>
> session.refresh(p)
>
> # Wasn't expecting the None here
> assert p.company is None
> assert p.company_id is None
>
> # Case 2: update the relation works as expected
> p.company = c
>
> assert p.company == c
> assert p.company_id is None
>
> session.flush()
>
> assert p.company == c
> assert p.company_id == c.company_id
>
> session.refresh(p)
>
> assert p.company == c
> assert p.company_id == c.company_id
> # --- END ---
>
> Sorry for bothering, but I'm really puzzled/stuck and feel like I've missed 
> something important,
> João
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/CAJGrhWZq%2Biw_X-7B9%2Bc0sdxj2fAebpMb-O1erXnfhYmDsWZ%3D4Q%40mail.gmail.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexePGmkOqqiTcZ8gEiUYM9b0Fmc4i3SeHGEEkX2Q-dMGzg%40mail.gmail.com.

Re: [sqlalchemy] conditionals inside column_property

[Simon King]

I think this page might explain it:

https://docs.sqlalchemy.org/en/13/core/sqlelement.html#column-elements-and-expressions

The "case" function, like many other SQLAlchemy functions, returns an
instance of a ClauseElement subclass. This instance is used later when
building SQL queries. Columns themselves are also ClauseElement
subclasses. You can construct them at any time and pass them around
like any other python object. When it comes time to query the
database, the SQLAlchemy compiler will build the query by traversing
all the ClauseElements and converting them to the appropriate SQL.

The column_property mechanism accepts a ClauseElement and causes it to
be added to the SQL when you later query the table.

Simon

On Fri, Dec 11, 2020 at 11:11 AM Jinghui Niu  wrote:
>
> Thanks. One thing to clarify, I noticed that here you used `case` without 
> using in a context of `select`. Is this considered a shorthand within 
> sqlalchemy?
>
> On Fri, Dec 11, 2020 at 2:16 AM Simon King  wrote:
>>
>> You can do it, but you need to use an SQL conditional rather than a
>> python one. In this case that would probably be a CASE expression:
>>
>> https://docs.sqlalchemy.org/en/13/core/sqlelement.html#sqlalchemy.sql.expression.case
>>
>> I think it would look something like this:
>>
>> from sqlalchemy.sql import case
>>
>> aggr = column_property(
>> case([(attr_a == 'go', attr_b + attr_c)],
>>  else_=attr_b - attr_c)
>> )
>>
>> Hope that helps,
>>
>> Simon
>>
>> On Fri, Dec 11, 2020 at 9:13 AM niuji...@gmail.com  
>> wrote:
>> >
>> > I have a mapped class:
>> >
>> > class Model(sqlalchemy.declarative_base()):
>> > attr_a = Column(String)
>> > attr_b = Column(Integer)
>> > attr_c = Column(Integer)
>> >
>> > aggr = column_property(attr_b + attr_c IF attr_a=='go' ELSE attr_b - 
>> > attr_c)
>> >
>> > Last line is pseoudo code that requires some conditional logic. Is such 
>> > logic allowed inside column_property? Thanks a lot!
>> >
>> > --
>> > SQLAlchemy -
>> > The Python SQL Toolkit and Object Relational Mapper
>> >
>> > http://www.sqlalchemy.org/
>> >
>> > To post example code, please provide an MCVE: Minimal, Complete, and 
>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> > description.
>> > ---
>> > You received this message because you are subscribed to the Google Groups 
>> > "sqlalchemy" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to sqlalchemy+unsubscr...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/sqlalchemy/736f0ded-e39a-47fd-a0db-8ed33057d2a3n%40googlegroups.com.
>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example.  See  http://stackoverflow.com/help/mcve for a full 
>> description.
>> ---
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "sqlalchemy" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/sqlalchemy/a_-Bqhh5wY0/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> sqlalchemy+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeO2qgWeKeWPN-WH9LD8_Zch4jQRYeiB-WNbZrcMBrZFQ%40mail.gmail.com.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/CAOQAhNeTVkH5m-XXotNyHV_wBrb3Ev9SJnJpTYF2sRuJvxmOFA%40mail.gmail.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexcZPm8fWn_5eGgKR28GOv94TgzW6aBSzRAYkpjV%3DJD%3D9A%40mail.gmail.com.

Re: [sqlalchemy] conditionals inside column_property

[Simon King]

You can do it, but you need to use an SQL conditional rather than a
python one. In this case that would probably be a CASE expression:

https://docs.sqlalchemy.org/en/13/core/sqlelement.html#sqlalchemy.sql.expression.case

I think it would look something like this:

from sqlalchemy.sql import case

aggr = column_property(
case([(attr_a == 'go', attr_b + attr_c)],
 else_=attr_b - attr_c)
)

Hope that helps,

Simon

On Fri, Dec 11, 2020 at 9:13 AM niuji...@gmail.com  wrote:
>
> I have a mapped class:
>
> class Model(sqlalchemy.declarative_base()):
> attr_a = Column(String)
> attr_b = Column(Integer)
> attr_c = Column(Integer)
>
> aggr = column_property(attr_b + attr_c IF attr_a=='go' ELSE attr_b - 
> attr_c)
>
> Last line is pseoudo code that requires some conditional logic. Is such logic 
> allowed inside column_property? Thanks a lot!
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/736f0ded-e39a-47fd-a0db-8ed33057d2a3n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeO2qgWeKeWPN-WH9LD8_Zch4jQRYeiB-WNbZrcMBrZFQ%40mail.gmail.com.

Re: [sqlalchemy] Setting the FK not performed when relationship is declared

[Simon King]

I can't see anything obviously wrong with what you've written (you
said "child.id is None", but I assume you meant child.parent_id). Can
you provide a runnable example?

Simon

On Thu, Dec 10, 2020 at 8:27 AM Nikola Radovanovic  wrote:
>
> Hi,
> I have a FK in child pointing to parent table. Also, there is a relationship 
> towards parent from children. I would like to create children and set FK to 
> parent explicitly, without previously loading the parent object. After 
> commit, FK of parent in child table is None. In case I remove relationship 
> from child to parent, FK is set properly.
>
> Here is the simplified code:
>
> class Child(Base):
> __tablename__ = 'children'
> id = Column(Integer, primary_key=True)
> parent_id = Column(Integer, ForeignKey('parent.id', ondelete="CASCADE"), 
> nullable=True)
>
>parent = relationship("Parent", uselist=False) ## <-- when removing this, 
> it works
>
> class Parent(Base):
> __tablename__ = 'parent'
> id = Column(Integer, primary_key=True)
>
> child = Child(parent_id=3)
> session.add(child)
> session.commit()
>
> # at this moment, child.id is None and not 3., although parent with id ==3 
> exists in the DB. DB is Postgres 12.
>
> I have tried different loading techniques, but nothing worked.
>
> Can someone please point me into the right direction?
>
> Thank you in advance.
>
> Kindest regards
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/4704d9db-2725-416f-9ab4-65077209b8f5n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfBrrXhbRZHg7abuPXL4nYYeVM43zR2CyDL9e5G0NpjGw%40mail.gmail.com.

Re: [sqlalchemy] object “is already present in this session” when using a new session with older objects

[Simon King]

I think your situation is described here:

https://docs.sqlalchemy.org/en/14/orm/session_state_management.html#merge-tips

I'm not certain, but my guess is that when you create westGate, you
cause a *copy* of typeDict['gate'] to be merged into the current
session (because the merge cascades across the objectType
relationship). This instance is marked as pending. Then when you try
to create eastGate, the same thing happens again, but now there is
*already* a pending copy of typeDict['gate'] in the session, so you
get an error.

The easiest solution would be to create a session-specific copy of
typeDict, either by re-querying from the database, or by using
session.merge with load=False.

Hope that helps,

Simon

On Thu, Nov 19, 2020 at 11:16 PM Vinit Shah  wrote:
>
> I posted this on StackOverflow a few days ago, but I haven't been able to 
> figure this one out yet. The original post can be found here: StackOverflow: 
> object "is already present in this session".
>
> I'm seeing the below error:
> sqlalchemy.exc.InvalidRequestError: Can't attach instance  0x10592fe50>; another instance with key ( , 
> (1,), None) is already present in this session.
>
> I'm seeing this issue when I try to instantiate a new object that has a 
> foreign key relationship with an existing object that was created in another 
> session.
>
> This happens in a few different cases in my actual code, but in the provided 
> sample it occurs with the following steps:
> 1. Add a new object into a new session
> 2. Close session and remove() from scoped_session
> 3. Reference the object in two newly constructed ones via their relationship
> 4. Error appears on the second object
>
> # typeDict just contains a pre-fetched ObjectTypes
> tokyo = Location(name="tokyo", objectType=typeDict['location'])
> tokyo = write(tokyo)
>
> # If I clear out the current session here, the error will occur
> scopedSessionFactory().close()
> scopedSessionFactory.remove()
>
> westGate = Gate(name="westGate", destination=tokyo, 
> objectType=typeDict['gate'])
> westGate = write(westGate)
>
> luggage = LocationInput(name="luggage", 
> objectType=typeDict['locationinput'])
> luggage = write(luggage)
>
> # This is the line where the error occurs
> eastGate = Gate(name="eastGate", origin=tokyo, destinationInput=luggage, 
> objectType=typeDict['gate'])
> eastGate = write(eastGate)
>
> I'm not sure what exactly causes this or way. For this example, I could just 
> reuse the same session, but I'd like to be able to take an object from one 
> closed session and add as a relationship field to another.
>
> Full code sample available here:
> https://gist.github.com/funseiki/a73424bebfb0d809e6d934f699a725bf
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/4d01df93-16ed-45a4-82f3-de04a8e57bcbn%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfk-xGOf5Je8Jubq0gtA6U6fFnwnBPj5SOFyArwJ51KbA%40mail.gmail.com.

Re: [sqlalchemy] NEED HELP! Using Hybrid property in SQLAlchemy filter throws error

[Simon King]

lazy='joined')
>> size = db.relationship('SizeMaster',
>>secondary='size_goods', passive_deletes=True, 
>> backref='size_goods', lazy='joined')
>>
>> size_chart = db.relationship('SizeChart',
>>  secondary='size_chart_goods', 
>> passive_deletes=True, backref='size_chart_goods', lazy='joined')
>>
>> # Foreign Key IDs for Unique Constraint
>>
>> product_category_id = db.Column(db.Integer, nullable=False)
>> fabric_combination_id = db.Column(db.Integer, nullable=False)
>> print_technique_id = db.Column(db.Integer, nullable=False)
>> design_number_id = db.Column(db.Integer, nullable=False)
>> uom_id = db.Column(db.Integer, nullable=False)
>> size_id = db.Column(db.Integer, nullable=False, default=None)
>> # End FK
>> title = db.Column(db.String(100), nullable=False)
>> description = db.Column(db.String(250), nullable=False)
>>
>> price = db.Column(db.Float, nullable=False,  default="0")
>> qty = db.Column(db.Float, nullable=False)
>> hold_qty = db.Column(db.Float, default=0)
>> pen_qty = db.Column(db.Float, default=0)
>>
>> gst = db.Column(db.Integer, nullable=False, default="0")
>> multiple = db.Column(db.Integer, nullable=False, default=1)
>> sku = db.Column(db.String(40), nullable=False)
>>
>> image = db.Column(db.String(250))
>>
>> children = db.relationship('FinishedGoodsChild',
>>passive_deletes=True, 
>> secondary='goods_child_sizes', backref='goods_child_sizes', lazy='joined')
>> balance = db.Column(db.Float , default= 0)
>>
>> __table_args__ = (db.UniqueConstraint(
>> 'product_category_id', 'fabric_combination_id', 
>> 'print_technique_id', 'design_number_id', 'sku', 
>> name='finished_goods_chk_id'), )
>>
>> def __init__(self,  product_category, fabric_combination, 
>> print_technique, design_number, uom, size, title, description, sku, price, 
>> qty, multiple):
>> self.product_category_id = product_category.id
>> self.product_category.append(product_category)
>>
>> self.fabric_combination_id = fabric_combination.id
>> self.fabric_combination.append(fabric_combination)
>>
>> self.print_technique_id = print_technique.id
>> self.print_technique.append(print_technique)
>>
>> self.design_number_id = design_number.id
>> self.design_number.append(design_number)
>>
>> self.uom_id = uom.id
>> self.uom.append(uom)
>>
>> self.size_id = size.id
>> self.size.append(size)
>>
>> self.title = title
>> self.description = description
>> self.price = price
>> self.qty = qty
>> self.multiple = multiple
>> self.sku = sku
>>
>> def get_gen_name(self):
>> goods_name = "{}/{}/{}/{}".format(
>> self.product_category[0].name, self.fabric_combination[0].name, 
>> self.print_technique[0].name, self.design_number[0].name)
>> return goods_name
>
>
> On Thu, Oct 22, 2020 at 3:53 PM Simon King  wrote:
>>
>> You don't need to add a parent_id column, you just need to write the
>> full relationship condition inside that "select" statement. It would
>> be much easier to explain if you can show your actual parent and child
>> classes, including the relationship between them and the association
>> table.
>>
>> Simon
>>
>> On Thu, Oct 22, 2020 at 7:23 AM Padam Sethia  wrote:
>> >
>> > Thanks for your input , the children have a many to many relationship with 
>> > the parent FinishedGoods  , with this how would I refer to parent_id , do 
>> > i need to create and add that also ?
>> >
>> > On Wed, 21 Oct 2020, 14:48 Simon King,  wrote:
>> >>
>> >> The "expression" part of a hybrid property is used whenever you write
>> >> "FinishedGoodsParent.balance". It operates in the context of the
>> >> class, not a single instance, and it needs to return an SQL expression
>> >> that can be used inside a larger query.
>> >>
>> >> In your version, you are trying to iterate over "cls.children", but
>> >> that's not possible because "cls.children" is not a list. Hybrid
>> >> properties that work across relationships can be a bit diffic

Re: [sqlalchemy] NEED HELP! Using Hybrid property in SQLAlchemy filter throws error

[Simon King]

You don't need to add a parent_id column, you just need to write the
full relationship condition inside that "select" statement. It would
be much easier to explain if you can show your actual parent and child
classes, including the relationship between them and the association
table.

Simon

On Thu, Oct 22, 2020 at 7:23 AM Padam Sethia  wrote:
>
> Thanks for your input , the children have a many to many relationship with 
> the parent FinishedGoods  , with this how would I refer to parent_id , do i 
> need to create and add that also ?
>
> On Wed, 21 Oct 2020, 14:48 Simon King,  wrote:
>>
>> The "expression" part of a hybrid property is used whenever you write
>> "FinishedGoodsParent.balance". It operates in the context of the
>> class, not a single instance, and it needs to return an SQL expression
>> that can be used inside a larger query.
>>
>> In your version, you are trying to iterate over "cls.children", but
>> that's not possible because "cls.children" is not a list. Hybrid
>> properties that work across relationships can be a bit difficult to
>> think about. The expression that you return needs to access another
>> table, so you need to consider how the query will join to that table:
>>
>> https://docs.sqlalchemy.org/en/13/orm/extensions/hybrid.html#working-with-relationships
>>
>> In your case, you actually need to perform an aggregating function
>> (sum) on the related table. The easiest way to do that would be to
>> follow the correlated subquery example from the docs:
>>
>> https://docs.sqlalchemy.org/en/13/orm/extensions/hybrid.html#correlated-subquery-relationship-hybrid
>>
>> Something like this:
>>
>> @balance.expression
>> def balance(cls):
>> return select([sa.func.sum(FinishedGoodsChild.qty)]).\
>> where(FinishedGoodsChild.parent_id==cls.id).\
>> label('balance')
>>
>> You probably need something a bit more complicated than that - I
>> didn't understand the join condition between parent and child in your
>> example so I made up the "parent_id" column.
>>
>> Hope that helps,
>>
>> Simon
>>
>> On Tue, Oct 20, 2020 at 4:57 PM Padam Sethia  wrote:
>> >
>> > Hello ,
>> >
>> > I'm having an issue with Hybrid methods - I still don't understand them 
>> > enough properly . So I have a parent and child many to many relationship
>> >
>> >
>> > This is the child model
>> >
>> > class FinishedGoodsChild(TimestampMixin, db.Model):
>> >   id = db.Column(db.Integer, primary_key=True)
>> >   hold_qty = db.Column(db.Float, default=0)
>> >   pen_qty = db.Column(db.Float, default=0)
>> >   qty = db.Column(db.Float, nullable=False, default=0)
>> >   sku = db.Column(db.String(40), nullable=False, default='')
>> >   size = db.relationship('SizeMaster',   
>> > secondary='size_goods_child', passive_deletes=True,   
>> > backref='size_goods_child', lazy='joined')
>> >   size_id = db.Column(db.Integer, nullable=False, default=None)
>> >
>> > This is the Parent model
>> >
>> >
>> > class FinishedGoodsChild(TimestampMixin, db.Model):
>> >   id = db.Column(db.Integer, primary_key=True)
>> >   qty =   db.Column(db.Float, default=0)
>> >   balance = db.Column(db.Float)
>> >   children = db.relationship('FinishedGoodsChild',   
>> > passive_deletes=True, secondary='goods_child_sizes', 
>> > backref='goods_child_sizes', lazy='joined')
>> >
>> >
>> > No I need to filter by the sum of the children qty
>> >
>> >
>> > Here is the hybrid property that I have set up , but throws not 
>> > implemented error
>> >
>> >
>> > @hybrid_property
>> > def balance(self):
>> >return sum(acc.qty for acc in self.children) @balance.expression
>> > def balance(cls):
>> >return sum(acc.qty for acc in cls.children)
>> >
>> > Help is much appreciated thanks!
>> >
>> > --
>> > SQLAlchemy -
>> > The Python SQL Toolkit and Object Relational Mapper
>> >
>> > http://www.sqlalchemy.org/
>> >
>> > To post example code, please provide an MCVE: Minimal, Complete, and 
>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> > description.
>> > ---
>> > You received this message because you ar

Re: [sqlalchemy] NEED HELP! Using Hybrid property in SQLAlchemy filter throws error

[Simon King]

The "expression" part of a hybrid property is used whenever you write
"FinishedGoodsParent.balance". It operates in the context of the
class, not a single instance, and it needs to return an SQL expression
that can be used inside a larger query.

In your version, you are trying to iterate over "cls.children", but
that's not possible because "cls.children" is not a list. Hybrid
properties that work across relationships can be a bit difficult to
think about. The expression that you return needs to access another
table, so you need to consider how the query will join to that table:

https://docs.sqlalchemy.org/en/13/orm/extensions/hybrid.html#working-with-relationships

In your case, you actually need to perform an aggregating function
(sum) on the related table. The easiest way to do that would be to
follow the correlated subquery example from the docs:

https://docs.sqlalchemy.org/en/13/orm/extensions/hybrid.html#correlated-subquery-relationship-hybrid

Something like this:

@balance.expression
def balance(cls):
return select([sa.func.sum(FinishedGoodsChild.qty)]).\
where(FinishedGoodsChild.parent_id==cls.id).\
label('balance')

You probably need something a bit more complicated than that - I
didn't understand the join condition between parent and child in your
example so I made up the "parent_id" column.

Hope that helps,

Simon

On Tue, Oct 20, 2020 at 4:57 PM Padam Sethia  wrote:
>
> Hello ,
>
> I'm having an issue with Hybrid methods - I still don't understand them 
> enough properly . So I have a parent and child many to many relationship
>
>
> This is the child model
>
> class FinishedGoodsChild(TimestampMixin, db.Model):
>   id = db.Column(db.Integer, primary_key=True)
>   hold_qty = db.Column(db.Float, default=0)
>   pen_qty = db.Column(db.Float, default=0)
>   qty = db.Column(db.Float, nullable=False, default=0)
>   sku = db.Column(db.String(40), nullable=False, default='')
>   size = db.relationship('SizeMaster',   
> secondary='size_goods_child', passive_deletes=True,   
> backref='size_goods_child', lazy='joined')
>   size_id = db.Column(db.Integer, nullable=False, default=None)
>
> This is the Parent model
>
>
> class FinishedGoodsChild(TimestampMixin, db.Model):
>   id = db.Column(db.Integer, primary_key=True)
>   qty =   db.Column(db.Float, default=0)
>   balance = db.Column(db.Float)
>   children = db.relationship('FinishedGoodsChild',   
> passive_deletes=True, secondary='goods_child_sizes', 
> backref='goods_child_sizes', lazy='joined')
>
>
> No I need to filter by the sum of the children qty
>
>
> Here is the hybrid property that I have set up , but throws not implemented 
> error
>
>
> @hybrid_property
> def balance(self):
>return sum(acc.qty for acc in self.children) @balance.expression
> def balance(cls):
>return sum(acc.qty for acc in cls.children)
>
> Help is much appreciated thanks!
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/ba1add47-9497-4b92-8cb8-926e03c958a5n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfvqQs6c%2BtR20pkojBt4JjToWB7gxW7U9O4Pb_gxpxC3A%40mail.gmail.com.

Re: [sqlalchemy] Cannot insert strings with a length greater than 2000 into columns with a datatype of Text, VARCHAR, or NVARCHAR using MS SQL Server 2017 and 2019

[Simon King]

Yep, I misunderstood what setinputsizes was doing. I thought it told
pyodbc how it should handle a particular datatype, rather than telling
it how to handle the set of parameters it is about receive in the next
execute() call...

Sorry for adding to the confusion,

Simon

On Fri, Oct 16, 2020 at 1:14 PM Mike Bayer  wrote:
>
>
>
> On Fri, Oct 16, 2020, at 3:53 AM, Nicolas Lykke Iversen wrote:
>
> Is it really necessary to use your very-subtle vendored version of the 
> set_input_sizes() hook? Why use it compared to Simon King's simple version?
>
>
> yes, because cursor.setinputsizes() must be passed an entry for every bound 
> parameter in your statement, in the order that they will be passed to 
> cursor.execute().this includes for all the numerics, dates, etc for which 
> you certainly don't want to pass those as "varchar".   so if the third 
> parameter in your statement was the textual version, you'd need to pass 
> cursor.setinputsizes([None, None, (pyodbc.SQL_WVARCHAR, None, None), ...]).   
> Also in my experimenation with this value you want to pass "None" for length, 
> if not otherwise specified, and not 0.
>
> Simon's version is hardcoding to passing varchar in all cases for a single 
> bound parameter, and I would not expect that recipe to work at all.
>
>
> Using Simon King's version I experience a weird issue: it works perfectly, 
> when using a single-threaded application, but when using multiprocessing it 
> doesn't work.
>
> In particular, if I execute:
>
> engine = sqlalchemy.create_engine(
>
> f'mssql+pyodbc://{user}:{pwd}@{host}:{port}/{db}?driver=ODBC+Driver+17+for+SQL+Server',
>pool_size=5,
>max_overflow=10,
>pool_pre_ping=True,
>isolation_level='READ_UNCOMMITTED',
>pool_recycle=900,
>echo=debug,
>connect_args={'connect_timeout': 10}
> )
>
> @sqlalchemy.event.listens_for(engine, 'before_cursor_execute')
> def receive_before_cursor_execute(conn, cursor, statement, parameters, 
> context, executemany):
>cursor.setinputsizes([(pyodbc.SQL_WVARCHAR, 0, 0), ])
>
> in the main application and:
>
> def db_init():
>engine = common.db.Session.get_bind()
>engine.dispose()
>
> in all the children, then the hook gets called in the children, but somehow 
> doesn't affect the INSERTs - the original error is produced for strings with 
> a. length longer than 2000 characters.
>
> Best regards
> Nicolas
>
>
>
> On Thursday, October 15, 2020 at 7:39:08 PM UTC+2 Mike Bayer wrote:
>
>
>
> On Thu, Oct 15, 2020, at 3:26 AM, Nicolas Lykke Iversen wrote:
>
> Hi Mike,
>
> I have created an issue for pyodbc: 
> https://github.com/mkleehammer/pyodbc/issues/835#issuecomment-708930870
>
> I've gotten really good feedback there from Microsoft, and a fix has been 
> proposed that works:
>
> "You can try to use setinputsizes on your parameter to tell it to send as 
> varchar(max): cursor.setinputsizes([(pyodbc.SQL_WVARCHAR,0,0),])"
> I'm by no means an SQLAlchemy expert, but shouldn't the pyodbc dialect be 
> updated to support varchar(max)using the proposed method? If not, how can I 
> execute cursor.setinputsizes([(pyodbc.SQL_WVARCHAR,0,0),]) using SQLAlchemy, 
> so that I can make use of  varchar(max)in my application?
>
>
>
> SQLAlchemy has some dialects that make use of setinputsizes() out of 
> necessity, but it's an area that is fraught with issues as it means 
> SQLAlchemy is second-guessing what the DBAPI is coming up with.
>
> It's actually news to me that pyodbc supports setinputsizes() as 
> historically, the cx_Oracle DBAPI was the only DBAPI that ever did so and 
> this method is usually not supported by any other DBAPI.   We have a hook 
> that calls upon setinputsizes() but right now it's hardcoded to cx_Oracle's 
> argument style, so the hook would need alterations to support different 
> calling styles on different dialects.
>
> In https://github.com/mkleehammer/pyodbc/issues/835#issuecomment-709385941 it 
> is suggested that there should be no need to use this "_SC" collation -  then 
> in https://github.com/mkleehammer/pyodbc/issues/835#issuecomment-709428774, 
> you stated "I previously experimented with non-_SC in my application, and it 
> caused errors.".   Can you be more specific of these errors?   At the moment, 
> this is suggesting a major architectural rework of the pyodbc dialect to 
> support a use case which has other workarounds.   The architecture of 
> SQLAlchemy's set_input_sizes() hook has changed and at best this would be 
> part of 1.4 which is not in beta release yet, a production release is not for 
> several months.
>
> From that point, there's an event hook at do_setinputsizes(): 
> https://docs.sqlalchemy.org/en/13/core/events.html?highlight=do_setinputsizes#sqlalchemy.events.DialectEvents.do_setinputsizes
>that would be usable so that you could set up rules like these on your 
> own, and we eventually would document the known workarounds for various 
> unusual issues.
>
> This issue is definitely unusual, it's never been 

Re: [sqlalchemy] Cannot insert strings with a length greater than 2000 into columns with a datatype of Text, VARCHAR, or NVARCHAR using MS SQL Server 2017 and 2019

[Simon King]

Do you know if there is a downside to calling setinputsizes like that?
To put it another way, why doesn't pyodbc configure itself like that
automatically? Why do you think this belongs in SQLAlchemy rather than
pyodbc?

I suspect the answer is that most applications don't need it and there
is a downside (perhaps a performance implication?).

I've never used SQL Server, but the fact that the error message refers
to these collations as "legacy" suggests that an alternative collation
might be better.
https://docs.microsoft.com/en-us/sql/relational-databases/collations/collation-and-unicode-support?view=sql-server-ver15#Supplementary_Characters
says:

SQL Server 2019 (15.x) extends supplementary character support to
the char and varchar data types with the new UTF-8 enabled collations
(_UTF8). These data types are also capable of representing the full
Unicode character range.

If you can restrict yourself to SQL Server 2019, that might be a better option.

Simon

On Thu, Oct 15, 2020 at 10:08 AM Nicolas Lykke Iversen
 wrote:
>
> Thank you, Simon.
>
> I'm curious whether this is the way to do it in the future, or whether 
> SQLAlchemy should implement varchar(max)properly?
>
> What would the argument be for not implementing varchar(max)in the pyodbc 
> dialect?
>
> On Thursday, October 15, 2020 at 11:05:32 AM UTC+2 Simon King wrote:
>>
>> You could call 'setinputsizes' in a handler for the
>> 'before_cursor_execute' event, something like this:
>>
>>
>> from sqlalchemy import event
>>
>> @event.listens_for(SomeEngine, 'before_cursor_execute')
>> def receive_before_cursor_execute(conn, cursor, statement,
>> parameters, context, executemany):
>> cursor.setinputsizes([(pyodbc.SQL_WVARCHAR,0,0),])
>>
>>
>> https://docs.sqlalchemy.org/en/13/core/events.html#sqlalchemy.events.ConnectionEvents
>> https://docs.sqlalchemy.org/en/13/core/events.html#sqlalchemy.events.ConnectionEvents.before_cursor_execute
>>
>> Hope that helps,
>>
>> Simon
>>
>>
>> On Thu, Oct 15, 2020 at 8:27 AM Nicolas Lykke Iversen  
>> wrote:
>> >
>> > Hi Mike,
>> >
>> > I have created an issue for pyodbc: 
>> > https://github.com/mkleehammer/pyodbc/issues/835#issuecomment-708930870
>> >
>> > I've gotten really good feedback there from Microsoft, and a fix has been 
>> > proposed that works:
>> >
>> > "You can try to use setinputsizes on your parameter to tell it to send as 
>> > varchar(max): cursor.setinputsizes([(pyodbc.SQL_WVARCHAR,0,0),])"
>> >
>> > I'm by no means an SQLAlchemy expert, but shouldn't the pyodbc dialect be 
>> > updated to support varchar(max)using the proposed method? If not, how can 
>> > I execute cursor.setinputsizes([(pyodbc.SQL_WVARCHAR,0,0),]) using 
>> > SQLAlchemy, so that I can make use of varchar(max)in my application?
>> >
>> > Can you recommend a hotfix for using varchar(max)in current SQLAlchemy 
>> > applications that need to handle Unicode supplementary characters (_SC)?
>> >
>> > I appreciate really appreciate your help.
>> >
>> > Best regards
>> > Nicolas
>> > On Wednesday, October 14, 2020 at 3:36:11 PM UTC+2 Mike Bayer wrote:
>> >>
>> >>
>> >>
>> >> On Wed, Oct 14, 2020, at 5:35 AM, Nicolas Lykke Iversen wrote:
>> >>
>> >> Hi Mike,
>> >>
>> >> I've now tested inserting strings with more than 2000 characters using 
>> >> Azure Data Studio (SQL Server GUI) and everything works.
>> >>
>> >> Furthermore, I've tested pyodbc (DBAPI) directly, and it only fails when 
>> >> inserting such strings using parameterised SQL queries (it succeeds 
>> >> without using parametrised queries).
>> >>
>> >>
>> >> that would be expected because all the datatype-related issues occur when 
>> >> bound parameters are passed.
>> >>
>> >>
>> >>
>> >> You can see my POC below, if you have any interest.
>> >>
>> >> I guess it should be submitted as a bug to pyodbc... Do you know if I can 
>> >> disable parametrisation for certain SQL queries in SQLAlchemy?
>> >>
>> >>
>> >> there is not and this is definitely an issue that has to be solved at the 
>> >> pyodbc level, either a bug on their end or something in your 
>> >> configuration that has to be changed.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Best regards (and t

Re: [sqlalchemy] Cannot insert strings with a length greater than 2000 into columns with a datatype of Text, VARCHAR, or NVARCHAR using MS SQL Server 2017 and 2019

[Simon King]

You could call 'setinputsizes' in a handler for the
'before_cursor_execute' event, something like this:


from sqlalchemy import event

@event.listens_for(SomeEngine, 'before_cursor_execute')
def receive_before_cursor_execute(conn, cursor, statement,
parameters, context, executemany):
cursor.setinputsizes([(pyodbc.SQL_WVARCHAR,0,0),])


https://docs.sqlalchemy.org/en/13/core/events.html#sqlalchemy.events.ConnectionEvents
https://docs.sqlalchemy.org/en/13/core/events.html#sqlalchemy.events.ConnectionEvents.before_cursor_execute

Hope that helps,

Simon


On Thu, Oct 15, 2020 at 8:27 AM Nicolas Lykke Iversen  wrote:
>
> Hi Mike,
>
> I have created an issue for pyodbc: 
> https://github.com/mkleehammer/pyodbc/issues/835#issuecomment-708930870
>
> I've gotten really good feedback there from Microsoft, and a fix has been 
> proposed that works:
>
> "You can try to use setinputsizes on your parameter to tell it to send as 
> varchar(max): cursor.setinputsizes([(pyodbc.SQL_WVARCHAR,0,0),])"
>
> I'm by no means an SQLAlchemy expert, but shouldn't the pyodbc dialect be 
> updated to support varchar(max)using the proposed method? If not, how can I 
> execute cursor.setinputsizes([(pyodbc.SQL_WVARCHAR,0,0),]) using SQLAlchemy, 
> so that I can make use of  varchar(max)in my application?
>
> Can you recommend a hotfix for using varchar(max)in current SQLAlchemy 
> applications that need to handle Unicode supplementary characters (_SC)?
>
> I appreciate really appreciate your help.
>
> Best regards
> Nicolas
> On Wednesday, October 14, 2020 at 3:36:11 PM UTC+2 Mike Bayer wrote:
>>
>>
>>
>> On Wed, Oct 14, 2020, at 5:35 AM, Nicolas Lykke Iversen wrote:
>>
>> Hi Mike,
>>
>> I've now tested inserting strings with more than 2000 characters using Azure 
>> Data Studio (SQL Server GUI) and everything works.
>>
>> Furthermore, I've tested pyodbc (DBAPI) directly, and it only fails when 
>> inserting such strings using parameterised SQL queries (it succeeds without 
>> using parametrised queries).
>>
>>
>> that would be expected because all the datatype-related issues occur when 
>> bound parameters are passed.
>>
>>
>>
>> You can see my POC below, if you have any interest.
>>
>> I guess it should be submitted as a bug to pyodbc... Do you know if I can 
>> disable parametrisation for certain SQL queries in SQLAlchemy?
>>
>>
>> there is not and this is definitely an issue that has to be solved at the 
>> pyodbc level, either a bug on their end or something in your configuration 
>> that has to be changed.
>>
>>
>>
>>
>>
>> Best regards (and thanks for your help and support!!!)
>> Nicolas
>>
>> System info:
>> python v. 3.8.5
>> pyodbc v. 4.0.30
>> msodbcsql17 v. 17.6.1.1
>>
>> POC:
>> import sys
>> import pyodbc
>>
>> host = 'tcp:127.0.0.1,1433'
>> db = 'pyodbc_test'
>> user = 'sa'
>> pwd = 'P@ssw0rd'
>>
>> print('started')
>>
>> cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL 
>> Server};SERVER='+host+';DATABASE='+'master'+';UID='+user+';PWD='+ pwd, 
>> autocommit=True)
>> cursor = cnxn.cursor()
>>
>> try:
>> cursor.execute(f'CREATE DATABASE {db} COLLATE 
>> Latin1_General_100_CI_AS_SC')
>> except pyodbc.ProgrammingError as e:
>> pass # database exists
>>
>> cursor.execute(f'USE {db}')
>>
>> try:
>> cursor.execute("""
>> CREATE TABLE msg (
>> id int identity(1,1) not null,
>> content varchar(max) not null
>> );""")
>> except pyodbc.ProgrammingError as exc:
>> pass # table exists
>>
>> content = 2000 * 'A'
>>
>> cursor.execute(f"""
>> INSERT INTO msg (content)
>> VALUES ('{content}')""")
>> print(f'non-param: {len(content)=}: success')
>>
>> sql = f"""
>>   INSERT INTO msg (content)
>>   VALUES (?)"""
>> cursor.execute(sql, (content))
>> print(f'param: {len(content)=}: success')
>>
>> content = 2001 * 'A'
>>
>> cursor.execute(f"""
>> INSERT INTO msg (content)
>> VALUES ('{content}')""")
>> print(f'non-param: {len(content)=}: success')
>>
>> # this fails!
>> sql = f"""
>>   INSERT INTO msg (content)
>>   VALUES (?)"""
>> cursor.execute(sql, (content))
>> print(f'param: {len(content)=}: success')
>>
>>
>> #cursor.execute('SELECT * FROM msg')
>> #rows = cursor.fetchall()
>> #for r in rows:
>> #print(r)
>>
>> print('finished')
>>
>>
>>
>> On Wednesday, October 14, 2020 at 12:43:25 AM UTC+2 Mike Bayer wrote:
>>
>>
>>
>> On Tue, Oct 13, 2020, at 4:57 PM, Nicolas Lykke Iversen wrote:
>>
>> Thank you, Mike - very much appreciated!
>>
>> Just to be clear, pyodbc is not a driver, it’s a ODBC-compliant DBAPI, 
>> right? I separately downloaded a driver for SQL Server from Microsoft, which 
>> pyodbc makes use of.
>>
>>
>> right the pyodbc is the DBAPI in this case, which we usually refer to as a 
>> "driver" but in the case of ODBC the "driver" is more specifically the 
>> separate ODBC driver component.
>>
>>
>> Do you suggest that changing pyodbc to another SQL Server DPAPI would solve 
>> the problem?
>>
>>
>> I 

Re: [sqlalchemy] extend automapped classes with a mixin

[Simon King]

On Mon, Oct 12, 2020 at 6:59 PM Imran Akbar  wrote:
>
> Hi,
>
> I'm using SQLAlchemy and am generating classes dynamically for my database 
> via the Automapping functionality.
>
> I need to add a Mixin class with various helper methods to each of these 
> automapped classes.
>
> I tried to create subclasses of the automapped class with the mixin class:
>
> db = create_engine(connection_string)
> automapper = automap_base()
> automapper.prepare(db, reflect=True)
> for class_variable in automapper.__subclasses__():
> new_class = type(class_variable.__name__, (class_variable, Mixins), {})
>
> but when I try to use these classes I get errors like:
>
> class _ is a subclass of AutomapBase. Mappings are not produced until the 
> .prepare() method is called on the class hierarchy.
>

If you want to add your methods to *every* automapped class, you can
change the base class used by automap_base(). Something like this:

class MySpecialBaseClass(object):
# define your helper methods here
pass

automapper = automap_base(cls=MySpecialBaseClass)

This works because automap_base passes most of its arguments on to
declarative_base:

https://docs.sqlalchemy.org/en/13/orm/extensions/automap.html#sqlalchemy.ext.automap.automap_base

...and declarative_base accepts a "cls" argument which is used as the
base class:

https://docs.sqlalchemy.org/en/13/orm/extensions/declarative/api.html#sqlalchemy.ext.declarative.declarative_base

Hope that helps,

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfK%2BnjPge%3DZB4f1vR8paFAbaFtyeA-GKM9BFQYQosHn0w%40mail.gmail.com.

Re: [sqlalchemy] ORM and objects with properties that need conversion to write to database

[Simon King]

On Thu, Oct 8, 2020 at 3:38 AM Richard Damon  wrote:
>
> I am working on a app using SQLAlchemy's ORM layer to interface to the
> database, but I am running into an issue that if an object has
> 'complicated' property, like a UUID, that SQLAlchemy doesn't know how to
> handle. One option would be to make the propery actually only hold a
> database suitable representation, and I see an option to add a
> 'reconstructor' to convert a value read from the database into an
> object. Is there a similar way that when writing the object, to indicate
> how to convert the object into a format that can be put into the database?
>

The usual pattern is to create a custom datatype using
sqlalchemy.types.TypeDecorator. You would override the
process_bind_baram and process_result_value methods to convert objects
between the python and database representations:

https://docs.sqlalchemy.org/en/13/core/custom_types.html#augmenting-existing-types

For the specific case of UUIDs, there is an example in the docs that
will use Postgres' UUID type against postgres, and CHAR against other
databases:

https://docs.sqlalchemy.org/en/13/core/custom_types.html#backend-agnostic-guid-type

Hope that helps,

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeVbT%3DAN%2BBTxNVoMY0PY3U7rTK73d1XHcbZYM%2BkQpp1RA%40mail.gmail.com.

Re: [sqlalchemy] How can I use a composite foreign-key constraint with a "mixin" class using declarative?

[Simon King]

You'd have to wait for a response from Mike to be certain, but it
seems overwhelmingly likely to me that dialect-specific options will
always be ignored by other dialects. One of the strengths of
SQLAlchemy is that it makes it easier to write code that works against
multiple database dialects, so I can't see that changing.

Simon

On Thu, Sep 3, 2020 at 2:00 PM Nicolas Lykke Iversen  wrote:
>
> Thanks Simon,
>
> Just to be clear:
>
>> dialect-specific options end up in table.dialect_options:
>> https://docs.sqlalchemy.org/en/13/core/metadata.html#sqlalchemy.schema.Table.dialect_options
>> But different dialects seem to handle them in different ways.
>
>
> Does this mean that it's not safe in general to mix dialect-specific options 
> in __table_args__, since it's arbitrary how they are handled? Indeed, in your 
> case it works (MySQL and SQLite), but that might not be the case for other 
> dialects like MSSQL?
>
> Best regards
> Nicolas
>
>
>
>
>
>
>
>
> Den tor. 3. sep. 2020 kl. 11.00 skrev Simon King :
>>
>> To be honest, I looked for documentation before I wrote my reply to
>> you, and couldn't find anything. I just know that I often use MySQL in
>> production but sqlite for tests, and sqlite never complained about the
>> mysql-specific options in the table args.
>>
>> dialect-specific options end up in table.dialect_options:
>>
>> https://docs.sqlalchemy.org/en/13/core/metadata.html#sqlalchemy.schema.Table.dialect_options
>>
>> But different dialects seem to handle them in different ways. For
>> example, here's how mysql consumes table-level options:
>>
>> https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/dialects/mysql/base.py#L1871
>>
>> whereas sqlite does this:
>>
>> https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/dialects/sqlite/base.py#L1120
>>
>> Simon
>>
>> On Thu, Sep 3, 2020 at 5:38 AM Nicolas Lykke Iversen  
>> wrote:
>> >
>> > Thank you, Simon.
>> >
>> > Yes, __table_args__ is the only reason I’m creating separate modules.
>> >
>> > Where do you see that arguments that don’t match the database dialect of 
>> > the engine get ignored? I looked at the source code for answering this 
>> > question myself, but couldn't find an answer to that question.
>> >
>> > Will constructs like table-level foreign keys also just get ignored for 
>> > databases that don’t support them?
>> >
>> > In what scenarios would it make sense to use multiple modules, like I do?
>> >
>> > If possible, please provide a link to relevant part of the documentation, 
>> > I’m eager to learn more.
>> >
>> > Best wishes
>> > Nicolas
>> >
>> > On Tue, 1 Sep 2020 at 10.49, Simon King  wrote:
>> >>
>> >> Is __table_args__ the only reason why you are creating separate
>> >>
>> >> modules for the different databases? You can specify parameters for
>> >>
>> >> different database dialects in __table_args__, and the ones that don't
>> >>
>> >> match the current engine will be ignored. For example:
>> >>
>> >>
>> >>
>> >> 
>> >>
>> >> import sqlalchemy as sa
>> >>
>> >> from sqlalchemy.ext.declarative import declarative_base
>> >>
>> >>
>> >>
>> >> Base = declarative_base()
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> class SomeTable(Base):
>> >>
>> >> __tablename__ = "sometable"
>> >>
>> >> __table_args__ = {
>> >>
>> >> "mysql_default_charset": "utf8",
>> >>
>> >> "mysql_engine": "InnoDB",
>> >>
>> >> "sqlite_autoincrement": True,
>> >>
>> >> }
>> >>
>> >> id = sa.Column(sa.Integer(), primary_key=True)
>> >>
>> >> name = sa.Column(sa.Text())
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> engine = sa.create_engine("sqlite:///", echo=True)
>> >>
>> >> Base.metadata.create_all(engine)
>> >>
>> >>
>> >>
>> >> 
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Simon
>> >>

Re: [sqlalchemy] SQLAlchemy MYSQL query utf8 character problem

[Simon King]

On Thu, Sep 3, 2020 at 9:55 AM chat...@gmail.com  wrote:
>
> Trying to query all items from a mysql (charset:utf8) table which has a field 
> that contains rows with chinese and other special characters I am taking the 
> above error
>
> items = session.query(Item).all()
>
> File 
> "/root/.local/share/virtualenvs/server-WesSANjA/lib/python3.8/site-packages/MySQLdb/cursors.py",
>  line 355, in _post_get_result self._rows = self._fetch_row(0) File 
> "/root/.local/share/virtualenvs/server-WesSANjA/lib/python3.8/site-packages/MySQLdb/cursors.py",
>  line 328, in _fetch_row return self._result.fetch_row(size, 
> self._fetch_type) File "/usr/local/lib/python3.8/encodings/cp1252.py", line 
> 15, in decode return codecs.charmap_decode(input,errors,decoding_table)
>
> UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 30: 
> character maps to
>

You mention utf8, but the error suggests that the data is being
decoded as cp1252. Are you declaring an explicit charset when you
create your engine, as suggested here:

https://docs.sqlalchemy.org/en/13/dialects/mysql.html#unicode

What does this output:

for row in dbsession.execute("show variables like 'character%'").fetchall():
print(row)

Warning: I've run an application for a long time where I didn't
specify the charset in the connection string. SQLAlchemy defaulted to
encoding strings as utf8 (because the dialect didn't support unicode
strings). However, my output from the above command looked something
like this:

('character_set_client', 'latin1')
('character_set_connection', 'latin1')
('character_set_database', 'utf8')
('character_set_filesystem', 'binary')
('character_set_results', 'latin1')
('character_set_server', 'latin1')
('character_set_system', 'utf8')
('character_sets_dir', '/usr/share/mysql/charsets/')

This meant that SQLAlchemy was sending utf-8 strings, but the database
was interpreting them as latin1. To make things worse, some of my
tables have a default charset of latin1, and the others are utf8. The
result is that the tables that are declared to hold latin1 data
actually hold utf8, and the tables that are declared to hold utf8
actually hold double-encoded utf8.

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfzHCGuwR3Xe-yBRLfMOm%3DjEOpY3KDrbkfPFA20zuB3Zw%40mail.gmail.com.

Re: [sqlalchemy] How can I use a composite foreign-key constraint with a "mixin" class using declarative?

[Simon King]

To be honest, I looked for documentation before I wrote my reply to
you, and couldn't find anything. I just know that I often use MySQL in
production but sqlite for tests, and sqlite never complained about the
mysql-specific options in the table args.

dialect-specific options end up in table.dialect_options:

https://docs.sqlalchemy.org/en/13/core/metadata.html#sqlalchemy.schema.Table.dialect_options

But different dialects seem to handle them in different ways. For
example, here's how mysql consumes table-level options:

https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/dialects/mysql/base.py#L1871

whereas sqlite does this:

https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/dialects/sqlite/base.py#L1120

Simon

On Thu, Sep 3, 2020 at 5:38 AM Nicolas Lykke Iversen  wrote:
>
> Thank you, Simon.
>
> Yes, __table_args__ is the only reason I’m creating separate modules.
>
> Where do you see that arguments that don’t match the database dialect of the 
> engine get ignored? I looked at the source code for answering this question 
> myself, but couldn't find an answer to that question.
>
> Will constructs like table-level foreign keys also just get ignored for 
> databases that don’t support them?
>
> In what scenarios would it make sense to use multiple modules, like I do?
>
> If possible, please provide a link to relevant part of the documentation, I’m 
> eager to learn more.
>
> Best wishes
> Nicolas
>
> On Tue, 1 Sep 2020 at 10.49, Simon King  wrote:
>>
>> Is __table_args__ the only reason why you are creating separate
>>
>> modules for the different databases? You can specify parameters for
>>
>> different database dialects in __table_args__, and the ones that don't
>>
>> match the current engine will be ignored. For example:
>>
>>
>>
>> 
>>
>> import sqlalchemy as sa
>>
>> from sqlalchemy.ext.declarative import declarative_base
>>
>>
>>
>> Base = declarative_base()
>>
>>
>>
>>
>>
>> class SomeTable(Base):
>>
>> __tablename__ = "sometable"
>>
>> __table_args__ = {
>>
>> "mysql_default_charset": "utf8",
>>
>> "mysql_engine": "InnoDB",
>>
>> "sqlite_autoincrement": True,
>>
>> }
>>
>> id = sa.Column(sa.Integer(), primary_key=True)
>>
>> name = sa.Column(sa.Text())
>>
>>
>>
>>
>>
>> engine = sa.create_engine("sqlite:///", echo=True)
>>
>> Base.metadata.create_all(engine)
>>
>>
>>
>> 
>>
>>
>>
>>
>>
>> Simon
>>
>>
>>
>> On Fri, Aug 28, 2020 at 10:35 AM Nicolas Lykke Iversen
>>
>>  wrote:
>>
>> >
>>
>> > Hi all,
>>
>> >
>>
>> > I need to create identical models (mapped classes) for several database 
>> > backends, e.g. MySQL and MSSQL, that take different __table_args__.
>>
>> >
>>
>> > Thus, I've opted for created one base for each database backend defining 
>> > the __table_args__ (base.py), while using common mixins for defining the 
>> > columns (mixin.py). The bases and mixins are then combined in mssql.py and 
>> > mysql.py to create the models.
>>
>> >
>>
>> > The problem is that I don't know how to create a table-level composite 
>> > foreign-key constraint (ForeignKeyConstraint) by reading the following 
>> > documentation:
>>
>> >
>>
>> > https://docs.sqlalchemy.org/en/13/orm/extensions/declarative/mixins.html#mixing-in-relationships
>>
>> >
>>
>> > Indeed, it can create column-level foreign-keys (ForeignKey), but defining 
>> > the ForeignKeyConstraint on any of the below classes yield errors, e.g.:
>>
>> >
>>
>> > class Project():
>>
>> >id = Column(Integer, primary_key=True)
>>
>> >scan_id = Column(Integer, nullable=False)
>>
>> >...
>>
>> >
>>
>> > class Project(Base, mixin.Project):
>>
>> >ForeignKeyConstraint(['project.scan_id'], ['stash_scan.id'])
>>
>> >
>>
>> > sqlalchemy.exc.NoForeignKeysError: Could not determine join condition 
>> > between parent/child tables on relationship Scan.projects - there are no 
>> > foreign keys linking these tables.  Ensure that referencing columns are 
>> > associated with a ForeignKey or ForeignKeyConstraint, or s

Re: [sqlalchemy] How can I use a composite foreign-key constraint with a "mixin" class using declarative?

[Simon King]

Is __table_args__ the only reason why you are creating separate
modules for the different databases? You can specify parameters for
different database dialects in __table_args__, and the ones that don't
match the current engine will be ignored. For example:


import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()


class SomeTable(Base):
__tablename__ = "sometable"
__table_args__ = {
"mysql_default_charset": "utf8",
"mysql_engine": "InnoDB",
"sqlite_autoincrement": True,
}
id = sa.Column(sa.Integer(), primary_key=True)
name = sa.Column(sa.Text())


engine = sa.create_engine("sqlite:///", echo=True)
Base.metadata.create_all(engine)




Simon

On Fri, Aug 28, 2020 at 10:35 AM Nicolas Lykke Iversen
 wrote:
>
> Hi all,
>
> I need to create identical models (mapped classes) for several database 
> backends, e.g. MySQL and MSSQL, that take different __table_args__.
>
> Thus, I've opted for created one base for each database backend defining the 
> __table_args__ (base.py), while using common mixins for defining the columns 
> (mixin.py). The bases and mixins are then combined in mssql.py and mysql.py 
> to create the models.
>
> The problem is that I don't know how to create a table-level composite 
> foreign-key constraint (ForeignKeyConstraint) by reading the following 
> documentation:
>
> https://docs.sqlalchemy.org/en/13/orm/extensions/declarative/mixins.html#mixing-in-relationships
>
> Indeed, it can create column-level foreign-keys (ForeignKey), but defining 
> the ForeignKeyConstraint on any of the below classes yield errors, e.g.:
>
> class Project():
>id = Column(Integer, primary_key=True)
>scan_id = Column(Integer, nullable=False)
>...
>
> class Project(Base, mixin.Project):
>ForeignKeyConstraint(['project.scan_id'], ['stash_scan.id'])
>
> sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between 
> parent/child tables on relationship Scan.projects - there are no foreign keys 
> linking these tables.  Ensure that referencing columns are associated with a 
> ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.
>
> Is it not possible to use ForeignKeyConstraint with the base/mixin design I'm 
> using?
>
> SQLAlchemy Version: 1.3.17.
>
> base.py:
> class SqlBase():
>@declared_attr
>   def __tablename__(cls):
>   return f'stash_{cls.__name__.lower()}'
>
>def __repr__(self):
>   return f'<{self.__class__.__name__}(id=\'{self.id}\')>'
>
> class MySqlBase(SqlBase):
>__table_args__ = {'mysql_default_charset': 'utf8',
>   'mysql_collate': 'utf8_bin'}
>
> class MsSqlBase(SqlBase):
>__table_args__ = {}
>
> mixin.py:
> class Project():
>id = Column(Integer, primary_key=True)
>key = Column(Text, nullable=False)
>name = Column(Text, nullable=False)
>href = Column(Text, nullable=False)
>
>@declared_attr
>def scan_id(cls):
>   return Column(Integer, ForeignKey('stash_scan.id', onupdate='CASCADE', 
> ondelete='CASCADE'), nullable=False)
>
>@declared_attr
>def scan(cls):
>   return relationship('Scan', back_populates='projects')
>
> mssql.py:
> Base = declarative_base(cls=db.MsSqlBase)
>
> class Scan(Base, mixin.Scan):
>   pass
>
> class Project(Base, mixin.Project):
>pass
>
> mysql.py:
> Base = declarative_base(cls=db.MySqlBase)
>
> class Scan(Base, mixin.Scan):
>   pass
>
> class Project(Base, mixin.Project):
>pass
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/39315b84-f595-47af-adc4-2b4afa508c67n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexewJhgRY6e_hQgFO48P-ac0HpfuwMgkjyx%3D8GPCa3LmXQ%40mail.gmail.com.

Re: [sqlalchemy] Getting column data from result set with column name as a string.

[Simon King]

If you want to get an attribute of an object where the name of the
attribute is variable, you can use the getattr function:

attrname = "lastname"
column = getattr(User, attrname)
for item in session.query(column):
print(item)

or:

attrname = "lastname"
for user in session.query(User):
value = getattr(user, attrname)
print(value)

Simon

On Wed, Aug 19, 2020 at 3:20 PM Mike Bayer  wrote:
>
> __table__ is public (private would be a single or double underscore prefix 
> only), but also you could use inspect(cls).column_attrs:
>
> https://docs.sqlalchemy.org/en/13/orm/mapping_api.html?highlight=column_attrs#sqlalchemy.orm.Mapper.column_attrs
>
>
>
> On Wed, Aug 19, 2020, at 1:22 AM, Dale Preston wrote:
>
> Thanks.  The label is an interesting option; I'll look into that.
>
> On a StackOverflow thread, I got row.__table__.columns which I can iterate 
> over and test the key, allowing me to get the column I need but I have to 
> loop through all the columns until I find the one I want for each row because 
> columns doesn't have an index either.  I also don't like using a private 
> property but I guess (hope) __table__ would always be there.
> On Tuesday, August 18, 2020 at 6:05:49 PM UTC-5 Mike Bayer wrote:
>
>
>
> On Tue, Aug 18, 2020, at 5:20 PM, Dale Preston wrote:
>
> I'm using sqlalchemy 1.3.18.  I'm trying to write an app that looks at data 
> from an ORM declarative table without necessarily knowing the table 
> definition.
>
> What I am looking for is a way to get a single object (row in resultSet), 
> having the name of column[1] is "lastname", and having "lastname" as a string 
> in memory, how can I get the value of the "lastname" field from the row in 
> resultSet?
>
> It's easy if I know in code that the row has a lastname property and I can 
> use row.lastname but I want to do something like row["lastname"] or 
> row.columns["lastname"] if there's a way.
>
>
> to get individual columns in the row you query for those columns directly:
>
>
> row = sess.query(User.lastname).first()
>
> print(row.lastname)
>
>
> otherwise you can always label a column if you need:
>
> row = sess.query(User.anything.label("lastname")).first()
>
> print(row.lastname)
>
>
>
>
>
>
>
>
> I'm using reflection to get the columns for the table.  Here's some code I 
> tried:
>
> class Users(Base):
> __tablename__ = 'users'
> userid = Column(String(80), primary_key=True)
> lastname = Column(String(40), nullable=False)
> firstname = Column(String(40), nullable=False)
> emailaddress = Column(String(80), nullable=False)
>
> def ReflectTableColumns(DbEngine, meta, targetTable):
> tableschema = Table(targetTable, meta, autoload=True, 
> autoload_with=DbEngine)
> cols = dict()
> for c in tableschema.columns:
> print("{0}\t|\t{1}".format(c.name, c.type))
> cols[c.name] = c.type
>
> return cols
>
> def GetUsers():
> DBSession = sessionmaker(bind=Engine)
> session = DBSession()
> ShowTableData(session.query(Users).all(), 'users')
>
>
> def ShowTableData(resultSet, tablename):
> columns = ReflectTableColumns(Engine, Base.metadata, tablename)
> columnNames = list(columns.keys())
> print (type(resultSet))
> for row in resultSet:
> print (row.items[columnNames[1]])
> print (row.columns[columnNames[1]])
> print (row[columnNames[1]])
>
> GetUsers()
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/f11bf60e-e872-489e-9a9b-03998440bbb1n%40googlegroups.com.
>
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/975d8626-39cf-439f-b5ed-df1e9680f7bfn%40googlegroups.com.
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message 

Re: [sqlalchemy] Re: Deletion of a row from an association table

[Simon King]

SQLAlchemy normally presents a many-to-many relationship as a list on
both sides. You've got "Machine.children", which is a list of Options,
and "Option.parents", which is a list of Machines.

If you remove one of the options from a machine.children list, you'll
find that SQLAlchemy removes the entry from the association table.
Something like this:

machine.children.remove(option_to_remove)

However, this does have the downside that when you access
"machine.children", SQLAlchemy will load all the Options for that
Machine from the database. This is a waste of effort if you are only
trying to delete one of them. (But if you've got them loaded anyway,
it doesn't matter)

The other option is to delete the row explicitly, something like this:

statement = Machine_Options.delete().where(
Machine_Options.c.machine_FK == machine.machine_ID,
Machine_Options.c.options_FK == option.option_ID
)
session.execute(statement)

But beware that if you do this, any machines or options already loaded
in your session won't be aware that the delete happened. If they had
already loaded their "parents" or "children" relationships, that
cached data will not match what is in the database.

Hope that helps,

Simon

On Wed, Aug 12, 2020 at 3:05 AM William Phillips  wrote:
>
> For the sake of completeness I am including the code to disconnect an option 
> from a machine using only python/SQLite code.
>
> def removeOption(bladeKey,  OptionKey):
>
> """
> DELETE from blade_options
> WHERE blade_FK == ?
>AND options_FK == ?
> """
> import sqlite3
> dbPath = config.database_path
> sqliteConnection = sqlite3.connect(dbPath)
> cursor = sqliteConnection.cursor()
> sql = 'DELETE from blade_options WHERE blades_ID == ? AND options_ID == 
> ?; '
> cursor.execute(sql, (bladeKey,  OptionKey, ))
> sqliteConnection.commit()
> sqliteConnection.close()
> return
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/45da4231-3550-4f5b-882e-9e61bef86bd5o%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdhhOPgkXi8owvS0t1hXE68zZYf9kam_MOYZ6PpjCe4Ew%40mail.gmail.com.

Re: [sqlalchemy] Flask SQlAlchemy BaseQuery Paginate not working correctly

[Simon King]

This is the same problem: you're writing a query that joins 3 tables
together, and then applying a "LIMIT 20" to that query. If you look
carefully at your 20 rows of psql output, I expect you'll see the same
aggregates_id appear more than once. There are less than 20 distinct
Aggregate objects. When SQLAlchemy receives those rows, it skips the
duplicates.

The SQL looks something like this:

SELECT ... FROM aggregates
LEFT JOIN aggregate_blocks ON ...
LEFT JOIN blocks ON ...
LIMIT 20

You need the LIMIT to apply just to the aggregates table, rather than
the joined tables. This is a bit tricky because you want to filter by
a field in one of those joined tables. You could use an EXISTS
function, something like this:

SELECT * FROM aggregates
WHERE EXISTS (
  SELECT 1
  FROM aggregate_blocks
  INNER JOIN blocks ON aggregate_blocks.block_id = block.id
  WHERE aggregate_blocks.aggregate_id = aggregates.id
  AND blocks.is_complete = false
)
LIMIT 20

If you run that in psql, I think you should get 20 *different*
aggregates rows back.

If that works, then to turn it into SQLAlchemy ORM syntax, you should
use Query.exists():

https://docs.sqlalchemy.org/en/13/orm/query.html#sqlalchemy.orm.query.Query.exists

Hope that helps,

Simon

On Fri, Aug 14, 2020 at 3:56 PM Prerna Pandit  wrote:
>
> Hey Simon,
>
> Thanks so much for replying to my question.  I reworked my code to use 
> sqlalchemy ORM and took off flask and paginate so I can narrow down the 
> issue. My models now extend from declarative_base.
>
> engine = 
> create_engine('postgresql://postgres:postgres@localhost:5400/postgres')
> Session = sessionmaker(bind=engine)
>
> Base = declarative_base()
> session = Session()
>
> Models
>
> class Aggregate(Base):
> __tablename__ = 'aggregates'
> id = Column(UUID(as_uuid=True), primary_key=True,
>server_default=text('uuid_generate_v4()'))
> site_id = Column(UUID(as_uuid=True), nullable=True)
> created_at = Column(DateTime, default=sa.func.now())
> created_by = Column(UUID(as_uuid=True), nullable=True)
> updated_at = Column(DateTime, default=sa.func.now(), 
> onupdate=sa.func.now())
> updated_by = Column(UUID(as_uuid=True), nullable=True)
> blocks = relationship('AggregateBlock', cascade='all, delete-orphan',
>   passive_deletes=True, 
> back_populates='aggregate')
>
>
> class Block(Base):
>
> __tablename__ = 'blocks'
> id = Column(UUID(as_uuid=True), primary_key=True,
>server_default=text('uuid_generate_v4()'))
> type = Column(Text, nullable=False)
> heading = Column(Text, nullable=True)
> subheading = Column(Text, nullable=True)
> label = Column(Text, nullable=True)
> is_complete = Column(Boolean, default=False)
> created_at = Column(DateTime, default=sa.func.now())
> created_by = Column(UUID(as_uuid=True), nullable=True)
> updated_at = Column(DateTime, default=sa.func.now(), 
> onupdate=sa.func.now())
> updated_by = Column(UUID(as_uuid=True), nullable=True)
> aggregates = relationship('AggregateBlock', cascade='all, delete-orphan',
>  passive_deletes=True, back_populates='block')
>
>
> class AggregateBlock(Base):
> __tablename__ = 'aggregate_blocks'
> id = Column(UUID(as_uuid=True), primary_key=True,
>server_default=text('uuid_generate_v4()'))
> block_id = Column(UUID(as_uuid=True),
>  ForeignKey('blocks.id', ondelete='CASCADE'), 
> nullable=False, index=True)
> aggregate_id = Column(UUID(as_uuid=True),
>  ForeignKey('aggregates.id', 
> ondelete='RESTRICT'), nullable=False)
> position = Column(Integer, nullable=False)
> block = relationship('Block', back_populates='aggregates')
> aggregate = relationship('Aggregate', back_populates='blocks')
>
>
> Query:
>
> select = session.query(Aggregate).order_by(Aggregate.created_at) \
> .join(AggregateBlock) \
> .join(Block) \
> .filter(Block.is_complete == complete) \
>
> all_results = select.all()
> limit_results = select.limit(20).all()
>
>
> I still get inconsistent results when I apply limit. Like select.all() will 
> return 47 rows but  with limit it'll return anywhere between 11 to 15.  If I 
> take the generated SQL query and run it directly in psql, I get the correct 
> count.
>
> SELECT aggregates.id AS aggregates_id, aggregates.site_id AS 
> aggregates_site_id, aggregates.created_at AS aggregates_created_at, 
> aggregates.created_by AS aggregates_created_by, aggregates.updated_at AS 
> aggregates_updated_at, aggregates.updated_by AS aggregates_updated_by
> FROM aggregates JOIN aggregat

Re: [sqlalchemy] Flask SQlAlchemy BaseQuery Paginate not working correctly

[Simon King]

"paginate" is not an SQLAlchemy function, so you'd be better off
asking the author of whatever is providing that feature.

However, I would guess that maybe paginate is naively applying
something like "LIMIT 20" to the query. This doesn't work properly
when you join along a one-to-many relationship, because if you have
(for example) 2 "parent" objects, each with 5 "child" objects, the
query will return 10 rows, but SQLAlchemy de-duplicates the results to
return just the 2 parent objects.

Simon

On Thu, Aug 13, 2020 at 3:31 PM Prerna Pandit  wrote:
>
> Hello, I've been struggling with this issue for the past couple of days and  
> would really, truly appreciate if someone could please give me pointers or 
> direction as to what I might be missing.
>
>
> Here are my models;
> class Aggregate(db.Model):
> id = db.Column(UUID(as_uuid=True), primary_key=True,
>server_default=db.text('uuid_generate_v4()'))
> blocks = db.relationship('AggregateBlock', cascade='all, delete-orphan',
>  passive_deletes=True, 
> back_populates='aggregate')..
>
>
> class AggregateBlock(db.Model):
>
> id = db.Column(UUID(as_uuid=True), primary_key=True,
>server_default=db.text('uuid_generate_v4()'))
> block_id = db.Column(UUID(as_uuid=True),
>  db.ForeignKey('blocks.id', ondelete='CASCADE'), 
> nullable=False, index=True)
> aggregate_id = db.Column(UUID(as_uuid=True),
>  db.ForeignKey('aggregates.id', 
> ondelete='RESTRICT'), nullable=False)
> block = db.relationship('Block', back_populates='aggregates')
> aggregate = db.relationship('Aggregate', back_populates='blocks')
>
>
>
>
> class Block(db.Model):
> id = db.Column(UUID(as_uuid=True), primary_key=True,
>server_default=db.text('uuid_generate_v4()'))
> is_complete = db.Column(db.Boolean, default=False)
> aggregates = db.relationship('AggregateBlock', cascade='all, 
> delete-orphan',
>  passive_deletes=True, back_populates='block')
>
>
> from flask_sqlalchemy import SQLAlchemy
> db = SQLAlchemy()
>
> select = 
> db.session.query(Aggregate).join(AggregateBlock).join(Block).filter(Block.is_complete
>  == complete)
>
> print(len(select.all())
>
> print(len(select.paginate(per_page=20).items())
>
>
> If I do a select.all(), I get the right number of rows which is 47.  However, 
> if I try to paginate for a per_page size say 20, I lot a less rows like 11.
> select.paginate(per_page=20).
> The number could go up to 21 or so as I increase the page size.  Why would 
> paginate decrease the number of returned records?
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/e5a14305-4e63-4467-9610-1faf3f8c8412o%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeg0K1URZZbi4oaQHtLFkCVbE3Fn2%2BY%3DDaaZymUp9S-qg%40mail.gmail.com.

Re: [sqlalchemy] Encrypt/Decrypt specific column(s)

[Simon King]

Not in the traditional sense, no. ORDER BY is implemented by the
database, and with client-side encryption, the database only ever sees
encrypted strings.

Simon

On Fri, Jul 10, 2020 at 8:41 AM Justin Van Vuuren  wrote:
>
> Also, regarding the client side approach, would one be able to do an order by 
> query?
>
> On Thu, 9 Jul 2020 at 21:28, Justvuur  wrote:
>>
>> A, ok ye, I understand, then I prefer client side.
>> I just need to sort out the encryption then... I'm not too familiar with 
>> encryption algorithms but at the moment its using a text key from a config 
>> file and the mode is CBC which I guess is what is generating a different 
>> value each time right?
>> In Mike's example he's using ECB mode which is the default.
>> What would be the best mode to use for my scenario? A good balance between 
>> secure and performance that works for client side?
>>
>> On Thursday, 9 July 2020 20:18:52 UTC+2, Jonathan Vanasco wrote:
>>>
>>>
>>>
>>> On Thursday, July 9, 2020 at 2:12:36 PM UTC-4, Justvuur wrote:

 I've done some more digging... It seems when I did the search for 
 "secrets", the text is encrypted and compared to the value in the columns,
>>>
>>>
>>> That is how client-side encryption works.  If you want to search for 
>>> "secrets", you need to use server-side encryption (which depends on the 
>>> database). In those systems, the server will decrypt the column in every 
>>> row when searching - which can be a performance issue.
>>>
 The thing is this type of comparison wont work, the algorithm generates a 
 different string each encryption for the same string.
>>>
>>>
>>>  What are you using for your encryption key? The key should be persistent, 
>>> and should always generate the same output for a given input.  In the 
>>> example from Michael Bayer, a random uuid is used as a placeholder.
>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> description.
>> ---
>> You received this message because you are subscribed to the Google Groups 
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to sqlalchemy+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sqlalchemy/58a78976-3ebe-428b-824f-165b15f61cb4o%40googlegroups.com.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/CAJK07SZwotU64v22pmRJn4SR6aV2cb%2B6U_tKMwJxgG9Pe0cUQA%40mail.gmail.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdJquCGHv0B5aAOUgkFBXijxtEM48Fo0YoSFrSOsNo4Zw%40mail.gmail.com.

Re: [sqlalchemy] convert subset to dictionary

[Simon King]

You should start by enabling SQLAlchemy logging to see the actual
queries that are being run. The easiest way is to pass "echo=True" to
your create_engine call. (You could also try echo="debug", but since
you've got hundreds of thousands of rows you'll be swamped)

Verify that the queries look correct (for example, they have the right
join conditions). If they look OK but they run slowly, use your
database's tools (eg. EXPLAIN or EXPLAIN ANALYZE) to understand why.

Simon

On Wed, Jul 8, 2020 at 8:22 AM Justvuur  wrote:
>
> I'd like to redesign the DB but that's not on the cards at the moment (or 
> ever hehe).
>
> Thanks for the feedback, I appreciate all the help, I really do.
> What puzzles me now is, why would the looping through each student (using 
> with_entities) and getting the subjects (using with_entities) for each 
> student be faster (from 2:25 min down to 0:19 min) than using the 
> student.subjects approach?
> I know I'm including 3 or 4 less columns but surely it wont cost 2 minutes to 
> have them included?
>
>
>
> On Tuesday, 7 July 2020 at 19:31:56 UTC+2 Jonathan Vanasco wrote:
>>
>> Based on what you shared above:
>>
>> * The "Subject" table is: `StudentId, SubjectCode, SubjectName`
>> * There are 181 subjects
>>
>> It looks like you don't have a "Subject" table, but a "StudentAndSubject" 
>> table.
>>
>> I think you'd have a bigger performance improvement by normalizing that data 
>> into two tables:
>>
>> Subject:  SubjectId (primary key), SubjectCode, SubjectName
>> Student2Subject: StudentId, SubjectId, (primary key is both)
>>
>> Assuming this can be done with your data... the database performance should 
>> improve because
>>
>> 1. The raw filestorage will decrease
>> 2. The in-memory dataset size will decrease
>>
>> You could then either
>>
>> 1. use the Subject table as part of a joined query to keep things simple, or
>> 2. just select off a join of Student+Student2Subject , and query all the 
>> Subjects separately.  Even if there are 2000 subjects total, it should only 
>> take a few ms to get all that into a python datastructure that is used to 
>> generate your csv
>>
>>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/3b701d26-2f3b-4407-bc69-2dc13df60caan%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexc40-E4ST6Bn5V1EOtgHt7nhoZXT1fHvQPL%2BdSJX%3D0WyQ%40mail.gmail.com.

Re: [sqlalchemy] convert subset to dictionary

[Simon King]

2:25 minutes, even for 1,267,000 rows, sounds like quite a long time
to me. Are you physically a long way from the database, querying
across the internet? Do the tables contain many columns, or perhaps
large BLOB or TEXT fields?

If there are large fields that you don't need very often, you could
consider marking them as "deferred", then they wouldn't be loaded by
default:

https://docs.sqlalchemy.org/en/13/orm/loading_columns.html#deferred-column-loading

I suppose if your database server was underpowered, loading over a
million rows might exhaust its memory and cause it to start swapping,
which would hurt performance, in which case querying for smaller
amounts of data might be better.

Simon

On Tue, Jul 7, 2020 at 12:53 PM Justvuur  wrote:
>
> I'm currently testing with 7000 students with 181 subjects.
> I first went over to the DB to run the query directly on there. I did an 
> innerjoin between the two tables.
> It took about 2:25 minutes. I then selected specific columns from the two 
> tables and the time dropped to about 2 minutes or so.
> I then tried a "select subjects where student ID in (...)" query and that 
> executed a lot quicker than the others.
>
> So then on the python side of things, I first fetched the students and only 
> selected id, created_on and updated_on columns using the with_entities option.
> That seemed to take less than 1 second. I then, for each student, executed a 
> DB query to fetch the subjects using the with_entities option.
> For some reason, this seems to work faster than using the  
> eager-loading/students.subjects/attribute_mapped_collection. They seem to 
> take 2 minutes longer than the above (what should be inefficient) approach.
>
> On Tuesday, 7 July 2020 at 12:50:13 UTC+2 Simon King wrote:
>>
>> How long is it taking? You mentioned 2000 students with 100 subjects
>> each, so there are something like 200,000 rows in the Subjects table,
>> and you need to load all of it. I wouldn't expect that to take longer
>> than a couple of seconds though.
>>
>> Simon
>>
>> On Mon, Jul 6, 2020 at 7:34 PM Justvuur  wrote:
>> >
>> > I added the eager-loading but it seems to slow down the SQL query quite a 
>> > lot.
>> > It's as if now, the SQL query is taking longer but the generating of the 
>> > file is quicker hehe... I guess now the queries are being fired before to 
>> > populate subjects.
>> > It's still taking relatively the same amount of time though.
>> >
>> >
>> > On Friday, 3 July 2020 17:07:43 UTC+2, Simon King wrote:
>> >>
>> >> Are you eager-loading the "student.subjects" relationship? If not,
>> >> that will give you the biggest performance increase. Without that, you
>> >> will be issuing a separate DB query for each of the students, to load
>> >> that student's subjects. Eager-loading allows you to preload the
>> >> subjects for every student in a single query:
>> >>
>> >> https://docs.sqlalchemy.org/en/13/orm/loading_relationships.html#joined-eager-loading
>> >>
>> >> Simon
>> >>
>> >> On Fri, Jul 3, 2020 at 1:36 PM Justvuur  wrote:
>> >> >
>> >> > Hi Simon, thanks for the help! I've never used that before, it's quite 
>> >> > handy.
>> >> >
>> >> > I'm looping through all the students and printing them and their 
>> >> > subject details to a CSV file.
>> >> > What makes things a tad complicated is the subjects must appear in a 
>> >> > specific order.
>> >> > There is a table that has the subject code and order number 
>> >> > (ordered_subjects used below is the resultset from it).
>> >> > I printed out the timing and found the problem to be with a nested for 
>> >> > loop.
>> >> >
>> >> > I was hoping to reduce that process time by using a map that 
>> >> > automatically gets populated instead of having to create it on the fly.
>> >> >
>> >> > Before - subjects_collection "attribute_mapped_collection":
>> >> > 
>> >> > for row in students:
>> >> > row_no += 1
>> >> >
>> >> > for subject in row.subjects:
>> >> > student_subjects[subject.code] = subject.value
>> >> >
>> >> > csv_row = [row_no]
>> >> > csv_row += [student_subjects.get(x.code, '') for x in ordered_subjects]
>> >> > csv_row +=

Re: [sqlalchemy] convert subset to dictionary

[Simon King]

How long is it taking? You mentioned 2000 students with 100 subjects
each, so there are something like 200,000 rows in the Subjects table,
and you need to load all of it. I wouldn't expect that to take longer
than a couple of seconds though.

Simon

On Mon, Jul 6, 2020 at 7:34 PM Justvuur  wrote:
>
> I added the eager-loading but it seems to slow down the SQL query quite a lot.
> It's as if now, the SQL query is taking longer but the generating of the file 
> is quicker hehe... I guess now the queries are being fired before to populate 
> subjects.
> It's still taking relatively the same amount of time though.
>
>
> On Friday, 3 July 2020 17:07:43 UTC+2, Simon King wrote:
>>
>> Are you eager-loading the "student.subjects" relationship? If not,
>> that will give you the biggest performance increase. Without that, you
>> will be issuing a separate DB query for each of the students, to load
>> that student's subjects. Eager-loading allows you to preload the
>> subjects for every student in a single query:
>>
>> https://docs.sqlalchemy.org/en/13/orm/loading_relationships.html#joined-eager-loading
>>
>> Simon
>>
>> On Fri, Jul 3, 2020 at 1:36 PM Justvuur  wrote:
>> >
>> > Hi Simon, thanks for the help! I've never used that before, it's quite 
>> > handy.
>> >
>> > I'm looping through all the students and printing them and their subject 
>> > details to a CSV file.
>> > What makes things a tad complicated is the subjects must appear in a 
>> > specific order.
>> > There is a table that has the subject code and order number 
>> > (ordered_subjects used below is the resultset from it).
>> > I printed out the timing and found the problem to be with a nested for 
>> > loop.
>> >
>> > I was hoping to reduce that process time by using a map that automatically 
>> > gets populated instead of having to create it on the fly.
>> >
>> > Before - subjects_collection "attribute_mapped_collection":
>> > 
>> > for row in students:
>> > row_no += 1
>> >
>> > for subject in row.subjects:
>> > student_subjects[subject.code] = subject.value
>> >
>> > csv_row = [row_no]
>> > csv_row += [student_subjects.get(x.code, '') for x in ordered_subjects]
>> > csv_row += [row.created_on, row.updated_on]
>> >
>> > writer.writerow([x.encode('utf-8') if type(x) == unicode else x for x 
>> > in csv_row])
>> >
>> >
>> > After adding the subjects_collection "attribute_mapped_collection", I 
>> > unfortunately did not see a change in performance.
>> >
>> > After - subjects_collection "attribute_mapped_collection":
>> > 
>> > for row in students:
>> > row_no += 1
>> > csv_row = [row_no]
>> > csv_row += [row.subjects_collection.get(x.code, '').value for x in 
>> > ordered_subjects]
>> > csv_row += [row.created_on, row.updated_on]
>> >
>> > writer.writerow([x.encode('utf-8') if type(x) == unicode else x for x 
>> > in csv_row])
>> >
>> >
>> > class Subject(db.Model):
>> > __tablename__ = 'subjects'
>> >
>> > student_id = db.Column(db.Integer, db.ForeignKey('students.id'), 
>> > primary_key=True)
>> >
>> > code = db.Column(db.String(50), primary_key=True)
>> >
>> > value= db.Column(db.String)
>> >
>> > def __init__(self, code , value):
>> > self.code = code
>> > self.value = value
>> >
>> >
>> > class Student(ResourceMixin, db.Model):
>> > __tablename__ = 'students'
>> >
>> > subjects= db.relationship('Subject', backref='student')
>> >
>> > id = db.Column(db.Integer, primary_key=True)
>> >
>> > subjects_collection = relationship("Subject", 
>> > collection_class=attribute_mapped_collection('code'))
>> >
>> > Can you see a way I can optimize this? Any ideas?
>> >
>> >
>> > On Friday, 3 July 2020 12:31:03 UTC+2, Simon King wrote:
>> >>
>> >> Are you trying to optimise the database access (ie. minimize the
>> >> number of queries), or provide a nice dictionary-style API for your
>> >> Student ob

Re: [sqlalchemy] convert subset to dictionary

[Simon King]

Are you eager-loading the "student.subjects" relationship? If not,
that will give you the biggest performance increase. Without that, you
will be issuing a separate DB query for each of the students, to load
that student's subjects. Eager-loading allows you to preload the
subjects for every student in a single query:

https://docs.sqlalchemy.org/en/13/orm/loading_relationships.html#joined-eager-loading

Simon

On Fri, Jul 3, 2020 at 1:36 PM Justvuur  wrote:
>
> Hi Simon, thanks for the help! I've never used that before, it's quite handy.
>
> I'm looping through all the students and printing them and their subject 
> details to a CSV file.
> What makes things a tad complicated is the subjects must appear in a specific 
> order.
> There is a table that has the subject code and order number (ordered_subjects 
> used below is the resultset from it).
> I printed out the timing and found the problem to be with a nested for loop.
>
> I was hoping to reduce that process time by using a map that automatically 
> gets populated instead of having to create it on the fly.
>
> Before - subjects_collection "attribute_mapped_collection":
> 
> for row in students:
> row_no += 1
>
> for subject in row.subjects:
> student_subjects[subject.code] = subject.value
>
> csv_row = [row_no]
> csv_row += [student_subjects.get(x.code, '') for x in ordered_subjects]
> csv_row += [row.created_on, row.updated_on]
>
> writer.writerow([x.encode('utf-8') if type(x) == unicode else x for x in 
> csv_row])
>
>
> After adding the subjects_collection "attribute_mapped_collection", I 
> unfortunately did not see a change in performance.
>
> After - subjects_collection "attribute_mapped_collection":
> 
> for row in students:
> row_no += 1
> csv_row = [row_no]
> csv_row += [row.subjects_collection.get(x.code, '').value for x in 
> ordered_subjects]
> csv_row += [row.created_on, row.updated_on]
>
> writer.writerow([x.encode('utf-8') if type(x) == unicode else x for x in 
> csv_row])
>
>
> class Subject(db.Model):
> __tablename__ = 'subjects'
>
> student_id = db.Column(db.Integer, db.ForeignKey('students.id'), 
> primary_key=True)
>
> code = db.Column(db.String(50), primary_key=True)
>
> value= db.Column(db.String)
>
> def __init__(self, code , value):
> self.code = code
> self.value = value
>
>
> class Student(ResourceMixin, db.Model):
> __tablename__ = 'students'
>
> subjects= db.relationship('Subject', backref='student')
>
> id = db.Column(db.Integer, primary_key=True)
>
> subjects_collection = relationship("Subject", 
> collection_class=attribute_mapped_collection('code'))
>
> Can you see a way I can optimize this? Any ideas?
>
>
> On Friday, 3 July 2020 12:31:03 UTC+2, Simon King wrote:
>>
>> Are you trying to optimise the database access (ie. minimize the
>> number of queries), or provide a nice dictionary-style API for your
>> Student objects? What do you mean when you say that looping over
>> student.subjects is quite heavy?
>>
>> An association proxy can be used to get dict-style access to a relationship:
>>
>> 
>> https://docs.sqlalchemy.org/en/13/orm/extensions/associationproxy.html#proxying-to-dictionary-based-collections
>>
>> There are also a couple of examples in the SQLAlchemy docs that
>> provide a dictionary-style API:
>>
>> 
>> https://docs.sqlalchemy.org/en/13/orm/examples.html#module-examples.dynamic_dict
>>
>> 
>> https://docs.sqlalchemy.org/en/13/orm/examples.html#module-examples.vertical
>>
>> Hope that helps,
>>
>> Simon
>>
>> On Thu, Jul 2, 2020 at 8:46 PM Justvuur  wrote:
>> >
>> > Hi there,
>> >
>> > I'm struggling to find an efficient way to get a two columned subset into 
>> > dictionary form.
>> >
>> > I have an entity that has a subset of data. The subset is linked to the 
>> > entity via Id. The order of the subset of data is defined in another table.
>> >
>> > Example:
>> > Student - Id, firstname, lastname
>> > Subjects - StudentId, SubjectCode, SubjectName
>> >
>> > At the moment I'm looping through the SqlAlchemy result of 
>> > "student.subjects" in python and creating a dictionary from that. It's 
>> > quite heavy, especially when ther

Re: [sqlalchemy] convert subset to dictionary

[Simon King]

Are you trying to optimise the database access (ie. minimize the
number of queries), or provide a nice dictionary-style API for your
Student objects? What do you mean when you say that looping over
student.subjects is quite heavy?

An association proxy can be used to get dict-style access to a relationship:


https://docs.sqlalchemy.org/en/13/orm/extensions/associationproxy.html#proxying-to-dictionary-based-collections

There are also a couple of examples in the SQLAlchemy docs that
provide a dictionary-style API:


https://docs.sqlalchemy.org/en/13/orm/examples.html#module-examples.dynamic_dict

https://docs.sqlalchemy.org/en/13/orm/examples.html#module-examples.vertical

Hope that helps,

Simon

On Thu, Jul 2, 2020 at 8:46 PM Justvuur  wrote:
>
> Hi there,
>
> I'm struggling to find an efficient way to get a two columned subset into 
> dictionary form.
>
> I have an entity that has a subset of data. The subset is linked to the 
> entity via Id. The order of the subset of data is defined in another table.
>
> Example:
> Student - Id, firstname, lastname
> Subjects - StudentId, SubjectCode, SubjectName
>
> At the moment I'm looping through the SqlAlchemy result of "student.subjects" 
> in python and creating a dictionary from that. It's quite heavy, especially 
> when there are 2000+ students with a potential of 100+ subjects each.
>
> For each student, how do I get the subjects as a dictionary for a student 
> where the key is the SubjectCode and the value is the SubjectName?
> Better yet, how can I get a result set: Id, firstname, lastname SubjectCode 
> x, SubjectCode y, etc etc (where the SubjectName becomes the value and the 
> SubjectCode becomes the column)?
>
> Regards,
> Justin
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/57b74c9a-a6e5-494b-b468-d0bdcbcce60co%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeFF0TvOMhapyPgFeZsPnwgsbOUWgiO%2B_YAtyPrd2JF6A%40mail.gmail.com.

Re: [sqlalchemy] SQL injection

[Simon King]

I don't understand. SQLAlchemy will not help you "generate SQL
injections". It's just a library for talking to databases.
Applications that *use* SQLAlchemy might be vulnerable to SQL
injections if they are careless, but if you use it in the recommended
way, SQL statements are parameterized and the parameters are sent
separately, so attackers can't change the syntactic form of the
statement.

Simon

On Wed, Jul 1, 2020 at 4:21 PM divyashi...@gmail.com
 wrote:
>
> SQL injections in the sense , malacious sql statements. or paylaod .
>
> On Wednesday, July 1, 2020 at 10:04:47 AM UTC-5 divyashi...@gmail.com wrote:
>>
>> Hi Simon ,  I am m trying to generate sql injections(in band SQLi  and 
>> outband sqli) for data analysis using sql code genrator Is there any sql 
>> query generator library in python that will help me in generating sql 
>> injections? or can I use SQLAlchemy for that purpose
>> I have tried to use the SQL penetration testing tools , but I am not happy 
>> with the results . Is there any way that I can generate SQL injections 
>> besides manual testing and pen testing .
>> Any information is appreciated
>>
>> On Wednesday, July 1, 2020 at 4:34:10 AM UTC-5 Simon King wrote:
>>>
>>> Hi,
>>>
>>> What do you mean by "SQL injection"?
>>>
>>> Thanks,
>>>
>>> Simon
>>>
>>> On Tue, Jun 30, 2020 at 10:12 PM Divya Shivakumar
>>>  wrote:
>>> >
>>> > Hey how do i generate new sql injections from sqlalchemy . Any links or 
>>> > information is much appreciated
>>> >
>>> > --
>>> > SQLAlchemy -
>>> > The Python SQL Toolkit and Object Relational Mapper
>>> >
>>> > http://www.sqlalchemy.org/
>>> >
>>> > To post example code, please provide an MCVE: Minimal, Complete, and 
>>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>>> > description.
>>> > ---
>>> > You received this message because you are subscribed to the Google Groups 
>>> > "sqlalchemy" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send an 
>>> > email to sqlalchemy+...@googlegroups.com.
>>> > To view this discussion on the web visit 
>>> > https://groups.google.com/d/msgid/sqlalchemy/fa4bac85-0fc2-42b5-b47b-11a35bfd7aa1o%40googlegroups.com.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/1dab490f-564b-4fc1-8a56-8becd3f13fd8n%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexc6d40deuG-p6XXyYLT0JLB%3DqHw2s8Ej1GpkFke2_BjUA%40mail.gmail.com.

Re: [sqlalchemy] SQL injection

[Simon King]

Hi,

What do you mean by "SQL injection"?

Thanks,

Simon

On Tue, Jun 30, 2020 at 10:12 PM Divya Shivakumar
 wrote:
>
> Hey how do i generate new sql injections from sqlalchemy . Any links or 
> information is much appreciated
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/fa4bac85-0fc2-42b5-b47b-11a35bfd7aa1o%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexcL%2BBjEHVD4K4khBxA8s_57T5kymzh7n8pP5TqxYOgATA%40mail.gmail.com.

Re: [sqlalchemy] Re: how can i remove an entry from relational database using sqlalchemy in python

[Simon King]

It looks like you've got a many-to-many relationship between Contract and
Permission, and you want to remove a Permission from a Contract (or vice
versa). Is that right?

If so, you can do something like this:

contract = 
permission = 

contract.permissions.remove(permission)

https://docs.sqlalchemy.org/en/13/orm/basic_relationships.html#deleting-rows-from-the-many-to-many-table

(note that accessing "contract.permissions" will load all the permissions
for that contract from the database, which might be inefficient depending
on your application)

Hope that helps,

Simon

On Fri, May 29, 2020 at 12:38 PM Mo Hus  wrote:

> Hey Jonathan, any luck? thanks
>
> On Thursday, May 28, 2020 at 4:15:38 PM UTC+1, Mo Hus wrote:
>>
>> [image: Capture.PNG]
>>
>> permission_id and contract_id have a relationship in the database
>>
>> How can i using remove a entry for example.. if permission_id = 2 and
>> contract_id = 2 exists in the same entry as shown on line one in database,
>> i want to be able to remove it from my database. (This entry is unique so
>> can only appear once)
>>
>> I have tried PermissionEntity.query.get(contract_id) and
>> PermissionEntity.query.get(permission_id) but doesnt seem to be working
>> as Its not stored in a permission entity.. My relationship does not have an
>> entity. the table i have provided a picture for has a relationship with
>> permissions table and contracts table..
>>
>> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/aac681c3-1a8f-4f99-9d0c-31ab8350518e%40googlegroups.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexe5ZW0stWL0PUyu_mLOTt5XQNctgYmDGyArky6PGhE1Aw%40mail.gmail.com.

Re: [sqlalchemy] Re: SQLAlchemy: UnicodeEncodeError: 'ascii' codec can't encode characters (db engine encoding ignored?)

[Simon King]

What are the values of "encoding" and "nencoding" on the connection object?

https://github.com/oracle/python-cx_Oracle/issues/36
https://stackoverflow.com/a/37600367/395053

You probably need to grab the raw dbapi connection:

https://docs.sqlalchemy.org/en/13/core/connections.html#working-with-raw-dbapi-connections

On Wed, May 6, 2020 at 11:46 AM Anno Nühm  wrote:
>
> ##
> Traceback (most recent call last):
>   File "/data/projects/Python/database/sqlalchemy/sqlalchemy_oracle.py", line 
> 45, in 
> df = pd.read_sql_query(u'SELECT owner, table_name FROM all_tables  WHERE 
> owner LIKE \'äöüßÄÖÜœ\'', con)
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/pandas/io/sql.py", 
> line 326, in read_sql_query
> return pandas_sql.read_query(
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/pandas/io/sql.py", 
> line 1218, in read_query
> result = self.execute(*args)
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/pandas/io/sql.py", 
> line 1087, in execute
> return self.connectable.execute(*args, **kwargs)
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
>  line 976, in execute
> return self._execute_text(object_, multiparams, params)
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
>  line 1145, in _execute_text
> ret = self._execute_context(
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
>  line 1287, in _execute_context
> self._handle_dbapi_exception(
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
>  line 1485, in _handle_dbapi_exception
> util.raise_(exc_info[1], with_traceback=exc_info[2])
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/util/compat.py",
>  line 178, in raise_
> raise exception
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
>  line 1247, in _execute_context
> self.dialect.do_execute(
>   File 
> "/opt/pyenv/versions/3.8.2/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
>  line 590, in do_execute
> cursor.execute(statement, parameters)
> UnicodeEncodeError: 'ascii' codec can't encode characters in position 60-66: 
> ordinal not in range(128)
> ##
>
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/38aa0bb8-93f3-4bc7-a771-19a84a17670d%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeTz2vmLzusfPWXeL9PhSrMnQPhbKY0xY9d1oUfPWkB0w%40mail.gmail.com.

Re: [sqlalchemy] SQLAlchemy: UnicodeEncodeError: 'ascii' codec can't encode characters (db engine encoding ignored?)

[Simon King]

It might help to display the stack trace when the encoding fails, so
we can see exactly where the error is coming from.

Simon

On Wed, May 6, 2020 at 9:01 AM Anno Nühm  wrote:
>
> I am currently engaged in evaluating SQLAlchemy for a new project. When 
> trying to execute queries containing non-ascii characters an exception is 
> raised.
>
> The SQL statement used for carrying out the evaluation:
>
> SELECT owner, table_name FROM all_tables  WHERE owner LIKE 'äöüßÄÖÜœ';
>
>
> Executing this statement in SQL*Plus, SQL Developer results--as expected--in 
> an empty list.
>
> In order to connect to an Oracle database the following code is being used:
>
> from sqlalchemy import create_engine, MetaData, Table, inspect, select
> import pandas as pd
> import keyring
>
> dbtype = 'Oracle'
> dbenv = 'LOCAL'
> dbname = 'MYDB'
> dbsys = '%s%s' % (dbtype, dbenv)
> dbusr = 'myusr'
> dbpwd = keyring.get_password(dbsys, dbusr)
> dbhost = 'mydbhost'
> dbport = 1521
> dbconstr = 'oracle+cx_oracle://%s:%s@%s:%s/%s' % (dbusr, dbpwd, dbhost, 
> dbport, dbname)
>
>
> To evaluate the database engine encoding:
>
> dbencs = ['UTF8', 'UTF-8', 'utf8', 'utf-8', 'latin1', 'ascii', None]
>
> for dbenc in dbencs:
> if dbenc is None:
> engine = create_engine(dbconstr)
> else:
> engine = create_engine(dbconstr, encoding=dbenc)
> con = engine.connect()
>
> try:
> df = pd.read_sql_query(u'SELECT owner, table_name FROM all_tables 
>  WHERE owner LIKE \'äöüßÄÖÜœ\'', con)
> print('SUCCESS: sql query with db encoding %s succeeded!' % dbenc)
> except Exception as e:
> print('ERROR: sql query with db encoding %s failed (%s)' % 
> (dbenc, e))
>
> con.close()
> engine.dispose()
>
>
> Regardless of the encoding specified when creating the db engine, every 
> single attempt to executed the query raises an exception
>
> ERROR: sql query with db encoding UTF8 failed ('ascii' codec can't encode 
> characters in position 60-66: ordinal not in range(128))
> ERROR: sql query with db encoding UTF-8 failed ('ascii' codec can't 
> encode characters in position 60-66: ordinal not in range(128))
> ERROR: sql query with db encoding utf8 failed ('ascii' codec can't encode 
> characters in position 60-66: ordinal not in range(128))
> ERROR: sql query with db encoding utf-8 failed ('ascii' codec can't 
> encode characters in position 60-66: ordinal not in range(128))
> ERROR: sql query with db encoding latin1 failed ('ascii' codec can't 
> encode characters in position 60-66: ordinal not in range(128))
> ERROR: sql query with db encoding ascii failed ('ascii' codec can't 
> encode characters in position 60-66: ordinal not in range(128))
> ERROR: sql query with db encoding None failed ('ascii' codec can't encode 
> characters in position 60-66: ordinal not in range(128))
>
>
> When connecting to the database directly with cx_Oracle (without SQLAlchemy)
>
> import cx_Oracle
> import pandas as pd
> import keyring
>
> dbtype = 'Oracle'
> dbenv = 'LOCAL'
> dbname = 'MYDB'
> dbsys = '%s%s' % (dbtype, dbenv)
> dbusr = 'myusr'
> dbpwd = keyring.get_password(dbsys, dbusr)
> dbhost = 'mydbhost'
> dbport = 1521
> dbconstr = '%s:%s/%s' % (dbhost, dbport, dbname)
>
>
> dbencs = ['UTF8', 'UTF-8', 'utf8', 'utf-8', 'latin1', 'ascii', None]
>
> for dbenc in dbencs:
> print('=' * 70)
> print('db encoding: %s' % dbenc)
> print('-' * 30)
>
> if dbenc is None:
> connection = cx_Oracle.connect(dbusr, dbpwd, dbconstr)
> else:
> connection = cx_Oracle.connect(dbusr, dbpwd, dbconstr, 
> encoding=dbenc)
> cursor = connection.cursor()
>
> try:
> r = cursor.execute("SELECT owner, table_name FROM all_tables  
> WHERE owner LIKE 'äöüßÄÖÜœ'")
> recs = list()
> for owner, table_name in cursor:
> recs.append({'owner': owner, 'table': table_name})
> df = pd.DataFrame(recs)
> print('SUCCESS: sql query with db encoding %s succeeded!' % dbenc)
> except Exception as e:
> print('ERROR: sql query with db encoding %s failed (%s)' % 
> (dbenc, e))
>
> cursor.close()
> connection.close()
>
>
> everything works as expected.
>
> SUCCESS: sql query with db encoding UTF8 succeeded!
> SUCCESS: sql query with db encoding UTF-8 succeeded!
> SUCCESS: sql query with db encoding utf8 succeeded!
> SUCCESS: sql query with db encoding utf-8 succeeded!
> SUCCESS: sql query with db encoding latin1 succeeded!
> ERROR: sql query with db encoding ascii failed ('ascii' codec can't 
> encode characters in position 60-66: ordinal not in range(128))
> ERROR: sql query with db encoding None failed ('ascii' codec can't encode 
> characters in position 

Re: [sqlalchemy] looking for help building relationship that references an intermediate mixer table

[Simon King]

Do you need it to be an actual relationship? It's common to use an
association proxy for this:

https://docs.sqlalchemy.org/en/13/orm/extensions/associationproxy.html#simplifying-association-objects

Hope that helps,

Simon

On Fri, Mar 20, 2020 at 7:47 PM Mark Aquino  wrote:
>
> I'd like to create a relationship that joins a table linked to a related 
> table on a model:
>
> I have a Vessel class, a Well class, and a Batch class.
>
> Vessel and Well are related via a FK on Well (well.vessel_id) but Batch is a 
> many to many relationship with Well, e.g.
>
> Batch(Base):
>   id = Column(Integer)
>
> Well(Base):
>id = Column(Integer)
>vessel_id = Column(Integer)
>batches = relationship("Batch", secondary="mix_well_tracked_entity")
>
> Is it possible to make a relationship on Vessel to the Batches linked to its 
> wells?
>
> Vessel(Base):
>id = Column(Integer)
>wells = relationship("Well")
>wells_batches = relationship("Batch", ...???)
>
>
> select * from vessel v join well w on w.vessel_id = v.id join 
> mix_well_tracked_entity mix1 on mix1.well_id = w.id join batch b on b.id = 
> mix1.tracked_entity_id;
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/54af97fe-9de7-4cb2-a4d9-96b33107c6af%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexc3eUFLWT0sPoqm%2B8RUM0eUKPhHq8kDiwFDPq6zBfn1Gw%40mail.gmail.com.

Re: [sqlalchemy] Cleaning metadata

[Simon King]

It's difficult to answer this question without knowing how your code
is structured. Are you reflecting your tables from the database, or
have you defined them statically?

What is the full stack trace when you get those errors?

Simon

On Wed, Mar 25, 2020 at 10:27 AM Javier Collado Jiménez
 wrote:
>
> Hello,
> I'm having a problem trying to cleanup sqlalchemy objects. My application has 
> a thread which handles DB connections. In some cases the thread dies and I 
> want to do a cleanup so, next time the thread is started it would be able to 
> reconnect again.
> The steps I tried are:
> self.metadata.clear()
> self.engine.dispose()
> self.session.close()
> self.conn.close()
> But when I start a new thread, errors like this appear:
> sqlalchemy.exc.ArgumentError: Column object 'column' already assigned to 
> Table 'table'
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/74117457-9968-4aa2-afa3-8ccb91e86937%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeUwzDhm1Hms7PfA752di6OmwG5BPoec5nfE6KKgw7D3g%40mail.gmail.com.

Re: [sqlalchemy] SQLAlchemy URI (in superset) to connect to SSL enabled DRUI UI

[Simon King]

I've never used Druid, but this is really a question for the pydruid
project, I don't know if any of those developers are on this list. It
looks like pydruid only recently started supporting self-signed
certificates (or allowing you to ignore certificate errors):

https://github.com/druid-io/pydruid/pull/180/files

The value of the "ssl_verify_cert" parameter gets passed as the
"verify" parameter to requests.post:

http://2.python-requests.org/en/v1.1.0/user/advanced/#ssl-cert-verification

Unfortunately, it doesn't look like the pydruid sqlalchemy adapter
allows the ssl_verify_cert option to be specified in the url:


https://github.com/druid-io/pydruid/blob/master/pydruid/db/sqlalchemy.py#L121

You might be able to use the connect_args option to create_engine:


https://docs.sqlalchemy.org/en/13/core/engines.html#custom-dbapi-connect-arguments

or you could use the "creator" argument to specify your own function
for creating the connection:


https://docs.sqlalchemy.org/en/13/core/engines.html#sqlalchemy.create_engine.params.creator

Hope that helps,

Simon

On Wed, Mar 25, 2020 at 12:15 AM Lakshman Pervatoj  wrote:
>
> Hi Everyone,
>
> Using SQLAlchemy I want to connect to Apache Druid DB through Druid UI which 
> is SSL enabled and also has local authentication
>
>
> druid+https://:@:/druid/v2/sql/
>
>
> And getting the below error:
>
>
> ERROR: {"error": "Connection failed!\n\nThe error message returned 
> was:\nHTTPSConnectionPool(host=‘, port=): Max retries exceeded 
> with url: /druid/v2/sql/ (Caused by SSLError(SSLCertVerificationError(1, 
> '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed 
> certificate
>
>
> How can we pass the certs
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/c95ac471-9bdc-42bb-8121-82c8a62466f7%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexc9H%2BmkbszLjKXUdtSvfTJ0Y%2BD2vMzwJhbDjxyhA0TRCw%40mail.gmail.com.

Re: [sqlalchemy] Bitwise AND operation in a select statement support in sqlalchemy

[Simon King]

SQLAlchemy overrides the & operator:


https://docs.sqlalchemy.org/en/13/core/sqlelement.html#sqlalchemy.sql.expression.and_

You can use the "op" function to get at the postgres & operator:


https://docs.sqlalchemy.org/en/13/core/sqlelement.html#sqlalchemy.sql.expression.ColumnElement.op

Something like this:

select([testa.c.id.op("&")(15)])

Hope that helps,

Simon

On Tue, Feb 25, 2020 at 5:02 PM Balukrishnan  wrote:
>
> Table definition
>
> from sqlalchemy import *
> testa = Table(
> "testa",
> metadata,
> Column("id", BigInteger, primary_key=True),
> Column("str_var_a", String, nullable=True),
> Colmn("bool_var_a", Boolean, nullable=True),
> )
>
> and I need to execute a query like.
>
> select([testa.c.id & 15])
>
> But while executing this query getting an error
>
> Traceback (most recent call last):
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/operators.py",
>  line 81, in __and__
> return self.operate(and_, other)
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/elements.py",
>  line 731, in operate
> return op(self.comparator, *other, **kwargs)
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/operators.py",
>  line 81, in __and__
> return self.operate(and_, other)
>   File "", line 1, in 
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py",
>  line 67, in operate
> return o[0](self.expr, op, *(other + o[1:]), **kwargs)
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/default_comparator.py",
>  line 147, in _conjunction_operate
> return and_(expr, other)
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/elements.py",
>  line 2098, in and_
> return cls._construct(operators.and_, True_, False_, *clauses)
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/elements.py",
>  line 2028, in _construct
> clauses = [
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/elements.py",
>  line 2029, in 
> _expression_literal_as_text(clause)
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/elements.py",
>  line 4569, in _expression_literal_as_text
> return _literal_as_text(element)
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/elements.py",
>  line 4592, in _literal_as_text
> return _literal_as(element, _no_text_coercion)
>   File 
> "/Users/users_name/Projects/x_men/lib/python3.8/site-packages/sqlalchemy/sql/elements.py",
>  line 4582, in _literal_as
> raise exc.ArgumentError(
> sqlalchemy.exc.ArgumentError: SQL expression object expected, got object of 
> type  instead
>
> But in Postgres using psql command I can perform the query SELECT id & 15 
> FROM testa;. Is there any support for this in sqlalchemy.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/37691a3d-826a-4311-bbdb-6ba3eb67b95a%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeLhoePjd2ibbTDxBtKhqE3AxwvBEGj4wKckm4PiesgpA%40mail.gmail.com.

Re: [sqlalchemy] how to "explicitly" combine columns, for example get the child class value from a query on parent class from same named column?

[Simon King]

On Fri, Feb 14, 2020 at 5:35 PM Mark Aquino  wrote:
>
> I have a polymorphic class structure like this, with a lot of classes 
> extending the parent class.
> In reality I'm using a Mixin that declares the visible_id column and it's 
> defined with @declared_attr.cascading, but for simplicity:
>
>
>
> class A(Base):
> __tablename__ = 'a'
> id = Column(Integer, primary_key=True)
>visible_id = Column(Integer)
>
> class B(A):
> __tablename__ = 'b'
> id = Column(Integer, ForeignKey("A.id"), primary_key=True)
> visible_id = Column(Integer)
>
>
> What I need for my application is to query A.visible_id and return the CHILD 
> values for B.visible_id (and all the others).
>
> The inheritance works fine, i.e. if i query all As in the database, my 
> response is a list of [B] objects, but unless I directly query B the 
> visible_id from A takes precedence and I cannot query A.visible_id if I 
> remove it from A.
>
> Can anyone tell me how to configure this?
>

Out of interest, what is the point of having a visible_id column in
the B table? I'm having difficulty imagining what it would mean to
have an instance of B (which due to inheritance is also an instance of
A) which has different values in A.visible_id and B.visible_id.

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexf305mqE_NRqxF2Pv1H9j9ngt6wVHpLmWaVRHi2Ot4o2g%40mail.gmail.com.

Re: [sqlalchemy] Re: SQL expression object expected, got object of type instead

[Simon King]

Can you show the real code that runs the query? I'm wondering whether
the thing that you are comparing against my_table.c.name is not
actually a simple string.

Simon

On Wed, Feb 12, 2020 at 11:01 PM Mark Wilkins  wrote:
>
> Some additional code incase its relevent:
>
> # Get DB connection
> engine = setup_env_db_engine(debug=True)
> connection = engine.connect()
>
> # Configure SQLalchemy
> meta = MetaData(bind=engine)
> meta.reflect()
>
> my_table = meta.tables.get('my_table')
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/839938dd-ed33-4f55-a836-185ec0689f2f%40googlegroups.com.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfyHn5OKiAm8k9QX3R3Gm4xAoZRy7%3Dg33L%3DdO%3DvSDumZg%40mail.gmail.com.

Re: [sqlalchemy] Bidirectional many-to-many without foreign key

[Simon King]

You'd be able to use traditional relationship definitions if you
defined an "AddressGroup" table where address_id was unique. Company
and Address would both have one-to-many relationships to the
AddressGroup table, and you could probably use association proxies for
Company.addresses and Address.companies. Would you be willing to
define that table?

Simon

On Tue, Jan 28, 2020 at 5:41 PM 'Radoslaw Krzak' via sqlalchemy
 wrote:
>
> Yes, exactly :)
>
> My goal was to set up read only relationship. I have no idea if writable one 
> is possible without the secondary table.
>
> On Tuesday, January 28, 2020 at 3:42:36 PM UTC, Simon King wrote:
>>
>> So conceptually, an address_id represents a *group* of addresses. A
>> company can be associated with exactly one group of addresses, and one
>> group of addresses can be shared by multiple companies. Is that right?
>>
>> A normal many-to-many relationship involves an association table. When
>> you add and remove items to one side or the other of these
>> relationships, SQLAlchemy will add and delete rows from the
>> association table. I'd be amazed if SQLAlchemy would be able to figure
>> out how to add and remove objects from your relationship. Do you need
>> it to be writable, or would a readonly relationship be good enough?
>>
>> Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfBqko98NuZ71gC0NEfbrv%3Db8NFeKDp338LQKw0%2B-4fWg%40mail.gmail.com.