[sqlalchemy] Re: Filling up a field in a database from a text file, flask

2022-08-22 Thread 'Jonathan Vanasco' via sqlalchemy
You should ask this in a Flask discussion group or stackoverflow. This is a sqlalchemy group and most users here have no experience with Flask. On Friday, August 19, 2022 at 4:13:50 PM UTC-4 nand...@gmail.com wrote: > I am trying to fill up a field in a table database with contents of a text

[sqlalchemy] Re: SQL Alchemy TypeDecorator

2022-08-09 Thread 'Jonathan Vanasco' via sqlalchemy
This is usually done in the ORM with functions, and possibly hybrids. See https://docs.sqlalchemy.org/en/14/orm/mapped_attributes.html On Tuesday, August 9, 2022 at 1:55:45 PM UTC-4 Justvuur wrote: > Hi there, > > Is there a way to pass/access model data for a row within the " >

[sqlalchemy] Re: SQLAlchemy exists() used with first() ?

2022-08-09 Thread 'Jonathan Vanasco' via sqlalchemy
I think you misunderstand `exists()` in SQLAlchemy and SQL. `exists()` is a convenience function to create a SQL `EXISTS` clause, which is an operator used for filtering subqueries. The 'from_exists' is just a subquery. It is supposed to be used within a query which would then limit the

[sqlalchemy] Re: Shared ORM objects between threads

2022-07-05 Thread 'Jonathan Vanasco' via sqlalchemy
> I'm guessing we shouldn't be passing ORM objects to threads, but rather just passing IDs and then querying the full object in the thread function Correct. Database Connections and Sessions are not threadsafe, they are thread-local. See

Re: [sqlalchemy] simple query takes to long

2022-06-08 Thread 'Jonathan Vanasco' via sqlalchemy
When you select in the database ui tool, you are just displaying raw data. When you select within your code snippets above, Python is creating pandas' DataFrame objects for the results. These two concepts are not comparable at all. Converting the SQL data to Python data structures in Pandas

[sqlalchemy] Re: SQLALCHEMY conncection to Sybase Adaptive Server Anywhere Version 7 via TCI/IP

2022-04-14 Thread 'Jonathan Vanasco' via sqlalchemy
thanks, gord! On Thursday, April 14, 2022 at 12:30:44 PM UTC-4 Gord Thompson wrote: > > Der Datenquellenname wurde nicht gefunden, und es wurde kein > Standardtreiber angegeben > > "The data source name was not found and no default driver was specified" > > Use > > import pyodbc > >

[sqlalchemy] Re: SQLALCHEMY conncection to Sybase Adaptive Server Anywhere Version 7 via TCI/IP

2022-04-13 Thread 'Jonathan Vanasco' via sqlalchemy
The Sybase dialect was deprecated from first-party support by SQLAlchemy and is currently unsupported. Gord Thompson, who is a frequent contributor to the core SQLAlchemy project, and has generously taken over responsibility for the original dialect as a third-party dialect::

[sqlalchemy] Re: create database name lowcase ?

2022-03-31 Thread 'Jonathan Vanasco' via sqlalchemy
I'm not aware of any recent changes in the libraries that would cause that behavior. It may be how you are using the libraries or raw sql. PostgreSQL will convert database names to lowercase UNLESS the database name is in quotes. These will all create `abc`: CREATE DATABASE abc;

Re: [sqlalchemy] Re: many-to-many orm warnings

2022-03-15 Thread 'Jonathan Vanasco' via sqlalchemy
I'm sorry you're getting bit by this messaging - but also glad that I'm not the only one. This got me a while ago too. SqlAlchemy just uses a bare field name when emitting the warning and accepting the `overlaps` arguments. In more complex models with 3+ tables that have standardize

[sqlalchemy] Re: Calculate rank of single row using subquery

2021-09-14 Thread 'Jonathan Vanasco' via sqlalchemy
> Is this the most efficient way to do this, or am I over-complicating it? That roughly looks like code that I've implemented in the past. If it works and you don't have issues, I wouldn't worry about efficiency. Stuff like this will often vary based on the underlying table data - the

[sqlalchemy] Re: AsyncEngine always returns UTC time for datetime column

2021-09-14 Thread 'Jonathan Vanasco' via sqlalchemy
Can you share the database drivers / dialects you use? The discrepancy could be there. On Tuesday, September 14, 2021 at 7:03:27 AM UTC-4 ivan.ran...@themeanalytics.com wrote: > Hi all, > > I am trying to figure it out why AsyncEngine always returns UTC time for > datetime column, any help is

Re: [sqlalchemy] Change in before_flush (maybe from 1.3 to 1.4?)

2021-09-09 Thread 'Jonathan Vanasco' via sqlalchemy
What version of 1.4 are you using? It is before 1.4.7? If so, please update to the latest (1.4.23 is current) There was a regression in some early 1.4s that affected flush/commit/transaction in some situations. That was solved in 1.4.7. On Thursday, September 9, 2021 at 8:52:59 AM UTC-4 Mike

[sqlalchemy] Re: sqlite setting foreign_keys=off temporarily

2021-08-10 Thread 'Jonathan Vanasco' via sqlalchemy
The first two things I would look into: 1. Check the sqlite install/version that SqlAlchemy uses. It is often NOT the same as the basic operating system install invoked in your terminal. Sometimes that version does not have the functionality you need. 2. Check the transactional isolation

[sqlalchemy] Re: Oracle connection problem

2021-08-06 Thread 'Jonathan Vanasco' via sqlalchemy
You should ensure the connection string does not have any reserved/escape characters in it. People have had similar issues in the past. If that is the case, there are some recent threads in this group and on the github issues that show how to overcome the issue by building a connection string

[sqlalchemy] Re: Testing and deprecation of nested transactions

2021-07-30 Thread 'Jonathan Vanasco' via sqlalchemy
I typically do local developer testing with sqlite3, and the switch the database to postgresql for build/deploy/ci testing in the cloud. For complex tests, I typically use a fresh database "image". e.g. a sqlite file or pgdump output that is tracked in git. This is not the solution you're

Re: [sqlalchemy] prevent (raise exceptions) on bytestring values for non-byte types

2021-07-30 Thread 'Jonathan Vanasco' via sqlalchemy
ibute. > > Hope that helps, > > Simon > > On Fri, Jul 30, 2021 at 5:10 PM 'Jonathan Vanasco' via sqlalchemy > wrote: > > > > Mike, thanks for replying but go back to vacation. > > > > Anyone else: I am thinking more about an event that can be used to > cat

Re: [sqlalchemy] prevent (raise exceptions) on bytestring values for non-byte types

2021-07-30 Thread 'Jonathan Vanasco' via sqlalchemy
my.org/en/14/core/custom_types.html#coercing-encoded-strings-to-unicode > > > > On Thu, Jul 29, 2021, at 5:17 PM, 'Jonathan Vanasco' via sqlalchemy wrote: > > I am finally at the tail end of migrating my largest (and hopefully last) > Python2 application to Python3. > > A

[sqlalchemy] prevent (raise exceptions) on bytestring values for non-byte types

2021-07-29 Thread 'Jonathan Vanasco' via sqlalchemy
I am finally at the tail end of migrating my largest (and hopefully last) Python2 application to Python3. An issue that has popped up a lot during this transition, is when a py3 bytestring gets submitted into SqlAlchemy. When that happens, it looks like SqlAlchemy just passes the value into

[sqlalchemy] Re: checking in

2021-06-21 Thread 'Jonathan Vanasco' via sqlalchemy
> If not I wonder why messages aren't arriving in my INBOX. Check your settings for this group. If you do not see the option on the menu, try visiting https://groups.google.com/g/sqlalchemy/membership Google sometimes has a product change de-selects the email delivery option. Sometimes users

[sqlalchemy] Re: (cx_Oracle.DatabaseError) ORA-00972

2021-03-22 Thread 'Jonathan Vanasco' via sqlalchemy
Try passing a small number to `label_length` in your `create_engine`. Something like `label_length=5` might work. I typically use 4-6 on Production/Staging servers, and no argument on Development. *

Re: [sqlalchemy] Injecting User info into _history table to track who performed the change

2021-03-15 Thread 'Jonathan Vanasco' via sqlalchemy
Going beyond what Simon did.. I typically make make a table like `user_transaction`, which has all of the relevant information for the transaction: * User ID * Timestamp * Remote IP Using the sqlalchemy hooks, I'll then do something like: * update the object table with the user_transaction id

Re: [sqlalchemy] Supporting Function Indexes on a Minimum Sqlite Version

2021-03-09 Thread 'Jonathan Vanasco' via sqlalchemy
Thank you so much, Mike! I roughly had that same @compiles in my tests, but I didn't trust myself... and the .dbapi looked like what I wanted, but I really wasn't sure! On Monday, March 8, 2021 at 4:36:03 PM UTC-5 Mike Bayer wrote: > > > On Mon, Mar 8, 2021, at 12:06 PM, 'Jonathan Van

[sqlalchemy] Supporting Function Indexes on a Minimum Sqlite Version

2021-03-08 Thread 'Jonathan Vanasco' via sqlalchemy
I have a project that, in a few rare situations, may run on a version of sqlite that does not support function indexes, and "need" to run a unique index on `lower(name)`. For simplicity, I'll just use a normal index on correct systems, I'm trying to figure out the best way to implement this.

Re: [sqlalchemy] relationship query_class in SQLAlchemy 1.4.0b3

2021-03-01 Thread 'Jonathan Vanasco' via sqlalchemy
"is it better to think of rebuilding medium+ projects for 2.0 while maintaining existing codebases for 1.3? In other words, how much will 2.0 be backward compatible with 1.3?" I am saying the following as a general user, and not a past contributor to this project: As per the Release Status

Re: [sqlalchemy] Batching INSERT statements

2021-02-12 Thread 'Jonathan Vanasco' via sqlalchemy
I'm not familiar with this exactly, but have a bit of experience in this area. I just took a look at this module (nice work!). It's VERY well documented in the docstrings (even nicer work!) I think the core bit of this technique looks to be in `_get_next_sequence_values` -

[sqlalchemy] Re: Relationship with 2 intermediary tables

2021-02-12 Thread 'Jonathan Vanasco' via sqlalchemy
This is, IMHO, one of the most complex parts of SQLAlchemy. In this public project, i have a handful of secondary/secondaryjoin examples that may help you https://github.com/aptise/peter_sslers/blob/main/peter_sslers/model/objects.py#L3778-L4714 There is a section in the docs that should help

Re: [sqlalchemy] SQLAlchemy transaction ID

2021-01-27 Thread 'Jonathan Vanasco' via sqlalchemy
FWIW, within the realm of pyramid_tm, the more common use-cases for two-phase transaction support are for sending mail and a dealing with task queues - not two separate databases. On Wednesday, January 27, 2021 at 2:40:21 PM UTC-5 Mike Bayer wrote: > > > On Wed, Jan 27, 2021, at 2:23 PM,

Re: [sqlalchemy] FAQ or Feature Ideas for ORM Object and Session

2021-01-27 Thread 'Jonathan Vanasco' via sqlalchemy
Ok. I'll generate a docs PR for sqlalchemy and pyramid. this comes up so much. On Wednesday, January 27, 2021 at 2:25:29 PM UTC-5 Mike Bayer wrote: > > > On Wed, Jan 27, 2021, at 1:12 PM, 'Jonathan Vanasco' via sqlalchemy wrote: > > I've been working with a handful of SQLAlch

[sqlalchemy] FAQ or Feature Ideas for ORM Object and Session

2021-01-27 Thread 'Jonathan Vanasco' via sqlalchemy
I've been working with a handful of SQLAlchemy and Pyramid based projects recently, and two situations have repeatedly come up: 1. Given a SQLAlchemy Object, access the SQLAlchemy Session 2. Given a SQLAlchemy Object or Session, access the Pyramid Request object The general solutions I've used

[sqlalchemy] Re: SQLAlchemy transaction ID

2021-01-27 Thread 'Jonathan Vanasco' via sqlalchemy
Thierry, Would you mind putting together a test-case on this? I haven't experienced that before, and I authored that feature in the debugtoolbar. If I can recreate it, I'll put together a fix and work with the pyramid team to get a new release out asap. On Wednesday, January 27, 2021 at

Re: [sqlalchemy] Differences between TableClause insert and Model.__table__ inserts?

2020-11-25 Thread 'Jonathan Vanasco' via sqlalchemy
This was not clear enough in Mike's post: `Foo.__table__` is the same type of object as `_foo = table(...)`. SQLAlchemy ORM is built on top of SQLAlchemy's Core, so the ORM's `.__table__` attribute is the Core's `table()` object. Since they're the same, the two will have the same performance

[sqlalchemy] Re: Dis/Reassociate objects with a db session.

2020-11-25 Thread 'Jonathan Vanasco' via sqlalchemy
Read the docs on State Management and pay attention to `merge`: https://docs.sqlalchemy.org/en/14/orm/session_state_management.html Also, to simplify this stuff a popular related pattern is to use a RevisionID or RevisionTimestamp on the objects. In the first session, you note the version

[sqlalchemy] Re: Zope.sqalchemy: AttributeError: '_thread._local' object has no attribute 'value'

2020-10-25 Thread 'Jonathan Vanasco' via sqlalchemy
Your new code is exactly what I have been running on several production systems, so it looks good to me! Long story short, `zope.sqlalchemy` had been using the `sqlalchemy` "extensions", which were deprecated in 2012 and are set to be removed (if they haven't been already). see

[sqlalchemy] Re: Update multiple rows in SQLite Databse

2020-08-28 Thread 'Jonathan Vanasco' via sqlalchemy
I believe your error is tied to this section of code: > for item in ingredDict: > ingredient_item = Ingredients(ingredientKey=item['ingredientKey'], > > ingredientDescription=item['ingredientDescription'], >

[sqlalchemy] Re: sqlalchemy messes up names with "_1" suffix

2020-07-10 Thread 'Jonathan Vanasco' via sqlalchemy
> i have this litte flask-admin game running, now out of nowwhere sqlalchemy has begun to add strange "_1" suffixes to the column names. i know sqlalchemy does this to keep names unique, but in my case the queries are failing SQLAlchemy does do this, for those reasons, and to the columns...

Re: [sqlalchemy] Encrypt/Decrypt specific column(s)

2020-07-09 Thread 'Jonathan Vanasco' via sqlalchemy
On Thursday, July 9, 2020 at 2:12:36 PM UTC-4, Justvuur wrote: > > I've done some more digging... It seems when I did the search for > "secrets", the text is encrypted and compared to the value in the columns, > That is how client-side encryption works. If you want to search for "secrets",

Re: [sqlalchemy] convert subset to dictionary

2020-07-07 Thread 'Jonathan Vanasco' via sqlalchemy
Based on what you shared above: * The "Subject" table is: `StudentId, SubjectCode, SubjectName` * There are 181 subjects It looks like you don't have a "Subject" table, but a "StudentAndSubject" table. I think you'd have a bigger performance improvement by normalizing that data into two

[sqlalchemy] Re: SQLAlchemy taking too much time to process the result

2020-07-06 Thread 'Jonathan Vanasco' via sqlalchemy
On Monday, July 6, 2020 at 2:14:33 PM UTC-4, Saylee M. wrote: > So, when I passed the query to MySQL directly, it took very less time > (around 0.016 seconds) but when I passed the same > query through SQLAlchemy connector, it took around 600 seconds > "query ... MySQL directly" Do you

[sqlalchemy] Re: Locking method used in SQLAlchemy (postgres)

2020-06-30 Thread 'Jonathan Vanasco' via sqlalchemy
On Monday, June 29, 2020 at 8:00:40 PM UTC-4, gbr wrote: > > > I'm using SQLAlchemy's Core to interface a postgres database (via > psycopg2) component alongside Flask-SQLAlchemy in a Flask app. Everything > was working fine until I recently discovered what seems to be a deadlock > state which

[sqlalchemy] Re: Am I doing this query of lots of columns correctly?

2020-06-26 Thread 'Jonathan Vanasco' via sqlalchemy
that should be `loaded_columns_as_dict()` , unless you decorate the method with @property. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See

[sqlalchemy] Re: Am I doing this query of lots of columns correctly?

2020-06-26 Thread 'Jonathan Vanasco' via sqlalchemy
I use a mixin class to handle this stuff. Example below. > So, my question: is it generally better practice to name every column that you want to pull, even if it's a long list? Not really. There is a "bundle" api here that might be better for you-

[sqlalchemy] Re: Can't find anything equivalent to bit_or in PostgreSQL

2020-06-16 Thread 'Jonathan Vanasco' via sqlalchemy
If this just needs this to be rendered for PostgreSQL, you can use the `func` generator: https://docs.sqlalchemy.org/en/13/core/sqlelement.html#sqlalchemy.sql.expression.func from sqlalchemy.sql.expression import func query = session.query(Foo).filter(func.bit_or(Foo.cola, Foo.colb)...