Re: [sqlalchemy] Help with a custom "seconds_interval()" construct

2016-05-29 Thread Mike Bayer
On 05/28/2016 09:44 AM, Kent wrote: I'm interested in being able to use second time intervals on PostgreSQL, Oracle 8 /and /modern Oracle versions, agnostically. The native python timedelta works great for the postgres and cx_Oracle drivers. However /cx_Oracle connected to Oracle 8 won't

Re: [sqlalchemy] Polymorphic inheritance with dynamically created tables: issues with undefined polymorphic identity

2016-05-29 Thread Mike Bayer
On 05/29/2016 01:18 AM, Radu Suciu wrote: I have a base Dataset class that I'd like to dynamically inherit from. When a user uploads a new dataset, I create a new table as shown below, and populate it with data. My two most common use-cases: 1. Retrieving entire dataset. I want to return

Re: [sqlalchemy] Joined/Eager loading into a non-relationship

2016-06-15 Thread Mike Bayer
On 06/15/2016 01:01 AM, Nick Whyte wrote: Hey, I'm working on a more complex problem with the ORM functionality of SQLA. I have a reasonably simple relationship, ie, class A(Base): id = sa.Column(sa.Integer(), primary_key=True) b_collection = sa.orm.relationship('B') class

Re: [sqlalchemy] How to update a DateTime column with the SQL server value `NOW()`?

2016-06-15 Thread Mike Bayer
On 06/15/2016 10:13 AM, Jean-Philippe Morin wrote: (I am using PostgreSQL) I want to use the TIMESTAMP values of the SQL server machine instead of the python `datetime.utcnow()` value of the WEB server machines. There could be latencies or time diffierences between machines, so I would like

Re: [sqlalchemy] MySQL server has gone away

2016-06-14 Thread Mike Bayer
Well I'd get off of OurSQL to start with since it is unmaintained for years now. The "gone away" error doesn't always mean the connection was actually dropped, in some old school situations it just means the client got out of sync with the MySQL protocol. ( waits ) still broken?

Re: [sqlalchemy] MySQL server has gone away

2016-06-14 Thread Mike Bayer
L server during query") sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT ...] Process finished with exit code 1 On 14 Jun 2016, at 15:15, Mike Bayer wrote: Well I'd get off of OurSQL to start with since

Re: [sqlalchemy] Applying joins sequentially to select objects

2016-06-16 Thread Mike Bayer
On 06/15/2016 09:53 PM, Kevin Murphy wrote: Hi, Is it possible to repeatedly add column expressions to a select object, where the column expression involves outer joins? Ideally I'd like there to be a several transformation functions that operate on the same select object in a chain. So far,

[sqlalchemy] SQLAlchemy 1.1.0b1 released

2016-06-16 Thread mike bayer
SQLAlchemy release 1.1.0b1 is now available. This is the first beta release in the new 1.1 series of SQLAlchemy. Users are encouraged to ensure that their pip installations are up-to-date, so that version 1.1.0b1 will only install if explcitly specified using the --pre flag. The 1.1 series

Re: [sqlalchemy] Force type conversions in newly created Declarative objects

2016-06-17 Thread Mike Bayer
On 06/17/2016 12:44 PM, Chuck Bearden wrote: I'm using SQLAlchemy 1.0.13 with Python 2.7.10. I load much of my data naively from text files, which means that for newly-created Declarative objects, all the values are strings, whatever the declared column type: | fromsqlalchemy.ext.declarative

Re: [sqlalchemy] Is there a way to create custom classes that inherit from sqlalchemy models in external library?

2016-06-22 Thread Mike Bayer
;sqlite://", echo=True) Base.metadata.create_all(e) s = Session(e) s.add(Bar(foo_bar=FooBar())) s.commit() s.close() b1 = s.query(Foo).first() print b1 print b1.foo_bar Thanks, Angie On Fri, Jun 3, 2016 at 12:38 PM, Mike Bayer <mike...@zzzcomputing.com <mailto:mike...@zzzcomputi

Re: [sqlalchemy] Nonefy arguments

2016-06-20 Thread Mike Bayer
On 06/17/2016 04:45 PM, fer mario wrote: Hello Guys, I was having issues cause some empty arguments were treated as "" (emtpy) instead of none, so the filter() was translating the query as 'where column=""' instead of "where column is NULL", so I'm trying to write some validation in my add

Re: [sqlalchemy] bulk insert from generator instead of list, possible?

2016-06-16 Thread Mike Bayer
On 06/16/2016 12:32 PM, Andy Crain wrote: Hi, I'm attempting to do a bulk insert from a large .csv file. I've read through the various options at http://docs.sqlalchemy.org/en/latest/faq/performance.html#i-m-inserting-400-000-rows-with-the-orm-and-it-s-really-slow, and I would like to

Re: [sqlalchemy] Update with where, ORM with joined table inheritance

2016-06-23 Thread Mike Bayer
On 06/23/2016 02:42 PM, Douglas Russell wrote: Hi, I have a situation where I need to check a condition during an update. In my case it is a timestamp and the function of the condition is to ensure that there was no intermittent update by another between the type the object was edited and

Re: [sqlalchemy] "RuntimeError: dictionary changed size during iteration" during configure_mappers

2016-06-23 Thread Mike Bayer
as the list is being iterated, which would cause the list to change size within the process. Chung On Thursday, June 23, 2016 at 1:26:38 PM UTC-7, Mike Bayer wrote: On 06/23/2016 03:30 PM, Chung wrote: > We're occasionally seeing this exception thrown

Re: [sqlalchemy] Boolean column type with MySQL has no mechanism to enforce 1/0 value

2016-06-23 Thread Mike Bayer
On 06/23/2016 05:49 PM, robert.picard via sqlalchemy wrote: This bug only appears when using SQLAlchemy with C extensions compiled and a database that does not have a native BOOLEAN type. If you have a boolean SQLAlchemy column with a MySQL database, when SQLAlchemy sets the value of this

Re: [sqlalchemy] "RuntimeError: dictionary changed size during iteration" during configure_mappers

2016-06-23 Thread Mike Bayer
On 06/23/2016 03:30 PM, Chung wrote: We're occasionally seeing this exception thrown out of mapper.configure_mappers when using SQLSoup in our webapp: | File"/home/glados/.virtualenvs/pod-glance/local/lib/python2.7/site-packages/sqlsoup.py",line 455,in__getattr__ returnself.entity(attr)

Re: [sqlalchemy] custom queries

2016-01-13 Thread Mike Bayer
ure" > be implemented in future sa releases? > > best regards, > richard. > > On 01/13/2016 04:41 PM, Mike Bayer wrote: >> or, place a marker in your own queries: >> >> query(User).execution_options(my_query=True).all() >> >> >> >> O

Re: [sqlalchemy] using mysql user variables

2016-01-13 Thread Mike Bayer
it would depend upon if this can be done within a single cursor.execute() or if it would require multiple invocations. At the very least, if the DBAPI that you're using can support it, I'd start with direct cursor access as illustrated at

Re: [sqlalchemy] custom queries

2016-01-13 Thread Mike Bayer
ion and i'm sorry, i was not clear enough in > my question. > > i would like to know if there's a way to tell if a query was fired by > sqlalchemy internals or by my coded query (programatically). > > > thanks a lot! > richard. > > On 01/13/2016 04:11 PM, Mike Baye

Re: [sqlalchemy] custom queries

2016-01-13 Thread Mike Bayer
or, place a marker in your own queries: query(User).execution_options(my_query=True).all() On 01/13/2016 01:40 PM, Mike Bayer wrote: > > > similar, you'd need to use sys.exc_info() and walk through the stack > trace to programmatically determine the origin of a Python statement.

Re: [sqlalchemy] PostgreSQL domains

2016-01-13 Thread Mike Bayer
On 01/13/2016 12:37 AM, Jonathan Rogers wrote: > I have several domains based on type TEXT to constrain values in > specific ways, such as to disallow empty values or only allow valid > email addresses. When I reflect tables with columns of such a domain, > SQLAlchemy simply considers their

Re: [sqlalchemy] a Python-side value or SQL expression is required

2016-01-18 Thread Mike Bayer
On 01/18/2016 02:23 AM, gbr wrote: > |I've upgraded from a SQLA version 0.9.x to 1.0.9. Previously, I did the > following when inserting new records: > > - Column('flag', Boolean, server_default=sql.expression.false()) |||I didn't > set those columns locally and didn't include them in the >

Re: [sqlalchemy] How to label columns of a union?

2016-01-18 Thread Mike Bayer
you'd probably want to specify literal SQL and use label(): literal_column('1').label('bar') On 01/17/2016 10:54 AM, 'Dr. Leo' via sqlalchemy wrote: > Hi, > > consider the following snippet: > > > from sqlalchemy import select, union > u=union(select([1]).alias('foo'),

Re: [sqlalchemy] Postgres 9.5 UPSERT and SQLAlchemy?

2016-01-14 Thread Mike Bayer
since I don't use UPSERT, I don't have a deep insight on an appropriate API for this, as all the backends have totally different systems, none of which behave the same. Whether a construct would attempt to work "the same" on different backends, or if there are just totally different constructs

Re: [sqlalchemy] Re: a Python-side value or SQL expression is required

2016-01-18 Thread Mike Bayer
On 01/18/2016 07:41 PM, gbr wrote: > Sorry, I'm having difficulties following your explanation. Would you > mind extending on it a bit? > > that error should only occur if the list of elements you're passing to > insert.values() contains inconsistent keys. As long as each dictionary >

Re: [sqlalchemy] 1.1.0b1: construct has no default compilation handler.

2016-06-27 Thread Mike Bayer
Not really sure, that URL itself won't allow a psycopg2 connection to even occur. The error means that the construct is trying to be generated as a string, like in a print statement. But 1.1 has a new feature that allows default stringification of pg.ARRAY and other constructs to actually

Re: [sqlalchemy] QueuePool negative overflow

2016-06-27 Thread Mike Bayer
On 06/27/2016 10:48 AM, davepuffin via sqlalchemy wrote: Hi QueuePool.status() is returning some odd looking negative values from one of our production applications. engine.pool.status() 'Pool size: 20 Connections in pool: 19 Current Overflow: -36 Current Checked out connections: -35' The

Re: [sqlalchemy] QueuePool negative overflow

2016-06-27 Thread Mike Bayer
:) I think you're right in that it must be threading related somehow, but it looks like all the monkeypatching has been applied correctly. If I examine the locking object used in the pool I get: type(engine.pool._overflow_lock) On Mon, Jun 27, 2016 at 4:21 PM, Mike Bayer <m

Re: [sqlalchemy] QueuePool negative overflow

2016-06-27 Thread Mike Bayer
of connections (from netstat) matches up with the pool status as expected. I will try and recreate the problem with a test. Many thanks! there's almost certainly a bug on our end on that, then. On Monday, June 27, 2016 at 4:36:21 PM UTC+1, Mike Bayer wrote: Well it is possible QueuePool

Re: [sqlalchemy] Update clause with alias name

2016-06-24 Thread Mike Bayer
On 06/24/2016 11:49 AM, T Johnson wrote: I'm using SQLAlchemy Core. A user prepares an "on" expression for a join query and passes it to a function. The goal of the function is to hide some messy details the must be done to perform the join. Part of these details include doing a subselect.

Re: [sqlalchemy] Is there a way to create custom classes that inherit from sqlalchemy models in external library?

2016-06-27 Thread Mike Bayer
ile ".../env/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 754, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File ".../env/lib/python2.7/site-packages/sqlalchemy/sql/selectable.py", line 553, in columns return self._columns.as_immutab

Re: [sqlalchemy] handling the results of `session.execute`

2016-02-05 Thread Mike Bayer
On 02/05/2016 05:32 PM, Jonathan Vanasco wrote: I've run into a few cases where I need to abandon the ORM and run `session.execute()` horrors Usually... I'm doing a bunch of nested queries and only pull out a `count` or a few rows of 1-2 id columns. Writing in pure sql is faster (for me),

Re: [sqlalchemy] Automap reference issue

2016-02-10 Thread Mike Bayer
On 02/10/2016 01:01 PM, Dermot O'Sullivan wrote: Hi guys, Been trying to work this out. I have an existing database that I would like to use sqlalchemy on. The code below is working in the sense that I can query individual tables and joins do not fail ( where a FK reference exists in the db )

Re: [sqlalchemy] Is it possible to use a Comparator to override a database/SQL-side comparison symmetrically?

2016-02-09 Thread Mike Bayer
On 02/09/2016 12:30 AM, Rudolf Cardinal wrote: Dear all, I've been trying to implement a datetime-style field in SQL Alchemy that, in the database backend, uses a specific ISO-8601 format (e.g. "2013-04-30T00:26:03.000+05:00"), and is a datetime at the Python end. The primary database engine

Re: [sqlalchemy] Session.flush() before Session.begin()

2016-02-04 Thread Mike Bayer
On 02/03/2016 11:10 PM, James Emerton wrote: im not sure why you need to use begin() at all? in autocommit mode, the user presses save, you just call flush(). flush() always uses a transaction internally. That's originally how the Session was meant to be used, having an explicitly

Re: [sqlalchemy] Thoughts on Column(unique=True, index=True) creating a unique constraint for postgres?

2016-02-09 Thread Mike Bayer
On 02/09/2016 04:47 PM, Jonathan Beluch wrote: Not sure about other DBs, but according to pg docs [1], it's preferred to make a unique constraint (and know that you get the index for free) versus creating a unique index. I'm not seeing the word "preferred" in that document?It is actually

Re: [sqlalchemy] TypeDecorator: find out column name in process_bind_parameter

2016-02-11 Thread Mike Bayer
On 02/11/2016 01:46 PM, immerrr again wrote: Hi everyone Say, I'm writing an API and I want to add a JsonDict TypeDecorator, just like in the example provided in the documentation [1]. MySQL database has a restriction on the size of BLOB columns and would complain if I try to write a bigger

Re: [sqlalchemy] Calling sqlalchemy.func on column

2016-02-11 Thread Mike Bayer
On 02/11/2016 09:39 PM, Balaji Pattewar wrote: Hi all, I have Sqlite table from which I can execute HEX to convert value to HEX. CREATE TABLE hosts( name TEXT, mac INTEGER, ipv4 INTEGER, } Running query "select HEX(mac) from hosts" give me address in HEX

Re: [sqlalchemy] SQL connections rising into the hundreds / thousands

2016-02-04 Thread Mike Bayer
On 02/04/2016 12:34 PM, Daniel Cochran wrote: Hi Michael -- thank you for the reply. everything looks fine, except what does get_sqlAlchemy() do? If that creates a new SqlAlchemy instance each time, then you're making new connection pools on every request. even then, when

Re: [sqlalchemy] Use order_by when using relationship() with secondary-parameter in a many-to-many?

2016-02-13 Thread Mike Bayer
On 02/13/2016 12:00 PM, c.bu...@posteo.jp wrote: I create a many-to-many relationship between a `Person` (as an _author_) and a `Reference` (as a _scientific publication_). ReferenceAuthor = sa.Table('ReferenceAuthor', _Base.metadata, sa.Column('ReferenceID', sa.Integer,

Re: [sqlalchemy] Re: Should RowProxy behave as a standard Python sequence?

2016-01-27 Thread Mike Bayer
On 01/27/2016 11:37 AM, Mike Bayer wrote: > > > On 01/27/2016 05:32 AM, Lele Gaifax wrote: >> Mike Bayer <mike...@zzzcomputing.com> writes: >> >>> Let's consider going with the third option I mentioned, I dont think >>> this will add too m

Re: [sqlalchemy] query.delete(fetch) gets confused by aliased

2016-02-01 Thread Mike Bayer
On 02/01/2016 11:18 AM, Rick Otten wrote: When I used "aliased" with query.delete(), the fetch query seems to get confused. * I'm using SQLAlchemy 1.0.11, which 'pip' tells me is the latest version. * My backend database is PostgreSQL 9.5 Code snippet: | fromsqlalchemy.orm

Re: [sqlalchemy] Is this a correct way to set up table inheritance? (circular dependency issue on delete attempt)

2016-02-02 Thread Mike Bayer
On 02/02/2016 04:27 PM, Brian Leach wrote: This question is sparked by some trouble that I am having with deleting model instances. Please see this question: http://stackoverflow.com/questions/35163325/sqlalchemy-circular-dependency-on-delete I have set up my models like below. I have

Re: [sqlalchemy] Postgrsql Array of CIDR and return type as String

2016-02-03 Thread Mike Bayer
On 02/03/2016 05:36 AM, Peter Hudec wrote: Hi, I have found, that sqlalcheny should convert poctgresql ARRAY fri LIST. This works fine with the TEXT. See example in some other post http://stackoverflow.com/questions/20699196/python-list-to-postgresql-array |(,"['new', 'updated',

Re: [sqlalchemy] Session.flush() before Session.begin()

2016-02-03 Thread Mike Bayer
On 02/03/2016 09:39 PM, jemer...@spiresystems.com wrote: We're using SQLAlchemy in a GUI application and have been experiencing some challenges integrating the ORM Session with our usage model. (This is difficult to change as we have ported from a legacy system to SQLAlchemy.) We're currently

Re: [sqlalchemy] query.delete(fetch) gets confused by aliased

2016-02-02 Thread Mike Bayer
On 02/02/2016 09:05 AM, Rick Otten wrote: > However, when the fetch query actually runs, it includes the non aliased > table name as well as the aliased table name in the from statement: > > select t.idfrom *some_table, some_table as t*

Re: [sqlalchemy] Re: Should RowProxy behave as a standard Python sequence?

2016-01-27 Thread Mike Bayer
On 01/27/2016 12:57 PM, Lele Gaifax wrote: > Mike Bayer <mike...@zzzcomputing.com> writes: > >> OK my change has revealed a bug in my test suite that wasn't catching >> that my change fails. The C implementation uses the keymap for >> integers. Where a

Re: [sqlalchemy] Dependency rule tried to blank-out primary key column when trying to update using association proxy

2016-02-23 Thread Mike Bayer
On 02/23/2016 10:47 AM, Piotr Dobrogost wrote: Hi! I'm getting AssertionError: Dependency rule tried to blank-out primary key column 'text_value.text_id' on instance '' error while trying to update row using association proxy ('values') like this: |

Re: [sqlalchemy] MSSQL ProgrammingError with aggregate functions

2016-02-24 Thread Mike Bayer
On 02/24/2016 10:13 AM, Alex Lowe wrote: Hi there, I'm receiving a ProgrammingError with certain types of query to MSSQL (they seem to work fine when querying SQLite though). Either my Google-fu is weak or there hasn't been a solution posted publicly, since the two most useful-looking pages

Re: [sqlalchemy] Asymmetry between engine.begin() and connection.begin()

2016-02-24 Thread Mike Bayer
get the connectable first: connection = engine_or_connection.connect() then do your context manager from that. the connection returned, if engine_or_connection is already a Connection, is "branched", meaning it is safe to call close() on it without affecting the original. So fully: with

Re: [sqlalchemy] Dependency rule tried to blank-out primary key column when trying to update using association proxy

2016-02-24 Thread Mike Bayer
On 02/24/2016 09:56 AM, Piotr Dobrogost wrote: On Wednesday, February 24, 2016 at 4:21:58 AM UTC+1, Mike Bayer wrote: in that way you can control exactly what __set__() does and it will never remove a TextValue object where the same identity is coming in on assignment. One more

Re: [sqlalchemy] "cache lookup failed" on reflection

2016-02-23 Thread Mike Bayer
On 02/23/2016 11:58 AM, Matt Smith wrote: Hi sqlalchemy! When using sqlalchemy (core) 1.0.8 with redshift-sqlalchemy 0.4.1, I encountered the following exception: | InternalError:(psycopg2.InternalError)cache lookup failed forrelation 3262644 | This is the query it failed on: | SELECT

Re: [sqlalchemy] Custom collection: nested dicts

2016-02-23 Thread Mike Bayer
On 02/23/2016 04:50 PM, Sergey Mozgovoy wrote: Recently I ran into two cases that required some custom collection behavior, and it turned out that what I needed did not exist in SQLAlchemy, so I ended up implementing custom collection classes: 1) Nested mapped collections

Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-23 Thread Mike Bayer
xc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Table2.table1_stuff - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression. Th

Re: [sqlalchemy] Dependency rule tried to blank-out primary key column when trying to update using association proxy

2016-02-23 Thread Mike Bayer
On 02/23/2016 06:11 PM, Piotr Dobrogost wrote: On Tuesday, February 23, 2016 at 5:44:45 PM UTC+1, Mike Bayer wrote: That the old TextValue objects need to be deleted I understand. What I don't see is what makes SA specifically blank-out primary key column 'text_value.text_id

Re: [sqlalchemy] Re: Can I make bulk update through association proxy?

2016-02-27 Thread Mike Bayer
On 02/27/2016 07:26 AM, Piotr Dobrogost wrote: On Friday, February 26, 2016 at 7:45:46 PM UTC+1, Jonathan Vanasco wrote: I'm not either, and I'm scared of the SQL that would be generated and the wire traffic/memory if there were. That would be subselects within subqueryloads

Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-27 Thread Mike Bayer
On 02/27/2016 08:48 AM, Will Angenent wrote: The code was just to reproduce the exact SQLA error by doing what alembic and SQLA are doing and additionally determining when the garbage collector runs. The theory is that garbage gets collected somewhere in between list(_mapper_registry) in

Re: [sqlalchemy] Trying to create fakes for unit testing app code

2016-02-28 Thread Mike Bayer
On 02/28/2016 08:51 PM, Nando Florestan wrote: Hello, Because even in-memory SQLite is too slow for unit tests (I have seen a couple large web applications with 7-minute+ test suites), I have started experimenting with new FakeSession and FakeQuery classes that store model instances in Python

Re: [sqlalchemy] Interaction between joined table inheritance and subquery correlate

2016-02-25 Thread Mike Bayer
On 02/24/2016 08:24 PM, Andrew Wansley wrote: Hey y'all, I'm trying to understand the interaction between joined table inheritance and a correlated subquery. The query seems to work as I expect if the same table doesn't have a joined table subclass, or if I circumvent the polymorphic code by

Re: [sqlalchemy] How to test functions that use sqlalchemy ORM?

2016-02-29 Thread Mike Bayer
sure see that at http://www.sqlalchemy.org/library.html#buildingtheapp. On 02/29/2016 04:01 AM, Abhijeet Rastogi wrote: Hi Mike, Thanks for the reply. Do you have any slides to share from that talk? Cheers! On Monday, February 22, 2016 at 9:54:38 PM UTC+5:30, Mike Bayer wrote: I

Re: [sqlalchemy] Date range query problem

2016-02-29 Thread Mike Bayer
On 02/29/2016 06:01 PM, Nana Okyere wrote: I have a model and one of its attributes is a column called last_updated_timestamp . It is a date column set to datetime.datetime.now() . That's all good. I'm trying to query for some roles based on a date range. So I do: results =

Re: [sqlalchemy] Bulk Insert Broken for Polymorphism?

2016-02-29 Thread Mike Bayer
On 02/29/2016 05:38 PM, Alex Hewson wrote: Hello All, I'm trying to use the new bulk_save_objects() to improve performance on bulk inserts, and have run into a problem. If bulk_save_objects() is used to save objects of a polymorphic class.. 1. They are created correctly in the DB, with

Re: [sqlalchemy] How to test functions that use sqlalchemy ORM?

2016-02-22 Thread Mike Bayer
I created a pretty comprehensive example of two ways to do this as part of a talk I did at Pycon some years ago. The example case is https://bitbucket.org/zzzeek/pycon2014_atmcraft and you can see a contrast of the "mock" approach and the "run tests in a transaction" approach at:

Re: [sqlalchemy] Association Proxy query reference ?

2016-02-22 Thread Mike Bayer
On 02/22/2016 06:51 AM, gio wrote: Hi I was trying to use the associationproxy feature like explained in the basic example http://docs.sqlalchemy.org/en/latest/orm/extensions/associationproxy.html Now my question is: Is it possible to do a query like and have as result an array of array of

Re: [sqlalchemy] prefix_with for queries with that eager load relationships

2016-02-23 Thread Mike Bayer
On 02/23/2016 04:00 AM, Daniel Kraus wrote: Hi, I want to use mysqls `SQL_CALC_FOUND_ROWS` but when I use `query.prefix_with(...)` it fails when the query eager loads a relationship because sqlalchemy puts the prefix not at the beginning. I'm not sure if I should file a bug report or if it's

Re: [sqlalchemy] SQLAlchemy dynamic & customized Collection Class

2016-02-23 Thread Mike Bayer
On 02/23/2016 01:41 AM, Edouard BERTHE wrote: Hello everybody ! This is a question I have already posted on Stack Overflow , but as none is answering I decided to post it here too :) However I invite

Re: [sqlalchemy] Re: Association Proxy query reference ?

2016-02-23 Thread Mike Bayer
On 02/23/2016 06:47 AM, gio wrote: Hi thanks for your answer Without filter i did not expect any big difference between query(Keyword) or a hypothetical query(User.keywords) but I was thinking it will be nice to do something like this session.query(User.keywords).filter(User ==4)

Re: [sqlalchemy] Memory leak? with connection & insert

2016-02-25 Thread Mike Bayer
(http://smira.ru/wp-content/uploads/2011/08/heapy.html) to see what kinds of objects are prevalent. *Randy Syring* Chief Executive Developer Direct: 502.276.0459 Office: 812.285.8766 Level 12 <https://www.level12.io/> On 02/25/2016 07:46 PM, Mike Bayer wrote: On 02/25/2016 06

Re: [sqlalchemy] duck_type_collection(_AssociationDict) is None?

2016-02-25 Thread Mike Bayer
duck_type_collection isn't really public API and also the association proxy objects don't fully emulate the collections they behave as, the issue at https://bitbucket.org/zzzeek/sqlalchemy/issues/3040/association-proxies-are-not-json talks about how it would be nice if these finally

Re: [sqlalchemy] Memory leak? with connection & insert

2016-02-25 Thread Mike Bayer
On 02/25/2016 06:31 PM, Randy Syring wrote: I'm working on a project to parse through a large text file (1GB) of records. Once parsed, each record gets sent to the DB. Due to the size of the file, I've been working on a streaming/functional approach that will keep my memory usage constant.

Re: [sqlalchemy] Re: pickled automap.metadata and AmbiguousForeignKeysError

2016-01-20 Thread Mike Bayer
im assuming this is https://bitbucket.org/zzzeek/sqlalchemy/issues/3632/immutablecolumncollection-creates-a a fix is posted but I'm not comfortable merging in 1.0.x since it will break existing pickled metadata files. On 01/20/2016 07:48 PM, Jeff Laughlin wrote: > I have the exact same issue. I

Re: [sqlalchemy] Searching from jsonb list

2016-01-19 Thread Mike Bayer
if it's JSONB then you'd need to CAST it as a CHAR first: func.lower(cast(column, CHAR)).contains(word.lower()) try it in SQL first at the psql command line to work it out fully. On 01/19/2016 01:30 PM, Sami Pietilä wrote: > I am not exactly sure where to add sql.func.lower(). Following

Re: [sqlalchemy] Searching from jsonb list

2016-01-21 Thread Mike Bayer
that style of PG function use isn't directly supported yet and there is a recipe to achieve this at https://bitbucket.org/zzzeek/sqlalchemy/issues/3566/figure-out-how-to-support-all-of-pgs#comment-22842678 On 01/21/2016 09:10 AM, Sami Pietilä wrote: > Sounds good. I was able to write following

Re: [sqlalchemy] Make ResultProxy JSON serializable

2016-01-21 Thread Mike Bayer
On 01/21/2016 06:23 PM, Nana Okyere wrote: > In my flask application, a view function has this piece of code: > > > sql_text = text(" \ > SELECT CONNECT_BY_ROOT \ > part_no as ROOT_PART_NO, \ > bc.part_no, \ >

Re: [sqlalchemy] Using a connection pool with multiple processes on the same machine

2016-01-21 Thread Mike Bayer
On 01/21/2016 08:43 PM, Maximilian Roos wrote: > We're using celery, a job distribution package. On a single machine, > there are 20+ celery workers running, each with their own Python > process. We had some issues with the processes attempting to use the > same SQLAlchemy connections (I think

Re: [sqlalchemy] Re: Should RowProxy behave as a standard Python sequence?

2016-01-25 Thread Mike Bayer
this is great. but you know throwing an isinstance() right into the otherwise no-latency __getitem__ (the pure python version) there is making me think this may be better as a key fallback, so the expense is limited just to the negative integer case (or another idea, the negative int keys could

Re: [sqlalchemy] Re: Should RowProxy behave as a standard Python sequence?

2016-01-25 Thread Mike Bayer
On 01/25/2016 01:41 PM, Lele Gaifax wrote: > Mike Bayer <mike...@zzzcomputing.com> writes: > >> throwing an isinstance() right into the otherwise no-latency __getitem__ >> (the pure python version) there is making me think this may be better as a >> key fallback,

Re: [sqlalchemy] Naming convention for primary key and Flask, Sqlalchemy

2016-01-25 Thread Mike Bayer
On 01/25/2016 02:51 PM, Gastón Avila wrote: > Hi all, > > I have set up specific naming conventions in a Flask-Sqlalchemy app > using the metadata argument and it all seems to work except for naming > PRIMARY KEY constraints. Here is what I have > > | > self.db.init_app(app) >

Re: [sqlalchemy] has anyone measured the performance of filter_by vs filter?

2016-01-20 Thread Mike Bayer
filter_by creates a SQL construct and calls filter(), so the difference should be negligible. On 01/20/2016 12:57 PM, Jonathan Vanasco wrote: > I'm auditing some code, and noticed that we've often flipped between the > two. > > has anyone run benchmarks on the two regarding performance /

Re: [sqlalchemy] Searching from jsonb list

2016-01-20 Thread Mike Bayer
you can use that, that's func.jsonb_array_elements_text(column) On 01/20/2016 12:12 PM, Sami Pietilä wrote: > I have been using this kind of sql to search case insensitive strings > from jsonb lists in postgresql. jsonb_array_elements_text unpacks the > list items as normal table rows from

Re: [sqlalchemy] Should RowProxy behave as a standard Python sequence?

2016-01-24 Thread Mike Bayer
On 01/24/2016 11:11 AM, Lele Gaifax wrote: > Hi all, > > I'm a bit surprised to see the following: > > >>> r = c.execute(q, idtask=reception_presence_05_01.idtask).fetchall() > >>> type(r[-1]) > > >>> r[-1] > (datetime.datetime(2016, 5, 1, 18, 0), datetime.datetime(2016, 5, 1, 22,

Re: [sqlalchemy] Searching from jsonb list

2016-01-20 Thread Mike Bayer
this is more of a "what SQL will work on Postgresql" question, I'd ask on StackOverflow or the PG list just in terms of the SQL query to use. Once you get a string SQL query that works we can do it through SQLAlchemy. On 01/20/2016 07:44 AM, Sami Pietilä wrote: > The generated SQL does not seem

Re: [sqlalchemy] How to execute multiple insert/update queries in one SQL using Python/SQLAlchemy and MSSQL?

2016-01-19 Thread Mike Bayer
On 01/19/2016 03:01 AM, Guoliang Li wrote: > Hi all, > > I’m new to python and SQLAlchemy, I'm trying to understand how to execute > multiple insert/update queries in one SQL using Python/SQLAlchemy: > > Requirement Execute multiple insert/update in one SQL: > > DECLARE @age INT = 160 > >

Re: [sqlalchemy] Searching from jsonb list

2016-01-19 Thread Mike Bayer
call func.lower(expr) on the expression that you're comparing towards so that it will render the SQL LOWER() function, then compare to a lower case Python value. On 01/19/2016 10:58 AM, Sami Pietilä wrote: > I think I got "contains" and "cast" working for case sensitive exact > string

Re: [sqlalchemy] Latest SA version which support Jython

2016-01-19 Thread Mike Bayer
I've heard people are running 1.0.x with Jython. It's more a matter of Jython fixing their bugs, so if they've been doing that, try it out. On 01/18/2016 09:23 PM, Jaimy Azle wrote: > Hi all, > > I know jython support was dropped from current version of sqlalchemy, > but here I'm stuck to

Re: [sqlalchemy] Using a connection pool with multiple processes on the same machine

2016-01-22 Thread Mike Bayer
On 01/22/2016 01:23 AM, Maximilian Roos wrote: > Great, thanks for the reply Mike. > > I looked further - Celery *is* using fork / multiprocessing, but the > forking occurs before any import of our libraries / sqlalchemy / > create_engine. Is there a risk of reusing connections in that state? >

Re: [sqlalchemy] proper attribute names for many-to-many relationships using automap

2016-02-14 Thread Mike Bayer
On 02/14/2016 05:01 PM, Brian Cherinka wrote: What is the proper way to get pluralized shortened names for many-to-many tables when using automap? I currently have it set to generate pluralized lowercase names for collections instead of the default "_collection". This is what I want for

Re: [sqlalchemy] Re: best way to declare one-to-one relationships with automap and non-explicit relationships

2016-02-14 Thread Mike Bayer
On 02/14/2016 06:13 PM, Brian Cherinka wrote: you'd need to implement a generate_relationship function as described at http://docs.sqlalchemy.org/en/rel_1_0/orm/extensions/automap.html#custom-relationship-arguments

[sqlalchemy] test message, please ignore

2016-02-15 Thread Mike Bayer
test -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com.

[sqlalchemy] SQLAlchemy 1.0.12 Released

2016-02-15 Thread Mike Bayer
SQLAlchemy release 1.0.12 is now available. Release 1.0.12 is a bug fix release resolving a handful of issues reported in the past few weeks. Primary development continues on the 1.1 series which is slated for initial releases in the spring. 1.0.12 includes relatively minor fixes,

Re: [sqlalchemy] How to correctly merge objects of type X or any subclass of type X into the session?

2016-02-16 Thread Mike Bayer
answered On 02/16/2016 09:08 AM, Michael Naber wrote: I would like to write code which can correctly merge objects of type X or any subclass of type X into the session. I have been doing session.merge(X(id=??)), which works fine for merging type X, but if the object ID references an instance

Re: [sqlalchemy] How to correctly merge objects of type X or any subclass of type X into the session?

2016-02-16 Thread Mike Bayer
to that session, which is dangerous. So you can either synchronize the threads on the point at which the receiving thread is calling merge() or you can detach the object from the Session before giving it to the receiving thread. Thank you, Michael On Tue, Feb 16, 2016 at 11:26 AM, Mike

Re: [sqlalchemy] Re: reflection taking a very long time?

2016-02-16 Thread Mike Bayer
turning on echo=True inside create_engine() will show you what queries are emitted as they occur so you can see which ones are taking long and/or hanging. On 02/16/2016 02:59 PM, Alex Hall wrote: Upon re-reading some of the docs, I realized that my problem may still be that initial

Re: [sqlalchemy] Re: reflection taking a very long time?

2016-02-16 Thread Mike Bayer
even get any queries. I get my own print statements, then the script tries to connect and hangs. I've added dbEngine.connect() just to be sure the problem is that first connection, and sure enough, it hangs on that line. On 2/16/16, Mike Bayer <clas...@zzzcomputing.com> wrote: turning o

Re: [sqlalchemy] Re: reflection taking a very long time?

2016-02-16 Thread Mike Bayer
ht=reflect#sqlalchemy.schema.MetaData.reflect.params.only for that. The columns reflected and/or mapped within each Table can be limited also but you need a little more code for that. Work on getting connected first :). On 2/16/16, Mike Bayer <clas...@zzzcomputing.com> wrote: well

Re: [sqlalchemy] hang on connect in forked process

2016-02-17 Thread Mike Bayer
On 02/17/2016 11:33 AM, Uri Okrent wrote: Maybe this is a psycopg question and if so please say so. I have a multi-threaded server which maintains a thread-pool (and a corresponding connection pool) for servicing requests. In order to mitigate python's high-water-mark memory usage behavior

Re: [sqlalchemy] hang on connect in forked process

2016-02-18 Thread Mike Bayer
in which it runs, so you'd need to put a timeout on it. On Wednesday, February 17, 2016 at 10:41:00 AM UTC-8, Mike Bayer wrote: On 02/17/2016 11:33 AM, Uri Okrent wrote: > Maybe this is a psycopg question and if so please say so. > > I have a multi-threaded serve

Re: [sqlalchemy] Weird SELECT when assigning to one-to-one relationship

2016-02-19 Thread Mike Bayer
so this is uselist=False e.g. one-to-one, you make a Contribution with a non-present timetable entry, when you get to the flush(), it knows nothing about what timetable_entry objects might exist for this row. Unsurprisingly then, if we do this: contrib = Contribution() s.add(contrib)

Re: [sqlalchemy] Accessing data from an already executed query in a hybrid_property (using SQLAlchemy Transformer in the query)

2016-02-19 Thread Mike Bayer
In general, if you want objects to be populated as part of the normal loading process there needs to be a relationship() to catch it. This can be achieved in this specific case like the following: class CustomerLatestShipmentTransformer(hybrid.Comparator): @property def cte(self):

Re: [sqlalchemy] relationship between declarative base and automap base

2016-02-19 Thread Mike Bayer
On Fri, Feb 19, 2016 at 7:09 PM, Brian Cherinka wrote: > Hi. > > I have two database schemas, with a table from each I would like to join. > The classes for one schema have been created as explicit declarative Bases, > while the classes for the other were all created via

<    3   4   5   6   7   8   9   10   11   12   >