Re: [sqlalchemy] How to map class to multiple tables (for sharding purpose)

2016-05-20 Thread Михаил Доронин
The first link you gave have exactly the thing I said I don't want to do. The second one is however looks very promising. Have I understand correctly that I need to create custom Session class and custom Query class for this session that would somehow produce correct from clause (with correct

Re: [sqlalchemy] mysql/sqlite date workaround

2016-05-19 Thread Михаил Доронин
Completely forgot! Thank you, Mike. That works. On Tuesday, 17 May 2016 00:43:07 UTC+3, Mike Bayer wrote: > > > > On 05/16/2016 04:34 PM, Михаил Доронин wrote: > > I've opened issue here > > < > https://bitbucket.org/zzzeek/sqlalchemy/issues/3714/sqlalche

[sqlalchemy] How to map class to multiple tables (for sharding purpose)

2016-05-19 Thread Михаил Доронин
Suppose I have models (with corresponding tables) in pseudocode table1 | some_unique_name | some_property | |--+---| | foo | 1 | |--+---| | bar | 2 | |--+---|

[sqlalchemy] mysql/sqlite date workaround

2016-05-16 Thread Михаил Доронин
I've opened issue here , please read it, it's very brief. Now Mike Bayer says that it's not a sqlalchemy bug, and says that I can use cast to hint to mysql to return datetime.date, but this won't

[sqlalchemy] Re: mysql/sqlite date workaround

2016-05-16 Thread Михаил Доронин
Yeah, I know that there's a lot of corner cases in SQL compatibility, although it should be standardised. So of course I'm testing most of the code both in unit tests with sqlite and then manually in mysql (with not so granural tests of course). I just hoped that SQLalchemy solves this

[sqlalchemy] MYSQL ON_DUPLICATE_KEY_UPDATE implementation

2017-05-22 Thread Михаил Доронин
I've rewritten tests to match other tests and have been able to run them successfully. I've also made pull request . I've tested sql generation in `test_compiler` and that values that should be updated have been updated and values that shouldn't

Re: [sqlalchemy] insert performance vs executemany and max_allowed_packet limit

2017-05-29 Thread Михаил Доронин
.values in mysql, because there is no values have been passed at this point. On Thursday, 25 May 2017 22:43:55 UTC+3, Mike Bayer wrote: > > > > On 05/25/2017 01:44 PM, Михаил Доронин wrote: > >> SQLAlchemy batches inserts in the ORM as is possible and with Core you > >

Re: [sqlalchemy] insert performance vs executemany and max_allowed_packet limit

2017-05-29 Thread Михаил Доронин
in mysql, because there are no values have been passed at this point. On Thursday, 25 May 2017 22:43:55 UTC+3, Mike Bayer wrote: > > > > On 05/25/2017 01:44 PM, Михаил Доронин wrote: > >> SQLAlchemy batches inserts in the ORM as is possible and with Core you > >>

[sqlalchemy] insert performance vs executemany and max_allowed_packet limit

2017-05-25 Thread Михаил Доронин
I've tried to benchmark alchemy performance when inserting a lot of data. The results wasn't that good for sqlalchemy. The difference was up to three times in median values. First of all the more elements inserted the more the difference between sqlalchemy and executemany (mysqlclient). I've

Re: [sqlalchemy] insert performance vs executemany and max_allowed_packet limit

2017-05-25 Thread Михаил Доронин
<https://c1.staticflickr.com/9/8245/8453257683_2f55e98120_b.jpg> Wow, I've never saw this thing about multiple statements. Thanks! RTFM On Thursday, 25 May 2017 22:43:55 UTC+3, Mike Bayer wrote: > > > > On 05/25/2017 01:44 PM, Михаил Доронин wrote: > >> SQLAlchemy

Re: [sqlalchemy] insert performance vs executemany and max_allowed_packet limit

2017-05-29 Thread Михаил Доронин
Umm, what I've meant is how to use postgresql on_conflict_do_update in such a way that sqlalchemy would use executemany behind the scenes. In examples it usage looks like this. stmt = insert(table, values) stmt = stmt.on_conflict_do_update(set_=dict(a=stmt.excluded.a)) excluded is generated

Re: [sqlalchemy] insert performance vs executemany and max_allowed_packet limit

2017-05-25 Thread Михаил Доронин
> SQLAlchemy batches inserts in the ORM as is possible and with Core you > do this explicitly, both make use of cursor.executemany() which is then > determined by how the DBAPI handles it. Ummm. If that is true, why this line is used when I pass a list of values (its from visit_insert)?

Re: [sqlalchemy] insert performance vs executemany and max_allowed_packet limit

2017-05-25 Thread Михаил Доронин
Sorry I can't post a benchmark here, because I'm not in the office. Maybe I will do that tomorrow. I use vmprof and can just share a link to the uploaded profile. Will that suite you? But I can say right now that I've passed only integers, not strings. Also mysqlclient encoding values if

Re: [sqlalchemy] insert performance vs executemany and max_allowed_packet limit

2017-05-25 Thread Михаил Доронин
For Christ sake how to post a new message here and not just reply? And how to edit the old post? Arg! -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example.