Amazing! :) Big thanks. ср, 13 июл. 2016 г. в 12:59, Simon King <[email protected]>:
> OK, I have to admit that I haven't fully understood the details of > what you are doing. However, it sounds like you are implementing > horizontal sharding, which SQLAlchemy has a bit of support for: > > http://docs.sqlalchemy.org/en/latest/orm/extensions/horizontal_shard.html > > > http://docs.sqlalchemy.org/en/latest/_modules/examples/sharding/attribute_shard.html > > In the example, the query_chooser function is inspecting the query to > see which backend database the query should be issued against, which I > guess is what your "_analyze_base_query" method does. > > Would this work for you? > > Simon > > > On Tue, Jul 12, 2016 at 3:03 PM, Антонио Антуан <[email protected]> > wrote: > > Ok, I define my situation. > > At first, I'm appologizing for my accent :) > > > > I have two databases, postgresql and vertica. > > I have some amount of large tables, besides other tables. > > > > Postgres stores only last two hours (or last week, depends on table) of > > large tables, vertica stores all other data. > > > > I implemented SplittedDataQuery, subclass of Query, which used with my > > Session like query_cls. > > The subclass overrides __iter__(), all(), count(), first(), one() > methods. > > It has attribute _base_query, which refers on self, _pg_query and > > _vertica_query. > > > > Before invoking them, SplittedDataQuery checks necessity (by whereclause > > inside query and `splitting_time` - moment when the data splitted between > > databases) of getting data only from Pg, or only from vertica, or from > both > > with data union. > > > > Data from vertica returns by _vertica_query, data from postgres - by > > _pg_query > > _vertica_query binds with SessionVertica. > > all other queries (_pg_query and _base_query) binds with Session > > > > Function, creates SessionVertica, has nothing unusual. > > > > Session for pg creates like this: > > def _create_session(self, conn_string, splitted_query_cls=True): > > engine = create_engine(conn_string) > > > > from core.splitted_data import SplittedDataQuery > > session = ScopedSession(sessionmaker(bind=engine, > > query_cls=SplittedDataQuery if splitted_query_cls else Query)) # > query_cls > > choosing only for SplittedDataQuery._pg_query > > > > return session > > > > > > There is overrided all(): > > def all(self): > > return self._get_result() > > > > Implementation of _get_result() looks like this: > > def _get_result(self): > > self._analyze_base_query() > > self._split_query() > > self._initialize_db_queries() > > result = self._vertica_query.all() > > if self._needs_to_get_from_pg: > > result.extend(self._pg_query.all()) > > > > _db_queries initialization inside SplittedDataQuery class: > > def __init_pg_query(self): > > original_session = > > _create_session(config['sqlalchemy']['conn_string'], > > splitted_query_cls=False) > > self._pg_query = self._query.filter() \ > > .with_session(original_session) > > > > > > def __init_vertica_query(self): > > from hasoffers.core.model.meta import SessionVertica > > self._vertica_query = self._query.filter() \ > > .with_session(SessionVertica) > > > > If I don't create _pg_query using splitted_query_cls=False (with std > Query > > class), then _pg_query invokation of methods (all(), one() and other) > will > > always apply overrided methods (because it uses SplittedDataQuery). > > But, if I create _pg_query by defined way, it creates new session and, > so, > > new connection to Postgres. > > > > As you see, I don't want nor the first, neither the second option :) > > вторник, 12 июля 2016 г., 13:35:48 UTC+3 пользователь Simon King написал: > >> > >> Could you describe what you are trying to achieve? There's nothing > >> about Mike's suggestion that means you need to create a new session - > >> you can reuse any existing session. > >> > >> What does your CustomQueryCls do? Perhaps there's another way of doing > >> what you want? > >> > >> Simon > >> > >> On Tue, Jul 12, 2016 at 11:09 AM, Антонио Антуан <[email protected]> > >> wrote: > >> > But it means that I should create one more session with one more > >> > connection > >> > to DB. This is not good for me :( > >> > > >> > понедельник, 11 июля 2016 г., 22:23:11 UTC+3 пользователь Антонио > Антуан > >> > написал: > >> >> > >> >> Can I specify query_cls only for a one query? I try to change > >> >> query.session._query_cls and, of course, it doesn't work... > >> >> > >> >> I want to perform something like this: > >> >> > >> >> query = Session.query(MyModel)... > >> >> default_querycls_result = query.all() > >> >> custom_querycls_result = query.change_query_cls(CustomQueryCls).all() > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "sqlalchemy" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> > an > >> > email to [email protected]. > >> > To post to this group, send email to [email protected]. > >> > Visit this group at https://groups.google.com/group/sqlalchemy. > >> > For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "sqlalchemy" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To post to this group, send email to [email protected]. > > Visit this group at https://groups.google.com/group/sqlalchemy. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. > -- Антон -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
