On Thu, Feb 5, 2015 at 3:16 PM, Carl Meyer <[email protected]> wrote: > On 02/05/2015 11:20 AM, Larry Martell wrote: >> On Thu, Feb 5, 2015 at 10:53 AM, Carl Meyer <[email protected]> wrote: >>> TransactionTestCase has been around for quite a long time (since 1.1, >>> IIRC). It's definitely in 1.5. >> >> I thought it was not in 1.5 because I went to >> https://docs.djangoproject.com/en/1.5/topics/testing/tools/#transactiontestcase >> and got a 404. > > Ah, yeah; I think the docs just moved around. > >>>> and so on. I need the temp table to get dropped between each get call. >>>> The best way to do this is if each is in its own MySQL session. Having >>>> the table truncated doesn't really accomplish this. I think I will >>>> have to modify the code to drop the temp table after it's used. I hate >>>> to muck with working production code to accommodate a test, but I also >>>> don't want the test branch to have a different code base from the >>>> production branch. I'll probably go with the former option. >>> >>> I agree, I don't think TransactionTestCase will help with this situation. >>> >>> I find the production design a bit questionable, >> >> Why do you say that? I recently added the temp table for performance. >> I have a query that was taking over an hour. When I broke it into 2 >> queries using a temp table it then ran in 10 seconds. If I've >> introduced something bad in my app, I'd really like to know about >> that. > > I don't know if it's bad or not, given your needs, but introducing the > constraint "every request must run in its own MySQL session which is > shut down after the request" would give me pause, since reusing > connections (using the built-in Django connection-reuse feature, which > isn't in 1.5 yet I don't think, or using an external connection pooler) > is often a quick and easy performance win. And because having every > request leave the connection in an unusable state makes testing > harder/slower, too. It just feels wrong :-)
I changed the main code to drop the temp table. This made the tests run, and also is much cleaner. Thanks for the impetus to do this. > I don't know exactly why restructuring the query made it faster. The main table being selected from is largish - 38 million rows with 282 columns - and it's joined with 6 other tables. > In PostgreSQL I would usually use a CTE (WITH clause) to allow the same > type of query decomposition without needing an actual temp table; not > sure if MySQL supports those. (I recently reworked an ORM query using > joins to raw SQL with CTEs and saw a similar many-orders-of-magnitude > performance improvement, because the version with joins was internally > generating a result-set that grew with the square of the size of a > particular table.) I wasn't familiar with CTEs - I just read up on them. I don't think MySQL supports those. > But if the temp table is the only approach that > works, I would probably prefer to have my production code clean it up to > make session reuse possible. (Ideally, by encapsulating the creation and > destruction of the temp table in a database stored procedure, so it > becomes what it ought to be, a query implementation detail, not > something that application logic needs to be concerned about.) > >>> but taking it as a >>> given that it meets your needs adequately and you don't want to change >>> it, you could also address this problem by simply dropping the temp >>> table in a tearDown() method, no? >> >> A tearDown() method would be run after the test, right? I am sending >> multiple requests within one test. > > Can you restructure your tests to avoid that? The more focused a test > is, usually the better. > > If not, then I think you just need a utility method to destroy the temp > table, which you either call explicitly from your tests after each > request, or perhaps build into a subclass of the test client so it's > automated. Although a bit ugly, I think that's still better than trying > to reconnect to the database for each test; that'll kill your > test-running speed. (But I'd still prefer fixing the problem at the source.) -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CACwCsY4c%2BH2WsrEWsaekofznaLomRQkk2ufQjfvUC%2BucQYB2DQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
