Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  Andriy
 |  Sokolovskiy
 Type:  New feature  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  threads  | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by felixxm):

 * status:  new => closed
 * resolution:   => fixed


Comment:

 Please don't reopen old tickets. You can create a new ticket, but Django
 1.11 is no longer supported so you need to first confirm this issue in
 Django 3.1+.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.70b96b77597b8a9a69ad4739df295965%40djangoproject.com.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  coldmind
 Type:  New feature  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  threads  | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham ):

 In [changeset:"6f653f759ac2ba735bf8fa3fbb634a431288c041" 6f653f75]:
 {{{
 #!CommitTicketReference repository=""
 revision="6f653f759ac2ba735bf8fa3fbb634a431288c041"
 [1.9.x] Refs #12118 -- Allowed "mode=memory" in SQLite test database
 names.

 Backport of 3543fec3b739864c52de0a116dde3b0e5e885799 from master
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.8e2eb1b1fb74bd66a85c27b96e8bc666%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  coldmind
 Type:  New feature  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  threads  | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham ):

 In [changeset:"e2db171e15b8d7d924cc4b1e0c67c96478c2644b" e2db171e]:
 {{{
 #!CommitTicketReference repository=""
 revision="e2db171e15b8d7d924cc4b1e0c67c96478c2644b"
 [1.8.x] Refs #12118 -- Allowed "mode=memory" in SQLite test database
 names.

 Backport of 3543fec3b739864c52de0a116dde3b0e5e885799 from master
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.333bbd918901becdc55436a93a093171%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  coldmind
 Type:  New feature  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  threads  | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham ):

 In [changeset:"3543fec3b739864c52de0a116dde3b0e5e885799" 3543fec3]:
 {{{
 #!CommitTicketReference repository=""
 revision="3543fec3b739864c52de0a116dde3b0e5e885799"
 Refs #12118 -- Allowed "mode=memory" in SQLite test database names.
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.9249ace36a3207115ee9fd80064ba4c0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  coldmind
 Type:  New feature  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  threads  | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by hjwp):

 Incidentally, in case it's of use to anyone else that's temporarily stuck
 on an earlier versions of django, you can hack a workaround for this by
 using a path in `/dev/shm` in `TEST_NAME`:


 {{{
 DATABASES['default']['TEST_NAME'] = '/dev/shm/myproject-
 djangotestdb.sqlite'
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.7fa6461bc538ca3543544e44dd91ce9f%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  coldmind
 Type:  New feature  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  threads  | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"8c99b7920e8187f98cf4d7dbd9918bd6c6da1238"]:
 {{{
 #!CommitTicketReference repository=""
 revision="8c99b7920e8187f98cf4d7dbd9918bd6c6da1238"
 Fixed #12118 -- Added shared cache support to SQLite in-memory testing.
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.fd3d2a4bfaa78ac5d296cecada711590%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  coldmind
 Type:  New feature  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  threads  | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by timgraham):

 * needs_better_patch:  0 => 1


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.c89119cee56cfd2ddbe5eaa3c0b5bb9e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  coldmind
 Type:  New feature  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  threads  |  Needs documentation:  0
Has patch:  1|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by coldmind):

 * has_patch:  0 => 1


Comment:

 Pull request here: https://github.com/django/django/pull/3677

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.546ba616435eb77ec951abb82758487c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  coldmind
 Type:  New feature  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  threads  |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by coldmind):

 * status:  new => assigned
 * owner:  nobody => coldmind


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.279b93aff42decd9aaf719a03f6a6cc2%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  threads  |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by timo):

 * version:  1.1 => master
 * type:  Uncategorized => New feature


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.d0ea08314de4f5e8bd5142ea68e8fd30%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.1
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  threads  |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by Alex):

 * stage:  Unreviewed => Accepted


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.e8a0ab3b4f1a9ee22768c66ee95f7dd9%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
-+-
 Reporter:  bluebird75   |Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.1
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:
 Keywords:  threads  |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by anonymous):

 * status:  closed => new
 * severity:   => Normal
 * type:   => Uncategorized
 * easy:   => 0
 * ui_ux:   => 0
 * resolution:  invalid =>


Comment:

 I vote to reopen this.

 sqlite, as of version 3.7.13 (released 2012-06-11) has the ability to
 share an in-memory database between multiple connections and threads.

 See: http://www.sqlite.org/releaselog/3_7_13.html

 Making this work with the Django testing framework should be pretty easy:

 In the sqlite database backend, instead of using the database name
 `:memory:`, we should use a name such as
 `file:testdb?mode=memory=shared` (where "testdb" can be anything and
 ideally should be unique so that multiple tests can run concurrently, each
 with its own individual database).

 As a bonus, doing this should allow removing the hacky sqlite-specific
 code from "LiveServerTestCase" (It contains a messy workaround for exactly
 the issue of this bug report)

 The only thing that might be a little tricky is making this update in the
 Django code to still support older vesions of sqlite by falling back to
 the current behavior (`:memory:`), but I imagine that this shouldn't be
 too difficult.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.1c052d2ddf4c9480e2dca17a0e9d1330%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
---+
  Reporter:  bluebird75| Owner:  nobody 
Status:  closed| Milestone: 
 Component:  Database layer (models, ORM)  |   Version:  1.1
Resolution:  invalid   |  Keywords:  threads
 Stage:  Unreviewed| Has_patch:  0  
Needs_docs:  0 |   Needs_tests:  0  
Needs_better_patch:  0 |  
---+
Changes (by kmtracey):

  * status:  new => closed
  * needs_better_patch:  => 0
  * resolution:  => invalid
  * needs_tests:  => 0
  * needs_docs:  => 0

Comment:

 Near as I can tell this is an sqlite or pysqlite restriction, though I
 haven't been able to find a definitive statement in sqlite or pysqlite
 doc.  SQLAlchemy, however,  appears to have come to the conclusion that
 you simply cannot share :memory databases between threads, see:
 http://www.sqlalchemy.org/docs/06/reference/dialects/sqlite.html
 #threading-behavior. Thus I think there is no bug in Django here, the
 answer is don't do that and if you really want to do it you'll need to
 take it up with the maintainers of the lower level code.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

[Django] #12118: in-memory test database does not work with threads

[Django]

#12118: in-memory test database does not work with threads
--+-
 Reporter:  bluebird75|   Owner:  nobody
   Status:  new   |   Milestone:
Component:  Database layer (models, ORM)  | Version:  1.1   
 Keywords:  threads   |   Stage:  Unreviewed
Has_patch:  0 |  
--+-
 When using the test configuration of the DB with XXX and accessing the DB
 from another thread, it fails miserably.

 Using this example script on the Poll tutorial:

 {{{
 import os
 os.environ[ 'DJANGO_SETTINGS_MODULE' ] = 'settings'
 import settings
 import datetime, threading

 #django  stuff
 from polls.models import *
 from django.core.mail import mail_admins
 from django.test.utils import *
 from django.db import connection


 def create_object():
 print 'Creating Poll'
 p = Poll()
 p.question = "What's up doc ?"
 p.pub_date = datetime.date.today()
 p.save()
 print 'Poll object saved. Id: %d' % p.id


 WITH_THREAD = False

 if __name__ == '__main__':
 setup_test_environment()
 old_db_name = settings.DATABASE_NAME
 new_db_name = connection.creation.create_test_db(verbosity=1)
 print 'New DATABASE:', new_db_name

 if WITH_THREAD:
 t = threading.Thread( target=create_object )
 t.start()
 t.join()
 else:
 create_object()

 teardown_test_environment()
 connection.creation.destroy_test_db( old_db_name )

 }}}

 If I run it with WITH_THREADS set to False:


 {{{
 phili...@pc-philippe /cygdrive/d/work/django/django-tuto/mysite $ python
 run_wi
 th_threads.py
 Creating test database...
 Creating table auth_permission
 Creating table auth_group
 Creating table auth_user
 Creating table auth_message
 Creating table django_admin_log
 Creating table django_content_type
 Creating table django_session
 Creating table django_site
 Creating table polls_poll
 Creating table polls_choice
 Installing index for auth.Permission model
 Installing index for auth.Message model
 Installing index for admin.LogEntry model
 Installing index for polls.Choice model
 New DATABASE: :memory:
 Creating Poll
 Poll object saved. Id: 1
 Destroying test database...

 }}}

 If I run it with WITH_THREADS set to True:


 {{{
 phili...@pc-philippe /cygdrive/d/work/django/django-tuto/mysite $ python
 run_wi
 th_threads.py
 Creating test database...
 Creating table auth_permission
 Creating table auth_group
 Creating table auth_user
 Creating table auth_message
 Creating table django_admin_log
 Creating table django_content_type
 Creating table django_session
 Creating table django_site
 Creating table polls_poll
 Creating table polls_choice
 Installing index for auth.Permission model
 Installing index for auth.Message model
 Installing index for admin.LogEntry model
 Installing index for polls.Choice model
 New DATABASE: :memory:
 Creating Poll
 Exception in thread Thread-1:
 Traceback (most recent call last):
   File "c:\Python26\lib\threading.py", line 522, in __bootstrap_inner
 self.run()
   File "c:\Python26\lib\threading.py", line 477, in run
 self.__target(*self.__args, **self.__kwargs)
   File "run_with_threads.py", line 19, in create_object
 p.save()
   File "c:\Python26\lib\site-packages\django\db\models\base.py", line 410,
 in save
 self.save_base(force_insert=force_insert, force_update=force_update)
   File "c:\Python26\lib\site-packages\django\db\models\base.py", line 495,
 in save_base
 result = manager._insert(values, return_id=update_pk)
   File "c:\Python26\lib\site-packages\django\db\models\manager.py", line
 177, in _insert
 return insert_query(self.model, values, **kwargs)
   File "c:\Python26\lib\site-packages\django\db\models\query.py", line
 1087, in
 insert_query
 return query.execute_sql(return_id)
   File "c:\Python26\lib\site-packages\django\db\models\sql\subqueries.py",
 line
 320, in execute_sql
 cursor = super(InsertQuery, self).execute_sql(None)
   File "c:\Python26\lib\site-packages\django\db\models\sql\query.py", line
 2369, in execute_sql
 cursor.execute(sql, params)
   File "c:\Python26\lib\site-packages\django\db\backends\util.py", line
 19, in execute
 return self.cursor.execute(sql, params)
   File "c:\Python26\lib\site-packages\django\db\backends\sqlite3\base.py",
 line
 193, in execute
 return Database.Cursor.execute(self, query, params)
 OperationalError: no such table: polls_poll

 Destroying test database...

 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send