Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-05 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  closed
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by Tim McCormack):

 I get the error even with `warnings.warn(message=("hi", dict),
 category=UserWarning)`. I suspect it depends on which warning filters are
 loaded in your environment.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.239d94b8981dafc8ff682170a2ec5d64%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  closed
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by Mariusz Felisiak ):

 In [changeset:"b2b2723512dd0024567fba0c680424ebb7fa1149" b2b2723]:
 {{{
 #!CommitTicketReference repository=""
 revision="b2b2723512dd0024567fba0c680424ebb7fa1149"
 [2.2.x] Fixed #31654 -- Fixed cache key validation messages.

 Backport of 926148ef019abcac3a9988c78734d9336d69f24e from master.
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.c37aab4cb7b4287000fe7913354a7ee1%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  closed
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by Mariusz Felisiak ):

 In [changeset:"e8723af44be82968f573a00277f40016b049f08e" e8723af4]:
 {{{
 #!CommitTicketReference repository=""
 revision="e8723af44be82968f573a00277f40016b049f08e"
 [3.0.x] Fixed #31654 -- Fixed cache key validation messages.

 Backport of 926148ef019abcac3a9988c78734d9336d69f24e from master
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.32c3bcdbab9a3410764eba7f742ff3c8%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  closed
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by Mariusz Felisiak ):

 In [changeset:"031a082d41ed86a75e5a0b53d9fcdfb3e3a6589f" 031a082]:
 {{{
 #!CommitTicketReference repository=""
 revision="031a082d41ed86a75e5a0b53d9fcdfb3e3a6589f"
 [3.1.x] Fixed #31654 -- Fixed cache key validation messages.

 Backport of 926148ef019abcac3a9988c78734d9336d69f24e from master
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.f3c75e354e1bf3ac6e6330c7ac25d8ba%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  closed
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+
Changes (by GitHub ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"926148ef019abcac3a9988c78734d9336d69f24e" 926148e]:
 {{{
 #!CommitTicketReference repository=""
 revision="926148ef019abcac3a9988c78734d9336d69f24e"
 Fixed #31654 -- Fixed cache key validation messages.
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.4d00b448ac3fc1ea4e42ee11faa2e925%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  assigned
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+
Changes (by felixxm):

 * has_patch:  0 => 1


Comment:

 [https://github.com/django/django/pull/13022 PR]

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.2b1d70c2e01f879bda8ad6203b974ed9%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  assigned
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by felixxm):

 I can reproduce this error when unpacking tuple
 {{{
 >>> warnings.warn(*("hi", str), RuntimeWarning)
 Traceback (most recent call last):
   File "", line 1, in 
 TypeError: 'type' object cannot be interpreted as an integer
 }}}
 because `RuntimeWarning` is passed as `stacklevel`, it must be somewhere
 in EdX. Nevertheless we will fix this message.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.d8c3de3f0549cac67e3a4156cb265a46%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  assigned
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by Simon Charette):

 Out of curiosity, do you get the same crash if passing `category` as a
 keyword argument? `warnings.warn(warning, category=CacheKeyWarning)`

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.2b074033ead8c0addf48844f4552e114%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  assigned
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by Tim McCormack):

 I don't understand it either! I'll take a little more time to poke at it
 today. (For anyone who wants to reproduce this strange test error in
 place: Python 3.5 virtualenv, `make requirements`, `pytest -s
 
common/djangoapps/xblock_django/tests/test_api.py::XBlockSupportTestCase::test_authorable_blocks`
 -- on commit 9f53525c.) The farthest I've gotten is that it's definitely
 coming from inside `warnings.warn(...)`:

 {{{
 (Pdb) warnings.warn(("hi", str), RuntimeWarning)
 *** TypeError: 'type' object cannot be interpreted as an integer
 }}}

 Why it doesn't happen on a fresh REPL with all dependencies loaded, I'm
 not sure. My best guess is that it has something to do with warning
 filtering, since that's configurable and appears to be what's inspecting
 the warning message. Probably not specifically relevant to Django or the
 edX code, though.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.42697ecdc4d8727bf89163c19551f04b%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  assigned
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Release blocker  |   Resolution:
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+
Changes (by Carlton Gibson):

 * severity:  Normal => Release blocker


Comment:

 Thanks for the follow up Tim.

 I still don't get the error using 3.5...


 {{{
 Python 3.5.9 (default, Jun  4 2020, 16:47:18)
 [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] on darwin
 Type "help", "copyright", "credits" or "license" for more information.
 >>> from django.core.cache.backends.base import CacheKeyWarning
 >>> import warnings
 >>> a_tuple = ('A string message, plus...', CacheKeyWarning)
 >>> warnings.warn(a_tuple, CacheKeyWarning)
 __main__:1: CacheKeyWarning: ('A string message, plus...', )
 }}}

 There must be something else going on. Maybe a custom warning formatting?

 (How else is the `TypeError: 'type' object cannot be interpreted as an
 integer` — the warning class being converted to an int? 樂)

 [https://github.com/edx/edx-platform/pull/24137 The EdX change] will stop
 the warning coming up, and is exactly the case that the patch was meant to
 catch.

 Treating CacheKeyWarnings as errors would be good — I wonder how many
 folks would be seeing those and ignoring them... 

 I think there's enough of a regression here to backport the fix.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.21f5f7ba3e934890b892c39438053982%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  assigned
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Normal   |   Resolution:
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by Tim McCormack):

 Ah, this was actually against the dummy backend. It was caught in tests:
 https://build.testeng.edx.org/job/edx-platform-python-pipeline-
 
pr/18276/testReport/junit/common.djangoapps.xblock_django.tests.test_api/XBlockSupportTestCase/Run_Tests___lms_unit___test_disabled_blocks/

 {{{
 self = 

 def setUp(self):
 super(XBlockSupportTestCase, self).setUp()

 # Set up XBlockConfigurations for disabled and deprecated states
 block_config = [
 ("poll", True, True),
 ("survey", False, True),
 ("done", True, False),
 ]

 for name, enabled, deprecated in block_config:
 >   XBlockConfiguration(name=name, enabled=enabled,
 deprecated=deprecated).save()

 common/djangoapps/xblock_django/tests/test_api.py:28:
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
 _ _ _
 ../edx-venv-3.5/edx-venv/lib/python3.5/site-
 packages/config_models/models.py:110: in save
 update_fields
 ../edx-venv-3.5/edx-venv/lib/python3.5/site-
 packages/django/db/models/base.py:741: in save
 force_update=force_update, update_fields=update_fields)
 ../edx-venv-3.5/edx-venv/lib/python3.5/site-
 packages/django/db/models/base.py:790: in save_base
 update_fields=update_fields, raw=raw, using=using,
 ../edx-venv-3.5/edx-venv/lib/python3.5/site-
 packages/django/dispatch/dispatcher.py:175: in send
 for receiver in self._live_receivers(sender)
 ../edx-venv-3.5/edx-venv/lib/python3.5/site-
 packages/django/dispatch/dispatcher.py:175: in 
 for receiver in self._live_receivers(sender)
 openedx/core/lib/cache_utils.py:187: in invalidate
 TieredCache.delete_all_tiers(key)
 ../edx-venv-3.5/edx-venv/lib/python3.5/site-
 packages/edx_django_utils/cache/utils.py:226: in delete_all_tiers
 django_cache.delete(key)
 ../edx-venv-3.5/edx-venv/lib/python3.5/site-
 packages/django/core/cache/backends/dummy.py:30: in delete
 self.validate_key(key)
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
 _ _ _

 self = 
 key = ":1:.xblock_django.api.deprecated_xblocks"

 def validate_key(self, key):
 """
 Warn about keys that would not be portable to the memcached
 backend. This encourages (but does not force) writing backend-
 portable
 cache code.
 """
 for warning in memcache_key_warnings(key):
 >   warnings.warn(warning, CacheKeyWarning)
 E   TypeError: 'type' object cannot be interpreted as an integer

 ../edx-venv-3.5/edx-venv/lib/python3.5/site-
 packages/django/core/cache/backends/base.py:250: TypeError
 }}}

 This error only started happening in tests with the Django 2.2.12 ->
 2.2.13 upgrade. I don't understand why we wouldn't have seen the
 InvalidCacheKey error in production, where we use memcached.

 (We were in fact creating an invalid cache key, and I have a patch ready
 for that on our side that unblocks the upgrade, so I'm not sure how much
 we'll end up digging into that mystery!)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.d9b9bdcaf90fd54d34686b9d621a5a7c%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message.

2020-06-04 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  assigned
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Normal   |   Resolution:
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+

Comment (by Carlton Gibson):

 > ... results in raising an exception rather than just producing a
 warning.

 The expected behaviour is to raise the InvalidCacheKey for memcached
 backends, addressing CVE-2020-13254.

 Tim, can you post the traceback?

 (How did you hit this?)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.0c05614274dedd724a3f32f9d834cab5%40djangoproject.com.

Re: [Django] #31654: Memcached key validation raises InvalidCacheKey with clunky message. (was: Memcached key validation raises TypeError on invalid characters)

2020-06-03 Thread Django 
#31654: Memcached key validation raises InvalidCacheKey with clunky message.
-+
 Reporter:  Tim McCormack|Owner:  felixxm
 Type:  Bug  |   Status:  assigned
Component:  Core (Cache system)  |  Version:  2.2
 Severity:  Normal   |   Resolution:
 Keywords:  memcached| Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+
Changes (by felixxm):

 * owner:  nobody => felixxm
 * status:  new => assigned
 * stage:  Unreviewed => Accepted


Comment:

 Thanks for the report, we've already noticed this, see
 [https://github.com/django/django/pull/13017 PR]. It raises
 `InvalidCacheKey` for me (even if a message is not perfect), e.g.
 {{{
 File "django/core/cache/backends/memcached.py", line 157, in validate_key
 raise InvalidCacheKey(warning)

 django.core.cache.backends.base.InvalidCacheKey: ("Cache key contains
 characters that will cause errors if used with memcached:
 '38_postgis_pull-requests-bionic_memcache:1:key with spaces and 清'",
 )
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.ac09885682d0bea14380f72c3ade2db7%40djangoproject.com.