Hi all,
We just fixed a bug in a library, edx-opaque-keys <https://pypi.python.org/pypi/edx-opaque-keys>, that we use in a few places in the edx ecosystem. Here’s the good news: if you’re using the Dogwood release, the Eucalyptus release, or any other stack whose edx-opaque-keys version is lower than 0.3.2, then no action is needed. You’re all set. In the rest of the document, I’ll describe what to do if your edx-platform is using version 0.3.2 of opaque-keys. If you don’t want to get into the details, here’s the TL;DR: 1. The risk of this bug is that opaque keys with trailing newline characters are written to the database. 2. Apply these two patches: patch1 <https://github.com/edx/edx-platform/commit/81ad1dc6df40019f5fa9c97ae2674f5b43a7c48e.patch>, patch2 <https://github.com/edx/edx-platform/commit/db25725f32a05f127376606a7a9302995f1ea36c.patch>. The first one fixes the bug (by updating opaque-keys to 0.3.3) and the second one logs if you encounter these newline-trailing keys when saving to or reading from the database. 3. Check your access logs for the URL-escaped newline character, “%0A”. If you don’t see them, you’re probably in the clear. 4. Check your application logs for the warnings from that second patch <https://github.com/edx/edx-platform/commit/db25725f32a05f127376606a7a9302995f1ea36c>. (query for the strings for “Removing trailing newline.” or “get_prep_value: Invalid key:”) 5. Check your database columns for trailing-newline keys: (list of columns and tables <https://docs.google.com/spreadsheets/d/1WwNkj99vI_FA1GMhNUWM2AJODItogKBAr26JHAwaNZA/edit#gid=0> to check). You might want to check student_courseenrollment and courseware_studentmodule first. If you don’t see any affected rows in all those tables, you’re definitely in the clear. 6. If you find warnings in step (4) or find newline-trailing keys in step (5), you should take steps to fix the data, either by removing or altering them. Here’s the more detailed story: In our stack, only edx-platform was on version 0.3.2 of edx-opaque-keys. If your version of edx-platform was using 0.3.2, it’s still pretty unlikely that you would have been affected by this issue. Nevertheless, we recommend you apply the following two patches: 1. To update opaque-keys to 0.3.3: https://github.com/edx/edx-platform/commit/81ad1dc6df40019f5fa9c97ae2674f5b43a7c48e.patch 2. To add logging to help tell if you need to take further remediation steps: https://github.com/edx/edx-platform/commit/db25725f32a05f127376606a7a9302995f1ea36c.patch The major effect of this bug is that it’s possible for opaque keys with trailing newline characters to be written to the edx-platform’s MySQL database. That is unlikely to happen, because it means that someone would have had to enter a course key, probably in a url, with a newline character at the end of it. To check to see if anyone tried that, you can search your nginx access logs for requests made to a url that has a course key that ends with the URL-escaped newline character, “%0A”. If you do find such requests, it’s very possible you were affected by this bug. If you don’t find such requests, you still may have been affected, but it’s unlikely. The second patch included above will also log every time a django model reads an opaque key with a trailing newline from the database. If you see these logs, then it’s definite that you have these trailing-newline keys in your database. So, if you don’t see any URL-escaped newline characters in your access logs, or any logs from that second patch, you’re probably ok. If you do see either of these, or you think you may still be affected, you should check your tables that have any columns that hold opaque keys. We’ve assembled a list of every table-column combination in edx-platform that might hold one of these keys in this spreadsheet <https://docs.google.com/spreadsheets/d/1WwNkj99vI_FA1GMhNUWM2AJODItogKBAr26JHAwaNZA/edit#gid=0> . For example, we found that we had rows in student_courseenrollment where entries in the course_id column ended with “\n”. If you do find these newline-trailing keys, you should take action to change or remove them. Thank you, and let us know if you have any questions. Best, Adam -- Adam Palay | edX a...@edx.org Lead Sustaining Engineer -- You received this message because you are subscribed to the Google Groups "General Open edX discussion" group. To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CAESNvkJYN9%3DSawL2VVJhbSwafW7781qjCS%2B%3D1Khqu%2B%3DJmfvFRw%40mail.gmail.com.
