Re: [mezzanine-users] Re: s3 and Bad Request 400 + thumbnails with s3
Ok cool, we might leave it for now since the new release is almost here - but thanks so much for documenting all your findings, no doubt it'll be useful. On Wed, Jul 1, 2015 at 1:01 PM, Lee H. popov.gh...@gmail.com wrote: Regarding a fix. I guess there a few points: 1) This was all in Django 1.6 with the latest stable release of Mezzanine (not the master dev branch). In Django 1.6, the suspicious operation gets swallowed, but in later version of Django 1.6, a technical 500 response is returned if the user has DEBUG True. The `django.core.handlers.base` code was changed to be if settings.DEBUG: return debug.technical_500_response(request, *sys.exc_info(), status_code=400) so the user gets a debug screen with Exception Type: SuspiciousOperation Exception Value: Attempted access to '/path/to/my/project/temp_media/uploads/' denied. and full trace, which makes identifying the issue A LOT easier (I verified this on the latest Mezzanine master branch and django 1.8). Hence there is an argument that nothing should be done, and that in future version Mezzanine that bind to django 1.6 the user will get this nicer debug screen anyway. 2) Whilst if I'd been using django 1.6 I wouldn't have had the headache. I'd personally say there is still room to make this more transparent for an end user who wants to use Mezzanine and s3. A user could quite easily forget to set `MEDIA_ROOT= ''` perhaps, and then be greeted with the above SuspiciousOperation and trace, which requires digging through a little bit to realise it's down to the MEDIA_ROOT. Could there perhaps be a misconfiguration warning coming in at the filebrowser-safe S3BotoStorageMixin level if MEDIA_ROOT is not ' ' . Or the test using self.location of S3BotoStorage and MEDIA_ROOT could be performed in the mixin. Typically say, self.location = media for a custom S3Storage media class, and MEDIA_ROOT might be naively set by the user as '/path/to/project/temp_media`, so comparing these two could also raise an error which asks the user to check the value of their MEDIA_ROOT setting if it looks like the filepath (generated using MEDIA_ROOT) is going to be outside of the location (in a very similar way to the check safe_join is currently doing, but with an exception that is more specific to this scenario, and more transparent: we can say it's not SuspciousOperation; it's just the user is mixing a cloud location with local filesystem MEDIA_ROOT). -- You received this message because you are subscribed to the Google Groups Mezzanine Users group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Stephen McDonald http://jupo.org -- You received this message because you are subscribed to the Google Groups Mezzanine Users group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [mezzanine-users] Adding pagination to Mezzanine TabularDynamicInlineAdmin
Hi Lee, it's definitely possible. Here's how it's done in the blog: https://github.com/stephenmcd/mezzanine/blob/master/mezzanine/blog/views.py#L54 Also, take a look at the template to see how it's used there: https://github.com/stephenmcd/mezzanine/blob/master/mezzanine/blog/templates/blog/blog_post_list.html Good luck! On Fri, Jun 26, 2015 at 12:05 PM, Lee H. popov.gh...@gmail.com wrote: On one of my pages I have a model called MapPlace (it's basically just a place name and some geo coords), I add it to the admin as an inline of the page like from mezzanine.core.admin import TabularDynamicInlineAdmin, class MapPlaceInline(TabularDynamicInlineAdmin): model = MapPlace Things look like http://www.awesomescreenshot.com/image/360673/6d18b2c7d906fead91ebe9696becb8f3 All well and good, but now I have over 30 MapPlaces added, and this list is becoming unwieldy, so I need to paginate these rows. Is this possible in Mezzanine? -- You received this message because you are subscribed to the Google Groups Mezzanine Users group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Mezzanine Users group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mezzanine-users] Re: Adding pagination to Mezzanine TabularDynamicInlineAdmin
Many thanks Josh, that's very helpful! -- You received this message because you are subscribed to the Google Groups Mezzanine Users group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mezzanine-users] Re: s3 and Bad Request 400 + thumbnails with s3
After killing myself to try and work out what throws this bad request (400)
error when filebrowser-safe is installed and when MEDIA_ROOT is set to non
''. I finally (through blood sweat and tears), worked out that it comes
from the safe_join function of the django-storages s3boto.py. Because I
have a custom storage class with 'location=media' for media files, and
another with location = static' for static files (to ensure static and
media are in sep dirs in the bucket), the safe_join function uses media
as the base dir. It finds that the path to the uploads dir is wherever I
set it in 'MEDIA_ROOT' and it doesn't like it (it's trying to prevent users
from uploading outside the MEDIA_ROOT of course). The exception is when
MEDIA_ROOT = '' or '/'.
if (not final_path.startswith(base_path) or
final_path[base_path_len:base_path_len + 1] not in ('', '/')):
raise ValueError('the joined path is located outside of the base
path'
' component')
This ValueError is caught by
_normalize_name and traded for a Supicious Operation.
Finally this gets caught by
`django.core.handlers.base.BaseHandler.get_response`, which if you have logging
turned on logs the suspicious operation
As attempt to access dir denied (I didnt). It then just sends this to a Bad
request (400) so the user is left blind. It's almost the same
as this https://code.djangoproject.com/ticket/21668#no2.
It is filebrowser-safe's get_directory, that calls isdir (of the
S3BotoStorageMixin) that calls isfile, that calls
_normalize_name of s3Botostorage, that finally calls safe_join. Hence why I
only saw the problem when filebrowser-safe was installed.
--
You received this message because you are subscribed to the Google Groups
Mezzanine Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mezzanine-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[mezzanine-users] Re: Performance: Filebrowser and S3
One option is just to `pip uninstall filebrowser-safe`, then Mezzanine reverts back to the usual Django file upload (no need to change settings). Of course then you can't manage the media library, only upload, but it's not slow at least. Can anyone confirm if the bottlebeck is due to the `isdir` method in the S3BotoStorageMixin after all? Any ideas how the methods in this mixin could be wrote more efficiently? -- You received this message because you are subscribed to the Google Groups Mezzanine Users group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mezzanine-users] Re: s3 and Bad Request 400 + thumbnails with s3
Just another note: if filebrowser-safe is uninstalled this bad request 400 does not occur, regardless if MEDIA_ROOT is set to some local fs, or empty, so I guess it must be a problem with filebrowser when MEDIA_ROOT is something other than ' ', and s3 storages being used. -- You received this message because you are subscribed to the Google Groups Mezzanine Users group. To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [mezzanine-users] Re: s3 and Bad Request 400 + thumbnails with s3
Any thoughts on a fix?
On Wed, Jul 1, 2015 at 3:45 AM, Lee H. popov.gh...@gmail.com wrote:
After killing myself to try and work out what throws this bad request
(400) error when filebrowser-safe is installed and when MEDIA_ROOT is set
to non ''. I finally (through blood sweat and tears), worked out that it
comes from the safe_join function of the django-storages s3boto.py.
Because I have a custom storage class with 'location=media' for media
files, and another with location = static' for static files (to ensure
static and media are in sep dirs in the bucket), the safe_join function
uses media as the base dir. It finds that the path to the uploads dir is
wherever I set it in 'MEDIA_ROOT' and it doesn't like it (it's trying to
prevent users from uploading outside the MEDIA_ROOT of course). The
exception is when MEDIA_ROOT = '' or '/'.
if (not final_path.startswith(base_path) or
final_path[base_path_len:base_path_len + 1] not in ('', '/')):
raise ValueError('the joined path is located outside of the base
path'
' component')
This ValueError is caught by
_normalize_name and traded for a Supicious Operation.
Finally this gets caught by
`django.core.handlers.base.BaseHandler.get_response`, which if you have
logging turned on logs the suspicious operation
As attempt to access dir denied (I didnt). It then just sends this to a
Bad request (400) so the user is left blind. It's almost the same
as this https://code.djangoproject.com/ticket/21668#no2.
It is filebrowser-safe's get_directory, that calls isdir (of the
S3BotoStorageMixin) that calls isfile, that calls
_normalize_name of s3Botostorage, that finally calls safe_join. Hence why I
only saw the problem when filebrowser-safe was installed.
--
You received this message because you are subscribed to the Google Groups
Mezzanine Users group.
To unsubscribe from this group and stop receiving emails from it, send an
email to mezzanine-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Stephen McDonald
http://jupo.org
--
You received this message because you are subscribed to the Google Groups
Mezzanine Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mezzanine-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
