Hi Stephan, I'm glad to hear you were able to find a workaround. However,
it's very similar to the solution I posted above, so I'm curious to hear if
you had tried that but were still finding trouble. Where I used instance.
val.storage, instance.val.name (as per the recommendation of the
stackexchange question I linked) to create a working deletion method, you
made the direct connection to the S3 bucket. It's nice to see another
approach.
At any rate, I think you could probably solve your double function issue by
using a try/except statement in the original arches version of the function
in models.py. It could look something like this:
@receiver(models.signals.post_delete)
def auto_delete_file_on_delete(sender, instance, **kwargs):
"""Deletes file from filesystem
when corresponding `Files` object is deleted.
"""
if instance.val:
try:
## this is the original deletion code. when it hits the error
we both found, it'll
## skip to the except block below
if os.path.isfile(instance.val.path):
os.remove(instance.val.path)
except:
## these two lines were my recommendation from before, to test
this method just
## uncomment them and comment out the three lines below from
your solution
#storage, name = instance.val.storage, instance.val.name
#storage.delete(name)
## these three lines are from your solution
bucket = aws_connection.get_bucket(settings.
AWS_STORAGE_BUCKET_NAME)
aws_connection = S3Connection(settings.AWS_ACCESS_KEY_ID,
settings.AWS_SECRET_ACCESS_KEY)
bucket.delete_key(instance.val)
Just replace the original function in the virtual environment arches
models.py file with this one, and you should be good to go. No change to
the ascat package necessary (comment out the function you have in there to
make sure that this one is used). If you can test the two different
alternate deletion methods, we'll have a good idea of what to use, because
we should get this permanently into the arches repository. As you notice,
the original code is still used if it causes no errors.
Cheers and thanks for posting!
Adam
On Wednesday, December 30, 2015 at 5:18:44 PM UTC-6, Stephan Gmur wrote:
>
> Here is an update on a work around that we have found to work for this
> problem but has not proven to be a long term solution.
>
> We have traced the error to the auto_delete_file_on_delete function within
> the arches models.py file. There are two event listeners that are called by
> django when a media file is either deleted or updated. The function uses
> the os library to delete files but since media is residing on an S3 bucket,
> it can't access the file and therefor hangs. The error that was being
> produced stated that absolute paths were not supported.
>
> The work around was to comment out this function within the arches
> application, move it to the ascat models.py file. The function was updated
> to use the boto library to delete media from the S3 bucket and functions
> properly. Below is the new code used within the auto_delete_file_on_delete
> function:
>
>
> @receiver(models.signals.post_delete)
> def auto_delete_file_on_delete(sender, instance, **kwargs):
> """Deletes file from filesystem
> when corresponding `Files` object is deleted.
> """
> aws_connection =
> S3Connection(settings.AWS_ACCESS_KEY_ID,settings.AWS_SECRET_ACCESS_KEY)
> bucket = aws_connection.get_bucket(settings.AWS_STORAGE_BUCKET_NAME)
> if instance.val:
> bucket.delete_key(instance.val)
>
>
> Where we are currently having troubles is having two functions with the
> same receiver action (models.signals.post_delete) living in the two
> applications: arches and ascat. It was suggested that we explore using the
> Signal.disconnect(receiver=None,
> sender=None, dispatch_uid=None) (
> https://docs.djangoproject.com/en/1.9/topics/signals/#django.dispatch.Signal.disconnect)
>
> to disconnect the arches receiver and reconnect django to the
> ascat receiver. The documentation on using this method primarily uses
> examples where the disconnect is a function within a class but this is not
> the case within the arches application. I am having troubles working with
> the methods to disconnect the receiver living in the arches application
> from the ascat application.
>
> Please let me know if you have any thoughts or might need
> additional clarification on some of the details I have provided.
>
> Stephan
>
> On Tuesday, November 17, 2015 at 1:30:28 PM UTC-8, Adam Cox wrote:
>>
>> Hi Andy, I'm not sure what's going on, because the behavior definitely
>> should have changed if you modified that models.py file. Maybe it would
>> be worth throwing a print statement in that function and then running the
>> development server just to make sure that the correct functions are being
>> called? You could test to see if the try/except blocks are working as we
>> want them to. Sorry, not a real answer, I know, but get us on the right
>> track.
>>
>> Adam
>>
>> On Mon, Nov 16, 2015 at 3:18 PM, Andy Graham <[email protected]> wrote:
>>
>>> Still no luck. I made the changes reinstalled and restarted apache and
>>> end up with the exact same error message.
>>>
>>>
>>> On Wednesday, November 4, 2015 at 11:39:32 AM UTC-8, Andy Graham wrote:
>>>>
>>>> Currently still using the Django development server and we work through
>>>> some issues, but will try it out on apache soon and let you know. Thanks.
>>>>
>>>> On Wednesday, November 4, 2015 at 6:24:12 AM UTC-8, Adam Cox wrote:
>>>>>
>>>>> Did you restart apache?
>>>>>
>>>>> On Tuesday, November 3, 2015 at 10:29:29 AM UTC-6, Andy Graham wrote:
>>>>>>
>>>>>> Posted a response earlier but it appears to have been deleted, not
>>>>>> sure why.
>>>>>>
>>>>>> Thanks much for the suggestion Adam, however, I am still having the
>>>>>> same issue. I have made the changes below but when I try and delete the
>>>>>> media nothing happens. The error I am getting via the developer tools
>>>>>> is
>>>>>> below:
>>>>>>
>>>>>> [email protected]:[email protected]:8152
>>>>>> [email protected]:12
>>>>>> [email protected]:[email protected]:4095
>>>>>>
>>>>>> The first issue above (k.cors.a.crossDomain.send) seems to be
>>>>>> erroring out on the below line in the jquery.js file:
>>>>>>
>>>>>> try {
>>>>>> // Do send the request (this may raise an exception)
>>>>>> xhr.send( options.hasContent && options.data || null );
>>>>>>
>>>>>>
>>>>>> Any suggestions are appreciated, thanks.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>> On Thursday, October 22, 2015 at 1:01:18 PM UTC-7, Adam Cox wrote:
>>>>>>>
>>>>>>> Thanks for posting on the forum about this... I added the S3 bucket
>>>>>>> info to the documentation, but didn't think to test deleting files from
>>>>>>> it. After trying it out a little bit ago, I got the same result as you
>>>>>>> did. What is happening is that the db resource is deleted, but Django
>>>>>>> hits
>>>>>>> an error while trying to delete the actual file on S3 (that's why the
>>>>>>> redirect didn't work). You'll see that the files you were trying to
>>>>>>> delete
>>>>>>> still exist in your bucket.
>>>>>>>
>>>>>>> I was able to fix it up, drawing mainly from this question
>>>>>>> <http://www.google.com/url?q=http%3A%2F%2Fstackoverflow.com%2Fquestions%2F5372934%2Fhow-do-i-get-django-admin-to-delete-files-when-i-remove-an-object-from-the-datab&sa=D&sntz=1&usg=AFQjCNFObrQVe4pUUGw_FqqUANaWBc269Q>
>>>>>>> (see
>>>>>>> the second answer, and its comments). It will be easy for you to do in
>>>>>>> this one case, but will require some more info in the documentation,
>>>>>>> and
>>>>>>> ultimately a modification in the arches code to accommodate S3 storage.
>>>>>>>
>>>>>>> For now, *carefully* head into your virtual environment, and open
>>>>>>> up this file:
>>>>>>> ENV/lib/python2.7/site-packages/arches/app/models/models.py.
>>>>>>> Be sure to back it up before you do! Around line 433 is a function
>>>>>>> called auto_delete_file_on_delete(). In that function, you'll see a
>>>>>>> line
>>>>>>> that says "if os.path.isfile(instance.val.path):" That line is causing
>>>>>>> a
>>>>>>> server error, because instance.val doesn't have a path property when
>>>>>>> you
>>>>>>> use S3 storage (it has a url property).
>>>>>>> To make the fix, replace the contents of the function with this
>>>>>>> (note that the same first line is used):
>>>>>>>
>>>>>>> if instance.val:
>>>>>>> try:
>>>>>>> if os.path.isfile(instance.val.path):
>>>>>>> os.remove(instance.val.path)
>>>>>>> except:
>>>>>>> storage, name = instance.val.storage, instance.val.name
>>>>>>> storage.delete(name)
>>>>>>>
>>>>>>> Now it'll still try the old method, but when it comes across an
>>>>>>> error (a.k.a. "exception") it'll skip to the new code that is necessary
>>>>>>> for
>>>>>>> dealing with S3.
>>>>>>>
>>>>>>> This is really a hack, not a fix, but if you are careful about the
>>>>>>> indentation I think you'll be able to manage just fine. Also, if you
>>>>>>> are
>>>>>>> using a text editor like notepad++, be sure that you are using 4 spaces
>>>>>>> (not tab) to make the indentation (4 spaces per indent).
>>>>>>>
>>>>>>> Good luck and thanks again for posting this issue!
>>>>>>>
>>>>>>> On Wednesday, October 21, 2015 at 3:38:32 PM UTC-5, Andy Graham
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello All,
>>>>>>>> A question about an error I am getting trying to delete Media
>>>>>>>> resources on S3. When I delete the resource via the Resource Manager
>>>>>>>> (Delete Resource in the left side tool bar) I get the message that
>>>>>>>> states
>>>>>>>> "You won't be able to undo this operation!", then I click on the
>>>>>>>> Delete
>>>>>>>> button and nothing happens, however, if I go back to the Map or Search
>>>>>>>> the
>>>>>>>> resource has been deleted.
>>>>>>>>
>>>>>>>> Using the developer tools for the browser I have traced the error
>>>>>>>> back to the delete-resource.js in the arches/app/media/js/views/forms
>>>>>>>> folder.
>>>>>>>>
>>>>>>>> Specifically:
>>>>>>>>
>>>>>>>> deleteResource: function() {
>>>>>>>> $.ajax({
>>>>>>>> method: 'DELETE',
>>>>>>>> url: '',
>>>>>>>> success: function() {
>>>>>>>> location.href = arches.urls.home;
>>>>>>>> }
>>>>>>>> });
>>>>>>>>
>>>>>>>> The error seems to be taking place at the url:, which isn't
>>>>>>>> populated. Does anyone have suggestions for the proper way to
>>>>>>>> reference
>>>>>>>> the S3 bucket here, and if there are any specific permissions that
>>>>>>>> need to
>>>>>>>> be set to allow deleting files from S3? Any help is appreciated,
>>>>>>>> thanks.
>>>>>>>>
>>>>>>>> Andy
>>>>>>>>
>>>>>>> --
>>> -- To post, send email to [email protected]. To
>>> unsubscribe, send email to [email protected].
>>> For more information, visit
>>> https://groups.google.com/d/forum/archesproject?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Arches Project" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
--
-- To post, send email to [email protected]. To unsubscribe, send
email to [email protected]. For more information,
visit https://groups.google.com/d/forum/archesproject?hl=en
---
You received this message because you are subscribed to the Google Groups
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
