[galaxy-dev] disk space warning
Hi, is there a way to warn users when the disk becomes full during a URL-based file upload (and local file upload)? I had to manually delete the temp files created during a large upload that hanged, after which Galaxy could not be re-started due to insufficient disk space. Thanks. -- Mutlu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] get wsgi headers
On 30 August 2012 14:23, Mutlu Dogruel mutludogr...@gmail.com wrote: On 29 August 2012 21:11, Nate Coraor n...@bx.psu.edu wrote: On Aug 24, 2012, at 9:48 AM, Mutlu Dogruel wrote: Hi folk, Do you know any method for reading (the extended) WSGI HTTP headers from a custom tool? If I put some debugging lines in $galaxy/lib/galaxy/web/framework/middleware/xforwardedhost.py, I can read the HTTP headers I want. But I need to access them under tools/my_script.py. Is there a way to reach the webapps http headers from there? Hi Mutlu, Do you mean the headers at the time of tool execution? Running tools (jobs) is intentionally decoupled from the state of the front end. However, you could modify the tool submission to store the header as a tool parameter and pass it in the command template. --nate Thanks. Mutlu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Thanks for the reply. Yes, this is what I am trying to achieve. But in my case, I need to read the headers before the tool is run, and to be more precise, just before dynamic_options creates a selection list. Because it's the dynamic_options script that needs the http header values. Essentially, I need to get some AWS keys from the http headers. We inject AWS keys and secrets into http headers on a proxy server that Galaxy is sitting behind. I have a customised upload.xml and upload.py for managing downloads from S3 buckets (I kept the original upload scripts as well). In this scenario, a list of available, permitted S3 files are generated by dynamics_options, according to the AWS keys. The user-selected file is later downloaded from Amazon using its URL. This is handled by Galaxy's native URL downloader. It works fine if I read the AWS keys from a local file, but of course, the keys do not have to be static. I just started to look into lib/galaxy/tools/parameters/dynamic_options.py, can this be a good starting point? Thanks, Best regards, -- Mutlu Hi, for those interested, this is how I solved this problem: I modified web/framework/__init__.py for setting the extra http headers we wanted. I added new objects to the app.py for holding the http headers sent by the proxy. Otherwise, it wasn't possible to retrieve them from os.environ at this depth of the code. Then I modified tools/parameters/basic.py for injecting these keys into the self.dynamic_options string variable that holds the user function's name. -- Mutlu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] get wsgi headers
On 29 August 2012 21:11, Nate Coraor n...@bx.psu.edu wrote: On Aug 24, 2012, at 9:48 AM, Mutlu Dogruel wrote: Hi folk, Do you know any method for reading (the extended) WSGI HTTP headers from a custom tool? If I put some debugging lines in $galaxy/lib/galaxy/web/framework/middleware/xforwardedhost.py, I can read the HTTP headers I want. But I need to access them under tools/my_script.py. Is there a way to reach the webapps http headers from there? Hi Mutlu, Do you mean the headers at the time of tool execution? Running tools (jobs) is intentionally decoupled from the state of the front end. However, you could modify the tool submission to store the header as a tool parameter and pass it in the command template. --nate Thanks. Mutlu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Thanks for the reply. Yes, this is what I am trying to achieve. But in my case, I need to read the headers before the tool is run, and to be more precise, just before dynamic_options creates a selection list. Because it's the dynamic_options script that needs the http header values. Essentially, I need to get some AWS keys from the http headers. We inject AWS keys and secrets into http headers on a proxy server that Galaxy is sitting behind. I have a customised upload.xml and upload.py for managing downloads from S3 buckets (I kept the original upload scripts as well). In this scenario, a list of available, permitted S3 files are generated by dynamics_options, according to the AWS keys. The user-selected file is later downloaded from Amazon using its URL. This is handled by Galaxy's native URL downloader. It works fine if I read the AWS keys from a local file, but of course, the keys do not have to be static. I just started to look into lib/galaxy/tools/parameters/dynamic_options.py, can this be a good starting point? Thanks, Best regards, -- Mutlu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Possible bug in dynamic options
Thanks for the reply but no, I wasn't using any filters. The relevant XML sections are like this: code file=listFiles.py/ ... param name = url_paste label = Select a file to copy type=select multiple=false dynamic_options = listFiles()/ Mutlu On 24 August 2012 17:28, John Chilton chil...@msi.umn.edu wrote: I cannot speak to the code running twi, but are using a unique_value filter? The dynamic_options code use to completely lock up our galaxy server for minutes at a time before I applied this optimization: https://bitbucket.org/jmchilton/galaxy-central-dynamic-options-fix/changeset/e4856cc362d92f7857699c5be6cab1a75c29932f It is only relevant if you are using a unique_value filter though. -John On Fri, Aug 24, 2012 at 11:07 AM, Mutlu Dogruel mutludogr...@gmail.com wrote: Hi, I wrote a script for dynamically populating a user form. It works but it's slow. When I checked the Galaxy log, I saw that the method called by my custom tool's xml is being run twice (the same debugging lines appear twice in the log, and that happens consistently). Is there any explanation for this? The script returns a list of strings fetched from a remote server. It's running a bit slow, taking up to 6 seconds, so it would be great if I can half that! For testing purposes, I wrote a dummy method that returns a few characters in a list of tuples. Again, my print statement in the Python script is called twice. Has anyone noticed such behaviour? Could it be related to this Galaxy bug: https://bitbucket.org/galaxy/galaxy-central/issue/491/data_column-parameter-bug-cannot-select The Galaxy script shown on that page says: NOTE: This is wasteful since dynamic options and dataset collection happens twice (here and when generating HTML). Thanks. Best regards, -- Mutlu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Mutlu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
