Re: [galaxy-dev] recovery: galaxy restart
Hi Nate, I have used the local.py logic for submission of my jobs, for a grid based architecture. But local.py does not have the recover() function so I was trying to use the concept of recover() from other modules. My job submission also uses the drmaa api's, so I thought of using the drmaa.py module. Moreover the local.py doesn't have the monitor() or check_watched_items() functions which means it doesn't use the concept of monitor_queue. So how can I create a recover() function in such a case. Can you suggest anything regarding this issue. Thanks Harendra On Wed, Jun 1, 2011 at 10:52 PM, Nate Coraor n...@bx.psu.edu wrote: Harendra chawla wrote: Hi Nate, Ya I have seen the function __check_jobs_at_startup(), it calls the recover function after checking the state of the job from the database and updating the jobWrapper. But what it does after calling the recover function and one more thing, does it use the monitor() and the check_watched_item() in the drmaa.py. Yes, notice that the last function of the recover() method is to insert the job state object into the DRM monitor queue with: self.monitor_queue.put( drm_job_state ) Which is the same as the final step of the submission process, the queue_job() method. Once this happens, the monitor thread will pick up the job state from monitor_queue (a Python Queue instance) and monitor it with monitor()/check_watched_items(). --nate Thanks Harendra On Wed, Jun 1, 2011 at 9:10 PM, Nate Coraor n...@bx.psu.edu wrote: Harendra chawla wrote: Hi Nate, I got your point but which part of the code is doing all these things, I mean how exactly this is done. Is it using any other function apart from recover? Yes, see __check_jobs_at_startup() in lib/galaxy/jobs/__init__.py --nate Regards Harendra On Wed, Jun 1, 2011 at 8:56 AM, Nate Coraor n...@bx.psu.edu wrote: Harendra chawla wrote: Hi everyone, I am trying to modify the *recover* function from the drmaa.py (/galaxy_central/lib/galaxy/job/runners/drmaa.py) as per my requirements. But I am not ale to understand the flow of that function. The recover function is called when the galaxy server is restarted. It first looks for the running jobs from the database. Then my problem is how it regains the same old state of the galaxy (specially the GUI) which was before the galaxy got restarted. Can anyone explain me the flow of the recover function and how the old state is regained. Hi Harendra, I'm not sure I understand what you mean by old state and the GUI - all that's really necessary here is to determine what Galaxy considers to be the state of the job (new, queued, running), recreate the in-memory job components (the JobWrapper), and place the job back in Galaxy's DRM-monitoring queue, which will then proceed with the process of finishing the job if it's finished in the DRM or waiting for it to finish if it's still queued or running in the DRM. --nate Regards Harendra ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Fwd: Request for pulling the updated FASTQ (de-)interlacer tool
Hi Kanwei, I think we'll get more feedback once it when it is added to Galaxy. I have run small tests (included in the functional tests) and real large datasets and have had no issues with the scripts and wrappers. Regards, Florent On 27/05/11 18:49, Kanwei Li wrote: Has anyone on the list tried this patch? Feedback would be appreciated. Thanks, K On Fri, May 27, 2011 at 1:27 AM, Florent Angly florent.an...@gmail.com mailto:florent.an...@gmail.com wrote: Can anyone comment on my email please? Florent On 22/05/11 14:03, Florent Angly wrote: Let me know if you have any questions about this. Thanks, Florent Original Message Subject:Request for pulling the updated FASTQ (de-)interlacer tool Date: Tue, 17 May 2011 17:47:13 +1000 From: Florent Angly florent.an...@gmail.com mailto:florent.an...@gmail.com To: galaxy-dev@lists.bx.psu.edu mailto:galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu mailto:galaxy-dev@lists.bx.psu.edu Dear Galaxy team, Please consider pulling these changes I made to the interlacer/de-interlacer tools into the main Galaxy repository: https://bitbucket.org/fangly/galaxy-central/changeset/7f7ec8f05d36 While the tools were handling mate pairs fine, I now added the ability to also keep single reads (reads that miss their mate) in a separate file. The functional tests were updated and pass. Thanks, Florent ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Blast2GO in Galaxy
On Tue, May 31, 2011 at 11:26 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, As I mentioned on Twitter, at the end of last week I wrapped Blast2GO for Galaxy, using the b2g4pipe program (Blast2GO for pipelines). See http://blast2go.org/ Currently current code is on bitbucket under my tools branch, https://bitbucket.org/peterjc/galaxy-central/src/tools/ Specifically files tools/ncbi_blast_plus/blast2go.* viewable here: https://bitbucket.org/peterjc/galaxy-central/src/tools/tools/ncbi_blast_plus/ I've using a Galaxy location file, tool-data/blast2go.loc, to offer one or more Blast2GO configurations (properties files), mapping this to the -prop argument. This way you could have for example the Spanish Blast2GO server with its current database (May 2010), and a local Blast2GO database. I want to setup a local database and try this before submitting the wrapper to the Tool Shed. I've done that using the current latest data from May 2011, so one year newer than the current default public Blast2GO database provided by the Blast2GO developers (dated May 2011). This requires downloaded some large data files and importing them into a MySQL database, and that took about 24 hours to process in all. The last step is done via their Java tool and took most of the time. For more details, see: http://blast2go.org/localgodb However, the end result is worth the effort as running Blast2GO is now much much faster. I need to try it on some larger files, but we're talking at least an order of magnitude quicker, maybe two. The input to the tool is a BLAST XML file, specifically blasting against a protein database like NR (so blastp or blastx, not blastn etc). I want to try some very large BLAST XML files to confirm b2g4pipe copes with the current BLAST+ output files - I gather there were some problems in this area in the past, so having the wrapper script fragment the XML might be a workaround. Currently the only real function of the wrapper script is to rename the output file - b2g4pipe insists on using the *.annot extension. Now that I have a local Blast2GO database, I'll be able to try out b2g4pipe on some bigger XML files (without having to wait ages). Right now the only output is a tabular three column *.annot file, which can be loaded into the Blast2GO GUI. For analysis within Galaxy, I'm wondering about an option to split the first column (which holds the original FASTA query's identifier and any description) in two. i.e. Split at the first white space to give the FASTA identifier, and any optional description as a separate column. That would make linking/joining/filtering on the ID much easier. If anyone has any comments or feedback now, that would be welcome. I've submitted the initial version to the Galaxy Tool Shed now, but would still welcome feedback and would even consider non-backwards compatible changes in the short term. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Updates to Galaxy Community Conference website
Hi all, I have a couple of minor suggestions for improving the website http://galaxy.psu.edu/gcc2011/Home.html First, please fill in a meaningful HTML titles since that will be used as the default caption in a bookmark, tab or window title. Second, please add a link to the presentation slides and videos, e.g. from the programme page and main page? http://wiki.g2.bx.psu.edu/GCC2011 I realise this isn't complete yet, but it is already a useful resource. Also once you have all the slides, could the links in the wiki table be made to point straight to the file? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] sqlite to postgres
Hi Harendra I am not aware of anybody who made this kind of transition without running into all kind of problems. I suggest the following workaround: - set up a new, independent Galaxy server using PostgreSQL. - transfer the histories you want to keep to the new server using the 'Export to File' option on the old server followed by the 'Import to File' option on the new server. Regards, Hans On 06/02/2011 12:01 PM, Harendra chawla wrote: Hi, I want to convert my database from sqlite to postgres. I am able to do it as far as schema is concerned, but I want to preserve the data as well. When I changed the database to postgres, all my history got lost. Is there any way I can preserve the data in sqlite database while converting to postgres. Regards Harendra ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] problems previewing certain files, downloading files and with login
Matthew Conte wrote: Hi Nate, I was able to finally track down the login issue. It had to do with the following setting in my universe_wsgi.ini: * * *cookie_path = /galaxy* Removing this out fixed the problem and I should be fine leaving it out out since I don't need to run more than one instance of Galaxy. I probably shouldn't have had this setting in the first place, though I'm not exactly sure what this caused the problem. Anyways, thanks for all the help tracking these issues down. Hi Matt, I was about to get back to this now that the conference is over and I was also wondering if that option was related - weird though, I'm not sure why it should have an effect, but I'm glad you've figured out what's up. I guess there's a bug with the cookie path I'll need to check out. --nate -Matt On Mon, May 16, 2011 at 5:26 PM, Matthew Conte mco...@umd.edu wrote: Yep. On May 16, 2011 7:14 AM, Nate Coraor n...@bx.psu.edu wrote: Matthew Conte wrote: On Fri, May 13, 2011 at 11:18 AM, Nate Coraor n...@bx.psu.edu wrote: That's correct, although it should've been fixed. Can you remove the contents of galaxy-dist/database/compiled_templates, clear cache, reload, and see if you still get the same results? Thanks, --nate I've removed the contents of that folder, cleared the browser cache, reloaded and still get the same results (nothing different in the log either). I'm not sure if this will help, but maybe I should mention that when I try to log in I do get the following screen to flash for a brief second before automatically taking me back to the welcome screen: [image: Screen shot 2011-05-13 at 1.44.25 PM.png] I'm going to try to set up a test environment to replicate this shortly. Does this happen with both Apache and nginx? --nate ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] recovery: galaxy restart
Harendra chawla wrote: Hi Nate, I have used the local.py logic for submission of my jobs, for a grid based architecture. But local.py does not have the recover() function so I was trying to use the concept of recover() from other modules. My job submission also uses the drmaa api's, so I thought of using the drmaa.py module. Moreover the local.py doesn't have the monitor() or check_watched_items() functions which means it doesn't use the concept of monitor_queue. So how can I create a recover() function in such a case. Can you suggest anything regarding this issue. Hi Harendra, If you're submitting jobs to a cluster, I'd suggest using drmaa.py. You could make a copy of it and make any site changes necessary and load that as a new job runner module. Hacking cluster support on to the back of local.py sort of defeats the purpose of Galaxy's cluster integration and prevents stuff like exactly what it is you're trying to do. Thanks, --nate Thanks Harendra On Wed, Jun 1, 2011 at 10:52 PM, Nate Coraor n...@bx.psu.edu wrote: Harendra chawla wrote: Hi Nate, Ya I have seen the function __check_jobs_at_startup(), it calls the recover function after checking the state of the job from the database and updating the jobWrapper. But what it does after calling the recover function and one more thing, does it use the monitor() and the check_watched_item() in the drmaa.py. Yes, notice that the last function of the recover() method is to insert the job state object into the DRM monitor queue with: self.monitor_queue.put( drm_job_state ) Which is the same as the final step of the submission process, the queue_job() method. Once this happens, the monitor thread will pick up the job state from monitor_queue (a Python Queue instance) and monitor it with monitor()/check_watched_items(). --nate Thanks Harendra On Wed, Jun 1, 2011 at 9:10 PM, Nate Coraor n...@bx.psu.edu wrote: Harendra chawla wrote: Hi Nate, I got your point but which part of the code is doing all these things, I mean how exactly this is done. Is it using any other function apart from recover? Yes, see __check_jobs_at_startup() in lib/galaxy/jobs/__init__.py --nate Regards Harendra On Wed, Jun 1, 2011 at 8:56 AM, Nate Coraor n...@bx.psu.edu wrote: Harendra chawla wrote: Hi everyone, I am trying to modify the *recover* function from the drmaa.py (/galaxy_central/lib/galaxy/job/runners/drmaa.py) as per my requirements. But I am not ale to understand the flow of that function. The recover function is called when the galaxy server is restarted. It first looks for the running jobs from the database. Then my problem is how it regains the same old state of the galaxy (specially the GUI) which was before the galaxy got restarted. Can anyone explain me the flow of the recover function and how the old state is regained. Hi Harendra, I'm not sure I understand what you mean by old state and the GUI - all that's really necessary here is to determine what Galaxy considers to be the state of the job (new, queued, running), recreate the in-memory job components (the JobWrapper), and place the job back in Galaxy's DRM-monitoring queue, which will then proceed with the process of finishing the job if it's finished in the DRM or waiting for it to finish if it's still queued or running in the DRM. --nate Regards Harendra ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Sharing tool definitions between XML files
On Wed, Oct 6, 2010 at 11:16 AM, Peter pe...@maubp.freeserve.co.uk wrote: On Tue, Sep 28, 2010 at 12:04 PM, Peter pe...@maubp.freeserve.co.uk wrote: Hi all, I'm wondering if there is any established way to share parameter definitions between tool wrapper XML files? For example, I am currently working on NCBI BLAST+ wrappers, and these tools have a lot in common. For example, the output format option will be a select parameter, and it will be the same for blastn, blastp, tblastn, etc. I can just cut and paste the definition but this seems inelegant and will be a long term maintenance burden if it ever needs updating (lots of places would need to be updated the same way). I'd like to have a shared XML snippet defining this parameter which I could then reference/include from each tool wrapper than needs it. I'm thinking of something like XML's !ENTITY ... might work. Is this possible? Peter Alex Bossers replied on the BLAST+ thread to express interest in sharing definitions between wrapper XML files: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-September/003376.html As I add more parameters to my BLAST+ wrappers, keeping all the files in sync is getting a bit tedious... Peter Hi all, I'm replying to this old thread after finding this old (fixed) issue on the bug tracker, https://bitbucket.org/galaxy/galaxy-central/issue/131/allow-xml-includes-in-all-xml-files Does this offer a solution to sharing definitions in tool XML files? Has anyone got an example using this? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error when adding datasets
Louise-Amélie Schmitt wrote: Hello everyone I have an issue when trying to import new datasets or when putting a dataset into a history. I saw Edward Kirton had the same problem but he got no answer: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002732.html Here is the error message I get when clicking the Add datasets button in a library, in the admin's Manage data libraries panel: UnmappedInstanceError: Class '__builtin__.NoneType' is not mapped URL: http://manni/galaxy/library_common/upload_library_dataset?library_id=f2db41e1fa331b3eshow_deleted=Falsecntrller=library_adminfolder_id=f2db41e1fa331b3euse_panels=False File '/g/funcgen/galaxy/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/g/funcgen/galaxy/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/g/funcgen/galaxy/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/g/funcgen/galaxy/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/g/funcgen/galaxy/lib/galaxy/web/framework/middleware/remoteuser.py', line 109 in __call__ return self.app( environ, start_response ) File '/g/funcgen/galaxy/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/g/funcgen/galaxy/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/g/funcgen/galaxy/lib/galaxy/web/controllers/library_common.py', line 907 in upload_library_dataset trans.sa_session.refresh( history ) File '/g/funcgen/galaxy/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py', line 127 in do return getattr(self.registry(), name)(*args, **kwargs) File '/g/funcgen/galaxy/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py', line 925 in refresh raise exc.UnmappedInstanceError(instance) UnmappedInstanceError: Class '__builtin__.NoneType' is not mapped Now when does it occur: I have two databases. One test database I created a month ago which works fine, even now. The other one I created recently which is supposed to be the final database. But the latter keeps triggering the above message, even when I drop it and create it all over again. I even tried to create a third one, all clean and new but the problem remains. I tried to trash all the eggs so Galaxy gets fresh new eggs, with no effect at all. The error's still there. If you have any clue, I'll be forever grateful. Okay, the problem is that the exact process you're taking after recreating the database is resulting in your user not having a history. This is pretty rare, but does occasionally happen. I just committed a fix for it in 5616:1f2870dc3ab0 and 5617:83103cff8757, but you can work around it without these changes pretty easily by hitting the Analysis view before uploading to make sure you have a history. --nate Cheers, L-A ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error: unable to read Galaxy config
Lewis, Brian Andrew wrote: After following the instructions in the wiki for setting up scaling/load balancing, I was testing out submitting a job and got this error when trying to pull data from USCS Main: Traceback (most recent call last): File /usr/local/galaxy-dist/tools/data_source/data_source.py, line 6, in from galaxy.util import gzip_magic File /usr/local/galaxy-dist/lib/galaxy/util/__init__.py, line 20, in pkg_resources.require( 'docutils' ) File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 400, in require c = Crate() File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 259, in __init__ self.galaxy_config = GalaxyConfig() File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 358, in __init__ raise Exception( error: unable to read Galaxy config from %s % GalaxyConfig.config_file ) Exception: error: unable to read Galaxy config from /usr/local/galaxy-dist/0 I'm running Galaxy on a RHEL6 server and I have one web server and one runner set up using Apache as a proxy server. Hi Brian, Due to some hardcoding that shouldn't exist, you need to leave a copy of universe_wsgi.ini behind in addition to the two new configs. I'll work on a fix for this ASAP. --nate Thanks, Brian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Support for subdirs in dataset extra_files_path
Patch Imported in 5618:b9fdb88da530.
Thanks!
Dan
On Mar 16, 2011, at 11:50 AM, Jim Johnson wrote:
Request is issue#494
https://bitbucket.org/galaxy/galaxy-central/issue/494/support-sub-dirs-in-extra_files_path-patch
I'm finding that some qiime metagenomics applications build HTML results with
an inherent directory structure. For some other applications, e.g. FastQC,
I've been able to flatten the hierarchy and edit the html, but that appears
problematic for qiime.
Galaxy hasn't supported a dataset extra_files_path hierarchy, though the
developers don't seem opposed to the idea:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-October/003605.html
I added a route in lib/galaxy/web/buildapp.pyand modified the dataset
download code to traverse a hierarchy in lib/galaxy/web/controllers/dataset.py
I don't think these add any security vulnerabilities, (I tried the obvious
../../ ).
$ hg diff lib/galaxy/web/buildapp.py
diff -r 6ae06d89fec7 lib/galaxy/web/buildapp.py
--- a/lib/galaxy/web/buildapp.pyWed Mar 16 09:01:57 2011 -0400
+++ b/lib/galaxy/web/buildapp.pyWed Mar 16 10:24:13 2011 -0500
@@ -94,6 +94,8 @@
webapp.add_route( '/async/:tool_id/:data_id/:data_secret',
controller='async', action='index', tool_id=None, data_id=None,
data_secret=None )
webapp.add_route( '/:controller/:action', action='index' )
webapp.add_route( '/:action', controller='root', action='index' )
+# allow for subdirectories in extra_files_path
+webapp.add_route( '/datasets/:dataset_id/display/{filename:.+?}',
controller='dataset', action='display', dataset_id=None, filename=None)
webapp.add_route( '/datasets/:dataset_id/:action/:filename',
controller='dataset', action='index', dataset_id=None, filename=None)
webapp.add_route(
'/display_application/:dataset_id/:app_name/:link_name/:user_id/:app_action/:action_param',
controller='dataset', action='display_application', dataset_id=None,
user_id=None, app_name = None, link_name = None, app_action = None,
action_param = None )
webapp.add_route( '/u/:username/d/:slug', controller='dataset',
action='display_by_username_and_slug' )
$
$ hg diff lib/galaxy/web/controllers/dataset.py
diff -r 6ae06d89fec7 lib/galaxy/web/controllers/dataset.py
--- a/lib/galaxy/web/controllers/dataset.py Wed Mar 16 09:01:57 2011 -0400
+++ b/lib/galaxy/web/controllers/dataset.py Wed Mar 16 10:24:29 2011 -0500
@@ -266,17 +266,18 @@
log.exception( Unable to add composite parent %s to
temporary library download archive % data.file_name)
msg = Unable to create archive for download, please
report this error
messagetype = 'error'
-flist = glob.glob(os.path.join(efp,'*.*')) # glob returns
full paths
-for fpath in flist:
-efp,fname = os.path.split(fpath)
-try:
-archive.add( fpath,fname )
-except IOError:
-error = True
-log.exception( Unable to add %s to temporary
library download archive % fname)
-msg = Unable to create archive for download, please
report this error
-messagetype = 'error'
-continue
+for root, dirs, files in os.walk(efp):
+for fname in files:
+fpath = os.path.join(root,fname)
+rpath = os.path.relpath(fpath,efp)
+try:
+archive.add( fpath,rpath )
+except IOError:
+error = True
+log.exception( Unable to add %s to temporary
library download archive % rpath)
+msg = Unable to create archive for download,
please report this error
+messagetype = 'error'
+continue
if not error:
if params.do_action == 'zip':
archive.close()
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Migration error: fields in MySQL
John Eppley wrote:
I had an error upgrading my galaxy instance. I got the following exception
while migrating the db (during step 64-65):
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1064, You have an error
in your SQL syntax; check the manual that corresponds to your MySQL server
version for the right syntax to use near 'fields FROM form_definition' at
line 1) u'SELECT id, fields FROM form_definition' []
It seems my version (4.1.22-log) of MySQL did not like 'fields' as a column
name. If I alias the formdefinition as f and us f.fields, the error goes
away. I also had to modify migration 76 for the same reason.
Hi John,
Thanks for the patch, I've committed it as 5619:b6689fb6532e.
--nate
Here is my diff of the migrations dir:
diff -r 50e249442c5a
lib/galaxy/model/migrate/versions/0065_add_name_to_form_fields_and_values.py
---
a/lib/galaxy/model/migrate/versions/0065_add_name_to_form_fields_and_values.py
Thu Apr 07 08:39:07 2011 -0400
+++
b/lib/galaxy/model/migrate/versions/0065_add_name_to_form_fields_and_values.py
Fri Apr 15 11:09:26 2011 -0400
@@ -39,7 +39,7 @@
return ''
# Go through the entire table and add a 'name' attribute for each field
# in the list of fields for each form definition
-cmd = SELECT id, fields FROM form_definition
+cmd = SELECT f.id, f.fields FROM form_definition f
result = db_session.execute( cmd )
for row in result:
form_definition_id = row[0]
@@ -53,7 +53,7 @@
field[ 'helptext' ] = field[ 'helptext' ].replace(',
'').replace('', )
field[ 'label' ] = field[ 'label' ].replace(', '')
fields_json = to_json_string( fields_list )
-cmd = UPDATE form_definition SET fields='%s' WHERE id=%i %(
fields_json, form_definition_id )
+cmd = UPDATE form_definition f SET f.fields='%s' WHERE f.id=%i
%( fields_json, form_definition_id )
db_session.execute( cmd )
# replace the values list in the content field of the form_values table
with a name:value dict
cmd = SELECT form_values.id, form_values.content,
form_definition.fields \
@@ -112,7 +112,7 @@
cmd = UPDATE form_values SET content='%s' WHERE id=%i %(
to_json_string( values_list ), form_values_id )
db_session.execute( cmd )
# remove name attribute from the field column of the form_definition
table
-cmd = SELECT id, fields FROM form_definition
+cmd = SELECT f.id, f.fields FROM form_definition f
result = db_session.execute( cmd )
for row in result:
form_definition_id = row[0]
@@ -124,5 +124,5 @@
for index, field in enumerate( fields_list ):
if field.has_key( 'name' ):
del field[ 'name' ]
-cmd = UPDATE form_definition SET fields='%s' WHERE id=%i %(
to_json_string( fields_list ), form_definition_id )
+cmd = UPDATE form_definition f SET f.fields='%s' WHERE id=%i
%( to_json_string( fields_list ), form_definition_id )
db_session.execute( cmd )
diff -r 50e249442c5a
lib/galaxy/model/migrate/versions/0076_fix_form_values_data_corruption.py
---
a/lib/galaxy/model/migrate/versions/0076_fix_form_values_data_corruption.py
Thu Apr 07 08:39:07 2011 -0400
+++
b/lib/galaxy/model/migrate/versions/0076_fix_form_values_data_corruption.py
Fri Apr 15 11:09:26 2011 -0400
@@ -32,7 +32,7 @@
def upgrade():
print __doc__
metadata.reflect()
-cmd = SELECT form_values.id as id, form_values.content as field_values,
form_definition.fields as fields \
+cmd = SELECT form_values.id as id, form_values.content as field_values,
form_definition.fields as fdfields \
+ FROM form_definition, form_values \
+ WHERE form_values.form_definition_id=form_definition.id \
+ ORDER BY form_values.id
@@ -46,7 +46,7 @@
except Exception, e:
corrupted_rows = corrupted_rows + 1
# content field is corrupted
-fields_list = from_json_string( _sniffnfix_pg9_hex( str(
row['fields'] ) ) )
+fields_list = from_json_string( _sniffnfix_pg9_hex( str(
row['fdfields'] ) ) )
field_values_str = _sniffnfix_pg9_hex( str( row['field_values']
) )
try:
#Encoding errors? Just to be safe.
-j
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Error when adding datasets
On Thu, 2 Jun 2011 11:48:55 -0400, Nate Coraor n...@bx.psu.edu wrote: Greg Von Kuster wrote: Hello Louise, I've CC'd Nate on this as he may be able to help - although no guarantees. I'm not expert enough in this area to know where to look for the cause. Perhaps someone in the community can help as well. It is likely that LDAP is playing a role in this behavior. On Apr 14, 2011, at 1:00 PM, Louise-Amélie Schmitt wrote: The thing is, we use LDAP logging so we can't even access the website without logging in. Moreover, when I logged in, I arrived on the data analysis page where the automatic unnamed history was obviously created in the history panel. I forgot to mention we have issues creating and deleting histories, like, we can't access some histories and everytime we delete histories, two extra unnamed histories are created. As I mentioned before, it is also impossible to load a dataset in any history, existing or not. Okay, somehow I missed all the followup on this question when I committed a fix a few minutes ago. Could you provide some details about the inaccessible histories (what happens when you try to access them, traceback, etc?) and loading data - you mean the upload tool fails? Is there also a traceback here? ooh it's been a while I haven't seen the issue since I not-so-elegantly avoided the problem making our two instances run on separate machines. But I'll try to answer your question as accurately as I can. I experienced a similar error (pretty much the same traceback as the one I pasted in a previous message, just yelling at me saying that the history is nonexistent) whenever I tried to: - upload data in the manage libraries admin panel and in the analyse data panel (upload tool) - load an already uploaded dataset in any kind of history (saved or new) - probably do something else, I can't really remember I also had issues dealing with histories but not getting any traceback, like I mentioned a little lower: Everytime I tried to delete any number of histories, the histories were deleted but two were created. I hope this will help, sorry I can't really be more precise. L-A Thanks, --nate Do you think it could be linked to our using LDAP? Thanks L-A On Thu, 14 Apr 2011 12:06:55 -0400, Greg Von Kuster g...@bx.psu.edu wrote: On Apr 14, 2011, at 11:49 AM, Louise-Amélie Schmitt wrote: Here is the result I got from the debug statements: galaxy.web.controllers.library_common DEBUG 2011-04-14 17:46:02,286 ### history: None This is the problem - when you registered normally, a history would have been automatically created for you. But somehow you don't have a history - no idea why / how this happened, but it is very unlikely this is a result of a bug within Galaxy. Try logging out and logging in again and a new history should be created for you. galaxy.web.controllers.library_common DEBUG 2011-04-14 17:46:02,286 ### trans: galaxy.web.framework.GalaxyWebUITransaction object at 0x29f40950 galaxy.web.controllers.library_common DEBUG 2011-04-14 17:46:02,286 ### trans.sa_session: sqlalchemy.orm.scoping.ScopedSession object at 0x19ab21d0 Thanks again L-A Le jeudi 14 avril 2011 à 11:05 -0400, Greg Von Kuster a écrit : I assume when you dropped the old database and recreated the new, empty database, you made sure the database connection string in universe_wsgi.ini was correctly set. if so, when you started up Galaxy, it would have created all of the required tables in the new database, and they would all be empty. When you first registered as the admin user, it would have automatically populated several tables with data, including the history table. One or more of these things must not have been successful. To attempt to narrow down the problem, I'll need you to do the following. Here are lines # 905 - 907 in ~/lib/galaxy/web/controllers/library-common.py # Send the current history to the form to enable importing datasets from history to library history = trans.get_history() trans.sa_session.refresh( history ) Please add the following debug statements: # Send the current history to the form to enable importing datasets from history to library history = trans.get_history() log.debug(### history: %s % str( history )) log.debug(### trans: %s % str( trans )) log.debug(### trans.sa_session: %s % str( trans.sa_session )) trans.sa_session.refresh( history ) Stop and restart your Galaxy server after making the above changes, and reply back with the output of the debug statements. Assuming you start your Galaxy instance with: sh run.sh you'll see the results of the debug statements in the log scrolling in the window in which you started Galaxy. Thanks On Apr 14, 2011, at 10:46
[galaxy-dev] Defining new file formats in Galaxy (for new tool wrappers)
Hi all, Something I've not needed to do until now is define a new file format in Galaxy. I understand the basic principle and defining a subclass in Python... however, how does this work with new tools on the Tool Shed? In particular, if an output format is likely to be used by more than one tool, can we get it added to the Galaxy core? As an example, the basic functionality of the Blast2GO for pipelines tool (b2g4pipe) takes a BLAST XML input file, and gives a tab separated annotation output file. Galaxy already has 'blastxml' and 'tabular' file formats defined, so I didn't need to do anything extra. However, the tool can also take (a directory of) InterProScan XML files as input, so here a new 'interproscanxml' format would useful. Then any wrapper using or producing InterProScan XML could take advantage of this. e.g. Konrad's InterProScan wrapper could then offer the XML output as an option in addition to or instead of the tabular output. Related to this example, why isn't there a generic base class for XML formats in general? https://bitbucket.org/galaxy/galaxy-central/issue/568/missing-xml-datatype-base-class Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error: unable to read Galaxy config
Lewis, Brian Andrew wrote: Nate - I checked and I still have a copy of universe_wsgi.ini within my galaxy-dist directory. I did also make a change in /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py according to the post here: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002637.html Ah, this would probably be problematic - when tools/data_source/data_source.py runs, its argv[2] is not going to be the universe_wsgi.ini file. What happens if you change it back to the original? --nate ~ Brian -Original Message- From: Nate Coraor [mailto:n...@bx.psu.edu] Sent: Thursday, June 02, 2011 11:51 AM To: Lewis, Brian Andrew Cc: galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] Error: unable to read Galaxy config Lewis, Brian Andrew wrote: After following the instructions in the wiki for setting up scaling/load balancing, I was testing out submitting a job and got this error when trying to pull data from USCS Main: Traceback (most recent call last): File /usr/local/galaxy-dist/tools/data_source/data_source.py, line 6, in from galaxy.util import gzip_magic File /usr/local/galaxy-dist/lib/galaxy/util/__init__.py, line 20, in pkg_resources.require( 'docutils' ) File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 400, in require c = Crate() File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 259, in __init__ self.galaxy_config = GalaxyConfig() File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 358, in __init__ raise Exception( error: unable to read Galaxy config from %s % GalaxyConfig.config_file ) Exception: error: unable to read Galaxy config from /usr/local/galaxy-dist/0 I'm running Galaxy on a RHEL6 server and I have one web server and one runner set up using Apache as a proxy server. Hi Brian, Due to some hardcoding that shouldn't exist, you need to leave a copy of universe_wsgi.ini behind in addition to the two new configs. I'll work on a fix for this ASAP. --nate Thanks, Brian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error: unable to read Galaxy config
Okay, yeah, when I changed it back to the original code it worked. ~ Brian -Original Message- From: Nate Coraor [mailto:n...@bx.psu.edu] Sent: Thursday, June 02, 2011 12:47 PM To: Lewis, Brian Andrew Cc: galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] Error: unable to read Galaxy config Lewis, Brian Andrew wrote: Nate - I checked and I still have a copy of universe_wsgi.ini within my galaxy-dist directory. I did also make a change in /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py according to the post here: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002637.html Ah, this would probably be problematic - when tools/data_source/data_source.py runs, its argv[2] is not going to be the universe_wsgi.ini file. What happens if you change it back to the original? --nate ~ Brian -Original Message- From: Nate Coraor [mailto:n...@bx.psu.edu] Sent: Thursday, June 02, 2011 11:51 AM To: Lewis, Brian Andrew Cc: galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] Error: unable to read Galaxy config Lewis, Brian Andrew wrote: After following the instructions in the wiki for setting up scaling/load balancing, I was testing out submitting a job and got this error when trying to pull data from USCS Main: Traceback (most recent call last): File /usr/local/galaxy-dist/tools/data_source/data_source.py, line 6, in from galaxy.util import gzip_magic File /usr/local/galaxy-dist/lib/galaxy/util/__init__.py, line 20, in pkg_resources.require( 'docutils' ) File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 400, in require c = Crate() File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 259, in __init__ self.galaxy_config = GalaxyConfig() File /usr/local/galaxy-dist/lib/galaxy/eggs/__init__.py, line 358, in __init__ raise Exception( error: unable to read Galaxy config from %s % GalaxyConfig.config_file ) Exception: error: unable to read Galaxy config from /usr/local/galaxy-dist/0 I'm running Galaxy on a RHEL6 server and I have one web server and one runner set up using Apache as a proxy server. Hi Brian, Due to some hardcoding that shouldn't exist, you need to leave a copy of universe_wsgi.ini behind in addition to the two new configs. I'll work on a fix for this ASAP. --nate Thanks, Brian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Separate histories between localhost and 127.0.0.1
For some odd reason, when I try to import data using the URL http://localhost:8081 or http://localhost:8081/galaxy none of my jobs will even show up. However if I go to http://127.0.0.1 or http://127.0.0.1/galaxy the jobs run fine. Here's a snip from my httpd.conf file: RewriteEngine on RewriteRule ^/static/style/(.*) /usr/local/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/static/scripts/(.*) /usr/local/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/static/(.*) /usr/local/galaxy-dist/static/$1 [L] RewriteRule ^/favicon.ico /usr/local/galaxy-dist/static/favicon.ico [L] RewriteRule ^/robots.txt /usr/local/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8081$1 [P] The host lines are commented out in my universe_wsgi.ini files, so I'm guessing they should be set to the default localhost. ~ Brian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Defining new file formats in Galaxy (for new tool wrappers)
On Jun 2, 2011, at 1:29 PM, Nate Coraor wrote: Peter Cock wrote: Hi all, Something I've not needed to do until now is define a new file format in Galaxy. I understand the basic principle and defining a subclass in Python... however, how does this work with new tools on the Tool Shed? In particular, if an output format is likely to be used by more than one tool, can we get it added to the Galaxy core? I think people have provided the new subclass as a patch with the tool, but probably many of them, if well written, could be added to the core. As an example, the basic functionality of the Blast2GO for pipelines tool (b2g4pipe) takes a BLAST XML input file, and gives a tab separated annotation output file. Galaxy already has 'blastxml' and 'tabular' file formats defined, so I didn't need to do anything extra. However, the tool can also take (a directory of) InterProScan XML files as input, so here a new 'interproscanxml' format would useful. Then any wrapper using or producing InterProScan XML could take advantage of this. e.g. Konrad's InterProScan wrapper could then offer the XML output as an option in addition to or instead of the tabular output. We will certainly include support for new data formats into the Galaxy core. In case you haven't seen it, details for adding new formats is available in our wiki at https://bitbucket.org/galaxy/galaxy-central/wiki/AddingDatatypes. It's fairly straightforward. However, glancing at the wiki, it looks like there is no mention of functional tests for the new format. If we could get a patch that includes a functional test for uploading the format as new method(s) in ~/test/functional/test_get_data.py, it would be great. Related to this example, why isn't there a generic base class for XML formats in general? https://bitbucket.org/galaxy/galaxy-central/issue/568/missing-xml-datatype-base-class It just hadn't been necessary in the past and no one had the time to write it, I agree it could be helpful since there are other more specific XML types. Yes, XML formats have not yet been abstracted, and certainly can be. Just a matter of bandwidth... --nate Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team g...@bx.psu.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Problem with Torque/Maui
Hi Marco, Thanks for all of the details, they make a big difference when troubleshooting. Sorry for the delay in response. Can you ensure that you don't have any of the pbs_* options set in universe_wsgi.ini? I noticed that there are stagein/stageouts set on the job. You may also need to set $usecp in mom_priv/config on your execution hosts to prevent pbs_mom from trying to rcp/scp the error and output files back to galaxy1. $usecp instructs pbs_mom to consider /path on the execution host to be the same filesystem as on the submission host, e.g.: $usecp *:/mnt/equallogic1 /mnt/equallogic1 --nate Marco Moretto wrote: Hi all, first of all, thanks to the Galaxy Team for this really useful software. Actually I don't really know if my problem is related with Galaxy or with Torque/Maui but I didn't find any solution looking in both Torque and Maui user lists, so I hope that some of you with more experience could give me some good advices. I'm trying to set up Galaxy in a small local virtual environment in order to test it. I started with 2 virtual ubuntu servers called galaxy1 and galaxy2. On galaxy1 I succesfully installed Galaxy, Apache, Torque and Maui. I'm using Postgres ad DBMS. It is installed on another real DB server. The virtual server galaxy2 is used as node. Galaxy is working like a charm locally but when I try to use Torque problems arise. Torque alone works correctly. That means that I can submit a job with qsub and everything works. The 2 virtual server (galaxy1 and galaxy2) share a directory (through NFS) in which I installed Galaxy following the unified method from the documentation. Now, as I said, Galaxy alone works, Torque/Maui alone works. When I put the two together nothing works. As a test I upload (using local runner) a gff file. Then I try to make a filter to the gff using Filter and sort - Extract features. When I run this tool the corresponding job on the Torque queue runs forever in Hold state. I report some output from diagnose programs: The diagnose -j reports the following: Name State Par Proc QOS WCLimit R Min User Group Account QueuedTime Network Opsys ArchMem Disk Procs Class Features 29 Hold DEF1 DEF 1:00:00 01 galaxy galaxy-00:02:36 [NONE] [NONE] [NONE]=0=0NC0 [batch:1] [NONE] While the showq command reports ACTIVE JOBS JOBNAMEUSERNAME STATE PROC REMAINING STARTTIME 0 Active Jobs 0 of1 Processors Active (0.00%) IDLE JOBS-- JOBNAMEUSERNAME STATE PROC WCLIMIT QUEUETIME 0 Idle Jobs BLOCKED JOBS JOBNAMEUSERNAME STATE PROC WCLIMIT QUEUETIME 29 galaxy Hold 1 1:00:00 Wed May 4 03:56:40 The checkjob reports: checking job 29 State: Hold Creds: user:galaxy group:galaxy class:batch qos:DEFAULT WallTime: 00:00:00 of 1:00:00 SubmitTime: Wed May 4 03:56:40 (Time Queued Total: 00:03:07 Eligible: 00:00:01) The qstat -f reports Job Id: 33.galaxy1.research.intra.ismaa.it Job_Name = 27_extract_features1_marco.more...@iasma.it Job_Owner = gal...@galaxy1.research.intra.ismaa.it job_state = W queue = batch server = galaxy1.research.intra.ismaa.it ctime = Wed May 4 04:56:36 2011 Error_Path = galaxy1:/mnt/equallogic1/galaxy/galaxy-dist/database/pbs/27.e exec_host = galaxy2/0 exec_port = 15003 Execution_Time = Wed May 4 05:26:41 2011 mtime = Wed May 4 04:56:37 2011 Output_Path = galaxy1:/mnt/equallogic1/galaxy/galaxy-dist/database/pbs/27. o qtime = Wed May 4 04:56:36 2011 Resource_List.neednodes = 1 Resource_List.nodect = 1 Resource_List.nodes = 1 Resource_List.walltime = 01:00:00 stagein = /mnt/equallogic1/galaxy/tmp/dataset_18.dat@galaxy1 :/mnt/equallog ic1/galaxy/galaxy-dist/database/files/000/dataset_18.dat, /mnt/equallogic1/galaxy/tmp/dataset_30.dat@galaxy1 :/mnt/equallogic1/g alaxy/galaxy-dist/database/files/000/dataset_30.dat stageout = /mnt/equallogic1/galaxy/galaxy-dist/database/files/000/dataset_ 30.dat@galaxy1 :/mnt/equallogic1/galaxy/galaxy-dist/database/files/000/ dataset_30.dat substate = 37 Variable_List = PBS_O_QUEUE=batch, PBS_O_HOST=galaxy1.research.intra.ismaa.it euser = galaxy egroup = galaxy hashname = 33.galaxy1.research.intra.ismaa.it queue_rank = 33 queue_type = E StartDate: -00:03:06 Wed May 4 03:56:41 Total Tasks: 1 Req[0] TaskCount: 1 Partition: DEFAULT Network: [NONE] Memory = 0 Disk = 0 Swap = 0 Opsys: [NONE] Arch: [NONE] Features: [NONE] IWD: [NONE] Executable: [NONE] Bypass: 0 StartCount: 1 PartitionMask: [ALL] PE: 1.00 StartPriority: 1 cannot select job 29 for
Re: [galaxy-dev] suggestion for multithreading
(moved to galaxy-dev) Nate Coraor wrote, On 06/02/2011 01:31 PM: Peter Cock wrote: On Thu, Jun 2, 2011 at 6:23 PM, Nate Coraor n...@bx.psu.edu wrote: pbs.py then knows to translate 'resource type=cores8/resource' to '-l nodes=1:ppn=8'. Your tool can access that value a bunch, like $__resources__.cores. The same should be possible for other consumables. Just a thought here: The actual parameters that are passed to the scheduler are not necessarily hard-coded. Meaning, at least with SGE, specifying the number of cores can be: qsub -pe threads=8 or qsub -pe cores=8 or qsub -pe jiffies=8 and same thing for memory limitation (e.g. -l virtual_free=800M). The reason is that those resources (e.g. threads, cores, virtual_free) are just identifiers, and they are created and configured by whomever installed SGE - they are not built-in or hard-coded). So just be careful in your design/implementation when automatically translating XML resources to hard-coded parameters. If you do hard-code them, just make sure the specifically document it (i.e. Galaxy expect the SGE threads parameter to be -pe threads=8 and nothing else). -gordon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Appropriate place to ask questions about your NGLIMS for Galaxy
scott.cou...@monash.edu wrote: Hi, I'm a newbie when it comes to this kind of thing, so I was wondering whether you can tell me the most appropriate place to ask questions about making some modifications to your nglims extension for galaxy? Should they be posted to the normal galaxy development list? Should I ask you directly? Hi Scott, It seems like maybe you intended to send this to Brad directly, but regardless, posting nglims questions here is completely fine. I don't think he has a seperate mailing list for it, and we're planning to integrate it with the Galaxy trunk code anyway. --nate Cheers, Scott. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error when adding datasets
Louise-Amélie Schmitt wrote: On Thu, 2 Jun 2011 11:48:55 -0400, Nate Coraor n...@bx.psu.edu wrote: Greg Von Kuster wrote: Hello Louise, I've CC'd Nate on this as he may be able to help - although no guarantees. I'm not expert enough in this area to know where to look for the cause. Perhaps someone in the community can help as well. It is likely that LDAP is playing a role in this behavior. On Apr 14, 2011, at 1:00 PM, Louise-Amélie Schmitt wrote: The thing is, we use LDAP logging so we can't even access the website without logging in. Moreover, when I logged in, I arrived on the data analysis page where the automatic unnamed history was obviously created in the history panel. I forgot to mention we have issues creating and deleting histories, like, we can't access some histories and everytime we delete histories, two extra unnamed histories are created. As I mentioned before, it is also impossible to load a dataset in any history, existing or not. Okay, somehow I missed all the followup on this question when I committed a fix a few minutes ago. Could you provide some details about the inaccessible histories (what happens when you try to access them, traceback, etc?) and loading data - you mean the upload tool fails? Is there also a traceback here? ooh it's been a while I haven't seen the issue since I not-so-elegantly avoided the problem making our two instances run on separate machines. When these were running on the same machine, were they both served as seperate directories under the same hostname? If so, did you set cookie_path in the config? If not, your cookies for one instance were being used with another instance, with unpredictable results. Although from recent discussion here on -dev, it looks like there might be a bug with cookie_path. --nate But I'll try to answer your question as accurately as I can. I experienced a similar error (pretty much the same traceback as the one I pasted in a previous message, just yelling at me saying that the history is nonexistent) whenever I tried to: - upload data in the manage libraries admin panel and in the analyse data panel (upload tool) - load an already uploaded dataset in any kind of history (saved or new) - probably do something else, I can't really remember I also had issues dealing with histories but not getting any traceback, like I mentioned a little lower: Everytime I tried to delete any number of histories, the histories were deleted but two were created. I hope this will help, sorry I can't really be more precise. L-A Thanks, --nate Do you think it could be linked to our using LDAP? Thanks L-A On Thu, 14 Apr 2011 12:06:55 -0400, Greg Von Kuster g...@bx.psu.edu wrote: On Apr 14, 2011, at 11:49 AM, Louise-Amélie Schmitt wrote: Here is the result I got from the debug statements: galaxy.web.controllers.library_common DEBUG 2011-04-14 17:46:02,286 ### history: None This is the problem - when you registered normally, a history would have been automatically created for you. But somehow you don't have a history - no idea why / how this happened, but it is very unlikely this is a result of a bug within Galaxy. Try logging out and logging in again and a new history should be created for you. galaxy.web.controllers.library_common DEBUG 2011-04-14 17:46:02,286 ### trans: galaxy.web.framework.GalaxyWebUITransaction object at 0x29f40950 galaxy.web.controllers.library_common DEBUG 2011-04-14 17:46:02,286 ### trans.sa_session: sqlalchemy.orm.scoping.ScopedSession object at 0x19ab21d0 Thanks again L-A Le jeudi 14 avril 2011 à 11:05 -0400, Greg Von Kuster a écrit : I assume when you dropped the old database and recreated the new, empty database, you made sure the database connection string in universe_wsgi.ini was correctly set. if so, when you started up Galaxy, it would have created all of the required tables in the new database, and they would all be empty. When you first registered as the admin user, it would have automatically populated several tables with data, including the history table. One or more of these things must not have been successful. To attempt to narrow down the problem, I'll need you to do the following. Here are lines # 905 - 907 in ~/lib/galaxy/web/controllers/library-common.py # Send the current history to the form to enable importing datasets from history to library history = trans.get_history() trans.sa_session.refresh( history ) Please add the following debug statements: # Send the current history to the form to enable importing datasets from history to library history = trans.get_history() log.debug(### history: %s % str( history ))
Re: [galaxy-dev] Update of cufflinks
Just FYI, With the recent version of cufflinks (at least 1.0.2), the developers added a 'feature' of automatic version check. While the idea is nice, the implications are not: cufflinks,cuffcompare and cuffdiff will connect to host cufflinks.cbcb.umd.edu every time you run them. I'm sure the intentions were good, but I don't like programs calling home, basically reporting usage statistics and origins. The check_update code is also not written with security in mind, and might be exploited (buffer overruns and such) - it's best to disable it. This will also generate a message to STDERR, which is just annoying (and bad, with galaxy). To disable it on the command line, add --no-update-check to every invocation. If you compiled it from source code, edit the file ./src/common.cpp and change the line: bool no_update_check = false; to: bool no_update_check = true; Then recompile and re-install. -gordon Jeremy Goecks wrote, On 05/10/2011 09:02 AM: We recently updated the Cufflinks/compare/diff wrappers to be compatible with v1.0.* ; the wrappers are available in galaxy-central and should be available in galaxy-dist in the next couple weeks as we're planning another release soon. New features in v1.0.* have not been implemented in these updates (and, in fact, there are features from 0.9.* that are still not available); the changes we made were simply to ensure that the wrappers' current functionality works with the new versions. In the next couple months we plan to extend the current wrappers to include new functionality. However, community contributions that extend the current wrappers to include new functionality would be most welcome, and we can integrate them into the galaxy code base if/when they are available. Best, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Early registration for BOSC ends tomorrow, Friday June 3
If you haven't already registered for BOSC, now is your chance--after June 3, prices will go up! Registration for BOSC is through the ISMB main conference website: http://www.iscb.org/ismbeccb2011-registration#sigs . Since BOSC is a two-day SIG, the price is 2x the one-day SIG price listed on the ISMB website. You can register for BOSC without registering for the main ISMB conference, if you want. The preliminary BOSC schedule (subject to change) is now up at http://www.open-bio.org/wiki/BOSC_2011_Schedule (more details will be added soon). There is also a two day Codefest proceeding BOSC; please add yourself to the list of attendees if you are interested: http://www.open-bio.org/wiki/Codefest_2011 The BOSC talks have already been chosen, but we have spaces for last-minute posters. If you'd like your poster abstract to appear in the BOSC program, you should submit it now--see http://www.open-bio.org/wiki/BOSC_2011#Abstract_Submission_Information We hope to see you at BOSC! Nomi Harris Co-Chair, BOSC 2011 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] how can i get all the output file names in local.py file
shashi shekhar wrote: Hi All, I am using standalone system for galaxy and am doing some modification in galaxy .i need the all output file names in local.py .How can i get all the output file names in local.py file. From the job wrapper object in local.py, job_wrapper.get_output_fnames() --nate Regards shashi shekhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Separate histories between localhost and 127.0.0.1
Lewis, Brian Andrew wrote: For some odd reason, when I try to import data using the URL http://localhost:8081 or http://localhost:8081/galaxy none of my jobs will even show up. However if I go to http://127.0.0.1 or http://127.0.0.1/galaxy the jobs run fine. Here's a snip from my httpd.conf file: RewriteEngine on RewriteRule ^/static/style/(.*) /usr/local/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/static/scripts/(.*) /usr/local/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/static/(.*) /usr/local/galaxy-dist/static/$1 [L] RewriteRule ^/favicon.ico /usr/local/galaxy-dist/static/favicon.ico [L] RewriteRule ^/robots.txt /usr/local/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8081$1 [P] The host lines are commented out in my universe_wsgi.ini files, so I'm guessing they should be set to the default localhost. Hi Brian, Since localhost and 127.0.0.1 are considered different from the server name perspective, the cookies will be different. If you access http://localhost or http://localhost:8081 you'll share cookies. Likewise, if you access http://127.0.0.1 or http://127.0.0.1:8081, those cookies will be shared. Mixing localhost and 127.0.0.1 results in seperate cookies being issues for both. If you're logged in, you can view your history with the localhost URLs by clicking Saved histories from the Options menu. --nate ~ Brian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
