Re: [galaxy-dev] Upload via FTP - Assertion Error on Change ownership of working dictory
Thanks for the tip Eric. I was able to figure it out. The key to this,
was you suggested was permission issue.
I had to modify sudoers so tty was not required. This will allow
scripts to change ownership to the correct user so SGE can submit
them.
Thanks!
-Adam
--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences
System Administrator, HPC Cluster
Office of Information Technology
http://hpc.oit.uci.edu/
University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
On Mon, Feb 3, 2014 at 9:57 AM, Eric Rasche rasche.e...@yandex.ru wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Looks like a permission issue. Is your FTP server set to use the same
user/group as galaxy?
For reference: my proftpd is configured to use LDAP user information,
but then it forces the user/group id of uploaded files to be that of galaxy
# Force those numbers even if LDAP finds a valid UID/GID
LDAPDefaultUID {galaxy's UID}
LDAPDefaultGID {galaxy group GID}
Hope that helps
On 02/03/2014 11:23 AM, Adam Brenner wrote:
Anyone have any ideas on this?
Thanks,
-Adam
--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences
System Administrator, HPC Cluster
Office of Information Technology
http://hpc.oit.uci.edu/
University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
On Thu, Jan 30, 2014 at 2:33 PM, Adam Brenner aebre...@uci.edu wrote:
Howdy,
I am having trouble uploading data on our local instance of Galaxy
here. The method is via FTP and here is the error I am seeing in the
logs:
galaxy.jobs.runners DEBUG 2014-01-30 14:29:31,987 execution of
external set_meta for job 1367 finished
galaxy.jobs DEBUG 2014-01-30 14:29:32,016 (1367) Changing ownership of
working directory with: /usr/bin/sudo -E
scripts/external_chown_script.py
/som/galaxy/database/job_working_directory/001/1367 root 0
galaxy.jobs ERROR 2014-01-30 14:29:32,058 (1367) Failed to change
ownership of /som/galaxy/database/job_working_directory/001/1367,
failing
Traceback (most recent call last):
File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line 870, in
finish
self.reclaim_ownership()
File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line
1316, in reclaim_ownership
self._change_ownership( self.galaxy_system_pwent[0], str(
self.galaxy_system_pwent[3] ) )
File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line
1302, in _change_ownership
assert p.returncode == 0
AssertionError
Any idea on how to fix the issue?
Thanks,
-Adam
--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences
System Administrator, HPC Cluster
Office of Information Technology
http://hpc.oit.uci.edu/
University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
- --
Eric Rasche
Programmer II
Center for Phage Technology
Texas AM University
College Station, TX 77843
404-692-2048
e...@tamu.edu
rasche.e...@yandex.ru
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
iQIcBAEBAgAGBQJS79hyAAoJEMqDXdrsMcpV2tIP/3k218ARn3VIvYlhTT0SRGsP
8Qgt1w0AgPUSv897MsoS3IQwVqNG4ZJtTPJWgyd+asi0DDPb1lERWSeCXHRr1QSn
gQmT3Q2sW2tM7sx1OINwMqQ34Hly0bOJiN3/ifOxNaqp3LYjqfWmEvHszQUb3OS1
mS4JGrrAuZC4z38KohLC93FRLRwA+tOtw6Cw35ViFJZ+MnAWXcalew+WdhnCJmUg
jqXqWHLk8+d3K/FKkn0xkSL5UK9MMn4d6g/TBV3UeG0vOIhFzY3r3sxQKEjtT5x6
/p4qeCAgYWZ8woFQn+mSyrLmYCC5U+jKvtlsuu4la/GhNxFTj9ohppi6SDRn5bsf
I8WDXQS7Hu8qkvVDi0THty0Mbx01/YyYkqfNTWCySEHPUJdwh0WeRW8G5vt374lm
Mq0eyvbQIrjqBDN9XjVdfagXrtC2lcot4Nn3bCcLYNG4DRdNMM+zZmyIa6tKq2l/
2TrbLxaFs0AMYpj0p99rFpjYyzWy+IofmsAMmdmZqs39rStetSh+V26dFY1nS6sM
o6cLL5tKBzDuP/gbU3qCLHRryBInUo+FKUxFHUy1FxUJGoAAL02R5eB1OhDozfCr
o2lFeKHfZuKToa5VPS4On5QuywWCSQghoD8LvJNcX9zumwjQz0sUbogPt5x9PDh7
h9mwsn570ZgbXWZ2FQmM
=0oH0
-END PGP SIGNATURE-
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http
Re: [galaxy-dev] Upload via FTP - Assertion Error on Change ownership of working dictory
Anyone have any ideas on this? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences System Administrator, HPC Cluster Office of Information Technology http://hpc.oit.uci.edu/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Jan 30, 2014 at 2:33 PM, Adam Brenner aebre...@uci.edu wrote: Howdy, I am having trouble uploading data on our local instance of Galaxy here. The method is via FTP and here is the error I am seeing in the logs: galaxy.jobs.runners DEBUG 2014-01-30 14:29:31,987 execution of external set_meta for job 1367 finished galaxy.jobs DEBUG 2014-01-30 14:29:32,016 (1367) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /som/galaxy/database/job_working_directory/001/1367 root 0 galaxy.jobs ERROR 2014-01-30 14:29:32,058 (1367) Failed to change ownership of /som/galaxy/database/job_working_directory/001/1367, failing Traceback (most recent call last): File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line 870, in finish self.reclaim_ownership() File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line 1316, in reclaim_ownership self._change_ownership( self.galaxy_system_pwent[0], str( self.galaxy_system_pwent[3] ) ) File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line 1302, in _change_ownership assert p.returncode == 0 AssertionError Any idea on how to fix the issue? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences System Administrator, HPC Cluster Office of Information Technology http://hpc.oit.uci.edu/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Upload via FTP - Assertion Error on Change ownership of working dictory
Howdy, I am having trouble uploading data on our local instance of Galaxy here. The method is via FTP and here is the error I am seeing in the logs: galaxy.jobs.runners DEBUG 2014-01-30 14:29:31,987 execution of external set_meta for job 1367 finished galaxy.jobs DEBUG 2014-01-30 14:29:32,016 (1367) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /som/galaxy/database/job_working_directory/001/1367 root 0 galaxy.jobs ERROR 2014-01-30 14:29:32,058 (1367) Failed to change ownership of /som/galaxy/database/job_working_directory/001/1367, failing Traceback (most recent call last): File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line 870, in finish self.reclaim_ownership() File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line 1316, in reclaim_ownership self._change_ownership( self.galaxy_system_pwent[0], str( self.galaxy_system_pwent[3] ) ) File /data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py, line 1302, in _change_ownership assert p.returncode == 0 AssertionError Any idea on how to fix the issue? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences System Administrator, HPC Cluster Office of Information Technology http://hpc.oit.uci.edu/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem With Galaxy
Is this a local galaxy setup that you host ... or the public one? It sounds like a cache issue on your end (clear your browsers cache). If this is a local setup that you host, perhaps you are not loading CSS/javascript? Are those being blocked? Are you running a reverse proxy? Have the redirects been setup to handle static content? To many unknowns ~Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Oct 28, 2013 at 3:59 PM, Caitlin Grube chg...@psu.edu wrote: I am having an issue trying to do anything. I click on a tool, the screen pops up in the middle but then when I move my mouse over to that screen, the drop down menus all pull down in a large black screen so that I cannot get to anything. The screen also does not go away if I try to click away from it, but only if I choose one of the options from the list. I would click analyze data which would take back to where I started. This happens for every tool that I try. I did not have this problem this morning or afternoon, but I've been trying to get it to work for almost an hour. As soon as my mouse hits that center area (top, bottom, middle), the screen comes down, but only after I select a tool. Please help. Caitlin Grube ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tardis job splitter
Alan, At first glance this look promising. I am a little leery of tools that claim to do parallel processing. However I would like to test it out on our HPC cluster here at UCI. Few questions: Could you explain how your tool actually does the parallel processing on something that is sequential? For example in your PDF you mention the fastq example, but I do not see the explanation as to how it splits up the work across multiple cores/nodes. Does it simply split the sequence string N times and then merges the results? * our current implementation is quite naive in the distributed compute API it uses - it supports launching condor job files (and also native sub-processes) - our plan is to replace that with using the drmaa API We are strictly a SGE (Son of Grid Engine) cluster with a lot of work done by Joseph Farran (check pointing, freeq system, etc). Using DRMAA APIs would be great. If this tool can parallel fastq jobs along with BAM as described, it would be a great improvement for a number of people here. ~Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Oct 28, 2013 at 7:39 PM, McCulloch, Alan alan.mccull...@agresearch.co.nz wrote: dear all, There have been a few posts lately about doing distributed computing via Galaxy – i.e. job splitters etc – below a contribution of some ideas we have developed and applied in our work, where we have arranged for some Galaxy tools to execute in parallel on our cluster. We have developed a job-splitter script tardis.py (available from https://bitbucket.org/agr-bifo/tardis), which takes marked-up standard unix commands that run an application or tool. The mark-up is prefixed to the input and output command-line options. Tardis strips off the mark-up, and re-writes the commands to refer to split inputs and outputs, which are then executed in parallel e.g. on a distributed compute resource. Tardis knows the output files to expect and how to join them back together. (This was referred to in our GCC2013 talk http://wiki.galaxyproject.org/Events/GCC2013/Abstracts#Events.2FGCC2013.2FAbstracts.2FTalks.A_layered_genotyping-by-sequencing_pipeline_using_Galaxy ) Any reasonable unix based data processing or analysis command may be marked up and run using tardis, though of course tardis needs to know how to split and join the data. Our approach also assumes a “symmetrical” HPC cluster configuration, in the sense that each node sees the same view of the file system (and has the required underlying application installed). We use tardis to support both Galaxy and command-line based compute. Background / design pattern / motivating analogy: Galaxy provides a high level end to end view of a workflow; the HPC cluster resource that one uses then involves spraying chunks of data out into parallel processes, usually in the form of some kind of distributed compute cluster - but an end-user looking at a Galaxy history, should ideally not be able to tell whether the workflow was run as a single process on the server, or via many parallel processes on the cluster (apart from the fact that when run in parallel on the cluster, its alot faster!). We noticed that the TCP / IP layered networking protocol stack provides a useful metaphor and design pattern - with the end to end topology of a Galaxy workflow corresponding to the transport layer of TCP/ IP; and the distribution of computation across a cluster corresponding to the next TCP/IP layer down - the packet-routing layer. This picture suggested a strongly layered approach to provisioning Galaxy with parallelised compute on split data, and hence to an approach in which the footprint in the Galaxy code-base, of parallel / distributed compute support, should ideally (from the layered-design point of view) be minimal and superficial. Thus in our approach so far, the only footprint is in the tool config files, where we arrange the templating to (optionally) prefix the required tardis mark-up to the input and output command options, and the tardis script name to the command as a whole. tardis then takes care of rewriting and launching all of the jobs, and finally joining the results back together and putting them where galaxy expects them to be (and also housekeeping such as collating and passing up stderr and stdout , and appropriate process exit codes). (For each galaxy job, tardis creates a working folder in a designated scratch area, where input files are uncompressed and split; job files and their output are stored; logging is done etc. Split data is cleaned up at the end unless there was an error in some part of the job, in which case everything is retained
[galaxy-dev] Local Galaxy Install - Debug Report FROM address as Admin and not User?
Howdy, On our local Galaxy install, everytime a user submits a bug report, it does email all the admins -- as defined in universe_wsgi.ini -- but the FROM address is not of the user, but rather the first email address from the admin list. Is this normal? May we have it changed so the FROM address is that of the user? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Galaxy Install - Debug Report FROM address as Admin and not User?
Martin, Thanks for the response. Yes, I do notice the reply-to address does include the user, but wouldn't it be better if the FROM address was from the actual user reporting the error? When checking my email, for example, I do not want to see emails that are FROM me, to MYSELF when its really from our users. Do you know which of the python files contain the code that relates to this? Or point me in the right direction so I can change this? Thanks! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Oct 1, 2013 at 10:52 AM, Martin Čech mar...@bx.psu.edu wrote: Hi Adam, you are right about the behaviour, this is what Galaxy does. However it sends the email to the user too, so Reply all will reach the user's email address. This is how the feature was meant to be used. Martin On Tue, Oct 1, 2013 at 12:59 PM, Adam Brenner aebre...@uci.edu wrote: Howdy, On our local Galaxy install, everytime a user submits a bug report, it does email all the admins -- as defined in universe_wsgi.ini -- but the FROM address is not of the user, but rather the first email address from the admin list. Is this normal? May we have it changed so the FROM address is that of the user? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Need your helps about Galaxy
You have to determine if the old version of galaxy ( galaxylocal.genenetwork.org:8080/) and the new one, if any database changes have occurred between them -- most likely. If not, and you can check, by looking at the hg history changes via the bitbucket website: https://bitbucket.org/galaxy/galaxy-central/src Assuming database has changed, you can upgrade your old galaxy: http://wiki.galaxyproject.org/Admin/Get%20Galaxy#Keep_your_code_up_to_datestart up the instance, upgrade the DB and then dump the database. Perhaps you can get away without upgrading your old galaxy instance by only importing a few tables (such as users) into the new galaxy DB..but I am not sure. Perhaps someone more familiar with the Galaxy DB can comment on this. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Sep 23, 2013 at 12:50 PM, Lei Yan leiyan2...@gmail.com wrote: Dear James, Thanks so much for your info. Dumping database is a good idea. But the problem is that the two Galaxy are running on different servers, and the Galaxy versions are very different. Thanks again. Lei Yan Center for Integrative and Translational Genomics UTHSC On Mon, Sep 23, 2013 at 2:37 PM, James Taylor ja...@taylorlab.org wrote: There is not a simple way to do this. You will need to dump the accounts from your old database and load into the new database. Are they running the same database server? Please send questions like this to the galaxy-dev mailing list. -- James Taylor, Associate Professor, Biology/CS, Emory University On Mon, Sep 23, 2013 at 3:31 PM, Lei Yan leiyan2...@gmail.com wrote: Dear James, I built a new cloud Galaxy instance today. Would you please tell me how to transfer some given accounts from an old Galaxy to this new one? This is old Galaxy: http://galaxylocal.genenetwork.org:8080/ This is new Galaxy: http://galaxyclass.genenetwork.org/ Thanks a lot. Lei Yan Center for Integrative and Translational Genomics UTHSC ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Limit number of Jobs to Trinity
Carlos, Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya ccanch...@gmail.comwrote: Hi guys, I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis? Bests, Carlos -- Carlos Canchaya ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
Setting the environment for each tool just prior to running the tool is clearly better. What I'm doing currently is loading an environment module just before running a tool in Galaxy. On our HPC cluster, I have configured Galaxy to source the following from our modules system: 1) Global file as defined in universe_wsgi.ini environment_setup_file=setup-modules-for-galaxy.sh ^^ This setups the module environment so commands like module load work. It does something similar to: source /data/apps/modules/Modules/default/init/bash 21 /dev/null 2) Configured tool_dependency_dir as described here: http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies tool_dependency_dir=tool_dependency For each requirement as specified the xml, galaxy will automatically look in tool_dependency/name/default/env.sh Each dependancy's env.sh then use our favorite command: module load blah/version Cumbersome? Yup! But I believe its a nice elegant way of getting thing to work with no mucking around in the xml file. This will enable my native EMBOSS wrapper to run the defined version of the (system-installed) EMBOSS suite. However, it would only be worth me writing this extension if there's a likelihood of getting it into Galaxy. Is there? I truly like the idea and think it would. However, stepping into the Galaxy devs shoes, a solution like this might only be best for those using modules (small subset of users). Hopefully others can give their thoughts. The con I see in this is that Galaxy already has the env.sh solution which in my setup described above, does support modules In the meantime, there's work going on in Galaxy land to package every application wanted in Galaxy, by writing install scripts, encapsulating environment variables in tool XML files, etc. (There is difficulty here in writing install scripts which work on every platform, which I have commented on previously.) I saw that post last week and while I like the idea, folks like me who really do care about the version of GCC we use (and compile our own versions of gcc) will not use it. However, that is going off topic now :-) Will wait and see what others say... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Sep 12, 2013 at 4:03 PM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Another vote for the excellent modules system from us at AgResearch Galaxy's dependency injection works in a similar way, without requiring all the external dependencies. We don't need unload since we compose a unique environment for every tool execution. This is much cleaner that loading everything into the environment before running Galaxy, we want tools to be executed with as clean an environment as possible. Setting the environment for each tool just prior to running the tool is clearly better. What I'm doing currently is loading an environment module just before running a tool in Galaxy. So far, I have done this in the tool XML file by prefixing the command with a module load command, e.g. command. /etc/profile; module load EMBOSS/5.0.0; infoseq -sequence [snipped] /command The ugly . /etc/profile is needed for technical reasons. I don't like this, as it's a bit hacky, but what it achieves is useful and flexible. I envisage an extension to the tool XML syntax which would let me write it like this, say: requirementsrequirement type=module version=5.0.0EMBOSS/requirement/requirements commandinfoseq -sequence [snipped] /command This will enable my native EMBOSS wrapper to run the defined version of the (system-installed) EMBOSS suite. However, it would only be worth me writing this extension if there's a likelihood of getting it into Galaxy. Is there? Here's where I'm coming from. I already have most of the applications I want installed on my system. I am working on updating the RPMs to support side-by-side installs of multiple versions, with version selection by the environment modules facility. This is a small change (and improvement) on what I have today. It's a work in progress, but it's a manageable amount of work. It's looking likely that this multi-version packaging approach may be adopted by an official CentOS Scientific Repo, and this could change the landscape in terms of packaging of scientific applications for major Linux distributions. Who knows, that's all in the future. In the meantime, there's work going on in Galaxy land to package every application wanted in Galaxy, by writing install scripts, encapsulating environment variables in tool XML files, etc. (There is difficulty here in writing install scripts which work on every platform, which I have commented on previously.) The big
Re: [galaxy-dev] PATH and local jobs
Ryan, On our HPC setup, we use the modules system but its the same issue you are running into. A number of methods exists for setting up the correct path, for you, I believe the easiest would be one of the two: 1) In your init script: source (or setup) PATHS as you have them in your .bash_profile. /etc/init.d/galaxy OR 2) Within Galaxy, edit universe_wsgi.ini and look for: environment_setup_file and create a file that will setup the correct $PATH for you. You can also invoke the Tool Dependency options for each specific tool you load as shown in[1]. [1]: http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Sep 11, 2013 at 10:18 AM, Ryan Davis rrda...@ucdavis.edu wrote: Hi, ** ** We have a local server running Galaxy and a majority of the tools work correctly (bowtie, BWA, samtools and etc…). I am having issues with some of the FASTX-Toolkit tools. I am starting Galaxy with an init script on Centos 6 and the tools are in my $PATH and work at the command line, just not in Galaxy. The wiki suggest to “set $PATH in the startup file or use.” It leaves me hanging and don’t know what the other option is. Can someone share with me their workaround? ** ** Here is the wiki page: http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies ** ** Here is the section I am hoping will help: ** ** Local Jobs Changes to $PATH can be persisted by setting them in your shell's startup file(s). This typically means ~/.bash_profile for bash, but please see the bash documentation on startup fileshttp://www.gnu.org/software/bash/manual/bashref.html#Bash-Startup-Files or the *INVOCATION* section of the bash(1) man page to understand the intricacies of how that file is read. Of particular importance, if you are starting Galaxy in a method other than manual invocation from a shell prompt (with sh run.sh) such as with an init script, it is likely that your startup file will not be read. In this instance, you should set $PATH in the startup file or use. ** ** ** ** Thanks! Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Stable Galaxy for local install
Galaxy development occurs in two repositories, galaxy-central and galaxy-dist. Changes are stabilized in galaxy-central and then pushed to galaxy-dist roughly once a month.[1] So $ hg clone https://bitbucket.org/galaxy/galaxy-dist/ $ cd galaxy-dist $ hg update stable [1]: http://wiki.galaxyproject.org/Admin/Get%20Galaxy -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Sep 11, 2013 at 8:42 AM, cngbgalaxy cngbgal...@gmail.com wrote: Hi Joachim, Thank you very much for your help. Actually I find some problems about such versions of Galaxy in https://bitbucket.org/galaxy/galaxy-dist/downloads . We can not view the data in history by clicking the eyes icon on the c42567f43aa7 version.The Select in Filter and Sort fail to work on the 1ae95b3aa98d (release_2013.08.12 ) version and the 524f246ca853 (release_2013.06.03 ),though I don't know whether it is a bug or not. As a result,I feel confused about which version of Galaxy to install and avoid the above problems. cngbgalaxy From: Joachim Jacob | VIB | Date: 2013-09-11 22:38 To: test galaxy CC: galaxy-dev; galaxy-dev Subject: Re: [galaxy-dev] Stable Galaxy for local install The latest code from the 'stable' branch of Galaxy is preferred. See the news brief accompanying the update (release_2013.08.12) on http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12 You need mercurial (http://mercurial.selenic.com/) for installing Galaxy. Report bugs at https://trello.com/b/75c1kASa/galaxy-development Cheers Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/11/2013 03:55 AM, test galaxy wrote: Hi everyone, I want to install my local Galaxy with less bugs,and I don't know which version is more stable and work well.Could anyone give same advise?Thank you very much. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Stable Galaxy for local install
What I want to do in a hurry is to fix the problem of Select in Filter and Sort .It is a step very important for me to Sort SAM file for Cufflinks.Thank you very much for all of your help. Per my last email:. $ hg clone https://bitbucket.org/galaxy/galaxy-dist/ $ cd galaxy-dist $ hg update stable If you are running into issues, can you be more specific...provide error message? Steps on how to reproduce your issue? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Sep 11, 2013 at 6:58 PM, cngbgal...@gmail.com cngbgal...@gmail.com wrote: Hi all, What I want to do in a hurry is to fix the problem of Select in Filter and Sort .It is a step very important for me to Sort SAM file for Cufflinks.Thank you very much for all of your help. cngbgalaxy From: Adam Brenner Date: 2013-09-12 00:41 To: cngbgalaxy CC: joachim.jacob; galaxy-dev@lists.bx.psu.edu; galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] Stable Galaxy for local install Galaxy development occurs in two repositories, galaxy-central and galaxy-dist. Changes are stabilized in galaxy-central and then pushed to galaxy-dist roughly once a month.[1] So $ hg clone https://bitbucket.org/galaxy/galaxy-dist/ $ cd galaxy-dist $ hg update stable [1]: http://wiki.galaxyproject.org/Admin/Get%20Galaxy -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Sep 11, 2013 at 8:42 AM, cngbgalaxy cngbgal...@gmail.com wrote: Hi Joachim, Thank you very much for your help. Actually I find some problems about such versions of Galaxy in https://bitbucket.org/galaxy/galaxy-dist/downloads . We can not view the data in history by clicking the eyes icon on the c42567f43aa7 version.The Select in Filter and Sort fail to work on the 1ae95b3aa98d (release_2013.08.12 ) version and the 524f246ca853 (release_2013.06.03 ),though I don't know whether it is a bug or not. As a result,I feel confused about which version of Galaxy to install and avoid the above problems. cngbgalaxy From: Joachim Jacob | VIB | Date: 2013-09-11 22:38 To: test galaxy CC: galaxy-dev; galaxy-dev Subject: Re: [galaxy-dev] Stable Galaxy for local install The latest code from the 'stable' branch of Galaxy is preferred. See the news brief accompanying the update (release_2013.08.12) on http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12 You need mercurial (http://mercurial.selenic.com/) for installing Galaxy. Report bugs at https://trello.com/b/75c1kASa/galaxy-development Cheers Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/11/2013 03:55 AM, test galaxy wrote: Hi everyone, I want to install my local Galaxy with less bugs,and I don't know which version is more stable and work well.Could anyone give same advise?Thank you very much. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Creating Galaxy Admin / Local Install Mailing List - Updates?
Howdy, I know this has been discussed before, but I can not find the email threads on this...but what was the status on creating a galaxy admin / local install mailing list? The Devel list is very chatty and while my filters do a good job, I think it will be easier for others to find help on their own if it was on a separate mailing list It would be easier to search and look back at previous threads. However, the added benefit of having the devel list is that we get feedback from Galaxy Developers As the saying godes: Just food for thought, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] questions on Galaxy installation under Apache
Jim, You need to enable Apache's proxy service. See my pervious post here: http://dev.list.galaxyproject.org/Running-Galaxy-through-Apache-td4659452.html Let us know if you run into further issues, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Sep 9, 2013 at 12:04 PM, Jingchao Zhang zh...@unl.edu wrote: Hello everyone, I installed both Galaxy and Apache in a Red Hat linux machine and they work fine individually. If I open http://localhost then a welcome page comes out showing the httpd is working. In galaxy I changed host=0.0.0.0 and sh run.sh, then the http://localhost:8080 will direct me to the Galaxy page. Now I want to put Galaxy under Apache and here is what I did: 1. First I comment the host=0.0.0.0 line to stop the build in web server in Galaxy and then sh run.sh to make Galaxy running. 2. Then I put these lines RewriteEngine On RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] into my httpd.conf file. Then I restart the httpd service. Also, I made changes to the universe_wsgi.ini file as instructed and restart Galaxy: [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /galaxy [app:main] filter-with = proxy-prefix cookie_path = /galaxy After I do these, either http://localhost:8080 nor http://localhost/galaxy works. I find two threads talked about this problem before: http://user.list.galaxyproject.org/Installing-galaxy-with-Apache-td4654499.html http://dev.list.galaxyproject.org/Running-Galaxy-behind-apache2-td4624545.html but my Red Hat httpd does not has this /etc/apache2/sites-available/default directory and only has /etc/httpd/conf/httpd.conf. Could someone please tell me where should I put those lines in the httpd.conf file? I have tries various places and none of them works. Any help would be deeply appreciated. Thanks, Jim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Typo in Wiki Page - Purge History and Datasets
http://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%20and%20Datasets There are 5 scripts included in the Galaxy distribution that can be used to clean up unwanted histories, libraries and datasets. There are located in the nwwl(GALAXY_ROOT)/scripts/cleanup_datasets directory and are named: Should be changed to 6 scripts Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] URL problem
Alper, Nginx or Apache? Did you modify universe_wsgi.ini and add the proxy settings? Does the information on http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Using_a_proxy_serverhelp you? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, Aug 30, 2013 at 12:48 PM, Jennifer Jackson j...@bx.psu.edu wrote: Hi Alper, There may be permissions problems if just for these two links, but I am not an expert on this. Moving post over to the galaxy-dev list, which will give it better exposure to the development team and community. You'll want to post here directly with any future local install questions. http://wiki.galaxyproject.org/**Support#Mailing_Listshttp://wiki.galaxyproject.org/Support#Mailing_Lists Best, Jen Galaxy team On 8/26/13 10:49 AM, Kucukural, Alper wrote: Hi, I have a problem in galaxy to get host/domain name in two different pages. First one is in the tool installation from toolshed, I got the error below, # Not Found The requested URL /admin_toolshed/prepare_for_**install was not found on this server. # The second one is in the saved histories. When I click the buttons of the saved histories. I got the similar error like below. # Not Found The requested URL /history/list was not found on this server. # I haven't seen these any other pages yet. My installation is working on LDAP authentication with Proxy. So, I could not find a place to set the domain or host name in these two places that they can actually find the requested URLs. In the paster.log file. I don't get any error when I install a tool or go to another history. It doesn't report any error. Thanks for your help, Alper Kucukural __**_ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/**listinfo/galaxy-devhttp://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/**search/mailinglists/http://galaxyproject.org/search/mailinglists/ -- Jennifer Hillman-Jackson http://galaxyproject.org __**_ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/**search/mailinglists/http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Instance and current website are not the same
Hamid, Read the information here: http://snpeff.sourceforge.net/SnpEff_manual.html#galaxy -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Aug 28, 2013 at 2:16 PM, Tevfik Hamdi Kitapci tkita...@usc.edu wrote: Hi, I installed a local instance of galaxy browser on my local machine using the steps in: http://wiki.galaxyproject.org/Admin/Get%20Galaxy I tried both the default (hg update default) and stable versions (hg update stable) but the current website has different features on the left panel then the local instance that I downloaded. Specifically I want to use the SnpEff feature under the NGS toolbox Beta on the left panel of the website. But I can't find this feature on the local instance is there anyway to install a local instance of the current version of the website? Thanks Best Regards Hamdi Kitapci ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Login hook
I am not aware of Galaxy having any sort of login hook as you described --- without writing custom code. This brings up the added challenge of keeping your custom modifications current with the latest release of galaxy. Another approach, and I am not sure how feasible this is on your setup, is to create all these added services during the user creation[1] process. On our HPC cluster this is the method we implemented. All our users can use SFTP, rsync, BBCP, etc to upload to the cluster and the user moves their data to galaxy by the FTP Trick[2] to their dropbox folder that galaxy has access to. Once data is moved to the dropbox folder, users login via the galaxy web interface and select the data via the Get Data - Upload File interface. This might be a simpler approach, create these services for each new user during their account creation. You will be forced to use the command line interface to create new users, but I believe its far simpler then using postgres notify or the other methods you described. [1]: http://dev.list.galaxyproject.org/user-creation-using-API-td4657726.html [2]: http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Aug 25, 2013 at 9:39 AM, Maddhi, Srinivas srinivas-mad...@uiowa.edu wrote: Hello, Does Galaxy provide a post-login hook that could be used to trigger an operating system level action upon successful login ? Here is the scenario and use-case where that would come in handy: * Local instance of Galaxy (Nov 2012 release) on a Linux server with a PostgreSQL DB * Upload via FTP feature is enabled * Users are authenticated via LDAP and nginx * Users should be able to securely upload files (to their sub-directory within the configured FTP upload directory) The current method being employed to fulfill this use-case is through a modification to Š/galaxy-dist/lib/galaxy/web/framework/__init__.py file that calls a a shell script which creates an OS-level user and associated FTP upload directory. Files are uploaded by users via sFTP (SSHD configured to restrict Galaxy users to a chroot'ed directory). Options considered: * Shell/Python script that periodically queries the galaxy_user table and creates an OS-level user and FTP upload directory (not yet tried; introduces delay between user creation and FTP capability unless frequency of polling is high) * ProFTPD with sFTP and LDAP modules (increased administrative overhead compared to using built-in SSHD; wasn't able to get auto directory creation to work due to TBD cause) * PostgreSQL listen-notify to call a sFTP user creation script when an insert occurs in the galaxy_user table (not yet tried) Thank you. -- Senior Systems Administrator Iowa Institute for Human Genetics ITS Research Services, The University of Iowa ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Help with installation
The issue here is the last line:
OperationalError: (OperationalError) disk I/O error u'PRAGMA
table_info(dataset)' ()
Do you have any security rules setup on the server, quotes, disk
issues, etc? You can always run a strace on run.sh for a better
understanding of whats going on.
--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences
Research Computing Support
Office of Information Technology
http://www.oit.uci.edu/rcs/
University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
On Thu, Aug 8, 2013 at 4:12 AM, Jurgens de Bruin debrui...@gmail.com wrote:
To all,
I am getting the following error when I run sh run.sh :
galaxy.model.migrate.check DEBUG 2013-08-08 07:52:39,479 pysqlite=2 egg
successfully loaded for sqlite dialect
Traceback (most recent call last):
File
/scratch/sysusers/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py,
line 35, in app_factory
app = UniverseApplication( global_conf = global_conf, **kwargs )
File /scratch/sysusers/galaxy/galaxy-dist/lib/galaxy/app.py, line 52, in
__init__
create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get(
'__file__', None ), self.config.database_engine_options, app=self )
File
/scratch/sysusers/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py,
line 51, in create_or_verify_database
dataset_table = Table( dataset, meta, autoload=True )
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py,
line 318, in __new__
table._init(name, metadata, *args, **kw)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py,
line 381, in _init
self._autoload(metadata, autoload_with, include_columns)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/schema.py,
line 408, in _autoload
self, include_columns, exclude_columns
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 2425, in run_callable
return conn.run_callable(callable_, *args, **kwargs)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1968, in run_callable
return callable_(self, *args, **kwargs)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py,
line 260, in reflecttable
return insp.reflecttable(table, include_columns, exclude_columns)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py,
line 370, in reflecttable
for col_d in self.get_columns(table_name, schema, **tblkw):
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py,
line 223, in get_columns
**kw)
File string, line 1, in lambda
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/reflection.py,
line 47, in cache
ret = fn(self, con, *args, **kw)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/sqlite/base.py,
line 713, in get_columns
(pragma, qtable)))
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1449, in execute
params)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1628, in _execute_text
statement, parameters
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1698, in _execute_context
context)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1691, in _execute_context
context)
File
/scratch/sysusers/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py,
line 331, in do_execute
cursor.execute(statement, parameters)
OperationalError: (OperationalError) disk I/O error u'PRAGMA
table_info(dataset)' ()
Please any help would be appreciated!!
--
Regards/Groete/Mit freundlichen Grüßen/recuerdos/meilleures salutations/
distinti saluti/siong/duì yú/привет
Jurgens de Bruin
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists
Re: [galaxy-dev] Dynamically decide where a Galaxy tool runs
This is something our research group is interested in, but it's not possible to specify which cluster the particular tool should run on ahead of time. Incorrect (at least to my knowledge). Using the new way jobs are handled within Galaxy via the XML[1] files you can send specific jobs to specific destinations which could be different clusters. For our cluster, we handle the file uploading tool to a different job runner then the cluster (SGE). This was accomplished by tools tool id=upload1 destination=local/ /tools where id is the galaxy tool id and destination is where the tool should be ran, cluster, local, etc. [1] http://wiki.galaxyproject.org/Admin/Config/Jobs -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, Jul 12, 2013 at 11:42 AM, Saliya Ekanayake esal...@gmail.com wrote: Hi, I see in documentation that Galaxy tools may be configured to run on a cluster (http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster). This is something our research group is interested in, but it's not possible to specify which cluster the particular tool should run on ahead of time. Therefore, is it possible to specify the cluster to run when the tool is actually being used when composing the workflow from Galaxy? Thank you, Saliya -- Saliya Ekanayake esal...@gmail.com Cell 812-391-4914 Home 812-961-6383 http://saliya.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Build the Galaxy under Cluster
May you make your question more specific? Have you read the online docs: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Jul 7, 2013 at 1:35 AM, shenwiyn shenw...@gmail.com wrote: Hi everyone, I want to install my local Galaxy under the Cluster with TORQUE.I have a question about how they work together successfully.In addition,after my local Galaxy and TORQUE install successfully,anything else should I do to the Cluster Galaxy?Thank you very much. shenwiyn ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Startup problem after upgrade
When you updated, did you download all the files? Was their any conflicts during the update? It appears you are missing a file. Cant locate template for uri 'root/index.mako' Run 'hg update'... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Jul 2, 2013 at 8:10 AM, Bertrand PITOLLAT bertrand.pitol...@cirad.fr wrote: Hi, I have just upgraded my galaxy instance. When trying to access my galaxy instance home page, I get the following error : 195.221.174.40 - - [02/Jul/2013:15:34:23 +0200] GET / HTTP/1.1 500 - - Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36 Error - class 'mako.exceptions.TopLevelLookupException': Cant locate template for uri 'root/index.mako' URL: http://marmadais.cirad.fr:7090/ File '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/SATA/bioinfo/galaxy_dev/galaxy_dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/SATA/bioinfo/galaxy_dev/galaxy_dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/webapps/galaxy/controllers/root.py', line 39 in index params=kwd ) File '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 957 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/SATA/bioinfo/galaxy_dev/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 964 in fill_template_mako template = self.webapp.mako_template_lookup.get_template( filename ) File '/SATA/bioinfo/galaxy_dev/galaxy_dist/eggs/Mako-0.4.1-py2.6.egg/mako/lookup.py', line 221 in get_template Cant locate template for uri %r % uri) TopLevelLookupException: Cant locate template for uri 'root/index.mako' Any help welcome. Bertrand. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Tool Dependency Check for eXpress - requirements syntax incorrect?
Howdy, One of our users is running the eXpress package under NGS: RNA Analysis - eXpress -- galaxy/dist/tools/ngs_rna/express_wrapper.* I noticed the syntax for express_wrapper.xml has the following: requirement type=binaryeXpress/requirement any reason why it was set to that, as opposed to: requirement type=packageeXpress/requirement In our setup, we need to source the env.sh file for express to setup the correct environmental variables. Changing from binary to package has not solved this and re-running the job never sources our env.sh file. What am I doing wrong? My tool dependency directory looks like so: galaxy/dist/tool_dependency/eXpress/ |---default -- 1.3.0 |1.3.0/env.sh Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool Dependency Check for eXpress - requirements syntax incorrect?
James, Thanks for the response. I had some sort of typo in my express_wrapper.xml file and all is well now. It is now sourcing the correct file. I went ahead and submitted a pull request to have eXpress updated to package, instead of binary: https://bitbucket.org/galaxy/galaxy-central/pull-request/192/express-tool-requirements-tag-updated/diff -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Jun 27, 2013 at 11:03 AM, James Taylor ja...@jamestaylor.org wrote: Hi Adam, yes, it should be package, the binary type has never been supported but keeps getting propagated for some reason. At some point it might be nice to have a configurable mapping from the other dependency types to packages. However, changing it to package *should* work. Can you send the relevant part of the paster log, it should log information about dependency resolution and the command line. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, Jun 27, 2013 at 1:38 PM, Adam Brenner aebre...@uci.edu wrote: Howdy, One of our users is running the eXpress package under NGS: RNA Analysis - eXpress -- galaxy/dist/tools/ngs_rna/express_wrapper.* I noticed the syntax for express_wrapper.xml has the following: requirement type=binaryeXpress/requirement any reason why it was set to that, as opposed to: requirement type=packageeXpress/requirement In our setup, we need to source the env.sh file for express to setup the correct environmental variables. Changing from binary to package has not solved this and re-running the job never sources our env.sh file. What am I doing wrong? My tool dependency directory looks like so: galaxy/dist/tool_dependency/eXpress/ |---default -- 1.3.0 |1.3.0/env.sh Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] DRMAA Runner URL Specify TORQUE host
Bart, I believe drmaa://-q somehost@queue-name will work. However I could be very wrong. It has been a while since I messed with the actual drmaa runners. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Jun 26, 2013 at 1:28 PM, Bart Gottschalk bgott...@umn.edu wrote: Is it possible to specify the torque host as part of a DRMAA runner URL? I haven't been able to find a native_options parameter to allow for this. I'm using the old style cluster configuration. drmaa://[native_options]/ Also, I haven't been able to find a list of native_options anywhere. Does such a list exist? If so, where? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Bowtie2 Specifying -k value (Alignment Search)
One of our users has requested to see if we could add specifying an alignment search option to Bowtie2 (-k and -a option http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#default-mode-search-for-multiple-alignments-report-the-best-one ) Is this something anyone else has added to their galaxy installs? If so, how? I was looking at galaxy/dist/tools/sr_mapping/bowtie2_wrapper.* and it does not seem to challenging to add another field where a user could specify some integer value that we simply pass to the the command tag in the xml file. Would this be the correct way to go about this? Thanks in advanced! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Config Questions
1. We want to use the NGS:Mapping tool 'Map with Bowtie for Illumina' which requires 'bowtie (0.12.7) with bowtie indexes'. Do we simply download the bowtie software, place it into /Users/administrator/galaxy-dist/tools, restart Galaxy, and the tool is available under NGS:Mapping ? Correct, download the bowtie software, install and verify that it works. Once it does, you can place the binaries (or a symbolic link) to it in the tools dependecy directory (not tools). You need to specify the tools dependency directory in your universe_wsgi.ini file. You can also, see my response to #3. More information: http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies 2. Continuing from question #1, does a type of 'package manager' exist for automating the installation of tools and dependencies? Can mercurial accomplish this or is this the purpose of the Tool Shed? Yes and no. Tool Shed is like a package manager, but not. More information here: http://wiki.galaxyproject.org/Tool%20Shed You will still need to install the tools, like bowtie, on your own and not via Galaxy. 3. What is the purpose of the Galaxy-specific environment file and how is it used? Going off your question one, you could install bowtie anywhere. I think on OSX its /Applications/Bowtie2.app/ or some other locations /opt/bowti2/ etc. Then simply have a reference to using env.sh to setup the correct environmental paths. In your tools dependency folder you would have the following setup: tools_dependecy/bowtie2/1.03/env.sh tools_dependecy/bowtie2/default -- 1.03/ Note: default is a symbolic link to 1.03 4. We have two hard drives: drive1 and drive2. Galaxy is installed on drive1 and our data files are on drive2. Will Galaxy know how to find the location of our data files on drive2? Also, will Galaxy know how to write the result of a job to drive2? Yes, see universe_wsgi.ini for this. Just update the paths. On OSX it could be /Volumes/someNameOfDrive/folder/on/drive 5. Does Galaxy come configured with an admin account and, if so, how is it accessed? Yes, see universe_wsgi.ini (admin_users). You need to add an email address of the admin account. 6. Does Galaxy use as many hardware resources as possible to execute each job (for example, all 12 cores)? If not, can it be configured to do so? You will need to configure your job runners and job workers for this. http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster Enjoy! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Jun 5, 2013 at 11:22 AM, Nicholas Kline nx...@case.edu wrote: Hi, Our lab is in the process of installing a local version of Galaxy on a computer with the following specs: - Mac Pro mid-2012 - OS X 10.8.3 - 2 2.4 GHz Intel Xeon Processors - 6 cores per processor, 12 cores total - 64 GB RAM This installation is meant to be used by a single user. We followed the instructions at http://wiki.galaxyproject.org/Admin/Get%20Galaxy and now have a working version of Galaxy available at localhost:8080, but we have a few questions before using it: 1. We want to use the NGS:Mapping tool 'Map with Bowtie for Illumina' which requires 'bowtie (0.12.7) with bowtie indexes'. Do we simply download the bowtie software, place it into /Users/administrator/galaxy-dist/tools, restart Galaxy, and the tool is available under NGS:Mapping ? 2. Continuing from question #1, does a type of 'package manager' exist for automating the installation of tools and dependencies? Can mercurial accomplish this or is this the purpose of the Tool Shed? 3. What is the purpose of the Galaxy-specific environment file and how is it used? 4. We have two hard drives: drive1 and drive2. Galaxy is installed on drive1 and our data files are on drive2. Will Galaxy know how to find the location of our data files on drive2? Also, will Galaxy know how to write the result of a job to drive2? 5. Does Galaxy come configured with an admin account and, if so, how is it accessed? 6. Does Galaxy use as many hardware resources as possible to execute each job (for example, all 12 cores)? If not, can it be configured to do so? Thank you for your help! ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use
Re: [galaxy-dev] Bypassing the proxy server when using the API, without breaking the automatic update of the history in the web interface
Why not use Allow All / Deny All rules? Location / ... your Auth Type stuff here ... Deny from all Allow from 127.0.0.1 # Assuming API calls are made locally Satisfy any /Location -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Jun 5, 2013 at 5:15 PM, Anthonius deBoer thondeb...@me.com wrote: Hi, I am trying desperately to setup my GALAXY system so that if you enter the website, you need to authenticate yourself using the proxy setup... I also would like to be able to BYPASS the proxy server when an API call is made... In the example proxy setting I have setup in the following way: ReWriteRule ^(/api/histories/.*) http://localhost:8080$1 [P,L] ReWriteRule ^(/api/.*) http://localhost:8081$1 [P,L] RewriteRule ^(.*) http://localhost:8080$1 [P] I then setup a PROXY setting in the apache galaxy.d in /etc/httpd/conf.d/galaxy.d to require authentication when a user is re-directed to the localhost:8080 and allow the calls to localhost:8081 to pass unhindered... This works brilliantly EXCEPT for when you want to make an API call that involves some histories The call fails, since /api/histories calls are routed through the proxy server and require authentication, which I want to avoid... It seems that histories are special so they have their own line in the configuration file above... I tried to change ReWriteRule ^(/api/histories/.*) http://localhost:8080$1 [P,L] to ReWriteRule ^(/api/histories/.*) http://localhost:8081$1 [L] Which DOES work, but it also breaks the automatic update of the history in the webbrowser... I have to reload the page manually to update the history... Is there anyway I can get the automatic update to work again, but don't require autentication? Thanks Thon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Storage question for Galaxy-Admins
Raj, I agree with the points Alex made. It would be much easier to start small and then work your way up. I recently setup galaxy for our HPC cluster for the entire school of medicine. Its only a few ~month into our setup but we are currently using ~300GB of storage. This is part of our larger cluster storage pool of 600TB. We use the gluster filesystem across 6 storage nodes. We are trying fhgfs and other file systems as well (lustre, ceph, etc). If you have any other questions, feel free to ask. -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, May 29, 2013 at 10:34 PM, Raj Ayyampalayam ra...@uga.edu wrote: Hello, I am trying to come up with a number for our local galaxy storage requirements when we go live (This is an installation for research at our university). I checked out the survey that was done a while back for the GalaxyAdmins. It seems that the bigger installations have allocated about 200TB for their installation. It would be of great help to me if I can get some feedback on storage used by local galaxy installations. If possible please use the following format: Total capacity: Used capacity: Storage Technology: Comments: Thank you all very much. -Raj ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] uploading large datasets into a new instance of Galaxy on a Mac Pro
How are you trying to upload the data to Galaxy? Are you using the web interface? Or...? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 31, 2013 at 1:59 PM, Elwood Linney ellin...@gmail.com wrote: Hello, I installed Galaxy on a new Mac Pro with 64gb of ram and a solid state first drive and 3 , 2tb normal drives. I can upload from my desktop a smaller file like a reference list or a cuffdiff file but when I try to load a 16gb fastq file it just does not load. Given that I have my files on the same computer that has Galaxy, and given that I am not well versed in unix, would anyone have a suggestion as to ways I could easily upload these large files. I would like to think I am just naive and that this has a simple solution. Hopefully this is the correct list to pose this question. Thank you for any help, el linney For those in the no, I did see this appear on the terminal after I tried to upload the file through the browser mechanism for upload: Overflow Error: signed integer is greater than maximum ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Invalid Galaxy URL: None - Installing Tools Shed
Greg, Thanks for the tip on Safari. I have switched to another browser and working now! Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, May 8, 2013 at 6:50 AM, Greg Von Kuster g...@bx.psu.edu wrote: In addition, it's helpful to know which browser you are using. Safari has fairly strict policies on 3rd-party cookies if you have them blocked. Chrome and firefox are less strict, so unblocking 3rd-party cookies if using Safari or simply switching to chrome or firefox will probably solve this issue. Greg Von Kuster On May 8, 2013, at 9:45 AM, Dave Bouvier wrote: Adam, Normally, the Galaxy URL should be automatically determined and set in a cookie during the repository installation process. To help track down this issue, could you provide the revision of Galaxy you're running, and the end of the paster.log file when this error occurs? --Dave B. On 5/7/13 20:13:38.000, Adam Brenner wrote: Howdy, I am trying to install a tools shed item, Emboss, but when I do this via the admin interface, I get the following text: Repository installation is not possible due to an invalid Galaxy URL: None. You may need to enable cookies in your browser. I have searched the universe_wsgi.ini file and could not find anything that looks like Galaxy URL. Any ideas on how to set the Galaxy URL? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Invalid Galaxy URL: None - Installing Tools Shed
Howdy, I am trying to install a tools shed item, Emboss, but when I do this via the admin interface, I get the following text: Repository installation is not possible due to an invalid Galaxy URL: None. You may need to enable cookies in your browser. I have searched the universe_wsgi.ini file and could not find anything that looks like Galaxy URL. Any ideas on how to set the Galaxy URL? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Taxonomy fetching not working
Deeps, It does not appear that you have installed taxBuilder on your system. What commands did you use to compile it on your system? If you type 'which taxBuilder' it should return no results (assuming installed incorrectly). You will need to 'make clean' then 'make all' then manually move the binary files it produces into some environmental $PATH. Hope that helps, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 3, 2013 at 3:58 PM, Deepthi Thomas Kannanayakal dttho...@ucalgary.ca wrote: Hi Galaxy team, I am having problems with taxonomy information fetching.(Metagenomic Analysses-Fetch taxonomic representation). According to the following link http://dev.list.galaxyproject.org/quot-Fetch-taxonomic-representation-quot-DOESN-T-WORK-td4565489.html I have installed libraries from https://bitbucket.org/natefoo/taxonomy I have installed them in my local directory(not in /usr/local). But I am still having problem /bin/sh: taxBuilder: command not found What am I missing here? Thanks, Deeps *Deepthi Theresa Thomas Kannanayakal Bioinformatics Database Specialist Sun Visual Genomic Centre, Suite 1150, Health Science Centre 3330 Hospital Drive NW Department of Biochemistry and Molecular Biology Faculty of Medicine, University of Calgary T2N 4N1 Calgary, Alberta, Canada** * *Ph: (403) - 210 6661 Fax: (403) - 210 9538* ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running Galaxy through Apache
Jeff,
Did you ever get this to work?
-Adam
--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences
Research Computing Support
Office of Information Technology
http://www.oit.uci.edu/rcs/
University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
On Wed, May 1, 2013 at 1:52 PM, Jeffrey Long jlo...@ualberta.ca wrote:
In the Apache access.log, a single access link results in a great many
output lines; the first two look like this:
128.233.109.35 - - [01/May/2013:14:31:29 -0600] GET /galaxy HTTP/1.1 200
5058 - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5)
AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13
128.233.109.35 - - [01/May/2013:14:31:29 -0600] GET
/galaxy/static/style/base.css?v=1367440234 HTTP/1.1 404 495
http://trove.usask.ca/galaxy; Mozilla/5.0 (Macintosh; Intel Mac OS X
10_7_5) AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4
Safari/536.29.13
If I'm reading this right, the second one involves a 404 error. Indeed, I
don't see any static/style directory in my galaxy location...should there
be one?
I don't get anything new in apache's error.log as a result of a single
access attempt.
-Jeff
On Wed, May 1, 2013 at 6:16 AM, Nate Coraor n...@bx.psu.edu wrote:
On Apr 30, 2013, at 5:15 PM, Jeffrey Long wrote:
1) Any idea which proxy settings in the universe_wsgi file need to be
set other than the ones on the website? Those are the only ones I changed.
I scrolled down to the 'Advanced Proxy' settings but didn't see anything
that looked relevant to me.
Probably I should have sent this earlier, but this is the traceback I
get; it looks like there's a url somewhere that's not getting passed along
or re-assembled properly.
Exception happened during processing of request from ('127.0.0.1',
47424)
Traceback (most recent call last):
File
/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py,
line 1053, in process_request_in_thread
self.finish_request(request, client_address)
File /usr/lib/python2.7/SocketServer.py, line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File /usr/lib/python2.7/SocketServer.py, line 638, in __init__
self.handle()
File
/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py,
line 432, in handle
BaseHTTPRequestHandler.handle(self)
File /usr/lib/python2.7/BaseHTTPServer.py, line 340, in handle
self.handle_one_request()
File
/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py,
line 427, in handle_one_request
self.wsgi_execute()
File
/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py,
line 287, in wsgi_execute
self.wsgi_start_response)
File
/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/config.py,
line 285, in __call__
return self.app(environ, start_response)
File
/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/urlmap.py,
line 193, in __call__
path_info = self.normalize_url(path_info, False)[1]
File
/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/urlmap.py,
line 117, in normalize_url
or self.domain_url_re.search(url)), URL fragments must start with
/ or http:// (you gave %r) % url
AssertionError: URL fragments must start with / or http:// (you gave
'.')
Hi Jeff,
Does the Apache log give any indication of what path is being accessed?
You may also want to enable mod_rewrite's debug logging to make sure it's
behaving correctly.
2) Are the static subdirectories strictly necessary, or can I worry
about one problem at a time? I've tried both commenting out the proxy
lines (in apache2.conf) and leaving them in, and I get the same behaviour.
You can exclude the static page rewrites for the purposes of getting it
working. Galaxy's internal HTTP server will serve the static pages itself.
--nate
Thanks, I'll try to remember to 'reply-all' here!
-Jeff
On Mon, Apr 29, 2013 at 1:23 PM, Adam Brenner aebre...@uci.edu wrote:
Hia Jeff,
I think this is related to the proxy settings in Galaxy
(universe_wsgi.ini)
What is happening is that if you look at the source code for your
galaxy website, it is trying to load css/images from:
http://trove.usask.ca/galaxy/static/style/base.css?v=1367254127 But if
you visit that link it offers you a 404 not found. Make sure the proxy
settings in your universe_wsgi.ini is set correctly.
If it is, it could be that you need to setup proxies for the
subdirectories as well. If you look at the nginx guide here:
http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy
you will notice that we provide alias for /static/*.
In terms of doing this in Apache, you may need to add more proxy pass
lines that correspond to the subdirectories
Re: [galaxy-dev] Running galaxy using Apache as proxy
Mike, Looking at the configuration, you are not proxy'ing anything. Take a look at my pervious post here and see if that helps: http://dev.list.galaxyproject.org/Running-Galaxy-through-Apache-td4659452.html Let us know how it goes! (I am planning on submitting a re-write for the Apache Proxy page as its missing information..feedback is appreciated). -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 3, 2013 at 11:25 AM, Michael Place mpl...@wisc.edu wrote: Hello, I am trying to run galaxy using Apache as the proxy server. This is intended to be internal to our lab. I have galaxy running and working. The problem is that the pages load slowly and then seem to time out after 10 min or so. If I login and click get data then go away for a few minutes , then I come back and click a tool I get: The connection to the server was reset while the page was loading. If I retry the page loads. This makes the page very clunky. httpd.conf changes: IfModule mod_proxy.c ProxyRequests On Proxy * Order deny,allow Deny from all Allow from * /Proxy VirtualHost *:8080 ServerName 192.168.0.240 RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /opt/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /opt/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /opt/galaxy-dist/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /opt/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /opt/galaxy-dist/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] /VirtualHost I have attached the universe_wsgi.ini and my httpd.conf I am running CentOS. Thank you, Mike ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] (no subject)
What version of Python are you using? I believe Python 2.7 is the recommend version. I suggest you grab a fresh copy of galaxy from the stable branch and try again. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, May 3, 2013 at 12:24 AM, sridhar srinivasan sridhar2bioi...@gmail.com wrote: Dear All, i am installing galaxy first time. i face problem while installing the python eggs. while running the Run.sh command it gives as /home/sbsuser/.python-eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg-tmp/csamtools.so:6: RuntimeWarning: __builtin__.file size changed, may indicate binary incompatibility � b �n 64 0� c � � � �� ? �� � ��U �#k � ! \ j Z o� [ � � � [ � � �9 - �� ra # ��� �� 8n � �C �U�; B � S l K (p� �� U � � P w 5 ≠ ' H š è π ø * ─ ␋ Ð Ä @ ê i couldn't read the message and also i couldn't open the galaxy page http://localhost:8080 Could you please suggest tth epossibilities to install. Thanks in advance Regards Sridhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Load balancing and job configuration
Oliver, The new job running file is also providing me a lot of headaches. The documentation on site is correct, but some of the items are not yet implemented, for example, DRMAA external scripts still need to be in the universe_wsgi.ini, yet the site say to put them in job_conf.xml file! Lot of hair pulling and waiting response from IRC to figure this out...wasn't fun. As for your issue, you want to specify your handlers, not your webserver for your handler types. handler id=handler0 tags=handlers/ handler id=handler1 tags=handlers/ Let me know if this helps, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, May 2, 2013 at 9:00 AM, Olivier Inizan olivier.ini...@versailles.inra.fr wrote: Dear galaxy-dev-list, I am trying to use new-style job configuration whith a load-balancing on 2 web servers and 2 job hanlders. I have configured load balancing as follows (universe_wsgi.ini): # Configuration of the internal HTTP server. [server:web0] use = egg:Paste#http port = 8083 host = 127.0.0.1 use_threadpool = true threadpool_workers = 7 [server:web1] use = egg:Paste#http port = 8082 host = 127.0.0.1 use_threadpool = true threadpool_workers = 7 [server:manager] use = egg:Paste#http port = 8079 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 This configuration works fine (manager.log): galaxy.jobs.manager DEBUG 2013-04-23 12:20:22,901 Starting job handler galaxy.jobs.runners DEBUG 2013-04-23 12:20:22,902 Starting 5 LocalRunner workers galaxy.jobs DEBUG 2013-04-23 12:20:22,904 Loaded job runner 'galaxy.jobs.runners.local:LocalJobRunner' as 'local' galaxy.jobs.runners DEBUG 2013-04-23 12:20:22,909 Starting 3 LWRRunner workers galaxy.jobs DEBUG 2013-04-23 12:20:22,911 Loaded job runner 'galaxy.jobs.runners.lwr:LwrJobRunner' as 'lwr' galaxy.jobs DEBUG 2013-04-23 12:20:22,911 Legacy destination with id 'local:///', url 'local:///' converted, got params: galaxy.jobs.handler DEBUG 2013-04-23 12:20:22,911 Loaded job runners plugins: lwr:local galaxy.jobs.handler INFO 2013-04-23 12:20:22,912 job handler stop queue started galaxy.jobs.handler INFO 2013-04-23 12:20:22,919 job handler queue started To use new-style job configuration I have create the following job_conf.xml: ?xml version=1.0? !-- A sample job config that explicitly configures job running the way it is configured by default (if there is no explicit config). -- job_conf plugins plugin id=local type=runner load=galaxy.jobs.runners.local:LocalJobRunner/ /plugins handlers default=handlers !-- Additional job handlers - the id should match the name of a [server:id] in universe_wsgi.ini. -- handler id=server:web0 tags=handlers/ handler id=server:web1 tags=handlers/ /handlers destinations default=local destination id=local runner=local/ /destinations /job_conf When I restart the instance the manager.log outputs: galaxy.jobs DEBUG 2013-05-02 17:25:28,402 Loading job configuration from ./job_conf.xml galaxy.jobs DEBUG 2013-05-02 17:25:28,402 Read definition for handler 'server:web0' galaxy.jobs DEBUG 2013-05-02 17:25:28,403 Read definition for handler 'server:web1' galaxy.jobs DEBUG 2013-05-02 17:25:28,403 handlers default set to child with id or tag 'handlers' galaxy.jobs DEBUG 2013-05-02 17:25:28,403 destinations default set to child with id or tag 'local' galaxy.jobs DEBUG 2013-05-02 17:25:28,403 Done loading job configuration So everything seems fine but when I try to run a tool, the corresponding job is in permanent waiting status (gray color in history). I think I have missed something in the configuration process, Thanks for your help, Olivier ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tophat for Illumina is not running
Lilach, Is this on your own install of Galaxy or the public ones? If its on the public ones offered by the Galaxy community, I think you need to post to the users list. If its own your own custom install of Galaxy, you need to check that the job runners have been configured correctly. -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 30, 2013 at 12:29 AM, lilach noy lilach...@gmail.com wrote: Hello, I'm new to Galaxy so I think i'm at the right place asking this, though please let me know if I got it all wrong. I sent a Tophat query on Thursday about 5 days ago and it is still waiting to ran (gray background). Is to O.k? A bug? Should i keep waiting or stop everything and restart? If this is not the address for reporting a suspicious bug, whom does this concern? Thank you very much, Lilach ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running Galaxy through Apache
Jeff, How / Where did you add the apache changes? Are you using VirutalHosts or ... ? Quick glance, everything appears correct. Its been a while since I used Apache (switched everything to nginx) however I believe if you want to setup a Reverse Proxy, you need to add / use the proxy pass lines which does not appear in the Galaxy Wiki. Something like this should work: (untested...) IfModule mod_proxy.c ProxyRequests Off Proxy * AddDefaultCharset off Order Deny,Allow Deny from all Allow from *.ualberta.ca # Change / Remove as needed /Proxy ProxyPass /galaxy http://localhost:8080/ ProxyPassReverse /galaxy http://localhost:8080/ # Line below is optional Redirect permanent /galaxy http://mymachine.com/galaxy ProxyVia On /IfModule Let us know, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Fri, Apr 26, 2013 at 2:56 PM, Jeffrey Long jlo...@ualberta.ca wrote: Hi folks, I am wanting to run a galaxy instance and serve it at a location other than the webserver root, i.e. at http://mymachine.com/galaxy rather than just at http://mymachine. To do this, I've followed the instructions about proxying Galaxy through Apache at http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy. I'm pretty sure I've followed the instructions exactly. I enabled the necessary apache proxy modules and then I added these lines to my apache.conf file: RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /myplace/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /myplace/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /myplace/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /myplace/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /myplace/galaxy-dist/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] ...and re-started the apache server without a problem (note: I also tried it commenting out everything but the first, second and last line in case there was an issue with the static stuff). I then made sure the following lines were present and uncommented in my universe_wsgi.ini file [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /galaxy [app:main] filter-with = proxy-prefix cookie_path = /galaxy I start up Galaxy. And still, I can only reach it on http://mymachine:8080, not at http://mymachine/galaxy as I intend. Any help would be appreciated! Thanks! -Jeff ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Command Line Tool for Creating New Users
Howdy All, I was wondering if anyone has created a command line tool that will add users to the Galaxy user database? I did look into switching to external user authentication via nginx, however, this will result in us using the campus wide kerberos system and allowing ~30,000 people access to Galaxy... and thus offering us less fine grain control to Galaxy. Anything anyone has to share would be much appreciated, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem using FTP upload
Um... Lets see Jillian. Can the user who is running the Galaxy process access the folders in question? Whats the chown/chmod on the folders? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Apr 22, 2013 at 11:56 PM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Hi Adam, Nope, still getting the 'no files found'. Is there any troubleshooting tips you could give me for file permissions? Best, Jillian From: Adam Brenner [aebre...@uci.edu] Sent: Sunday, April 21, 2013 9:42 PM To: Jillian Rowe Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Problem using FTP upload Jillian, Make sure import/users is the full path. IE: /storage/import/users. If you are using the add-on provided by Jon in: http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html make sure you add: ftp_upload_dir_identifier = username to universe_wsgi.ini. Save and restart. We have the same setup on our HPC cluster. Users copy their data to the large storage bricks via scp, rsync, etc... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 21, 2013 at 12:00 AM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Hello, I would like to use the FTP upload option without actually having an FTP service setup. Instead users will scp to the directory. ftp_upload_dir = import/users ftp_upload_site = notnothing if a create a import/users/username with username as my login I still get the message: Your FTP upload directory contains no files. This Galaxy server allows you to upload files via FTP. To upload some files, log in to the FTP server at localhost using your Galaxy credentials (email address and password). According to this email tot he mailing list http://gmod.827538.n3.nabble.com/Configuring-Galaxy-for-FTP-upload-td3934194.html I thought I could do a setup such as I am trying now, but now I am not so sure. Any ideas? Best, Jillian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem using FTP upload
This is what I have configured for our cluster: ftp_upload_dir = /som/galaxy/users/ ftp_upload_dir_identifier = username (requires Jon's modification http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html ) ftp_upload_site = None ^^ can not be left blank. Any value should work. The folder setup/structure for /som/galaxy/users/ looks like so. /som/galaxy/users/aebrenne /som/galaxy/users/panteater . Both aebrenne and panteater are users within Galaxy, they just have the @domain.tld part left out. If the folders contain no files, then nothing will be displayed in Galaxy. It does support nested folders, etc... Hopefully that helps, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 23, 2013 at 1:11 AM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Well, I just tried as the root user and it still doesn't see them. I'm sure root has access everywhere. From: Adam Brenner [aebre...@uci.edu] Sent: Tuesday, April 23, 2013 10:13 AM To: Jillian Rowe Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Problem using FTP upload Um... Lets see Jillian. Can the user who is running the Galaxy process access the folders in question? Whats the chown/chmod on the folders? -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Mon, Apr 22, 2013 at 11:56 PM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Hi Adam, Nope, still getting the 'no files found'. Is there any troubleshooting tips you could give me for file permissions? Best, Jillian From: Adam Brenner [aebre...@uci.edu] Sent: Sunday, April 21, 2013 9:42 PM To: Jillian Rowe Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Problem using FTP upload Jillian, Make sure import/users is the full path. IE: /storage/import/users. If you are using the add-on provided by Jon in: http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html make sure you add: ftp_upload_dir_identifier = username to universe_wsgi.ini. Save and restart. We have the same setup on our HPC cluster. Users copy their data to the large storage bricks via scp, rsync, etc... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 21, 2013 at 12:00 AM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Hello, I would like to use the FTP upload option without actually having an FTP service setup. Instead users will scp to the directory. ftp_upload_dir = import/users ftp_upload_site = notnothing if a create a import/users/username with username as my login I still get the message: Your FTP upload directory contains no files. This Galaxy server allows you to upload files via FTP. To upload some files, log in to the FTP server at localhost using your Galaxy credentials (email address and password). According to this email tot he mailing list http://gmod.827538.n3.nabble.com/Configuring-Galaxy-for-FTP-upload-td3934194.htmlI thought I could do a setup such as I am trying now, but now I am not so sure. Any ideas? Best, Jillian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Run Jobs as Real User - Unable to get working with Drmaa (SGE) and job_conf.xml
Anyone have any ideas as to why its not running? Surely must be a few users who run Galaxy via a Scheduler using DRMAA... My updated job_conf.xml is like so: http://pastebin.com/raw.php?i=QJbay57i Jobs do get submitted to DRMAA, but the scripts are never called to run as real user. I have DEBUG set to true in universe_wsgi.ini and nothing regarding errors or debug message are created in paster.log about this. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 16, 2013 at 4:55 PM, Adam Brenner aebre...@uci.edu wrote: I have the following job_conf.xml file setup for our cluster: http://pastebin.com/raw.php?i=0uNhcVJk When running jobs from my admin account via Galaxy it runs under the same user as galaxy is, for testing, this as root. I need for it to run under the real user. The email address / galaxy account I am using to test with does match with a user on the cluster -- u...@example.com where user is a valid user account with correct permissions to run jobs on our cluster and queue. I followed the information here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User Any ideas as to why galaxy_external* is not being called? The paster.log file does not show anything useful regarding calling external scripts. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Run Jobs as Real User - Unable to get working with Drmaa (SGE) and job_conf.xml
Okay! I found out what the issue was. I am not sure if this is related to the new Galaxy setup (requiring job_conf.xml) but, I added the following lines to my universe_wsgi.ini, saved, restarted and it worked. drmaa_external_runjob_script = scripts/drmaa_external_runner.py drmaa_external_killjob_script = scripts/drmaa_external_killer.py external_chown_script = scripts/external_chown_script.py So the bigger question is: Is the current documentation wrong and adding drmaa to the job_conf.xml incorrect? If not, then what I am doing wrong in my file? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 23, 2013 at 10:47 AM, Adam Brenner aebre...@uci.edu wrote: Anyone have any ideas as to why its not running? Surely must be a few users who run Galaxy via a Scheduler using DRMAA... My updated job_conf.xml is like so: http://pastebin.com/raw.php?i=QJbay57i Jobs do get submitted to DRMAA, but the scripts are never called to run as real user. I have DEBUG set to true in universe_wsgi.ini and nothing regarding errors or debug message are created in paster.log about this. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Tue, Apr 16, 2013 at 4:55 PM, Adam Brenner aebre...@uci.edu wrote: I have the following job_conf.xml file setup for our cluster: http://pastebin.com/raw.php?i=0uNhcVJk When running jobs from my admin account via Galaxy it runs under the same user as galaxy is, for testing, this as root. I need for it to run under the real user. The email address / galaxy account I am using to test with does match with a user on the cluster -- u...@example.com where user is a valid user account with correct permissions to run jobs on our cluster and queue. I followed the information here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User Any ideas as to why galaxy_external* is not being called? The paster.log file does not show anything useful regarding calling external scripts. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem using FTP upload
Jillian, Make sure import/users is the full path. IE: /storage/import/users. If you are using the add-on provided by Jon in: http://dev.list.galaxyproject.org/user-names-with-authentication-td4659271.html make sure you add: ftp_upload_dir_identifier = username to universe_wsgi.ini. Save and restart. We have the same setup on our HPC cluster. Users copy their data to the large storage bricks via scp, rsync, etc... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 21, 2013 at 12:00 AM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Hello, I would like to use the FTP upload option without actually having an FTP service setup. Instead users will scp to the directory. ftp_upload_dir = import/users ftp_upload_site = notnothing if a create a import/users/username with username as my login I still get the message: Your FTP upload directory contains no files. This Galaxy server allows you to upload files via FTP. To upload some files, log in to the FTP server at localhost using your Galaxy credentials (email address and password). According to this email tot he mailing list http://gmod.827538.n3.nabble.com/Configuring-Galaxy-for-FTP-upload-td3934194.html I thought I could do a setup such as I am trying now, but now I am not so sure. Any ideas? Best, Jillian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] user names with authentication
We would also be looking forward to seeing a solution (or hack) like this. We currently have a a gluster filesystem setup where users drop big data to /gl/tmp/user/. Keeping this format would be best. However, I believe a simple symbolic link would also work with Galaxy. for i in $(ls /gl/tmp/user); do ln -s /gl/tmp/user /gl/tmp/u...@uci.edu ;done; -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 14, 2013 at 1:12 AM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Hello, I have a galaxy instance authentication using LDAP. The authentication works fine, and I would like to enable the user upload option. I would like to have: user_library_import_dir = /a/path and for galaxy to look for users with just the username. So if I log in with au...@example.com I want for galaxy to look in /a/path/auser instead of /a/path/ au...@example.com Is there anywhere I can set this? Best, Jillian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] user names with authentication
John, that worked! I was able to upload some sample files using usernames, rather then full email address. No errors, no bugs. Thanks! -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 18, 2013 at 2:18 PM, John Chilton chil...@msi.umn.edu wrote: Crazy coincidence, I had the same problem and hadn't even seen the original e-mail from Jillian. My pull request implementing this was merged by Nate earlier this week. https://bitbucket.org/galaxy/galaxy-central/pull-request/157/increased-flexibility-for-ftp-directory/diff You can just grab the three changesets out of there and apply them to your local galaxy repository. Then you just need to set the new `ftp_upload_dir_identifier` option in universe_wsgi.ini to `username`. Hope this helps. -John On Thu, Apr 18, 2013 at 4:07 PM, Adam Brenner aebre...@uci.edu wrote: We would also be looking forward to seeing a solution (or hack) like this. We currently have a a gluster filesystem setup where users drop big data to /gl/tmp/user/. Keeping this format would be best. However, I believe a simple symbolic link would also work with Galaxy. for i in $(ls /gl/tmp/user); do ln -s /gl/tmp/user /gl/tmp/u...@uci.edu;done; -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 14, 2013 at 1:12 AM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Hello, I have a galaxy instance authentication using LDAP. The authentication works fine, and I would like to enable the user upload option. I would like to have: user_library_import_dir = /a/path and for galaxy to look for users with just the username. So if I log in with au...@example.com I want for galaxy to look in /a/path/auser instead of /a/path/au...@example.com Is there anywhere I can set this? Best, Jillian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
So I finally figured it out (thanks to the help from the IRC folks, bag and natefoo). I incorrectly had my job_conf.xml setup where no job runners were set. All is working now. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sun, Apr 14, 2013 at 1:54 PM, Adam Brenner aebre...@uci.edu wrote: Anyone have any ideas? Still have not sorted this out -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 2:24 PM, Adam Brenner aebre...@uci.edu wrote: FYI: Here are the entries in paster.log. If you notice no job_handlers are being sent. I performed tests on both nginx reverse proxy and direct localhost:8080 (limited to our private network on campus) 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] GET /tool_runner?tool_id=upload1 HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/root/tool_menu; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool upload1 created job id 8 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] POST /tool_runner/index HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] GET /history HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/tool_runner/index; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] GET /api/histories/56959efe847bb360 HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/history; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner aebre...@uci.edu wrote: Howdy Galaxy Admins, I am installing Galaxy on our HPC cluster and running into a strange issue that I can no longer debug further. I am running the latest galaxy build as of Tuesday April 9th. hg clone https://bitbucket.org/galaxy/galaxy-dist/ hg update stable When a user tries to upload a file via the web interface, nginx reverse proxy setup as described here http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the upload stays in a state of Job is waiting to run. The job stays like this for both FTP uploads, and uploads via browser. The dataset is a 319 bytes (both FTP and upload via browser) My universe_wsgi.ini variables are like so: .. under [app:main] new_file_path = /som/galaxy/tmp file_path = /som/galaxy/datasets/files track_jobs_in_database = True job_manager = manager job_handlers = handler0,handler1 ... before [app:main] in [server:main] [server:main] use = egg:Paste#http port = 8080 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:manager] use = egg:Paste#http port = 8079 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 Galaxy, for now, is running under the root user and is installed in /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS system under /som/galaxy/. I have tried removing the extra job managers and handlers so I just have server:main and this still continues to happen. The paster.log files (with DEBUG = True) does not show any errors from python or issues writing to file. All POST and GET responses are recorded and nothing else. What I find strange is that if I look in new_file_path I do see my dataset as upload_file_data_Zx_1rK and cat'ing the file shows the correct contents. The items listed in my file_path (the dataset_*.dat) files are empty/blank..but are created. Any ideas as to why its failing? Is it nginx? I only have
[galaxy-dev] Run Jobs as Real User - Unable to get working with Drmaa (SGE) and job_conf.xml
I have the following job_conf.xml file setup for our cluster: http://pastebin.com/raw.php?i=0uNhcVJk When running jobs from my admin account via Galaxy it runs under the same user as galaxy is, for testing, this as root. I need for it to run under the real user. The email address / galaxy account I am using to test with does match with a user on the cluster -- u...@example.com where user is a valid user account with correct permissions to run jobs on our cluster and queue. I followed the information here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User Any ideas as to why galaxy_external* is not being called? The paster.log file does not show anything useful regarding calling external scripts. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
Anyone have any ideas? Still have not sorted this out -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 2:24 PM, Adam Brenner aebre...@uci.edu wrote: FYI: Here are the entries in paster.log. If you notice no job_handlers are being sent. I performed tests on both nginx reverse proxy and direct localhost:8080 (limited to our private network on campus) 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] GET /tool_runner?tool_id=upload1 HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/root/tool_menu; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool upload1 created job id 8 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] POST /tool_runner/index HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] GET /history HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/tool_runner/index; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] GET /api/histories/56959efe847bb360 HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/history; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner aebre...@uci.edu wrote: Howdy Galaxy Admins, I am installing Galaxy on our HPC cluster and running into a strange issue that I can no longer debug further. I am running the latest galaxy build as of Tuesday April 9th. hg clone https://bitbucket.org/galaxy/galaxy-dist/ hg update stable When a user tries to upload a file via the web interface, nginx reverse proxy setup as described here http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the upload stays in a state of Job is waiting to run. The job stays like this for both FTP uploads, and uploads via browser. The dataset is a 319 bytes (both FTP and upload via browser) My universe_wsgi.ini variables are like so: .. under [app:main] new_file_path = /som/galaxy/tmp file_path = /som/galaxy/datasets/files track_jobs_in_database = True job_manager = manager job_handlers = handler0,handler1 ... before [app:main] in [server:main] [server:main] use = egg:Paste#http port = 8080 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:manager] use = egg:Paste#http port = 8079 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 Galaxy, for now, is running under the root user and is installed in /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS system under /som/galaxy/. I have tried removing the extra job managers and handlers so I just have server:main and this still continues to happen. The paster.log files (with DEBUG = True) does not show any errors from python or issues writing to file. All POST and GET responses are recorded and nothing else. What I find strange is that if I look in new_file_path I do see my dataset as upload_file_data_Zx_1rK and cat'ing the file shows the correct contents. The items listed in my file_path (the dataset_*.dat) files are empty/blank..but are created. Any ideas as to why its failing? Is it nginx? I only have it redirect 8080, and none of the other job managers / handlers...is that an issue? Do you think its permissions on the my file_path or new_file_path directories? A very stumped Adam. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu
[galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
Howdy Galaxy Admins, I am installing Galaxy on our HPC cluster and running into a strange issue that I can no longer debug further. I am running the latest galaxy build as of Tuesday April 9th. hg clone https://bitbucket.org/galaxy/galaxy-dist/ hg update stable When a user tries to upload a file via the web interface, nginx reverse proxy setup as described here http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the upload stays in a state of Job is waiting to run. The job stays like this for both FTP uploads, and uploads via browser. The dataset is a 319 bytes (both FTP and upload via browser) My universe_wsgi.ini variables are like so: .. under [app:main] new_file_path = /som/galaxy/tmp file_path = /som/galaxy/datasets/files track_jobs_in_database = True job_manager = manager job_handlers = handler0,handler1 ... before [app:main] in [server:main] [server:main] use = egg:Paste#http port = 8080 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:manager] use = egg:Paste#http port = 8079 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 Galaxy, for now, is running under the root user and is installed in /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS system under /som/galaxy/. I have tried removing the extra job managers and handlers so I just have server:main and this still continues to happen. The paster.log files (with DEBUG = True) does not show any errors from python or issues writing to file. All POST and GET responses are recorded and nothing else. What I find strange is that if I look in new_file_path I do see my dataset as upload_file_data_Zx_1rK and cat'ing the file shows the correct contents. The items listed in my file_path (the dataset_*.dat) files are empty/blank..but are created. Any ideas as to why its failing? Is it nginx? I only have it redirect 8080, and none of the other job managers / handlers...is that an issue? Do you think its permissions on the my file_path or new_file_path directories? A very stumped Adam. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
FYI: Here are the entries in paster.log. If you notice no job_handlers are being sent. I performed tests on both nginx reverse proxy and direct localhost:8080 (limited to our private network on campus) 128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] GET /tool_runner?tool_id=upload1 HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/root/tool_menu; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool upload1 created job id 8 128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] POST /tool_runner/index HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] GET /history HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/tool_runner/index; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] GET /api/histories/56959efe847bb360 HTTP/1.0 200 - http://ghtf-hpc.oit.uci.edu/history; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner aebre...@uci.edu wrote: Howdy Galaxy Admins, I am installing Galaxy on our HPC cluster and running into a strange issue that I can no longer debug further. I am running the latest galaxy build as of Tuesday April 9th. hg clone https://bitbucket.org/galaxy/galaxy-dist/ hg update stable When a user tries to upload a file via the web interface, nginx reverse proxy setup as described here http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the upload stays in a state of Job is waiting to run. The job stays like this for both FTP uploads, and uploads via browser. The dataset is a 319 bytes (both FTP and upload via browser) My universe_wsgi.ini variables are like so: .. under [app:main] new_file_path = /som/galaxy/tmp file_path = /som/galaxy/datasets/files track_jobs_in_database = True job_manager = manager job_handlers = handler0,handler1 ... before [app:main] in [server:main] [server:main] use = egg:Paste#http port = 8080 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:manager] use = egg:Paste#http port = 8079 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 Galaxy, for now, is running under the root user and is installed in /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS system under /som/galaxy/. I have tried removing the extra job managers and handlers so I just have server:main and this still continues to happen. The paster.log files (with DEBUG = True) does not show any errors from python or issues writing to file. All POST and GET responses are recorded and nothing else. What I find strange is that if I look in new_file_path I do see my dataset as upload_file_data_Zx_1rK and cat'ing the file shows the correct contents. The items listed in my file_path (the dataset_*.dat) files are empty/blank..but are created. Any ideas as to why its failing? Is it nginx? I only have it redirect 8080, and none of the other job managers / handlers...is that an issue? Do you think its permissions on the my file_path or new_file_path directories? A very stumped Adam. -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
