Re: [galaxy-dev] approximate line numbers shown in history items can be very imprecise
Hi Dannon, Thanks for the information. Yes, I know the line number tool, but running this for each file a user is interested in getting a line number is a pain (and confusing because the history items double), because in our case (probably not rare for real analysis tasks), this information is almost always needed to watch if anything funny happens. Maybe it is possible to add a link near the approximate line number, so that the user can get the accurate numbers more easily if desired? Chaolin On Jul 31, 2011, at 11:38 AM, Dannon Baker wrote: Chaolin, You guessed correctly as to why we implemented this, getting exact line counts on very large files is a time consuming process. You can still get an exact line count using the Line/Word/Character count tool in the Text Manipulation section. If you're interested in the way it currently works, the first 1MB of a large file is read, and a line number approximation is made from that and the assumption that line lengths don't vary dramatically throughout the file. It would slow down the metadata setting, but for a personal galaxy instance you could certainly increase that number, or disable the estimation entirely. -Dannon On Jul 31, 2011, at 11:11 AM, Chaolin Zhang wrote: Hi, I noticed that the current version of galaxy shows approximate number of lines for history items, when it is relatively big. I guess this is due to a consideration of performance, but it is quite annoying, because the exact line numbers provide a very easy way for users to get simple statistics. Sometimes the approximation can be really off. For instance, for one file with 1 M lines, it shows ~850,000 lines. Any thought? Chaolin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] approximate line numbers shown in history items can be very imprecise
Hi Dannon, We do have a local mirror here. How to disable the estimation? Thanks! Chaolin On Jul 31, 2011, at 11:38 AM, Dannon Baker wrote: Chaolin, You guessed correctly as to why we implemented this, getting exact line counts on very large files is a time consuming process. You can still get an exact line count using the Line/Word/Character count tool in the Text Manipulation section. If you're interested in the way it currently works, the first 1MB of a large file is read, and a line number approximation is made from that and the assumption that line lengths don't vary dramatically throughout the file. It would slow down the metadata setting, but for a personal galaxy instance you could certainly increase that number, or disable the estimation entirely. -Dannon On Jul 31, 2011, at 11:11 AM, Chaolin Zhang wrote: Hi, I noticed that the current version of galaxy shows approximate number of lines for history items, when it is relatively big. I guess this is due to a consideration of performance, but it is quite annoying, because the exact line numbers provide a very easy way for users to get simple statistics. Sometimes the approximation can be really off. For instance, for one file with 1 M lines, it shows ~850,000 lines. Any thought? Chaolin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] exporting environment variables to SGE in galaxy
Hi Shantanu,
Thanks for the information! This did the trick.
Chaolin
On Jul 30, 2011, at 1:28 AM, Shantanu Pavgi wrote:
On Jul 29, 2011, at 11:28 PM, Chaolin Zhang wrote:
Hi,
I am setting up SGE in our galaxy mirror. One problem I have is that I
cannot export environment variables of the specific users running the galaxy
service. On command line, I did this by qsub -V script.sh. or add a line
#$ -V
in script.sh
I tried to change
/lib/galaxy/jobs/runners/sge.py and lib/galaxy/jobs/runners/drmaae.py
by adding a line
#$ -V
under
sge_template = #!/bin/sh
#$ -S /bin/sh
but this did not help.
Any idea is very appreciated!
Chaolin
Chaolin,
I don't think you will need to modify source code for this. Adding -V
option to the drmaa URL should be sufficient. For example:
{{{
drmaa:// -V /
}}}
Hope this helps.
--
Shantanu.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] exporting environment variables to SGE in galaxy
Hi, I am setting up SGE in our galaxy mirror. One problem I have is that I cannot export environment variables of the specific users running the galaxy service. On command line, I did this by qsub -V script.sh. or add a line #$ -V in script.sh I tried to change /lib/galaxy/jobs/runners/sge.py and lib/galaxy/jobs/runners/drmaae.py by adding a line #$ -V under sge_template = #!/bin/sh #$ -S /bin/sh but this did not help. Any idea is very appreciated! Chaolin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] organizing histories in galaxy
Hi, Is there a way to organize related histories together into a project or folder? I realize in a lot of cases, a pipeline is designed for processing of a single sample, while a study typically consists of multiple samples that go through the same processing. They are then put together for more downstream analysis. It seems to me that it would be logical to put the processing of each individual file into a history, and the combined analysis into another, which would be bundled together. I, and other users here have a growing list of histories, and it is becoming more and more difficult to organize them ... Thanks! Chaolin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] how to purge histories/datasets not accessed/updated for a certain time
Hi, We have a local mirror of the galaxy system and the disk is occupied really quickly. Is there a way to purge histories/datasets not accessed/updated for a certain period of time, no matter if they are deleted by the user? It looks like the current scripts for clean up only purges deleted histories/datasets. Thanks! Chaolin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
