Re: [galaxy-dev] Unsorted BAM file --> Galaxy crash?

2016-05-25 Thread Raymond Wan
Hello Peter and everyone,

Just wanted to update all of you that replacing samtools with 1.2
solved the problem.  As you suggested, we didn't replace the
system-installed one at /usr/bin (though I suppose we could have) but
modified the environment variable.

I don't know the details of the recovery since I look after Galaxy and
not the overall system (i.e., I'm not the root user) but it sounds
like the administrators didn't need to perform any job deletions
and/or Galaxy database editing.  Once samtools was replaced, the
system managed to restart correctly.

Honestly, we never would have guessed that using an old version of
samtools would cause Galaxy to crash.  Even with the log files
alluding to it, we were initially in disbelief.

And thanks to Christian for pointing out the main cause of the problem
in the source code.  We had thought (hoped?) that Galaxy was a
self-contained system.  So, this experience helped us better
understand how Galaxy works.

Thank you all for your prompt help!

Ray


On Tue, May 24, 2016 at 7:04 PM, Peter Briggs
 wrote:
> Just to clarify, for us it wasn't dependent on the tool - it was actually
> the version of samtools installed in the 'galaxy' user's environment i.e. on
> the 'galaxy' user's PATH.
>
> I set it explicitly by creating a file called "local_env.sh" in the 'config'
> directory of the Galaxy installation with the following content:
>
> $ cat local_env.sh
> # Prepend samtools 1.2
> export PATH=$HOME/apps/samtools/1.2/bin:$PATH
> ##
> #
>
> which is automatically picked up by 'run.sh' when Galaxy is started (I don't
> know how that plays if you're using a uwsgi/supervisor setup).
>
> Best wishes
>
> Peter
>
>
> On 24/05/16 11:46, Raymond Wan wrote:
>>
>> Hi Peter,
>>
>>
>> On Tue, May 24, 2016 at 6:29 PM, Peter Briggs
>>  wrote:
>>>
>>> One of my colleagues encountered a similar-sounding error on our local
>>> test
>>> instance, where an unsorted BAM file seemed to crash the Galaxy handler
>>> processes and prevented them from restarting.
>>>
>>> In our case the default samtools version in the Galaxy environment was
>>> 0.1.18; we found that updating this to samtools 1.2 fixed the crash
>>> problem.
>>
>>
>>
>> Ah!  That does sound familiar.  As I just replied in another message,
>> we seem to have at least 5 versions of samtools in the system.  I'm
>> not sure if that is normal or not for a Galaxy instance...
>>
>> Let me see if I can figure out (through logs, but most likely by
>> asking the user) what tool he was using and figure out which samtools
>> executable was being run.  If it is 0.1.18, I'll look into update
>> it...thank you for sharing your colleague's experience!
>>
>> Ray
>>
>
> --
> Peter Briggs peter.bri...@manchester.ac.uk
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Question: How to limit the number of dispatched jobs in Galaxy?

2016-05-25 Thread Dannon Baker
Hey George,

There sure is.  Let me know if this wiki page (
https://wiki.galaxyproject.org/Admin/Config/Jobs#Limiting_Job_Resource_Usage
)  doesn't have all the info you need and I can try to help more.

-Dannon

On Wed, May 25, 2016 at 4:04 PM George Weingart 
wrote:

> Hi
>
> Is there a way to  control the demand that Galaxy is putting on the server
> by limiting the number of jobs that will dispatch at a time ?
>
> Thanks!
> George Weingart PhD
> Huttenhower Lab
> Harvard School of Public Health
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Question: How to limit the number of dispatched jobs in Galaxy?

2016-05-25 Thread George Weingart
Hi

Is there a way to  control the demand that Galaxy is putting on the server
by limiting the number of jobs that will dispatch at a time ?

Thanks!
George Weingart PhD
Huttenhower Lab
Harvard School of Public Health
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] STACKS error

2016-05-25 Thread Ralf Weber
Hi Yvan,

Thanks for including me in this conversation. I'll keep an eye on this and
will make the necessary changes in our next commit/version.

Best wishes,

Ralf


On 20 May 2016 at 23:28, Yvan Le Bras  wrote:

> Thank you John for the rapid answer and the PR!
>
> I have had the same issue trying installing and use a Galaxy-M instance (
> 
> https://gigascience.biomedcentral.com/articles/10.1186/s13742-016-0115-8)
> . I cc Ralf concerning this.
>
> Cheers,
>
> Yvan
>
> --
>
> *De: *"John Chilton" 
> *À: *"Yvan Le Bras" 
> *Cc: *"Amogelang Raphenya" ,
> galaxy-dev@lists.galaxyproject.org
> *Envoyé: *Vendredi 20 Mai 2016 17:15:19
> *Objet: *Re: [galaxy-dev] STACKS error
>
> This is one the reasons (though not the only one) I generally
> discourage putting datatypes in the tool shed - Galaxy isn't very
> clear about what interface it provides to datatypes and it would seem
> very difficult from my perspective to maintain that interface and grow
> Galaxy at the same time.
>
> This checkers module though is clearly useful to datatypes and it is
> quite trivial to provide backward compatibility so I've opened a PR to
> the new 16.04 release to restore this module -
> https://github.com/galaxyproject/galaxy/pull/2389.
>
> Sorry for the inconvenience.
>
> -John
>
>
> On Fri, May 20, 2016 at 10:45 AM, Yvan Le Bras 
> wrote:
> > Hi Amos,
> >
> > Thanks to use Stacks on Galaxy! I already seen this error. This is due, I
> > think, to newest Galaxy versions who don't have the checkers.py anymore
> on
> > the datatypes folder Thios can be fixed using an older Galaxy
> version...
> > Otherwise, we are working on a new version of the Stacks Galaxy tools...
> who
> > will be usable in the upcoming days or not ;)
> >
> > Cheers,
> >
> > Yvan
> >
> >
> > 
> >
> > De: "Amogelang Raphenya" 
> > À: galaxy-dev@lists.galaxyproject.org
> > Envoyé: Vendredi 20 Mai 2016 16:37:04
> > Objet: [galaxy-dev] STACKS error
> >
> >
> > Anyone encountered this error?
> >
> > Fatal error: Exit code 1 (Error in Stacks Process radtag execution)
> > Traceback (most recent call last):
> >   File
> > "/galaxylab/production/new/shed_tools/
> toolshed.g2.bx.psu.edu/repos/cmonjeau/stacks/0e0ff9e9c761/stacks/STACKS_procrad.py
> ",
> > line 16, in 
> > from stacks import *
> >   File
> > "/galaxylab/production/new/shed_tools/
> toolshed.g2.bx.psu.edu/repos/cmonjeau/stacks/0e0ff9e9c761/stacks/stacks.py
> ",
> > line 19, in 
> > from galaxy.datatypes.checkers import *
> > ImportError: No module named checkers
> >
> >
> > Cheers.
> >
> >
> > Amos Raphenya, B.Eng
> > Bioinformatics Software Developer
> > Department of Biochemistry & Biomedical Sciences
> > McMaster University, MDCL 2317
> > p: (905) 525-9140
> > a: 1280 Main St W.,Hamilton, Ontario, Canada L8S 4K1
> > e:  raphe...@mcmaster.ca
> > skype: amos_raphenya
> >
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >   https://lists.galaxyproject.org/
> >
> > To search Galaxy mailing lists use the unified search at:
> >   http://galaxyproject.org/search/mailinglists/
> >
> >
> >
> >
> > --
> >
> -
> > Yvan Le Bras, PhD
> > @Yvan2935
> > <°><
> >   e-Biogenouest project
> > http://www.e-biogenouest.org
> >CNRS UMR 6074 IRISA-INRIA, Campus de Beaulieu,
> 35042
> > Rennes Cedex
> >  tél.:  +33 (0) 2 99 84 71 79 /
>  +33
> > (0) 6.10.43.96.51
> >
> > yvan.le_b...@irisa.fr
> >
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >   https://lists.galaxyproject.org/
> >
> > To search Galaxy mailing lists use the unified search at:
> >   http://galaxyproject.org/search/mailinglists/
>
>
>
>
> --
>
> -
> Yvan Le Bras, PhD
>
> @Yvan2935
>
>   <°><
>   e-Biogenouest project
>http://www.e-biogenouest.org
>CNRS UMR 6074 

[galaxy-dev] setting up a local galaxy instance - need sftp help

2016-05-25 Thread Jessica Grant
Hello,

I have a local instance running - I see the page with instructions for
setting up an ftp server, but my institution prefers I use sftp, which is
already installed.   Can I get help/info about configuring galaxy to accept
this?

Thanks!

Jessica





-- 
Jessica Grant
Research Associate, Williams Lab

205C Ford Hall
Smith College
Northampton, MA
413-585-4341
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] sftp over proftpd and postgresql

2016-05-25 Thread Durdevic Marija
Dear all,

I am trying to setup sftp connection for my new Galaxy server. I managed to 
configure sftp_module and I am able to connect to the server as an system user. 
The problem is that I am not able to login there as Galaxy user. I guess that 
the problem is with settings for backend and postgresql, but I don't know how 
to solve it.

My configuration for sql is:

# Do not authenticate against real (system) users
AuthPAM off

# Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1
SQLLogFile  /home/galaxy/galaxy/log_files/sqlLog.txt
SQLPasswordEngine   on
SQLPasswordEncoding hex

# Set up mod_sql to authenticate against the Galaxy database
SQLEngine   on

SQLBackend  postgres
SQLConnectInfo  galaxy@localhost:5432 galaxy
SQLAuthTypesSHA1
SQLAuthenticate users

# An empty directory in case chroot fails
SQLDefaultUID   1001
SQLDefaultGID   1001

SQLDefaultHomedir   /var/lib/proftpd/empty


# Define a custom query for lookup that returns a passwd-like entry.  UID and 
GID should match your Galaxy user.
SQLUserInfo custom:/LookupGalaxyUser
SQLNamedQuery   LookupGalaxyUser SELECT 
"email,password,'1001','1001','/home/data/galaxy_ftp/%U','/bin/bash' FROM 
galaxy_user WHERE email='%U'"


Db name is galaxy, user is galaxy, no pass.
Any recommendation, please?
Regards,


Mag. Marija Đurđević
Core Facility Computational Bioanalytics

Medical University of Graz
Center for Medical Research
Stiftingtalstraße 24, A-8010 Graz
Austria

Phone: +43 316/385-73024
Fax:+43 316/385-73009

Email: marija.djurdje...@medunigraz.at
Email: marija.djurdje...@klinikum-graz.at

Web: https://zmf.medunigraz.at/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] not sure what I can do about this...

2016-05-25 Thread Fernandez Edgar
Hello Peter,

For now, my new installation of galaxy v16.01 is on one test desktop running 
Oracle Linux 6.
Before changing my production installation, I'm doing my due diligent.

Let me know what you think...

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
I  Bur. : 1-514-343-6111 poste 16568



-Original Message-
From: Peter Cock [mailto:p.j.a.c...@googlemail.com] 
Sent: May-24-16 5:40 PM
To: Fernandez Edgar 
Cc: galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] not sure what I can do about this...

Hi Edgar,

Are your Galaxy server AND your cluster are all running Oracle Linux 6?

If yes, my guess about the cluster using an older version of Linux compared to 
the Galaxy server is unrelated.

Peter

On Tue, May 24, 2016 at 7:47 PM, Fernandez Edgar  
wrote:
> Hello Peter,
>
> What would your suggestion be?
> Wait until a new version of galaxy is compatible with my Oracle Linux 6 ?
>
> Best regards,
>
> Edgar Fernandez
> System Administrator (Linux)
> Direction Générale des Technologies de l'Information et de la 
> Communication Université de Montréal
>
> PAVILLON ROGER-GAUDRY, bureau X-210
> I  Bur. : 1-514-343-6111 poste 16568
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Current history via the REST API (or blend4j)

2016-05-25 Thread Suderman Keith

> On May 24, 2016, at 11:14 PM, Carl Eberhard  wrote:
> 
> most_recently_used *was* broken - thanks for finding that.

Glad I could be of assistance!  Just to clarify, the user has to actually *do* 
something with a history (i.e. add something to the history) for it to be 
considered the MRU; simply selecting switching to a history (without 
using/adding to it) is not sufficient to make it the MRU history. Correct?

> A planning issue for a session API is here: 
> https://github.com/galaxyproject/galaxy/issues/2404 
> 
I will let you folks decide if a session API is appropriate, however I was 
surprised that something like /api/histories/current wasn’t an available 
endpoint. For my purposes I think this would be sufficient and this way I 
wouldn’t have to keep track of the user’s API key as well as an additional 
session cookie.

Also, I should likely open an issue for this, but trying to access 
http://my.galaxy.instance/history/current_history_json?key=myapikey results in 
a "HTTP 500 Server Error” response.  I realize (now) that this shouldn’t work, 
but it likely should result in a server error either.

Cheers,
Keith

> 
> On Tue, May 24, 2016 at 4:16 PM, Carl Eberhard  > wrote:
> Yeah - I understand what you're trying to do now and you're right. In an 
> effort to make the API stateless I think we've introduced a gap in 
> functionality for a good, practical use case.
> 
> I've been wanting to replace the history/current_history* endpoints for a 
> while now with a session API (that would include a 'current' history) so I'll 
> bump that up in priority.
> 
> The most_recently_used may be broken - I'll check that now. Note, though, 
> that changing the current history doesn't change the update time for a given 
> history. You have to alter/change it for the time to change - then it will 
> become the 'most recently used'. It's confusing and doesn't really match user 
> expectations. I'll see what I can do about that and possibly make it an alias 
> to the above API when/if that makes it to a release. In any event, again, 
> I'll double check that it's at least doing that.
> 
> I'll also take a look and see if there's some other option for something like 
> this if most_recently_used won't fit the use case.
> 
> 
> 
> On Tue, May 24, 2016 at 2:16 AM, Suderman Keith  > wrote:
> 
>> On May 23, 2016, at 3:34 PM, Carl Eberhard > > wrote:
>> 
>> There's a way to get the current history data through a non-api url:
>> /history/current_history_json
> 
> That works, but to get the current history for a user I need to set the 
> galaxysession cookie. So it is not really a good solution unless I can find 
> an api to determine the session cookie for the user.  I’ve also noticed that 
> /api/histories/most_recently_used is not returning the most recently used 
> history!  A bug maybe?
> 
> Keith
> 
>> 
>> ...but, I don't believe those are accessible through any of the bio/blend 
>> libraries.
>> 
>> Failing the above, is there some way to get the history ID from the UI?
>> 
>> You can click the information button ('i' with a circle around it) for any 
>> dataset in a history. The encoded history API id will be listed under: 
>> history api id.
>> 
>> On Sat, May 21, 2016 at 7:03 PM, Suderman Keith > > wrote:
>> I have been looking at the JavaDoc for Blend4J [1] and the REST API [2], but 
>> I don’t see any way to obtain the “current” history as selected by the user 
>> in the Galaxy UI.  I see `/api/histories/most_recently_used`, but it seems 
>> that simply selecting a history as the current history in the UI does not 
>> make it the “most recently used” (MRU) history; I guess the user has to 
>> actually _do_ something with before it becomes the MRU history.  Am I 
>> missing something?
>> 
>> Failing the above, is there some way to get the history ID from the UI?
>> 
>> Thanks,
>> Keith
>> 
>> REFERENCES
>> 
>> 1.http://jmchilton.github.io/blend4j/apidocs/ 
>> 
>> 2. 
>> https://galaxy.readthedocs.io/en/master/lib/galaxy.webapps.galaxy.api.html#module-galaxy.webapps.galaxy.api.histories
>>  
>> 
>> 
>> --
>> Research Associate
>> Department of Computer Science
>> Vassar College
>> Poughkeepsie, NY
>> 
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/ 
>>