Re: [galaxy-dev] Dataset Cleanup

[McCully, Dwayne (NIH/NIAMS) [C]]

Strange since I've deleted almost everything from my Galaxy account and it 
still says 59 GB and 
another user is having the same problem.   Is there a way to tie the dataset or 
history to the user?
I can then ask them to delete them.

Dwayne

-Original Message-
From: Björn Grüning [mailto:bjoern.gruen...@gmail.com] 
Sent: Thursday, January 05, 2017 9:43 AM
To: McCully, Dwayne (NIH/NIAMS) [C] ; 
galaxy-dev@lists.galaxyproject.org
Subject: Re: [galaxy-dev] Dataset Cleanup

Hi,

this script only deletes files that are marked as deleted. So it might be that 
users have old histories and this data will not be removed.

For you account and the used-space. Please go to your admin section, to users, 
your user-name and there should be an option to recalculate the disc-usage for 
your user.

Hope this helps,
Bjoern

Am 03.01.2017 um 16:41 schrieb McCully, Dwayne (NIH/NIAMS) [C]:
> Hello Everyone,
> 
>  
> 
> I have datasets that go back over a year in the files/000 directory 
> but running the following script does not delete the datasets.
> 
> Also, my account says I have 59 GB used and can’t figure out where 
> it’s getting that number.  I have almost everything deleted
> 
> in my Galaxy account.
> 
>  
> 
> galaxy_clean.sh
> 
> #!/bin/sh -x
> 
>  
> 
> /home/niamsgalaxy/galaxy/scripts/cleanup_datasets/delete_userless_hist
> ories.sh
> 
> /home/niamsgalaxy/galaxy/scripts/cleanup_datasets/purge_histories.sh
> 
> /home/niamsgalaxy/galaxy/scripts/cleanup_datasets/purge_libraries.sh
> 
> /home/niamsgalaxy/galaxy/scripts/cleanup_datasets/purge_folders.sh
> 
> /home/niamsgalaxy/galaxy/scripts/cleanup_datasets/delete_datasets.sh
> 
> /home/niamsgalaxy/galaxy/scripts/cleanup_datasets/purge_datasets.sh
> 
> Exit
> 
>  
> 
>  
> 
> Any information on this issue would be appreciated.
> 
>  
> 
> Dwayne
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this and other 
> Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Dataset Cleanup

[McCully, Dwayne (NIH/NIAMS) [C]]

Hello Everyone,

I have datasets that go back over a year in the files/000 directory but running 
the following script does not delete the datasets.
Also, my account says I have 59 GB used and can't figure out where it's getting 
that number.  I have almost everything deleted
in my Galaxy account.

galaxy_clean.sh
#!/bin/sh -x

/home/niamsgalaxy/galaxy/scripts/cleanup_datasets/delete_userless_histories.sh
/home/niamsgalaxy/galaxy/scripts/cleanup_datasets/purge_histories.sh
/home/niamsgalaxy/galaxy/scripts/cleanup_datasets/purge_libraries.sh
/home/niamsgalaxy/galaxy/scripts/cleanup_datasets/purge_folders.sh
/home/niamsgalaxy/galaxy/scripts/cleanup_datasets/delete_datasets.sh
/home/niamsgalaxy/galaxy/scripts/cleanup_datasets/purge_datasets.sh
Exit


Any information on this issue would be appreciated.

Dwayne
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Uploading files to Galaxy with nginx proxy server

[McCully, Dwayne (NIH/NIAMS) [C]]

Hello Everyone,

Uploading a file does not complete with a listing of the file in the history 
section.  The file upload does reach 100 percent.
Files are successfully being upload but not displayed.

I'm getting the below error In my pastore.log file.


galaxy.tools ERROR 2015-10-30 11:00:53,471 Checking parameter files failed. 
'UploadDataset' object has no attribute 'from_html'
galaxy.tools ERROR 2015-10-30 11:00:53,589 tools::to_json() - Skipping 
parameter expansion for files.
Traceback (most recent call last):
  File "/home/niamsgalaxy/galaxy/lib/galaxy/tools/__init__.py", line 2550, in 
iterate
tool_dict = input.to_dict(trans, other_values=other_values)
TypeError: to_dict() got an unexpected keyword argument 'other_values'

Any hints would be appreciated.

Dwayne
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Uploading files to Galaxy with nginx proxy server

[McCully, Dwayne (NIH/NIAMS) [C]]

Hello Everyone,

Uploading a file does not complete with a listing of the file in the history 
section.  The file upload does reach 100 percent.

I'm getting the below error In my pastore.log file.


galaxy.tools ERROR 2015-10-30 11:00:53,471 Checking parameter files failed. 
'UploadDataset' object has no attribute 'from_html'
galaxy.tools ERROR 2015-10-30 11:00:53,589 tools::to_json() - Skipping 
parameter expansion for files.
Traceback (most recent call last):
  File "/home/niamsgalaxy/galaxy/lib/galaxy/tools/__init__.py", line 2550, in 
iterate
tool_dict = input.to_dict(trans, other_values=other_values)
TypeError: to_dict() got an unexpected keyword argument 'other_values'

Any hints would be appreciated.

Dwayne
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] MS Active Directory Authentication

[McCully, Dwayne (NIH/NIAMS) [C]]

Thanks Nicola,

Updated the auth_config.xml  file but got the following message in the 
paster.log.   Any idea how to fix?

DEBUG 2015-10-13 14:18:11,644 LDAP authenticate: could not load ldap module

Dwayne

From: Nicola Soranzo [mailto:nicola.sora...@gmail.com] On Behalf Of Nicola 
Soranzo
Sent: Tuesday, October 13, 2015 1:44 PM
To: McCully, Dwayne (NIH/NIAMS) [C] ; 
galaxy-dev@lists.galaxyproject.org
Subject: Re: [galaxy-dev] MS Active Directory Authentication

Hi Dwayne,

see https://github.com/galaxyproject/galaxy/blob/dev/config/auth_conf.xml.sample

ldap is equivalent to activedirectory

Something like this may work for you:



activedirectory

False
True
ldap://dc1.example.com
True
sAMAccountName,mail
dc=dc1,dc=example,dc=com

(&(objectClass=user)(sAMAccountName={username}))
jsmith
mysecret
{sAMAccountName}
{password}
{sAMAccountName}
{mail}




Cheers,
Nicola
On 13/10/15 17:44, McCully, Dwayne (NIH/NIAMS) [C] wrote:
Hello Everyone,

Is there an example of the auth_config.xml for a MS Active directory connection?
I would like to all my users to register and login to Galaxy with their AD 
username and password.

Note: We don’t use a “@example.com” extension for our usernames just “username”.

Thanks in advance

Dwayne




___

Please keep all replies on the list by using "reply all"

in your mail client.  To manage your subscriptions to this

and other Galaxy lists, please use the interface at:

  https://lists.galaxyproject.org/



To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] MS Active Directory Authentication

[McCully, Dwayne (NIH/NIAMS) [C]]

Hello Everyone,

Is there an example of the auth_config.xml for a MS Active directory connection?
I would like to all my users to register and login to Galaxy with their AD 
username and password.

Note: We don't use a "@example.com" extension for our usernames just "username".

Thanks in advance

Dwayne
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Tophat run stops

[McCully, Dwayne (NIH/NIAMS) [C]]

When running a Tophat job for over a hour, I get the following message and the 
job stops.
Would anyone know why and is there a way to stop Galaxy from cleaning up the 
job working
directory so that I can check the error log files?

Dwayne

galaxy.jobs.runners.local DEBUG 2015-07-14 16:54:10,478 execution finished: 
/data/galaxy/galaxy_prod/database/job_working_directory/000/26/galaxy_26.sh
galaxy.jobs.output_checker INFO 2015-07-14 16:54:10,595 Job 26: Log: tool 
progress
galaxy.jobs.output_checker INFO 2015-07-14 16:54:10,595 Job 26: Log: tool 
progress
galaxy.datatypes.metadata DEBUG 2015-07-14 16:54:10,675 setting metadata 
externally failed for HistoryDatasetAssociation 32: [Errno 2] No such file or 
directory
galaxy.jobs.runners.local ERROR 2015-07-14 16:54:10,754 Job wrapper finish 
method failed
Traceback (most recent call last):
  File "/data/galaxy/galaxy_prod/lib/galaxy/jobs/runners/local.py", line 129, 
in queue_job
job_wrapper.finish( stdout, stderr, exit_code )
  File "/data/galaxy/galaxy_prod/lib/galaxy/jobs/__init__.py", line 1126, in 
finish
dataset.datatype.set_meta( dataset, overwrite=False )  # call 
datatype.set_meta directly for the initial set_meta call during dataset creation
  File "/data/galaxy/galaxy_prod/lib/galaxy/datatypes/binary.py", line 268, in 
set_meta
proc = subprocess.Popen( args=command, stderr=open( stderr_name, 'wb' ) )
  File "/opt/python/2.7.9/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
  File "/opt/python/2.7.9/lib/python2.7/subprocess.py", line 1335, in 
_execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Not seeing reference genome - Bowtie2

[McCully, Dwayne (NIH/NIAMS) [C]]

Hello Everyone,

Performed a data retrieval using the rsync method for mm10 and also downloaded 
the .loc files per the "Data Integration" page
to the tool-data directory.  The paths match in the .loc file for bowtie2 of 
the mm10 download directory.

My problem is that I'm not seeing mm10 in the "Select Reference Genome" from 
within Bowtie2.

Any tips would be appreciated

Dwayne
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] MySQL InnoDB - Execution Order

[McCully, Dwayne (NIH/NIAMS) [C]]

Hello Everyone,

The running Galaxy in a production environment documentation states:

"If you are using MySQL with 
MyISAM table 
engine when Galaxy is in multiprocess configuration, workflow steps will get 
executed out of 
order
 and fail.
Use InnoDB 
engine instead or switch to PostgreSQL."

I've enabled InnoDB for my MySQL database via the my.cnf file but the default 
database and database to log all database transactions were created with MyISAM 
files.
If this is normal,  when will Galaxy create InnoDB tables?   I don't believe 
you can create a database with InnoDB.

Dwayne
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] MySQL or Postgres size and growth?

[McCully, Dwayne (NIH/NIAMS) [C]]

When setting up a Galaxy site, what should I expect for MySQL or Postgres 
database initial size and growth?

Dwayne
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/