Re: [galaxy-dev] Galaxy Cleanup

2018-04-27 Thread Previti 
Thanks! Super useful!

Cheers,

Christopher


On 04/27/2018 04:12 PM, Nicola Soranzo wrote:
> See also:
>
> https://galaxyproject.github.io/training-material/topics/admin/tutorials/monitoring-maintenance/slides.html
>
>
> Cheers,
> Nicola
>
> On 27/04/18 15:05, Hans-Rudolf Hotz wrote:
>> Hi Christopher
>>
>>
>> It depends on the setting of "cleanup_job" ('always', 'onsuccess',
>> 'never'), whether the temporary files are cleaned up or not. Though,
>> to be honest, I don't know, if this setting as any influence, when
>> you 'delete' a job.
>>
>> For the quotas, only dataset in the histories are counted
>>
>> The temporary files are very handy for debugging, so I have
>> 'onsuccess' on our production server and 'never' on our development
>> server. I just regularly check the total size of the working
>> directories and do the clean up manually
>>
>>
>> Regards, Hans-Rudolf
>>
>>
>>
>>
>> On 04/27/2018 01:51 PM, Previti wrote:
>>> Hi everybody,
>>>
>>> I have a hopefully basic questions regarding temporary output files.
>>>
>>> A user was generating >4TB output files via the program bedtools
>>> intersectbed. The jobs were cancelled, but I still see the (temporary)
>>> output files that were generated.
>>>
>>> Does Galaxy clean these up? Does it do this regularly?
>>>
>>> Would a quota prevent something like this from happening again or would
>>> the jobs have to finish before being "counted"?
>>>
>>> Thanks and best regards,
>>>
>>> Christopher
>>>
>>> -- 
>>> *Dr. Christopher Previti*
>>> Genomics and Proteomics Core Facility
>>> High Throughput Sequencing (W190)
>>> Bioinformatician
>>>
>>> German Cancer Research Center (DKFZ)
>>> Foundation under Public Law
>>> Im Neuenheimer Feld 580
>>> 69120 Heidelberg
>>> Germany
>>> Room: B2.102 (INF580/TP3)
>>> Phone: +49 6221 42-4661
>>>
>>> christopher.prev...@dkfz.de 
>>> www.dkfz.de 
>>>
>>> Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
>>> VAT-ID No.: DE143293537
>>>
>>> Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
>>> Personen bestimmt, an die sie adressiert ist.
>>> Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
>>> Informationen enthalten. Sollten Sie nicht
>>> der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
>>> Absender und löschen Sie die Mitteilung.
>>> Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
>>> untersagt.
>>>
>>>
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>    https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>    http://galaxyproject.org/search/
>>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/
>

-- 
*Dr. Christopher Previti*
Genomics and Proteomics Core Facility
High Throughput Sequencing (W190)
Bioinformatician

German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 580
69120 Heidelberg
Germany
Room: B2.102 (INF580/TP3)
Phone: +49 6221 42-4661

christopher.prev...@dkfz.de 
www.dkfz.de 

Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
VAT-ID No.: DE143293537

Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
Personen bestimmt, an die sie adressiert ist.
Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
Informationen enthalten. Sollten Sie nicht
der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
Absender und löschen Sie die Mitteilung.
Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
untersagt.


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Galaxy Cleanup

2018-04-27 Thread Nicola Soranzo 

See also:

https://galaxyproject.github.io/training-material/topics/admin/tutorials/monitoring-maintenance/slides.html

Cheers,
Nicola

On 27/04/18 15:05, Hans-Rudolf Hotz wrote:

Hi Christopher


It depends on the setting of "cleanup_job" ('always', 'onsuccess', 
'never'), whether the temporary files are cleaned up or not. Though, 
to be honest, I don't know, if this setting as any influence, when you 
'delete' a job.


For the quotas, only dataset in the histories are counted

The temporary files are very handy for debugging, so I have 
'onsuccess' on our production server and 'never' on our development 
server. I just regularly check the total size of the working 
directories and do the clean up manually



Regards, Hans-Rudolf




On 04/27/2018 01:51 PM, Previti wrote:

Hi everybody,

I have a hopefully basic questions regarding temporary output files.

A user was generating >4TB output files via the program bedtools
intersectbed. The jobs were cancelled, but I still see the (temporary)
output files that were generated.

Does Galaxy clean these up? Does it do this regularly?

Would a quota prevent something like this from happening again or would
the jobs have to finish before being "counted"?

Thanks and best regards,

Christopher

--
*Dr. Christopher Previti*
Genomics and Proteomics Core Facility
High Throughput Sequencing (W190)
Bioinformatician

German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 580
69120 Heidelberg
Germany
Room: B2.102 (INF580/TP3)
Phone: +49 6221 42-4661

christopher.prev...@dkfz.de 
www.dkfz.de 

Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
VAT-ID No.: DE143293537

Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
Personen bestimmt, an die sie adressiert ist.
Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
Informationen enthalten. Sollten Sie nicht
der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
Absender und löschen Sie die Mitteilung.
Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
untersagt.




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/

Re: [galaxy-dev] Galaxy Cleanup

2018-04-27 Thread Hans-Rudolf Hotz 

Hi Christopher


It depends on the setting of "cleanup_job" ('always', 'onsuccess', 
'never'), whether the temporary files are cleaned up or not. Though, to 
be honest, I don't know, if this setting as any influence, when you 
'delete' a job.


For the quotas, only dataset in the histories are counted

The temporary files are very handy for debugging, so I have 'onsuccess' 
on our production server and 'never' on our development server. I just 
regularly check the total size of the working directories and do the 
clean up manually



Regards, Hans-Rudolf




On 04/27/2018 01:51 PM, Previti wrote:

Hi everybody,

I have a hopefully basic questions regarding temporary output files.

A user was generating >4TB output files via the program bedtools
intersectbed. The jobs were cancelled, but I still see the (temporary)
output files that were generated.

Does Galaxy clean these up? Does it do this regularly?

Would a quota prevent something like this from happening again or would
the jobs have to finish before being "counted"?

Thanks and best regards,

Christopher

--
*Dr. Christopher Previti*
Genomics and Proteomics Core Facility
High Throughput Sequencing (W190)
Bioinformatician

German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 580
69120 Heidelberg
Germany
Room: B2.102 (INF580/TP3)
Phone: +49 6221 42-4661

christopher.prev...@dkfz.de 
www.dkfz.de 

Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
VAT-ID No.: DE143293537

Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
Personen bestimmt, an die sie adressiert ist.
Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
Informationen enthalten. Sollten Sie nicht
der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
Absender und löschen Sie die Mitteilung.
Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
untersagt.




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/

[galaxy-dev] Galaxy Cleanup

2018-04-27 Thread Previti 
Hi everybody,

I have a hopefully basic questions regarding temporary output files.

A user was generating >4TB output files via the program bedtools
intersectbed. The jobs were cancelled, but I still see the (temporary)
output files that were generated.

Does Galaxy clean these up? Does it do this regularly?

Would a quota prevent something like this from happening again or would
the jobs have to finish before being "counted"?

Thanks and best regards,

Christopher

-- 
*Dr. Christopher Previti*
Genomics and Proteomics Core Facility
High Throughput Sequencing (W190)
Bioinformatician

German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 580
69120 Heidelberg
Germany
Room: B2.102 (INF580/TP3)
Phone: +49 6221 42-4661

christopher.prev...@dkfz.de 
www.dkfz.de 

Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
VAT-ID No.: DE143293537

Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
Personen bestimmt, an die sie adressiert ist.
Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
Informationen enthalten. Sollten Sie nicht
der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
Absender und löschen Sie die Mitteilung.
Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
untersagt.


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/