subject:"\[Freesurfer\] Longitudinal analysis using SLURM \{Disarmed\}"

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed} (Martin Reuter)

2018-04-16 Thread Martin Reuter

Hi Lea,

sorry for the late reply, just in case this is still of interest:

1. It is possible to process all cross data with 5.3 and then base and 
long with 6.0. As long as you do the same for all subjects it should be 
fine, just don't mix across subjects. Also depending on the size if only 
half the cross are already processed it may make sense to re-do 
everything with 6.0 to have a clean and easy setup.

2. Basically yes, but there are some situations where things can go 
wrong, for example

  - you restart long_submit_jobs, but not all scheduled jobs are running 
(e.g. some are still in the queue), it would then resubmit those jobs

- Jobs die and the is_running file still exists in the freesurfer 
scripts dir, in that case long_submit_jobs will not re-submit those jobs


Best, Martin


Am 27.02.2018 um 11:23 schrieb Backhausen, Lea:
> Hi Martin,
>
> thank you so much for your detailed answer. I will see how we can implement 
> these modifications.
>
> Just two quick follow up questions:
> 1) I have half of the data already processed with the CROSS step using FS 
> 5.3.0. Is it possible to use this data when running the TEMPLATE and LONG 
> steps with FS 6.0.0 or would this corrupt results?
> 2) Our HPC system is known to have problems from time to time and jobs end up 
> being killed. If the long_submit_jobs job gets killed prematurely would it 
> just resume processing the data that hasn't been processed yet if I submit 
> this job/script again?
>
> Thanks again for your help!
>
> Best,
> Lea Backhausen
> Research assistant
>
> Klinik und Poliklinik für Kinder- und Jugendpsychiatrie und –psychotherapie
>
> Universitätsklinikum Carl Gustav Carus
> an der Technischen Universität Dresden
> Anstalt des öffentlichen Rechts des Freistaates Sachsen
> Fetscherstraße 74, 01307 Dresden
> http://www.uniklinikum-dresden.de
>
>
> -Original Message-
> From: freesurfer-boun...@nmr.mgh.harvard.edu 
> [mailto:freesurfer-boun...@nmr.mgh.harvard.edu] On Behalf Of 
> freesurfer-requ...@nmr.mgh.harvard.edu
> Sent: Thursday, February 22, 2018 5:33 PM
> To: freesurfer@nmr.mgh.harvard.edu
> Subject: Freesurfer Digest, Vol 168, Issue 34
>
> Send Freesurfer mailing list submissions to
>   freesurfer@nmr.mgh.harvard.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
>   https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
> or, via email, send a message with subject or body 'help' to
>   freesurfer-requ...@nmr.mgh.harvard.edu
>
> You can reach the person managing the list at
>   freesurfer-ow...@nmr.mgh.harvard.edu
>
> When replying, please edit your Subject line so it is more specific than "Re: 
> Contents of Freesurfer digest..."
>
>
> Today's Topics:
>
> 1. Re: Search angle limitation in the flirt registration
>(Douglas N Greve)
> 2. Re: Extracting the centroids from Desikan-Killiany atlas
>(zuxfoucault Wong)
> 3. Unsubscribe Mailing List (Duy Nguyen)
> 4. Unknown Label in following postprocessing with mne-python
>(Daniel van de Velden)
> 5. Re: Surface Errors (Emily Schwartz)
> 6. Re: Problem with cortical parcellation of rh but not lh
>(Amal Achaibou)
> 7. Re: Longitudinal analysis using SLURM {Disarmed} (Martin Reuter)
> 8. Re: Longitudinal analysis using SLURM {Disarmed} (Satrajit Ghosh)
> 9. Re: Longitudinal analysis using SLURM {Disarmed} (Martin Reuter)
>10. Re: Freesurfer equivalent to fslmaths? (Bruce Fischl)
>11. asymmetric parcellation error (Dijkshoorn, A.B.C. (Aicha))
>
>
> --
>
> Message: 1
> Date: Wed, 21 Feb 2018 18:26:32 -0500
> From: Douglas N Greve 
> Subject: Re: [Freesurfer] Search angle limitation in the flirt
>   registration
> To: freesurfer@nmr.mgh.harvard.edu
> Message-ID: <1e86b3d8-54c6-c9ad-54b3-7cc8df81c...@nmr.mgh.harvard.edu>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> you should post your question to the FSL list (they developed and support 
> flirt)
>
>
> On 02/21/2018 05:34 PM, Yixin Ma wrote:
>> Hi freesurfer developers and users,
>>
>> I'm writing to ask about the search angle limitation (--searchrx, --
>> searchry and --searchrz) in the flirt function. I use the search angle
>> option a lot for limited angle search, which usually generate a better
>> registration result compared with the broad angle search. But I still
>> don't quite understand how rx,ry and rz are defined and how negative
>> and positive angles affect the result. Because sometimes I find the
>> result after the angle limitation does not align well with my
>> expectation of rotation angle.
>>
>> Thank you for your help,
>>
>> Yixin Ma
>>
>> Brain Image and Analysis Center
>>
>> Duke university
>>
>>
>>
>> ___
>> Freesurfer mailing list
>> Freesurfer@nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed} (Martin Reuter)

2018-02-27 Thread Backhausen, Lea

Hi Martin, 

thank you so much for your detailed answer. I will see how we can implement 
these modifications. 

Just two quick follow up questions:
1) I have half of the data already processed with the CROSS step using FS 
5.3.0. Is it possible to use this data when running the TEMPLATE and LONG steps 
with FS 6.0.0 or would this corrupt results?
2) Our HPC system is known to have problems from time to time and jobs end up 
being killed. If the long_submit_jobs job gets killed prematurely would it just 
resume processing the data that hasn't been processed yet if I submit this 
job/script again?

Thanks again for your help!

Best,
Lea Backhausen
Research assistant

Klinik und Poliklinik für Kinder- und Jugendpsychiatrie und –psychotherapie 

Universitätsklinikum Carl Gustav Carus
an der Technischen Universität Dresden
Anstalt des öffentlichen Rechts des Freistaates Sachsen
Fetscherstraße 74, 01307 Dresden
http://www.uniklinikum-dresden.de

-Original Message-
From: freesurfer-boun...@nmr.mgh.harvard.edu 
[mailto:freesurfer-boun...@nmr.mgh.harvard.edu] On Behalf Of 
freesurfer-requ...@nmr.mgh.harvard.edu
Sent: Thursday, February 22, 2018 5:33 PM
To: freesurfer@nmr.mgh.harvard.edu
Subject: Freesurfer Digest, Vol 168, Issue 34

Send Freesurfer mailing list submissions to
freesurfer@nmr.mgh.harvard.edu

To subscribe or unsubscribe via the World Wide Web, visit
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
or, via email, send a message with subject or body 'help' to
freesurfer-requ...@nmr.mgh.harvard.edu

You can reach the person managing the list at
freesurfer-ow...@nmr.mgh.harvard.edu

When replying, please edit your Subject line so it is more specific than "Re: 
Contents of Freesurfer digest..."

Today's Topics:

   1. Re: Search angle limitation in the flirt registration
  (Douglas N Greve)
   2. Re: Extracting the centroids from Desikan-Killianyatlas
  (zuxfoucault Wong)
   3. Unsubscribe Mailing List (Duy Nguyen)
   4. Unknown Label in following postprocessing withmne-python
  (Daniel van de Velden)
   5. Re: Surface Errors (Emily Schwartz)
   6. Re: Problem with cortical parcellation of rh but not lh
  (Amal Achaibou)
   7. Re: Longitudinal analysis using SLURM {Disarmed} (Martin Reuter)
   8. Re: Longitudinal analysis using SLURM {Disarmed} (Satrajit Ghosh)
   9. Re: Longitudinal analysis using SLURM {Disarmed} (Martin Reuter)
  10. Re: Freesurfer equivalent to fslmaths? (Bruce Fischl)
  11. asymmetric parcellation error (Dijkshoorn, A.B.C. (Aicha))

--

Message: 1
Date: Wed, 21 Feb 2018 18:26:32 -0500
From: Douglas N Greve 
Subject: Re: [Freesurfer] Search angle limitation in the flirt
registration
To: freesurfer@nmr.mgh.harvard.edu
Message-ID: <1e86b3d8-54c6-c9ad-54b3-7cc8df81c...@nmr.mgh.harvard.edu>
Content-Type: text/plain; charset=utf-8; format=flowed

you should post your question to the FSL list (they developed and support flirt)

On 02/21/2018 05:34 PM, Yixin Ma wrote:
> Hi freesurfer developers and users,
>
> I'm writing to ask about the search angle limitation (--searchrx, -- 
> searchry and --searchrz) in the flirt function. I use the search angle 
> option a lot for limited angle search, which usually generate a better 
> registration result compared with the broad angle search. But I still 
> don't quite understand how rx,ry and rz are defined and how negative 
> and positive angles affect the result. Because sometimes I find the 
> result after the angle limitation does not align well with my 
> expectation of rotation angle.
>
> Thank you for your help,
>
> Yixin Ma
>
> Brain Image and Analysis Center
>
> Duke university
>
>
>
> ___
> Freesurfer mailing list
> Freesurfer@nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

-- 
Douglas N. Greve, Ph.D.
MGH-NMR Center
gr...@nmr.mgh.harvard.edu
Phone Number: 617-724-2358
Fax: 617-726-7422

Bugs: surfer.nmr.mgh.harvard.edu/fswiki/BugReporting
FileDrop: https://gate.nmr.mgh.harvard.edu/filedrop2
www.nmr.mgh.harvard.edu/facility/filedrop/index.html
Outgoing: ftp://surfer.nmr.mgh.harvard.edu/transfer/outgoing/flat/greve/

--

Message: 2
Date: Thu, 22 Feb 2018 02:23:36 +
From: zuxfoucault Wong 
Subject: Re: [Freesurfer] Extracting the centroids from
Desikan-Killianyatlas
To: Freesurfer support list 
Message-ID:

Content-Type: text/plain; charset="utf-8"

Yes! I got better results now! Thanks!!

On Thu, Feb 22, 2018 at 12:57 AM Kirstie Whitaker  wrote:

> Hi Foucalt,
>
> I think you need to set the threshold to slightly larger than 0 otherwise
> you're taking the center of mass of the whole

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed}

2018-02-22 Thread Martin Reuter


Could be all that is needed is to add this:


mycluster ='sbatch --mail-type=FAIL --mail-user= -N 1 
--ntasks-per-node=10 --partition=work --export=ALL -o 
/%(username)s/logs/job.%J.out- e 
/%(username)/logs/job.%J.err "%(command)s"'


and comment the lines

# if queue is not None ...
#   pbcmd = ...

with a hashtag.

you can also simplify the slurm command (remove the --mail flags and the 
-o logfile location flags) your cluster partition may not be called 
"work" either.


Best, Martin


Am 22.02.2018 um 16:43 schrieb Martin Reuter:


Hi Lea,


I wrote that script to simplify processing on our cluster which uses 
qsub PBS for submission. I have never used SLURM so far.



There is two functions that you (or someone who knows this stuff) 
would need to modify:


def submit

and

def wait_jobs


The submit procedure basically issues a submit command (stored in 
pbcmd).  The examples there should help anyone who programs python to 
modify it for a slurm submission.



The wait_jobs scripts runs a different command (here qstat and grep 
user) to see how many jobs you are already running and waits until 
this number sinks below maxjobs, then it continues submitting new 
jobs. If you don't care about the number of submitted jobs (because 
your cluster does the scheduling anyway), you can just write



"return" as the first line (before the "users = os." line).


Still the submit block would have to be rewritten for slurm.


Best, Martin



Am 17.02.2018 um 12:13 schrieb Backhausen, Lea:

Dear FS experts,

I am trying to do a longitudinal analysis of around 250 adolescents 
with up to 4 data points each. Given the complexity I would like to 
use the long_submit_jobs script which is described here: 
https://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/Scripts 



Unfourtunately it does only run on launchpad and I would do the 
analyses with SLURM using bash system.
Has anybody adopted the script so i will run with SLURM or does 
anybody know where I should look to get information on this?


I have little scripting experience so any help would be greatly 
appreciated!


Best,

Lea Backhausen, B.Sc.

Research assistant

Klinik und Poliklinik für Kinder- und Jugendpsychiatrie und 
–psychotherapie

Tel. +49 (0)351 458-7166
Fax +49 (0)351 458-5754

Universitätsklinikum Carl Gustav Carus
an der Technischen Universität Dresden
Anstalt des öffentlichen Rechts des Freistaates Sachsen
Fetscherstraße 74, 01307 Dresden
*MailScanner has detected a possible fraud attempt from 
"deref-web-02.de" claiming to be* _http://www.uniklinikum-dresden.de_ 


Vorstand: Prof. Dr. med. D. M. Albrecht (Sprecher), Wilfried E. B. Winzer
Vorsitzender des Aufsichtsrates: Prof. Dr. med. Peter C. Scriba
USt.-IDNr.: DE 140 135 217, St.-Nr.: 203 145 03113


___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer




___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed}

2018-02-22 Thread Satrajit Ghosh

hi lea,

in addition you could consider using the bids app for freesurfer:
https://github.com/BIDS-Apps/freesurfer

and then submit each subject's longitudinal pipeline as a separate slurm
job.

cheers,

satra

On Thu, Feb 22, 2018 at 10:43 AM, Martin Reuter  wrote:

> Hi Lea,
>
>
> I wrote that script to simplify processing on our cluster which uses qsub
> PBS for submission. I have never used SLURM so far.
>
>
> There is two functions that you (or someone who knows this stuff) would
> need to modify:
>
> def submit
>
> and
>
> def wait_jobs
>
>
> The submit procedure basically issues a submit command (stored in pbcmd).
> The examples there should help anyone who programs python to modify it for
> a slurm submission.
>
>
> The wait_jobs scripts runs a different command (here qstat and grep user)
> to see how many jobs you are already running and waits until this number
> sinks below maxjobs, then it continues submitting new jobs. If you don't
> care about the number of submitted jobs (because your cluster does the
> scheduling anyway), you can just write
>
>
> "return" as the first line (before the "users = os." line).
>
>
> Still the submit block would have to be rewritten for slurm.
>
>
> Best, Martin
>
>
>
> Am 17.02.2018 um 12:13 schrieb Backhausen, Lea:
>
> Dear FS experts,
>
> I am trying to do a longitudinal analysis of around 250 adolescents with
> up to 4 data points each. Given the complexity I would like to use the
> long_submit_jobs script which is described here: https://surfer.nmr.mgh.
> harvard.edu/fswiki/FsTutorial/Scripts
>
> Unfourtunately it does only run on launchpad and I would do the analyses
> with SLURM using bash system.
> Has anybody adopted the script so i will run with SLURM or does anybody
> know where I should look to get information on this?
>
> I have little scripting experience so any help would be greatly
> appreciated!
>
> Best,
>
> Lea Backhausen, B.Sc.
>
> Research assistant
>
> Klinik und Poliklinik für Kinder- und Jugendpsychiatrie und –psychotherapie
> Tel. +49 (0)351 458-7166 <+49%20351%204587166>
> Fax +49 (0)351 458-5754 <+49%20351%204585754>
>
> Universitätsklinikum Carl Gustav Carus
> an der Technischen Universität Dresden
> Anstalt des öffentlichen Rechts des Freistaates Sachsen
> Fetscherstraße 74, 01307 Dresden
> *MailScanner has detected a possible fraud attempt from "deref-web-02.de"
> claiming to be* *http://www.uniklinikum-dresden.de*
> 
> Vorstand: Prof. Dr. med. D. M. Albrecht (Sprecher), Wilfried E. B. Winzer
> Vorsitzender des Aufsichtsrates: Prof. Dr. med. Peter C. Scriba
> USt.-IDNr.: DE 140 135 217, St.-Nr.: 203 145 03113
>
>
> ___
> Freesurfer mailing 
> listfreesur...@nmr.mgh.harvard.eduhttps://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
>
>
>
> ___
> Freesurfer mailing list
> Freesurfer@nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
>
>
> The information in this e-mail is intended only for the person to whom it
> is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
>
>
___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed}

2018-02-22 Thread Martin Reuter


Hi Lea,


I wrote that script to simplify processing on our cluster which uses 
qsub PBS for submission. I have never used SLURM so far.



There is two functions that you (or someone who knows this stuff) would 
need to modify:


def submit

and

def wait_jobs


The submit procedure basically issues a submit command (stored in 
pbcmd).  The examples there should help anyone who programs python to 
modify it for a slurm submission.



The wait_jobs scripts runs a different command (here qstat and grep 
user) to see how many jobs you are already running and waits until this 
number sinks below maxjobs, then it continues submitting new jobs. If 
you don't care about the number of submitted jobs (because your cluster 
does the scheduling anyway), you can just write



"return" as the first line (before the "users = os." line).


Still the submit block would have to be rewritten for slurm.


Best, Martin



Am 17.02.2018 um 12:13 schrieb Backhausen, Lea:

Dear FS experts,

I am trying to do a longitudinal analysis of around 250 adolescents 
with up to 4 data points each. Given the complexity I would like to 
use the long_submit_jobs script which is described here: 
https://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/Scripts 



Unfourtunately it does only run on launchpad and I would do the 
analyses with SLURM using bash system.
Has anybody adopted the script so i will run with SLURM or does 
anybody know where I should look to get information on this?


I have little scripting experience so any help would be greatly 
appreciated!


Best,

Lea Backhausen, B.Sc.

Research assistant

Klinik und Poliklinik für Kinder- und Jugendpsychiatrie und 
–psychotherapie

Tel. +49 (0)351 458-7166
Fax +49 (0)351 458-5754

Universitätsklinikum Carl Gustav Carus
an der Technischen Universität Dresden
Anstalt des öffentlichen Rechts des Freistaates Sachsen
Fetscherstraße 74, 01307 Dresden
*MailScanner has detected a possible fraud attempt from 
"deref-web-02.de" claiming to be* _http://www.uniklinikum-dresden.de_ 


Vorstand: Prof. Dr. med. D. M. Albrecht (Sprecher), Wilfried E. B. Winzer
Vorsitzender des Aufsichtsrates: Prof. Dr. med. Peter C. Scriba
USt.-IDNr.: DE 140 135 217, St.-Nr.: 203 145 03113


___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

[Freesurfer] Longitudinal analysis using SLURM {Disarmed}

2018-02-17 Thread Backhausen, Lea

Dear FS experts,

I am trying to do a longitudinal analysis of around 250 adolescents with up to 
4 data points each. Given the complexity I would like to use the 
long_submit_jobs script which is described here: 
https://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/Scripts

Unfourtunately it does only run on launchpad and I would do the analyses with 
SLURM using bash system.
Has anybody adopted the script so i will run with SLURM or does anybody know 
where I should look to get information on this?

I have little scripting experience so any help would be greatly appreciated!

Best,

Lea Backhausen, B.Sc.

Research assistant

Klinik und Poliklinik für Kinder- und Jugendpsychiatrie und –psychotherapie
Tel. +49 (0)351 458-7166
Fax +49 (0)351 458-5754

Universitätsklinikum Carl Gustav Carus
an der Technischen Universität Dresden
Anstalt des öffentlichen Rechts des Freistaates Sachsen
Fetscherstraße 74, 01307 Dresden
http://www.uniklinikum-dresden.de
Vorstand: Prof. Dr. med. D. M. Albrecht (Sprecher), Wilfried E. B. Winzer
Vorsitzender des Aufsichtsrates: Prof. Dr. med. Peter C. Scriba
USt.-IDNr.: DE 140 135 217, St.-Nr.: 203 145 03113
___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed} (Martin Reuter)

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed} (Martin Reuter)

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed}

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed}

Re: [Freesurfer] Longitudinal analysis using SLURM {Disarmed}

[Freesurfer] Longitudinal analysis using SLURM {Disarmed}

6 matches

Site Navigation

Mail list logo

Footer information