Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
On 09 Dec 2014 at 12:29, "Book, Gregory" wrote: > > We’ve run into a weird problem when using the GenericfMRIVolume batch > script to process more than 2 tasks at a time. When we setup any more > than 2 tasks (we’ve tried 3 and 4 tasks), the first 2 run correctly > and without error, but any after that crash in the TopUp section with > the following error (the failed tasks all get to the same point): ... On 09 Dec 2014 at 15:11, "Xu, Junqian" wrote: > > After a second thought, maybe convert the list to array specifically > for the comparison purpose so the rest of the pipeline script don't > need to be modified to use array indexing. > ... Gordon and Gregory, Comments indicating that the Tasklist and PhaseEncodinglist should have the same number of space-delimited elements, and a test to verify that this is true before starting the main processing loop have been added to the GenericfMRIVolumeProcessingPipelineBatch.sh script that is in the example scripts directory. This change will be included in the next released version of the HCP Pipeline Scripts. Gregory, thank you for pointing out the issue. Gordon, thank you for the suggested changes. Tim -- Timothy B. Brown Business & Technology Application Analyst III Pipeline Developer (Human Connectome Project) tbbrown(at)wustl.edu The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. ___ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users
Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
After a second thought, maybe convert the list to array specifically for the
comparison purpose so the rest of the pipeline script don’t need to be modified
to use array indexing.
Tasklist="domino0 domino1 domino2 domino3"
PhaseEncodinglist="y y- y y-“
TaskArray=($Tasklist)
PhaseEncodingArray=($PhaseEcondinglist)
nTaskArray=${#TaskArray[@]}
nPhaseEncodingArray=${#PhaseEncodingArray[@]}
if [ $nTaskArray -ne $nPhaseEncodingArray ]; then
fi
Gordon
> On Dec 9, 2014, at 4:05 PM, Glasser, Matthew wrote:
>
> Thanks Gordon, that would help too.
>
> On 12/9/14, 3:03 PM, "Xu, Junqian" wrote:
>
>> Tim,
>>
>> A patch here to use array instead of space delimited string might be
>> helpful for error detection:
>>
>> e.g.
>>
>> Tasklist=(domino0 domino1 domino2 domino3)
>> PhaseEncodinglist=(y y- y y-)
>>
>> nTasklist=${#Tasklist[@]}
>> nPhaseEncodinglist=${#PhaseEncodinglist[@]}
>>
>> if [ $nTasklist -ne $nPhaseEncodinglist ]; then
>>
>> fi
>>
>>
>> Gordon
>>
>>> On Dec 9, 2014, at 3:33 PM, Glasser, Matthew
>>> wrote:
>>>
>>> You need to expand the PhaseEncodinglist variable to have the same
>>> length as the Tasklist variable (i.e. same number of elements separated
>>> by spaces). This probably should be made clearer in the comments for
>>> these variables.
>>>
>>> Peace,
>>>
>>> Matt.
>>>
>>> From: , Gregory
>>> Date: Tuesday, December 9, 2014 at 2:19 PM
>>> To: Matt Glasser ,
>>> "hcp-users@humanconnectome.org"
>>> Subject: RE: [HCP-Users] error when processing more than 2 tasks with
>>> GenericfMRIVolume script
>>>
>>> Thanks, the script is attached. Also the directory structure is after
>>> the script.
>>>
>>> #!/bin/bash
>>>
>>> get_batch_options() {
>>>local arguments=($@)
>>>
>>>unset command_line_specified_study_folder
>>>unset command_line_specified_subj_list
>>>unset command_line_specified_run_local
>>>
>>>local index=0
>>>local numArgs=${#arguments[@]}
>>>local argument
>>>
>>>while [ ${index} -lt ${numArgs} ]; do
>>>argument=${arguments[index]}
>>>
>>>case ${argument} in
>>>--StudyFolder=*)
>>>command_line_specified_study_folder=${argument/*=/""}
>>>index=$(( index + 1 ))
>>>;;
>>>--Subjlist=*)
>>>command_line_specified_subj_list=${argument/*=/""}
>>>index=$(( index + 1 ))
>>>;;
>>>--runlocal)
>>>command_line_specified_run_local="TRUE"
>>>index=$(( index + 1 ))
>>>;;
>>>esac
>>>done
>>> }
>>>
>>> get_batch_options $@
>>>
>>>
>>> StudyFolder="/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoS
>>> kyra"
>>> Subjlist="analysis"
>>> EnvironmentScript="/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh"
>>>
>>> if [ -n "${command_line_specified_study_folder}" ]; then
>>>StudyFolder="${command_line_specified_study_folder}"
>>> fi
>>>
>>> if [ -n "${command_line_specified_subj_list}" ]; then
>>>Subjlist="${command_line_specified_subj_list}"
>>> fi
>>>
>>> # Requirements for this script
>>> # installed versions of: FSL (version 5.0.6), FreeSurfer (version
>>> 5.3.0-HCP) , gradunwarp (HCP version 1.0.1)
>>> # environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR ,
>>> PATH (for gradient_unwarp.py)
>>>
>>> #Set up pipeline environment variables and software
>>> . ${EnvironmentScript}
>>>
>>> # Log the originating call
>>> echo "$@"
>>>
>>> #if [ X$SGE_ROOT != X ] ; then
>>>QUEUE="-q long.q"
>>> #fi
>>>
>>> PRINTCOM=""
>>> #PRINTCOM="echo"
>>> #QUEUE="-q veryshort.q"
>>>
>>> ## INPUTS
>>> ##
>>>
>>> #Scripts called by this script do NOT assume anythi
Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
Thanks Gordon, that would help too.
On 12/9/14, 3:03 PM, "Xu, Junqian" wrote:
>Tim,
>
>A patch here to use array instead of space delimited string might be
>helpful for error detection:
>
>e.g.
>
>Tasklist=(domino0 domino1 domino2 domino3)
>PhaseEncodinglist=(y y- y y-)
>
>nTasklist=${#Tasklist[@]}
>nPhaseEncodinglist=${#PhaseEncodinglist[@]}
>
>if [ $nTasklist -ne $nPhaseEncodinglist ]; then
>
>fi
>
>
>Gordon
>
>> On Dec 9, 2014, at 3:33 PM, Glasser, Matthew
>>wrote:
>>
>> You need to expand the PhaseEncodinglist variable to have the same
>>length as the Tasklist variable (i.e. same number of elements separated
>>by spaces). This probably should be made clearer in the comments for
>>these variables.
>>
>> Peace,
>>
>> Matt.
>>
>> From: , Gregory
>> Date: Tuesday, December 9, 2014 at 2:19 PM
>> To: Matt Glasser ,
>>"hcp-users@humanconnectome.org"
>> Subject: RE: [HCP-Users] error when processing more than 2 tasks with
>>GenericfMRIVolume script
>>
>> Thanks, the script is attached. Also the directory structure is after
>>the script.
>>
>> #!/bin/bash
>>
>> get_batch_options() {
>> local arguments=($@)
>>
>> unset command_line_specified_study_folder
>> unset command_line_specified_subj_list
>> unset command_line_specified_run_local
>>
>> local index=0
>> local numArgs=${#arguments[@]}
>> local argument
>>
>> while [ ${index} -lt ${numArgs} ]; do
>> argument=${arguments[index]}
>>
>> case ${argument} in
>> --StudyFolder=*)
>> command_line_specified_study_folder=${argument/*=/""}
>> index=$(( index + 1 ))
>> ;;
>> --Subjlist=*)
>> command_line_specified_subj_list=${argument/*=/""}
>> index=$(( index + 1 ))
>> ;;
>> --runlocal)
>> command_line_specified_run_local="TRUE"
>> index=$(( index + 1 ))
>> ;;
>> esac
>> done
>> }
>>
>> get_batch_options $@
>>
>>
>>StudyFolder="/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoS
>>kyra"
>> Subjlist="analysis"
>> EnvironmentScript="/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh"
>>
>> if [ -n "${command_line_specified_study_folder}" ]; then
>> StudyFolder="${command_line_specified_study_folder}"
>> fi
>>
>> if [ -n "${command_line_specified_subj_list}" ]; then
>> Subjlist="${command_line_specified_subj_list}"
>> fi
>>
>> # Requirements for this script
>> # installed versions of: FSL (version 5.0.6), FreeSurfer (version
>>5.3.0-HCP) , gradunwarp (HCP version 1.0.1)
>> # environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR ,
>>PATH (for gradient_unwarp.py)
>>
>> #Set up pipeline environment variables and software
>> . ${EnvironmentScript}
>>
>> # Log the originating call
>> echo "$@"
>>
>> #if [ X$SGE_ROOT != X ] ; then
>> QUEUE="-q long.q"
>> #fi
>>
>> PRINTCOM=""
>> #PRINTCOM="echo"
>> #QUEUE="-q veryshort.q"
>>
>> ## INPUTS
>>##
>>
>> #Scripts called by this script do NOT assume anything about the form of
>>the input names or paths.
>> #This batch script assumes the HCP raw data naming convention, e.g. for
>>tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:
>>
>> #
>>${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_t
>>fMRI_EMOTION_LR.nii.gz
>> #
>>${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_t
>>fMRI_EMOTION_LR_SBRef.nii.gz
>>
>> #
>>${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_t
>>fMRI_EMOTION_RL.nii.gz
>> #
>>${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_t
>>fMRI_EMOTION_RL_SBRef.nii.gz
>>
>> #
>>${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_S
>>pinEchoFieldMap_LR.nii.gz
>> #
>>${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_S
>>pinEchoFieldMap_RL.nii.gz
>>
>> #
>>${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/
Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
Tim,
A patch here to use array instead of space delimited string might be helpful
for error detection:
e.g.
Tasklist=(domino0 domino1 domino2 domino3)
PhaseEncodinglist=(y y- y y-)
nTasklist=${#Tasklist[@]}
nPhaseEncodinglist=${#PhaseEncodinglist[@]}
if [ $nTasklist -ne $nPhaseEncodinglist ]; then
fi
Gordon
> On Dec 9, 2014, at 3:33 PM, Glasser, Matthew wrote:
>
> You need to expand the PhaseEncodinglist variable to have the same length as
> the Tasklist variable (i.e. same number of elements separated by spaces).
> This probably should be made clearer in the comments for these variables.
>
> Peace,
>
> Matt.
>
> From: , Gregory
> Date: Tuesday, December 9, 2014 at 2:19 PM
> To: Matt Glasser , "hcp-users@humanconnectome.org"
>
> Subject: RE: [HCP-Users] error when processing more than 2 tasks with
> GenericfMRIVolume script
>
> Thanks, the script is attached. Also the directory structure is after the
> script.
>
> #!/bin/bash
>
> get_batch_options() {
> local arguments=($@)
>
> unset command_line_specified_study_folder
> unset command_line_specified_subj_list
> unset command_line_specified_run_local
>
> local index=0
> local numArgs=${#arguments[@]}
> local argument
>
> while [ ${index} -lt ${numArgs} ]; do
> argument=${arguments[index]}
>
> case ${argument} in
> --StudyFolder=*)
> command_line_specified_study_folder=${argument/*=/""}
> index=$(( index + 1 ))
> ;;
> --Subjlist=*)
> command_line_specified_subj_list=${argument/*=/""}
> index=$(( index + 1 ))
> ;;
> --runlocal)
> command_line_specified_run_local="TRUE"
> index=$(( index + 1 ))
> ;;
> esac
> done
> }
>
> get_batch_options $@
>
> StudyFolder="/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoSkyra"
> Subjlist="analysis"
> EnvironmentScript="/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh"
>
> if [ -n "${command_line_specified_study_folder}" ]; then
> StudyFolder="${command_line_specified_study_folder}"
> fi
>
> if [ -n "${command_line_specified_subj_list}" ]; then
> Subjlist="${command_line_specified_subj_list}"
> fi
>
> # Requirements for this script
> # installed versions of: FSL (version 5.0.6), FreeSurfer (version 5.3.0-HCP)
> , gradunwarp (HCP version 1.0.1)
> # environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR , PATH (for
> gradient_unwarp.py)
>
> #Set up pipeline environment variables and software
> . ${EnvironmentScript}
>
> # Log the originating call
> echo "$@"
>
> #if [ X$SGE_ROOT != X ] ; then
> QUEUE="-q long.q"
> #fi
>
> PRINTCOM=""
> #PRINTCOM="echo"
> #QUEUE="-q veryshort.q"
>
> ## INPUTS
> ##
>
> #Scripts called by this script do NOT assume anything about the form of the
> input names or paths.
> #This batch script assumes the HCP raw data naming convention, e.g. for
> tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:
>
> #
> ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR.nii.gz
> #
> ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR_SBRef.nii.gz
>
> #
> ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL.nii.gz
> #
> ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL_SBRef.nii.gz
>
> #
> ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
> #
> ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz
>
> #
> ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
> #
> ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz
>
> #Change Scan Settings: Dwelltime, FieldMap Delta TE (if using), and
> $PhaseEncodinglist to match your images
> #These are set to match the HCP Protocol by default
>
> #If using gradient distortion correction, use the coefficents from your
> scanner
> #The HCP gradient distortion coefficents are only available through Siemens
> #
Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
Yes it is the EPI phase encoding direction. The positive and negative phase encoding directions are conventional (see the notes next to the variables for the convention). Basically, you want them in the order that produces a field map whose signs look
like a regular field map (as opposed to looking the opposite of it). Then all that remains is to specify the EPI phase encoding direction. If you were to reverse the spin echo image order, you would also have to reverse the EPI phase encoding direction (then
the correction would still work fine, but the topup field map would not have the same +/- convention as a regular field map.
Peace,
Matt.
From: , Gregory <gregory.b...@hhchealth.org>
Date: Tuesday, December 9, 2014 at 2:49 PM
To: Matt Glasser <glass...@wusm.wustl.edu>, "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org>
Cc: "Brown, Tim" <bro...@mir.wustl.edu>
Subject: RE: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
So the phase encoding list corresponds to the EPI encoding direction, not the spin echo direction. How does TopUp know the encoding directions of the spin echos? Does it assume the positive image is the same
encoding direction as the EPI and the negative image is the opposite direction?
From: Glasser, Matthew [mailto:glass...@wusm.wustl.edu]
Sent: Tuesday, December 09, 2014 3:33 PM
To: Book, Gregory; hcp-users@humanconnectome.org
Cc: Brown, Tim
Subject: Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
You need to expand the PhaseEncodinglist variable to have the same length as the Tasklist variable
(i.e. same number of elements separated by spaces). This probably should be made clearer in the comments for these variables.
Peace,
Matt.
From: , Gregory <gregory.b...@hhchealth.org>
Date: Tuesday, December 9, 2014 at 2:19 PM
To: Matt Glasser <glass...@wusm.wustl.edu>, "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org>
Subject: RE: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
Thanks, the script is attached. Also the directory structure is after the script.
#!/bin/bash
get_batch_options() {
local arguments=($@)
unset command_line_specified_study_folder
unset command_line_specified_subj_list
unset command_line_specified_run_local
local index=0
local numArgs=${#arguments[@]}
local argument
while [ ${index} -lt ${numArgs} ]; do
argument=${arguments[index]}
case ${argument} in
--StudyFolder=*)
command_line_specified_study_folder=${argument/*=/""}
index=$(( index + 1 ))
;;
--Subjlist=*)
command_line_specified_subj_list=${argument/*=/""}
index=$(( index + 1 ))
;;
--runlocal)
command_line_specified_run_local="TRUE"
index=$(( index + 1 ))
;;
esac
done
}
get_batch_options $@
StudyFolder="/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoSkyra"
Subjlist="analysis"
EnvironmentScript="/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh"
if [ -n "${command_line_specified_study_folder}" ]; then
StudyFolder="${command_line_specified_study_folder}"
fi
if [ -n "${command_line_specified_subj_list}" ]; then
Subjlist="${command_line_specified_subj_list}"
fi
# Requirements for this script
# installed versions of: FSL (version 5.0.6), FreeSurfer (version 5.3.0-HCP) , gradunwarp (HCP version 1.0.1)
# environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR , PATH (for gradient_unwarp.py)
#Set up pipeline environment variables and software
. ${EnvironmentScript}
# Log the originating call
echo "$@"
#if [ X$SGE_ROOT != X ] ; then
QUEUE="-q long.q"
#fi
PRINTCOM=""
#PRINTCOM="echo"
#QUEUE="-q veryshort.q"
## INPUTS ##
#Scripts called by this script do NOT assume anything about the form of the input names or paths.
#This batch script assumes the HCP raw data naming convention, e.g. for tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR_SBRef.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL_SBRef.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tf
Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
So the phase encoding list corresponds to the EPI encoding direction, not the
spin echo direction. How does TopUp know the encoding directions of the spin
echos? Does it assume the positive image is the same encoding direction as the
EPI and the negative image is the opposite direction?
From: Glasser, Matthew [mailto:glass...@wusm.wustl.edu]
Sent: Tuesday, December 09, 2014 3:33 PM
To: Book, Gregory; hcp-users@humanconnectome.org
Cc: Brown, Tim
Subject: Re: [HCP-Users] error when processing more than 2 tasks with
GenericfMRIVolume script
You need to expand the PhaseEncodinglist variable to have the same length as
the Tasklist variable (i.e. same number of elements separated by spaces). This
probably should be made clearer in the comments for these variables.
Peace,
Matt.
From: , Gregory
mailto:gregory.b...@hhchealth.org>>
Date: Tuesday, December 9, 2014 at 2:19 PM
To: Matt Glasser mailto:glass...@wusm.wustl.edu>>,
"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>"
mailto:hcp-users@humanconnectome.org>>
Subject: RE: [HCP-Users] error when processing more than 2 tasks with
GenericfMRIVolume script
Thanks, the script is attached. Also the directory structure is after the
script.
#!/bin/bash
get_batch_options() {
local arguments=($@)
unset command_line_specified_study_folder
unset command_line_specified_subj_list
unset command_line_specified_run_local
local index=0
local numArgs=${#arguments[@]}
local argument
while [ ${index} -lt ${numArgs} ]; do
argument=${arguments[index]}
case ${argument} in
--StudyFolder=*)
command_line_specified_study_folder=${argument/*=/""}
index=$(( index + 1 ))
;;
--Subjlist=*)
command_line_specified_subj_list=${argument/*=/""}
index=$(( index + 1 ))
;;
--runlocal)
command_line_specified_run_local="TRUE"
index=$(( index + 1 ))
;;
esac
done
}
get_batch_options $@
StudyFolder="/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoSkyra"
Subjlist="analysis"
EnvironmentScript="/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh"
if [ -n "${command_line_specified_study_folder}" ]; then
StudyFolder="${command_line_specified_study_folder}"
fi
if [ -n "${command_line_specified_subj_list}" ]; then
Subjlist="${command_line_specified_subj_list}"
fi
# Requirements for this script
# installed versions of: FSL (version 5.0.6), FreeSurfer (version 5.3.0-HCP) ,
gradunwarp (HCP version 1.0.1)
# environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR , PATH (for
gradient_unwarp.py)
#Set up pipeline environment variables and software
. ${EnvironmentScript}
# Log the originating call
echo "$@"
#if [ X$SGE_ROOT != X ] ; then
QUEUE="-q long.q"
#fi
PRINTCOM=""
#PRINTCOM="echo"
#QUEUE="-q veryshort.q"
## INPUTS
##
#Scripts called by this script do NOT assume anything about the form of the
input names or paths.
#This batch script assumes the HCP raw data naming convention, e.g. for
tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR_SBRef.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL_SBRef.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz
#Change Scan Settings: Dwelltime, FieldMap Delta TE (if using), and
$PhaseEncodinglist to match your images
#These are set to match the HCP Protocol by default
#If using gradient distortion correction, use the coefficents from your scanner
#The HCP gradient distortion coefficents are only available through Siemens
#Gradient distortion in standard scanners like the Trio is much less than for
the HCP Skyra.
#To get accurate EPI distortion correction with TOPUP, the flags in
PhaseEncodinglist must match the phase encoding
#direction of the EPI scan, and you must have used the correct images in
SpinEchoPhase
Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
You need to expand the PhaseEncodinglist variable to have the same length as the Tasklist variable (i.e. same number of elements
separated by spaces). This probably should be made clearer in the comments for these variables.
Peace,
Matt.
From: , Gregory <gregory.b...@hhchealth.org>
Date: Tuesday, December 9, 2014 at 2:19 PM
To: Matt Glasser <glass...@wusm.wustl.edu>, "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org>
Subject: RE: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
Thanks, the script is attached. Also the directory structure is after the script.
#!/bin/bash
get_batch_options() {
local arguments=($@)
unset command_line_specified_study_folder
unset command_line_specified_subj_list
unset command_line_specified_run_local
local index=0
local numArgs=${#arguments[@]}
local argument
while [ ${index} -lt ${numArgs} ]; do
argument=${arguments[index]}
case ${argument} in
--StudyFolder=*)
command_line_specified_study_folder=${argument/*=/""}
index=$(( index + 1 ))
;;
--Subjlist=*)
command_line_specified_subj_list=${argument/*=/""}
index=$(( index + 1 ))
;;
--runlocal)
command_line_specified_run_local="TRUE"
index=$(( index + 1 ))
;;
esac
done
}
get_batch_options $@
StudyFolder="/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoSkyra"
Subjlist="analysis"
EnvironmentScript="/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh"
if [ -n "${command_line_specified_study_folder}" ]; then
StudyFolder="${command_line_specified_study_folder}"
fi
if [ -n "${command_line_specified_subj_list}" ]; then
Subjlist="${command_line_specified_subj_list}"
fi
# Requirements for this script
# installed versions of: FSL (version 5.0.6), FreeSurfer (version 5.3.0-HCP) , gradunwarp (HCP version 1.0.1)
# environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR , PATH (for gradient_unwarp.py)
#Set up pipeline environment variables and software
. ${EnvironmentScript}
# Log the originating call
echo "$@"
#if [ X$SGE_ROOT != X ] ; then
QUEUE="-q long.q"
#fi
PRINTCOM=""
#PRINTCOM="echo"
#QUEUE="-q veryshort.q"
## INPUTS ##
#Scripts called by this script do NOT assume anything about the form of the input names or paths.
#This batch script assumes the HCP raw data naming convention, e.g. for tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR_SBRef.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL_SBRef.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz
#Change Scan Settings: Dwelltime, FieldMap Delta TE (if using), and $PhaseEncodinglist to match your images
#These are set to match the HCP Protocol by default
#If using gradient distortion correction, use the coefficents from your scanner
#The HCP gradient distortion coefficents are only available through Siemens
#Gradient distortion in standard scanners like the Trio is much less than for the HCP Skyra.
#To get accurate EPI distortion correction with TOPUP, the flags in PhaseEncodinglist must match the phase encoding
#direction of the EPI scan, and you must have used the correct images in SpinEchoPhaseEncodeNegative and Positive
#variables. If the distortion is twice as bad as in the original images, flip either the order of the spin echo
#images or reverse the phase encoding list flag. The pipeline expects you to have used the same phase encoding
#axis in the fMRI data as in the spin echo field map data (x/-x or y/-y).
# DO WORK ##
Tasklist=" domino0 domino1 domino2 domino3"
PhaseEncodinglist="y y-"
for Subject in $Subjlist ; do
echo $Subject
i=1
for fMRIName in $Taskl
Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
xrwx 8,551,575
/analysis/unprocessed/3T/domino0/analysis_3T_SpinEchoFieldMap_LR.nii.gz Dec
9, 2014 15:52:11rwxr-xr-x1,119,399
/analysis/unprocessed/3T/domino0/analysis_3T_SpinEchoFieldMap_RL.nii.gz Dec
9, 2014 15:52:11rwxr-xr-x1,127,080
/analysis/unprocessed/3T/domino0/analysis_3T_domino0.nii.gz Dec 9,
2014 15:51:37rwxr-xr-x 486,903,684
/analysis/unprocessed/3T/domino0/analysis_3T_domino0_SBRef.nii.gz
Dec 9, 2014 15:52:10rwxr-xr-x446,348
/analysis/unprocessed/3T/domino1/analysis_3T_SpinEchoFieldMap_LR.nii.gz Dec
9, 2014 15:52:11rwxr-xr-x1,119,399
/analysis/unprocessed/3T/domino1/analysis_3T_SpinEchoFieldMap_RL.nii.gz Dec
9, 2014 15:52:11rwxr-xr-x1,127,080
/analysis/unprocessed/3T/domino1/analysis_3T_domino1.nii.gz Dec 9,
2014 15:51:46rwxr-xr-x 488,162,639
/analysis/unprocessed/3T/domino1/analysis_3T_domino1_SBRef.nii.gz
Dec 9, 2014 15:52:11rwxr-xr-x447,898
/analysis/unprocessed/3T/domino2/analysis_3T_SpinEchoFieldMap_LR.nii.gz Dec
9, 2014 15:52:11rwxr-xr-x1,119,399
/analysis/unprocessed/3T/domino2/analysis_3T_SpinEchoFieldMap_RL.nii.gz Dec
9, 2014 15:52:11rwxr-xr-x1,127,080
/analysis/unprocessed/3T/domino2/analysis_3T_domino2.nii.gz Dec 9,
2014 15:51:58rwxr-xr-x 486,336,577
/analysis/unprocessed/3T/domino2/analysis_3T_domino2_SBRef.nii.gz
Dec 9, 2014 15:52:11rwxr-xr-x447,153
/analysis/unprocessed/3T/domino3/analysis_3T_SpinEchoFieldMap_LR.nii.gz Dec
9, 2014 15:52:11rwxr-xr-x1,119,399
/analysis/unprocessed/3T/domino3/analysis_3T_SpinEchoFieldMap_RL.nii.gz Dec
9, 2014 15:52:12rwxr-xr-x1,127,080
/analysis/unprocessed/3T/domino3/analysis_3T_domino3.nii.gz Dec 9,
2014 15:52:10rwxr-xr-x 484,561,229
/analysis/unprocessed/3T/domino3/analysis_3T_domino3_SBRef.nii.gz
Dec 9, 2014 15:52:11rwxr-xr-x445,265
From: Glasser, Matthew [mailto:glass...@wusm.wustl.edu]
Sent: Tuesday, December 09, 2014 2:43 PM
To: Book, Gregory; hcp-users@humanconnectome.org
Subject: Re: [HCP-Users] error when processing more than 2 tasks with
GenericfMRIVolume script
Can you post the text of your launcher script? Only one task is run at a time
inside the pipeline, so the issue will be there.
Peace,
Matt.
From: , Gregory
mailto:gregory.b...@hhchealth.org>>
Date: Tuesday, December 9, 2014 at 12:29 PM
To: "hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>"
mailto:hcp-users@humanconnectome.org>>
Subject: [HCP-Users] error when processing more than 2 tasks with
GenericfMRIVolume script
We've run into a weird problem when using the GenericfMRIVolume batch script to
process more than 2 tasks at a time. When we setup any more than 2 tasks (we've
tried 3 and 4 tasks), the first 2 run correctly and without error, but any
after that crash in the TopUp section with the following error (the failed
tasks all get to the same point):
# RUN TOPUP
# Needs FSL (version 5.0.6)
${FSLDIR}/bin/topup --imain=${WD}/BothPhases --datain=$txtfname
--config=${TopupConfig} --out=${WD}/Coefficents --iout=${WD}/Magnitudes
--fout=${WD}/TopupField --dfout=${WD}/WarpField --rbmout=${WD}/MotionMatrix
--jacout=${WD}/Jacobian -v
Could not open matrix file
/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt
Part of FSL (build 506)
topup
Usage:
topup --imain= --datain= --config= --coutname=my_field
Compulsory arguments (You MUST set one or more of):
--imain name of 4D file with images
--datainname of text file with PE
directions/times
Optional arguments (You may optionally specify one or more of):
--out base-name of output files (spline
coefficients (Hz) and movement parameters)
--foutname of image file with field (Hz)
--iout name of 4D image file with unwarped
images
--logoutName of log-file
--warpres(approximate) resolution (in mm) of warp
basis for the different sub-sampling levels, default 10
--subsamp sub-sampling scheme, default 1
--fwhm FWHM (in mm) of gaussian smoothing
kernel, default 8
--configName of config file specifying command
line arguments
--miter Max # of non-linear iterations,
default 5
--lambda
Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
Can you post the text of your launcher script? Only one task is run at a time inside the pipeline, so the issue will be there.
Peace,
Matt.
From: , Gregory <gregory.b...@hhchealth.org>
Date: Tuesday, December 9, 2014 at 12:29 PM
To: "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org>
Subject: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
We’ve run into a weird problem when using the GenericfMRIVolume batch script to process more than 2 tasks at a time. When we setup any more than 2 tasks (we’ve tried 3 and 4 tasks), the first 2 run correctly and without error, but any after
that crash in the TopUp section with the following error (the failed tasks all get to the same point):
# RUN TOPUP
# Needs FSL (version 5.0.6)
${FSLDIR}/bin/topup --imain=${WD}/BothPhases --datain=$txtfname --config=${TopupConfig} --out=${WD}/Coefficents --iout=${WD}/Magnitudes --fout=${WD}/TopupField --dfout=${WD}/WarpField --rbmout=${WD}/MotionMatrix --jacout=${WD}/Jacobian
-v
Could not open matrix file /home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt
Part of FSL (build 506)
topup
Usage:
topup --imain= --datain= --config= --coutname=my_field
Compulsory arguments (You MUST set one or more of):
--imain name of 4D file with images
--datain name of text file with PE directions/times
Optional arguments (You may optionally specify one or more of):
--out base-name of output files (spline coefficients (Hz) and movement parameters)
--fout name of image file with field (Hz)
--iout name of 4D image file with unwarped images
--logout Name of log-file
--warpres (approximate) resolution (in mm) of warp basis for the different sub-sampling levels, default 10
--subsamp sub-sampling scheme, default 1
--fwhm FWHM (in mm) of gaussian smoothing kernel, default 8
--config Name of config file specifying command line arguments
--miter Max # of non-linear iterations, default 5
--lambda Weight of regularisation, default depending on --ssqlambda and --regmod switches. See user documetation.
--ssqlambda If set (=1), lambda is weighted by current ssq, default 1
--regmod Model for regularisation of warp-field [membrane_energy bending_energy], default bending_energy
--estmov Estimate movements if set, default 1 (true)
--minmet Minimisation method 0=Levenberg-Marquardt, 1=Scaled Conjugate Gradient, default 0 (LM)
--splineorder Order of spline, 2->Qadratic spline, 3->Cubic spline. Default=3
--numprec Precision for representing Hessian, double or float. Default double
--interp Image interpolation model, linear or spline. Default spline
--scale If set (=1), the images are individually scaled to a common mean, default 0 (false)
--regrid If set (=1), the calculations are done in a different grid, default 1 (true)
-h,--help display help info
-v,--verbose Print diagonostic information while running
-h,--help display help info
TopupFileIO:: msg=TopupDatafileReader:: error reading file /home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt
set -- --path=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST --subject=analysis --fmriname=shop1 --fmritcs=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_shop1.nii.gz
--fmriscout=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_shop1_SBRef.nii.gz --SEPhaseNeg=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_SpinEchoFieldMap_LR.nii.gz
--SEPhasePos=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_SpinEchoFieldMap_RL.nii.gz --fmapmag=NONE --fmapphase=NONE --echospacing=0.00055 --echodiff=NONE --unwarpdir=
--fmrires=2 --dcmethod=TOPUP --gdcoeffs=/opt/HCP/coeff_AS098.grad --topupconfig=/opt/HCP/HCP/global/config/b02b0.cnf --printcom=
. /opt/HCP/HCP/Examples
[HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script
We've run into a weird problem when using the GenericfMRIVolume batch script to
process more than 2 tasks at a time. When we setup any more than 2 tasks (we've
tried 3 and 4 tasks), the first 2 run correctly and without error, but any
after that crash in the TopUp section with the following error (the failed
tasks all get to the same point):
# RUN TOPUP
# Needs FSL (version 5.0.6)
${FSLDIR}/bin/topup --imain=${WD}/BothPhases --datain=$txtfname
--config=${TopupConfig} --out=${WD}/Coefficents --iout=${WD}/Magnitudes
--fout=${WD}/TopupField --dfout=${WD}/WarpField --rbmout=${WD}/MotionMatrix
--jacout=${WD}/Jacobian -v
Could not open matrix file
/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt
Part of FSL (build 506)
topup
Usage:
topup --imain= --datain= --config= --coutname=my_field
Compulsory arguments (You MUST set one or more of):
--imain name of 4D file with images
--datainname of text file with PE
directions/times
Optional arguments (You may optionally specify one or more of):
--out base-name of output files (spline
coefficients (Hz) and movement parameters)
--foutname of image file with field (Hz)
--iout name of 4D image file with unwarped
images
--logoutName of log-file
--warpres(approximate) resolution (in mm) of warp
basis for the different sub-sampling levels, default 10
--subsamp sub-sampling scheme, default 1
--fwhm FWHM (in mm) of gaussian smoothing
kernel, default 8
--configName of config file specifying command
line arguments
--miter Max # of non-linear iterations,
default 5
--lambda Weight of regularisation, default
depending on --ssqlambda and --regmod switches. See user documetation.
--ssqlambda If set (=1), lambda is weighted by current
ssq, default 1
--regmod Model for regularisation of warp-field
[membrane_energy bending_energy], default bending_energy
--estmov Estimate movements if set, default 1 (true)
--minmet Minimisation method 0=Levenberg-Marquardt,
1=Scaled Conjugate Gradient, default 0 (LM)
--splineorder Order of spline, 2->Qadratic spline, 3->Cubic
spline. Default=3
--numprec Precision for representing Hessian, double
or float. Default double
--interpImage interpolation model, linear or
spline. Default spline
--scale If set (=1), the images are
individually scaled to a common mean, default 0 (false)
--regrid If set (=1), the calculations are done
in a different grid, default 1 (true)
-h,--help display help info
-v,--verbose Print diagonostic information while running
-h,--help display help info
TopupFileIO:: msg=TopupDatafileReader:: error reading file
/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt
set -- --path=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST
--subject=analysis --fmriname=shop1
--fmritcs=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_shop1.nii.gz
--fmriscout=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_shop1_SBRef.nii.gz
--SEPhaseNeg=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_SpinEchoFieldMap_LR.nii.gz
--SEPhasePos=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_SpinEchoFieldMap_RL.nii.gz
--fmapmag=NONE --fmapphase=NONE --echospacing=0.00055
--echodiff=NONE --unwarpdir= --fmrires=2 --dcmethod=TOPUP
--gdcoeffs=/opt/HCP/coeff_AS098.grad
--topupconfig=/opt/HCP/HCP/global/config/b02b0.cnf --printcom=
. /opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh
_
Gregory Book
Senior Technology Manager
Olin Neuropsychiatry Research Center, Institute of Living, Hartford Hospital
200 Retreat Avenue
Hartford, CT 06106
Tel: 860-545-7267 Fax: 860-545-7797
gregory.b...@hhchealth.org
http://nidb.sourceforge.net
This e-mail message, including any attachments, is for the sole use of t
