Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script

2014-12-10 Thread Timothy B. Brown 


On 09 Dec 2014 at 12:29, Book, Gregory
gregory.b...@hhchealth.org wrote:

 We’ve run into a weird problem when using the GenericfMRIVolume batch
 script to process more than 2 tasks at a time. When we setup any more
 than 2 tasks (we’ve tried 3 and 4 tasks), the first 2 run correctly
 and without error, but any after that crash in the TopUp section with
 the following error (the failed tasks all get to the same point): ...

On 09 Dec 2014 at 15:11, Xu, Junqian junqian...@mssm.edu wrote:

 After a second thought, maybe convert the list to array specifically
 for the comparison purpose so the rest of the pipeline script don't
 need to be modified to use array indexing. 
 ...

Gordon and Gregory,

Comments indicating that the Tasklist and PhaseEncodinglist should have
the same number of space-delimited elements, and a test to verify that
this is true before starting the main processing loop have been added to
the GenericfMRIVolumeProcessingPipelineBatch.sh script that is in the
example scripts directory.

This change will be included in the next released version of the HCP
Pipeline Scripts.

Gregory, thank you for pointing out the issue. Gordon, thank you for the
suggested changes.

Tim
--
 Timothy B. Brown
 Business  Technology Application Analyst III
 Pipeline Developer (Human Connectome Project)
 tbbrown(at)wustl.edu


The material in this message is private and may contain Protected Healthcare 
Information (PHI). 
If you are not the intended recipient, be advised that any unauthorized use, 
disclosure, copying 
or the taking of any action in reliance on the contents of this information is 
strictly prohibited. 
If you have received this email in error, please immediately notify the sender 
via telephone or 
return mail.


___
HCP-Users mailing list
HCP-Users@humanconnectome.org
http://lists.humanconnectome.org/mailman/listinfo/hcp-users

[HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script

2014-12-09 Thread Book, Gregory 
We've run into a weird problem when using the GenericfMRIVolume batch script to 
process more than 2 tasks at a time. When we setup any more than 2 tasks (we've 
tried 3 and 4 tasks), the first 2 run correctly and without error, but any 
after that crash in the TopUp section with the following error (the failed 
tasks all get to the same point):

# RUN TOPUP
# Needs FSL (version 5.0.6)
${FSLDIR}/bin/topup --imain=${WD}/BothPhases --datain=$txtfname 
--config=${TopupConfig} --out=${WD}/Coefficents --iout=${WD}/Magnitudes 
--fout=${WD}/TopupField --dfout=${WD}/WarpField --rbmout=${WD}/MotionMatrix 
--jacout=${WD}/Jacobian -v
Could not open matrix file 
/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt

Part of FSL (build 506)
topup

Usage:
topup --imain= --datain= --config= --coutname=my_field


Compulsory arguments (You MUST set one or more of):
--imain  name of 4D file with images
--datainname of text file with PE 
directions/times

Optional arguments (You may optionally specify one or more of):
--out  base-name of output files (spline 
coefficients (Hz) and movement parameters)
--foutname of image file with field (Hz)
--iout name of 4D image file with unwarped 
images
--logoutName of log-file
--warpres(approximate) resolution (in mm) of warp 
basis for the different sub-sampling levels, default 10
--subsamp  sub-sampling scheme, default 1
--fwhm FWHM (in mm) of gaussian smoothing 
kernel, default 8
--configName of config file specifying command 
line arguments
--miter  Max # of non-linear iterations, 
default 5
--lambda  Weight of regularisation, default 
depending on --ssqlambda and --regmod switches. See user documetation.
--ssqlambda   If set (=1), lambda is weighted by current 
ssq, default 1
--regmod Model for regularisation of warp-field 
[membrane_energy bending_energy], default bending_energy
--estmov Estimate movements if set, default 1 (true)
--minmet Minimisation method 0=Levenberg-Marquardt, 
1=Scaled Conjugate Gradient, default 0 (LM)
--splineorder Order of spline, 2-Qadratic spline, 3-Cubic 
spline. Default=3
--numprec  Precision for representing Hessian, double 
or float. Default double
--interpImage interpolation model, linear or 
spline. Default spline
--scale   If set (=1), the images are 
individually scaled to a common mean, default 0 (false)
--regrid If set (=1), the calculations are done 
in a different grid, default 1 (true)
-h,--help  display help info
-v,--verbose   Print diagonostic information while running
-h,--help  display help info



TopupFileIO:: msg=TopupDatafileReader:: error reading file 
/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt
set -- --path=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST
   --subject=analysis   --fmriname=shop1   
--fmritcs=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_shop1.nii.gz
   
--fmriscout=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_shop1_SBRef.nii.gz
   
--SEPhaseNeg=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_SpinEchoFieldMap_LR.nii.gz
   
--SEPhasePos=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_SpinEchoFieldMap_RL.nii.gz
   --fmapmag=NONE   --fmapphase=NONE   --echospacing=0.00055   
--echodiff=NONE   --unwarpdir=   --fmrires=2   --dcmethod=TOPUP 
  --gdcoeffs=/opt/HCP/coeff_AS098.grad   
--topupconfig=/opt/HCP/HCP/global/config/b02b0.cnf   --printcom=
. /opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh

_
Gregory Book
Senior Technology Manager
Olin Neuropsychiatry Research Center, Institute of Living, Hartford Hospital
200 Retreat Avenue
Hartford, CT 06106
Tel: 860-545-7267 Fax: 860-545-7797
gregory.b...@hhchealth.orgmailto:gregory.b...@hhchealth.org
http://nidb.sourceforge.net


This e-mail message, including any attachments, is for the sole use of the

Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script

2014-12-09 Thread Glasser, Matthew 



Can you post the text of your launcher script? Only one task is run at a time inside the pipeline, so the issue will be there.


Peace,


Matt.




From: Book, Gregory gregory.b...@hhchealth.org
Date: Tuesday, December 9, 2014 at 12:29 PM
To: hcp-users@humanconnectome.org hcp-users@humanconnectome.org
Subject: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script








We’ve run into a weird problem when using the GenericfMRIVolume batch script to process more than 2 tasks at a time. When we setup any more than 2 tasks (we’ve tried 3 and 4 tasks), the first 2 run correctly and without error, but any after
 that crash in the TopUp section with the following error (the failed tasks all get to the same point):

# RUN TOPUP
# Needs FSL (version 5.0.6)
${FSLDIR}/bin/topup --imain=${WD}/BothPhases --datain=$txtfname --config=${TopupConfig} --out=${WD}/Coefficents --iout=${WD}/Magnitudes --fout=${WD}/TopupField --dfout=${WD}/WarpField --rbmout=${WD}/MotionMatrix --jacout=${WD}/Jacobian
 -v 
Could not open matrix file /home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt

Part of FSL (build 506)
topup

Usage: 
topup --imain= --datain= --config= --coutname=my_field


Compulsory arguments (You MUST set one or more of):
 --imain name of 4D file with images
 --datain name of text file with PE directions/times

Optional arguments (You may optionally specify one or more of):
 --out base-name of output files (spline coefficients (Hz) and movement parameters)
 --fout name of image file with field (Hz)
 --iout name of 4D image file with unwarped images
 --logout Name of log-file
 --warpres (approximate) resolution (in mm) of warp basis for the different sub-sampling levels, default 10
 --subsamp sub-sampling scheme, default 1
 --fwhm FWHM (in mm) of gaussian smoothing kernel, default 8
 --config Name of config file specifying command line arguments
 --miter Max # of non-linear iterations, default 5
 --lambda Weight of regularisation, default depending on --ssqlambda and --regmod switches. See user documetation.
 --ssqlambda If set (=1), lambda is weighted by current ssq, default 1
 --regmod Model for regularisation of warp-field [membrane_energy bending_energy], default bending_energy
 --estmov Estimate movements if set, default 1 (true)
 --minmet Minimisation method 0=Levenberg-Marquardt, 1=Scaled Conjugate Gradient, default 0 (LM)
 --splineorder Order of spline, 2-Qadratic spline, 3-Cubic spline. Default=3
 --numprec Precision for representing Hessian, double or float. Default double
 --interp Image interpolation model, linear or spline. Default spline
 --scale If set (=1), the images are individually scaled to a common mean, default 0 (false)
 --regrid If set (=1), the calculations are done in a different grid, default 1 (true)
 -h,--help display help info
 -v,--verbose Print diagonostic information while running
 -h,--help display help info



TopupFileIO:: msg=TopupDatafileReader:: error reading file /home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/shop1/DistortionCorrectionAndEPIToT1wReg_FLIRTBBRAndFreeSurferBBRbased/FieldMap/acqparams.txt
set -- --path=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST --subject=analysis --fmriname=shop1 --fmritcs=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_shop1.nii.gz
 --fmriscout=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_shop1_SBRef.nii.gz --SEPhaseNeg=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_SpinEchoFieldMap_LR.nii.gz
 --SEPhasePos=/home/pipeline/onrc/data/pipeline/S8931EBN/2/HCPfMRI-SMARTTEST/analysis/unprocessed/3T/shop1/analysis_3T_SpinEchoFieldMap_RL.nii.gz --fmapmag=NONE --fmapphase=NONE --echospacing=0.00055 --echodiff=NONE --unwarpdir=
 --fmrires=2 --dcmethod=TOPUP --gdcoeffs=/opt/HCP/coeff_AS098.grad --topupconfig=/opt/HCP/HCP/global/config/b02b0.cnf --printcom=
. /opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh

_
Gregory Book
Senior Technology Manager
Olin Neuropsychiatry Research Center, Institute of Living, Hartford Hospital
200 Retreat Avenue
Hartford, CT 06106
Tel: 860-545-7267 Fax: 860-545-7797
gregory.b...@hhchealth.org
http://nidb.sourceforge.net



This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure,
 or distribution is prohibited. If you are not the intended recipient, or an employee or agent responsible for delivering the message to the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message, including
 any attachments

Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script

2014-12-09 Thread Glasser, Matthew 



You need to expand thePhaseEncodinglistvariable to have the same length as theTasklistvariable (i.e. same number of elements
 separated by spaces). This probably should be made clearer in the comments for these variables. 


Peace,


Matt.




From: Book, Gregory gregory.b...@hhchealth.org
Date: Tuesday, December 9, 2014 at 2:19 PM
To: Matt Glasser glass...@wusm.wustl.edu, hcp-users@humanconnectome.org hcp-users@humanconnectome.org
Subject: RE: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script








Thanks, the script is attached. Also the directory structure is after the script.

#!/bin/bash 

get_batch_options() {
 local arguments=($@)

 unset command_line_specified_study_folder
 unset command_line_specified_subj_list
 unset command_line_specified_run_local

 local index=0
 local numArgs=${#arguments[@]}
 local argument

 while [ ${index} -lt ${numArgs} ]; do
 argument=${arguments[index]}

 case ${argument} in
 --StudyFolder=*)
 command_line_specified_study_folder=${argument/*=/}
 index=$(( index  1 ))
 ;;
 --Subjlist=*)
 command_line_specified_subj_list=${argument/*=/}
 index=$(( index  1 ))
 ;;
 --runlocal)
 command_line_specified_run_local=TRUE
 index=$(( index  1 ))
 ;;
 esac
 done
}

get_batch_options $@

StudyFolder=/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoSkyra
Subjlist=analysis
EnvironmentScript=/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh

if [ -n ${command_line_specified_study_folder} ]; then
 StudyFolder=${command_line_specified_study_folder}
fi

if [ -n ${command_line_specified_subj_list} ]; then
 Subjlist=${command_line_specified_subj_list}
fi

# Requirements for this script
# installed versions of: FSL (version 5.0.6), FreeSurfer (version 5.3.0-HCP) , gradunwarp (HCP version 1.0.1)
# environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR , PATH (for gradient_unwarp.py)

#Set up pipeline environment variables and software
. ${EnvironmentScript}

# Log the originating call
echo $@

#if [ X$SGE_ROOT != X ] ; then
 QUEUE=-q long.q
#fi

PRINTCOM=
#PRINTCOM=echo
#QUEUE=-q veryshort.q

## INPUTS ##


#Scripts called by this script do NOT assume anything about the form of the input names or paths.
#This batch script assumes the HCP raw data naming convention, e.g. for tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:

# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR_SBRef.nii.gz

# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL_SBRef.nii.gz

# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz

# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz

#Change Scan Settings: Dwelltime, FieldMap Delta TE (if using), and $PhaseEncodinglist to match your images
#These are set to match the HCP Protocol by default

#If using gradient distortion correction, use the coefficents from your scanner
#The HCP gradient distortion coefficents are only available through Siemens
#Gradient distortion in standard scanners like the Trio is much less than for the HCP Skyra.

#To get accurate EPI distortion correction with TOPUP, the flags in PhaseEncodinglist must match the phase encoding
#direction of the EPI scan, and you must have used the correct images in SpinEchoPhaseEncodeNegative and Positive
#variables. If the distortion is twice as bad as in the original images, flip either the order of the spin echo
#images or reverse the phase encoding list flag. The pipeline expects you to have used the same phase encoding
#axis in the fMRI data as in the spin echo field map data (x/-x or y/-y).


# DO WORK ##

Tasklist= domino0 domino1 domino2 domino3
PhaseEncodinglist=y y-

for Subject in $Subjlist ; do
 echo $Subject

 i=1
 for fMRIName in $Tasklist ; do
 echo  ${fMRIName}
 UnwarpDir=`echo $PhaseEncodinglist | cut -d   -f $i`
 fMRITimeSeries=${StudyFolder}/${Subject}/unprocessed/3T/${fMRIName}/${Subject}_3T_${fMRIName}.nii.gz
fMRISBRef=${StudyFolder}/${Subject}/unprocessed/3T/${fMRIName}/${Subject}_3T_${fMRIName}_SBRef.nii.gz #A single band reference image (SBRef) is recommended if using multiband, set to NONE if you want to use
 the first volume of the timeseries for motion correction
 DwellTime=0.00055 #Echo Spacing or Dwelltime of fMRI image, set to NONE if not used. Dwelltime = 1/(BandwidthPerPixelPhaseEncode * # of phase encoding samples

Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script

2014-12-09 Thread Book, Gregory 
So the phase encoding list corresponds to the EPI encoding direction, not the 
spin echo direction. How does TopUp know the encoding directions of the spin 
echos? Does it assume the positive image is the same encoding direction as the 
EPI and the negative image is the opposite direction?

From: Glasser, Matthew [mailto:glass...@wusm.wustl.edu]
Sent: Tuesday, December 09, 2014 3:33 PM
To: Book, Gregory; hcp-users@humanconnectome.org
Cc: Brown, Tim
Subject: Re: [HCP-Users] error when processing more than 2 tasks with 
GenericfMRIVolume script

You need to expand the PhaseEncodinglist variable to have the same length as 
the Tasklist variable (i.e. same number of elements separated by spaces).  This 
probably should be made clearer in the comments for these variables.

Peace,

Matt.

From: Book, Gregory 
gregory.b...@hhchealth.orgmailto:gregory.b...@hhchealth.org
Date: Tuesday, December 9, 2014 at 2:19 PM
To: Matt Glasser glass...@wusm.wustl.edumailto:glass...@wusm.wustl.edu, 
hcp-users@humanconnectome.orgmailto:hcp-users@humanconnectome.org 
hcp-users@humanconnectome.orgmailto:hcp-users@humanconnectome.org
Subject: RE: [HCP-Users] error when processing more than 2 tasks with 
GenericfMRIVolume script

Thanks, the script is attached. Also the directory structure is after the 
script.

#!/bin/bash

get_batch_options() {
local arguments=($@)

unset command_line_specified_study_folder
unset command_line_specified_subj_list
unset command_line_specified_run_local

local index=0
local numArgs=${#arguments[@]}
local argument

while [ ${index} -lt ${numArgs} ]; do
argument=${arguments[index]}

case ${argument} in
--StudyFolder=*)
command_line_specified_study_folder=${argument/*=/}
index=$(( index + 1 ))
;;
--Subjlist=*)
command_line_specified_subj_list=${argument/*=/}
index=$(( index + 1 ))
;;
--runlocal)
command_line_specified_run_local=TRUE
index=$(( index + 1 ))
;;
esac
done
}

get_batch_options $@

StudyFolder=/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoSkyra
Subjlist=analysis
EnvironmentScript=/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh

if [ -n ${command_line_specified_study_folder} ]; then
StudyFolder=${command_line_specified_study_folder}
fi

if [ -n ${command_line_specified_subj_list} ]; then
Subjlist=${command_line_specified_subj_list}
fi

# Requirements for this script
#  installed versions of: FSL (version 5.0.6), FreeSurfer (version 5.3.0-HCP) , 
gradunwarp (HCP version 1.0.1)
#  environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR , PATH (for 
gradient_unwarp.py)

#Set up pipeline environment variables and software
. ${EnvironmentScript}

# Log the originating call
echo $@

#if [ X$SGE_ROOT != X ] ; then
QUEUE=-q long.q
#fi

PRINTCOM=
#PRINTCOM=echo
#QUEUE=-q veryshort.q

## INPUTS 
##

#Scripts called by this script do NOT assume anything about the form of the 
input names or paths.
#This batch script assumes the HCP raw data naming convention, e.g. for 
tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:

# 
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR_SBRef.nii.gz

# 
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL.nii.gz
#
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL_SBRef.nii.gz

# 
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
# 
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz

# 
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
# 
${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz

#Change Scan Settings: Dwelltime, FieldMap Delta TE (if using), and 
$PhaseEncodinglist to match your images
#These are set to match the HCP Protocol by default

#If using gradient distortion correction, use the coefficents from your scanner
#The HCP gradient distortion coefficents are only available through Siemens
#Gradient distortion in standard scanners like the Trio is much less than for 
the HCP Skyra.

#To get accurate EPI distortion correction with TOPUP, the flags in 
PhaseEncodinglist must match the phase encoding
#direction of the EPI scan, and you must have used the correct images in 
SpinEchoPhaseEncodeNegative and Positive
#variables.  If the distortion is twice as bad as in the original images, flip 
either the order of the spin echo

Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script

2014-12-09 Thread Glasser, Matthew 



Yes it is the EPI phase encoding direction. The positive and negative phase encoding directions are conventional (see the notes next to the variables for the convention). Basically, you want them in the order that produces a field map whose signs look
 like a regular field map (as opposed to looking the opposite of it). Then all that remains is to specify the EPI phase encoding direction. If you were to reverse the spin echo image order, you would also have to reverse the EPI phase encoding direction (then
 the correction would still work fine, but the topup field map would not have the same /- convention as a regular field map.


Peace,


Matt.




From: Book, Gregory gregory.b...@hhchealth.org
Date: Tuesday, December 9, 2014 at 2:49 PM
To: Matt Glasser glass...@wusm.wustl.edu, hcp-users@humanconnectome.org hcp-users@humanconnectome.org
Cc: Brown, Tim bro...@mir.wustl.edu
Subject: RE: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script








So the phase encoding list corresponds to the EPI encoding direction, not the spin echo direction. How does TopUp know the encoding directions of the spin echos? Does it assume the positive image is the same
 encoding direction as the EPI and the negative image is the opposite direction?



From: Glasser, Matthew [mailto:glass...@wusm.wustl.edu]

Sent: Tuesday, December 09, 2014 3:33 PM
To: Book, Gregory; hcp-users@humanconnectome.org
Cc: Brown, Tim
Subject: Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script




You need to expand thePhaseEncodinglistvariable to have the same length as theTasklistvariable
 (i.e. same number of elements separated by spaces). This probably should be made clearer in the comments for these variables. 





Peace,





Matt.





From: Book, Gregory gregory.b...@hhchealth.org
Date: Tuesday, December 9, 2014 at 2:19 PM
To: Matt Glasser glass...@wusm.wustl.edu, hcp-users@humanconnectome.org hcp-users@humanconnectome.org
Subject: RE: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script






Thanks, the script is attached. Also the directory structure is after the script.

#!/bin/bash 

get_batch_options() {
 local arguments=($@)

 unset command_line_specified_study_folder
 unset command_line_specified_subj_list
 unset command_line_specified_run_local

 local index=0
 local numArgs=${#arguments[@]}
 local argument

 while [ ${index} -lt ${numArgs} ]; do
 argument=${arguments[index]}

 case ${argument} in
 --StudyFolder=*)
 command_line_specified_study_folder=${argument/*=/}
 index=$(( index  1 ))
 ;;
 --Subjlist=*)
 command_line_specified_subj_list=${argument/*=/}
 index=$(( index  1 ))
 ;;
 --runlocal)
 command_line_specified_run_local=TRUE
 index=$(( index  1 ))
 ;;
 esac
 done
}

get_batch_options $@

StudyFolder=/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoSkyra
Subjlist=analysis
EnvironmentScript=/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh

if [ -n ${command_line_specified_study_folder} ]; then
 StudyFolder=${command_line_specified_study_folder}
fi

if [ -n ${command_line_specified_subj_list} ]; then
 Subjlist=${command_line_specified_subj_list}
fi

# Requirements for this script
# installed versions of: FSL (version 5.0.6), FreeSurfer (version 5.3.0-HCP) , gradunwarp (HCP version 1.0.1)
# environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR , PATH (for gradient_unwarp.py)

#Set up pipeline environment variables and software
. ${EnvironmentScript}

# Log the originating call
echo $@

#if [ X$SGE_ROOT != X ] ; then
 QUEUE=-q long.q
#fi

PRINTCOM=
#PRINTCOM=echo
#QUEUE=-q veryshort.q

## INPUTS ##


#Scripts called by this script do NOT assume anything about the form of the input names or paths.
#This batch script assumes the HCP raw data naming convention, e.g. for tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:

# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR_SBRef.nii.gz

# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL_SBRef.nii.gz

# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz

# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
# ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz

#Change Scan Settings: Dwelltime, FieldMap Delta TE (if using), and $PhaseEncodinglist to match your images
#These are set to match the HCP Protocol by default

#If using gradient distortion correction, use the coefficents from your

Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script

2014-12-09 Thread Xu, Junqian 
Tim,

A patch here to use array instead of space delimited string might be helpful 
for error detection:

e.g.

Tasklist=(domino0 domino1 domino2 domino3)
PhaseEncodinglist=(y y- y y-)

nTasklist=${#Tasklist[@]}
nPhaseEncodinglist=${#PhaseEncodinglist[@]}

if [ $nTasklist -ne $nPhaseEncodinglist ]; then
   Error message
fi


Gordon

 On Dec 9, 2014, at 3:33 PM, Glasser, Matthew glass...@wusm.wustl.edu wrote:
 
 You need to expand the PhaseEncodinglist variable to have the same length as 
 the Tasklist variable (i.e. same number of elements separated by spaces).  
 This probably should be made clearer in the comments for these variables.  
 
 Peace,
 
 Matt.
 
 From: Book, Gregory gregory.b...@hhchealth.org
 Date: Tuesday, December 9, 2014 at 2:19 PM
 To: Matt Glasser glass...@wusm.wustl.edu, hcp-users@humanconnectome.org 
 hcp-users@humanconnectome.org
 Subject: RE: [HCP-Users] error when processing more than 2 tasks with 
 GenericfMRIVolume script
 
 Thanks, the script is attached. Also the directory structure is after the 
 script.
  
 #!/bin/bash 
  
 get_batch_options() {
 local arguments=($@)
  
 unset command_line_specified_study_folder
 unset command_line_specified_subj_list
 unset command_line_specified_run_local
  
 local index=0
 local numArgs=${#arguments[@]}
 local argument
  
 while [ ${index} -lt ${numArgs} ]; do
 argument=${arguments[index]}
  
 case ${argument} in
 --StudyFolder=*)
 command_line_specified_study_folder=${argument/*=/}
 index=$(( index + 1 ))
 ;;
 --Subjlist=*)
 command_line_specified_subj_list=${argument/*=/}
 index=$(( index + 1 ))
 ;;
 --runlocal)
 command_line_specified_run_local=TRUE
 index=$(( index + 1 ))
 ;;
 esac
 done
 }
  
 get_batch_options $@
  
 StudyFolder=/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoSkyra
 Subjlist=analysis
 EnvironmentScript=/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh
  
 if [ -n ${command_line_specified_study_folder} ]; then
 StudyFolder=${command_line_specified_study_folder}
 fi
  
 if [ -n ${command_line_specified_subj_list} ]; then
 Subjlist=${command_line_specified_subj_list}
 fi
  
 # Requirements for this script
 #  installed versions of: FSL (version 5.0.6), FreeSurfer (version 5.3.0-HCP) 
 , gradunwarp (HCP version 1.0.1)
 #  environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR , PATH (for 
 gradient_unwarp.py)
  
 #Set up pipeline environment variables and software
 . ${EnvironmentScript}
  
 # Log the originating call
 echo $@
  
 #if [ X$SGE_ROOT != X ] ; then
 QUEUE=-q long.q
 #fi
  
 PRINTCOM=
 #PRINTCOM=echo
 #QUEUE=-q veryshort.q
  
 ## INPUTS 
 ##
  
 #Scripts called by this script do NOT assume anything about the form of the 
 input names or paths.
 #This batch script assumes the HCP raw data naming convention, e.g. for 
 tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:
  
 # 
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR.nii.gz
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_tfMRI_EMOTION_LR_SBRef.nii.gz
  
 # 
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL.nii.gz
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_tfMRI_EMOTION_RL_SBRef.nii.gz
  
 # 
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
 # 
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz
  
 # 
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_LR.nii.gz
 # 
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_SpinEchoFieldMap_RL.nii.gz
  
 #Change Scan Settings: Dwelltime, FieldMap Delta TE (if using), and 
 $PhaseEncodinglist to match your images
 #These are set to match the HCP Protocol by default
  
 #If using gradient distortion correction, use the coefficents from your 
 scanner
 #The HCP gradient distortion coefficents are only available through Siemens
 #Gradient distortion in standard scanners like the Trio is much less than for 
 the HCP Skyra.
  
 #To get accurate EPI distortion correction with TOPUP, the flags in 
 PhaseEncodinglist must match the phase encoding
 #direction of the EPI scan, and you must have used the correct images in 
 SpinEchoPhaseEncodeNegative and Positive
 #variables.  If the distortion is twice as bad as in the original images, 
 flip either the order of the spin echo
 #images or reverse the phase encoding list flag.  The pipeline expects you to 
 have used the same phase encoding

Re: [HCP-Users] error when processing more than 2 tasks with GenericfMRIVolume script

2014-12-09 Thread Xu, Junqian 
After a second thought, maybe convert the list to array specifically for the 
comparison purpose so the rest of the pipeline script don’t need to be modified 
to use array indexing.

Tasklist=domino0 domino1 domino2 domino3
PhaseEncodinglist=y y- y y-“

TaskArray=($Tasklist)
PhaseEncodingArray=($PhaseEcondinglist)

nTaskArray=${#TaskArray[@]}
nPhaseEncodingArray=${#PhaseEncodingArray[@]}

if [ $nTaskArray -ne $nPhaseEncodingArray ]; then
 Error message
fi

Gordon

 On Dec 9, 2014, at 4:05 PM, Glasser, Matthew glass...@wusm.wustl.edu wrote:
 
 Thanks Gordon, that would help too.
 
 On 12/9/14, 3:03 PM, Xu, Junqian junqian...@mssm.edu wrote:
 
 Tim,
 
 A patch here to use array instead of space delimited string might be
 helpful for error detection:
 
 e.g.
 
 Tasklist=(domino0 domino1 domino2 domino3)
 PhaseEncodinglist=(y y- y y-)
 
 nTasklist=${#Tasklist[@]}
 nPhaseEncodinglist=${#PhaseEncodinglist[@]}
 
 if [ $nTasklist -ne $nPhaseEncodinglist ]; then
  Error message
 fi
 
 
 Gordon
 
 On Dec 9, 2014, at 3:33 PM, Glasser, Matthew glass...@wusm.wustl.edu
 wrote:
 
 You need to expand the PhaseEncodinglist variable to have the same
 length as the Tasklist variable (i.e. same number of elements separated
 by spaces).  This probably should be made clearer in the comments for
 these variables.
 
 Peace,
 
 Matt.
 
 From: Book, Gregory gregory.b...@hhchealth.org
 Date: Tuesday, December 9, 2014 at 2:19 PM
 To: Matt Glasser glass...@wusm.wustl.edu,
 hcp-users@humanconnectome.org hcp-users@humanconnectome.org
 Subject: RE: [HCP-Users] error when processing more than 2 tasks with
 GenericfMRIVolume script
 
 Thanks, the script is attached. Also the directory structure is after
 the script.
 
 #!/bin/bash
 
 get_batch_options() {
local arguments=($@)
 
unset command_line_specified_study_folder
unset command_line_specified_subj_list
unset command_line_specified_run_local
 
local index=0
local numArgs=${#arguments[@]}
local argument
 
while [ ${index} -lt ${numArgs} ]; do
argument=${arguments[index]}
 
case ${argument} in
--StudyFolder=*)
command_line_specified_study_folder=${argument/*=/}
index=$(( index + 1 ))
;;
--Subjlist=*)
command_line_specified_subj_list=${argument/*=/}
index=$(( index + 1 ))
;;
--runlocal)
command_line_specified_run_local=TRUE
index=$(( index + 1 ))
;;
esac
done
 }
 
 get_batch_options $@
 
 
 StudyFolder=/home/pipeline/onrc/data/pipeline/S3777AUX/4/HCPfMRI-dominoS
 kyra
 Subjlist=analysis
 EnvironmentScript=/opt/HCP/HCP/Examples/Scripts/SetUpHCPPipeline.sh
 
 if [ -n ${command_line_specified_study_folder} ]; then
StudyFolder=${command_line_specified_study_folder}
 fi
 
 if [ -n ${command_line_specified_subj_list} ]; then
Subjlist=${command_line_specified_subj_list}
 fi
 
 # Requirements for this script
 #  installed versions of: FSL (version 5.0.6), FreeSurfer (version
 5.3.0-HCP) , gradunwarp (HCP version 1.0.1)
 #  environment: FSLDIR , FREESURFER_HOME , HCPPIPEDIR , CARET7DIR ,
 PATH (for gradient_unwarp.py)
 
 #Set up pipeline environment variables and software
 . ${EnvironmentScript}
 
 # Log the originating call
 echo $@
 
 #if [ X$SGE_ROOT != X ] ; then
QUEUE=-q long.q
 #fi
 
 PRINTCOM=
 #PRINTCOM=echo
 #QUEUE=-q veryshort.q
 
 ## INPUTS
 ##
 
 #Scripts called by this script do NOT assume anything about the form of
 the input names or paths.
 #This batch script assumes the HCP raw data naming convention, e.g. for
 tfMRI_EMOTION_LR and tfMRI_EMOTION_RL:
 
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_t
 fMRI_EMOTION_LR.nii.gz
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_t
 fMRI_EMOTION_LR_SBRef.nii.gz
 
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_t
 fMRI_EMOTION_RL.nii.gz
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_t
 fMRI_EMOTION_RL_SBRef.nii.gz
 
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_S
 pinEchoFieldMap_LR.nii.gz
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_LR/${Subject}_3T_S
 pinEchoFieldMap_RL.nii.gz
 
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_S
 pinEchoFieldMap_LR.nii.gz
 #
 ${StudyFolder}/${Subject}/unprocessed/3T/tfMRI_EMOTION_RL/${Subject}_3T_S
 pinEchoFieldMap_RL.nii.gz
 
 #Change Scan Settings: Dwelltime, FieldMap Delta TE (if using), and
 $PhaseEncodinglist to match your images
 #These are set to match the HCP Protocol by default
 
 #If using gradient distortion correction, use the coefficents from your
 scanner
 #The HCP gradient distortion coefficents are only available through
 Siemens
 #Gradient distortion in standard scanners like