Dear HCP experts, I wrote to this mailing list about a month ago with a problem that the FreeSurfer Pipeline would get interrupted after finishing recon-all, and starting FreeSurferHiResWhite script. The error log showed START: FreeSurferHighResWhite Unmatched ". and below set -- --subject=VIMS_HC_070 --subjectDIR=/home/lisak/Desktop/NMDA_HCP/controls/VIMS_HC_070/T1w --t1=/home/lisak/Desktop/NMDA_HCP/controls/VIMS_HC_070/T1w/T1w_acpc_dc_restore.nii.gz --t1brain=/home/lisak/Desktop/NMDA_HCP/controls/VIMS_HC_070/T1w/T1w_acpc_dc_restore_brain.nii.gz --t2=/home/lisak/Desktop/NMDA_HCP/controls/VIMS_HC_070/T1w/T2w_acpc_dc_restore.nii.gz --printcom= . /home/lisak/Desktop/Pipelines/Examples/Scripts/SetUpHCPPipeline.sh
The suggestion I got was that I probably removed or added a quote somewhere while editing the scripts. However, after looking carefully and multiple times at FreeSurferPipelineBatch.sh, FreeSurferPipeline.sh, as well as FreeSurferHiresWhite.sh, I have not been able to find any unmatched quotes. Looking at the message above the only thing that seems a bit weird is a dot between the equal sign and the beginning of the file path for the SetUp HCPPipeline script, after the --printcom segment. We don't see that dot and space after other headings, such as subject, t1, etc. However, I haven't been able to identify what is causing that space and dot to appear only after printcom. The corresponding segments of my script files seem identical to the scripts that are in the latest release from the GitHub website. Does this seem like a reasonable line of thought (that a space and a dot between the equal sign and the beginning of the path are not normal)? And if so, do you have any idea what might be causing it? Thanks in advance for any suggestions! Best, Darko On Mon, Jan 29, 2018 at 10:39 PM, Glasser, Matthew <glass...@wustl.edu> wrote: > gedit with bash shell highlighting (sh). > > Matt > > On 1/29/18, 11:50 AM, "Darko Komnenić" <komnen...@gmail.com> wrote: > > >Hi Matt, > >could you please tell me which editor with shell syntax highlighting > >you are using? I don't see any option to change the view/display mode > >of the text in the one I have installed on my computer. > >Thanks, > >Darko > > > >On 1/26/18, Glasser, Matthew <glass...@wustl.edu> wrote: > >> Here is the relevant part of the log: > >> > >> START: FreeSurferHighResWhite > >> Unmatched ". > >> > >> You don't ever get a message that this script finishes. If I paste the > >> current version of the script from GitHub into a editor with shell > >>syntax > >> highlighting there are no unopposed quotes, so you must have done > >>something > >> to your version of the script. > >> > >> Peace, > >> > >> Matt. > >> > >> From: Darko Komnenić <komnen...@gmail.com<mailto:komnen...@gmail.com>> > >> Date: Friday, January 26, 2018 at 11:58 AM > >> To: Matt Glasser <glass...@wustl.edu<mailto:glass...@wustl.edu>> > >> Cc: > >>"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: Re: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> If I interpreted the error message correctly, the unmatched quote error > >> appears after recon-all, and right after the pipeline tries to perform > >> FreeSurferHiresWhite.sh > >> This is what that part of script looks like in my case: > >> #Highres white stuff and Fine Tune T2w to T1w Reg > >> log_Msg "High resolution white matter and fine tune T2w to T1w > >> registration" > >> "$PipelineScripts"/FreeSurferHiresWhite.sh "$SubjectID" "$SubjectDIR" > >> "$T1wImage" "$T2wImage" > >> > >> There doesn't seem to me to be a quote problem, but I'm not sure? Also, > >>the > >> FreeSurferHiresWhite.sh script was never edited, and I've had the > >>pipelines > >> run successfully before, so I doubt that the problem is there. > >> Any ideas on where I should keep looking (i.e. which particular script > >>or > >> section)? Thanks! > >> > >> On Fri, Jan 26, 2018 at 6:34 PM, Darko Komnenić > >> <komnen...@gmail.com<mailto:komnen...@gmail.com>> wrote: > >> I did the usual changes of subject list and folder paths, as well as the > >> number of cores in the FreeSurfer script (not the > >>FreeSurferPipelineBatch, > >> but the one that's found in the Pipelines/FreeSurfer folder), I will go > >>over > >> them now, thanks! > >> > >> On Fri, Jan 26, 2018 at 6:24 PM, Glasser, Matthew > >> <glass...@wustl.edu<mailto:glass...@wustl.edu>> wrote: > >> Did you accidentally change on of the pipeline scripts? If you look at > >>the > >> end of that log file it complaining about an unmatched quote. > >> > >> Peace, > >> > >> Matt. > >> > >> From: > >> > >><hcp-users-boun...@humanconnectome.org<mailto:hcp > -users-bounces@humanconn > >>ectome.org>> > >> on behalf of Darko Komnenić > >> <komnen...@gmail.com<mailto:komnen...@gmail.com>> > >> Date: Friday, January 26, 2018 at 10:58 AM > >> > >> Cc: > >>"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: Re: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> Dear HCP experts, I reran the PreFS and FS pipeline, with an added > >>argument > >> for generating error and output logs. Once again, wmparc_1mm is not > >>inside > >> the T1w subfolder within individual partcipants' folders. I am now > >>sending > >> the log attached. If someone could take a look and tell me what went > >>wrong, > >> I'd be very grateful! > >> Best, > >> Darko > >> > >> On Mon, Jan 22, 2018 at 8:25 PM, Harms, Michael > >> <mha...@wustl.edu<mailto:mha...@wustl.edu>> wrote: > >> > >> If you want to separate stderr and stdout at the terminal you can also > >>do > >> (in an 'sh/bash' shell): > >> > >> ./FreeSurferPipelineBatch.sh 2> stderr.txt 1> stdout.txt > >> > >> cheers, > >> -MH > >> > >> -- > >> Michael Harms, Ph.D. > >> ----------------------------------------------------------- > >> Conte Center for the Neuroscience of Mental Disorders > >> Washington University School of Medicine > >> Department of Psychiatry, Box 8134 > >> 660 South Euclid > >> > >>Ave<https://maps.google.com/?q=660+South+Euclid+Ave&entry=gmail&source=g > > > >>. > >> Tel: 314-747-6173<tel:(314)%20747-6173> > >> St. Louis, MO 63110 Email: > >> mha...@wustl.edu<mailto:mha...@wustl.edu> > >> > >> From: "Glasser, Matthew" <glass...@wustl.edu<mailto:glass...@wustl.edu > >> > >> Date: Monday, January 22, 2018 at 12:58 PM > >> To: Darko Komnenić <komnen...@gmail.com<mailto:komnen...@gmail.com>>, > >> "Harms, Michael" <mha...@wustl.edu<mailto:mha...@wustl.edu>> > >> > >> Cc: > >>"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: Re: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> > >> ./FreeSurferPipelineBatch.sh > log.txt 2>&1 would collect both stderr > >>and > >> stdout into a single file. One of the nice things about using SGE or a > >> similar software is they automatically put stderr and stdout into > >>separate > >> log files. > >> > >> Peace, > >> > >> Matt. > >> > >> From: Darko Komnenić <komnen...@gmail.com<mailto:komnen...@gmail.com>> > >> Date: Monday, January 22, 2018 at 12:55 PM > >> To: "Harms, Michael" <mha...@wustl.edu<mailto:mha...@wustl.edu>> > >> Cc: Matt Glasser <glass...@wustl.edu<mailto:glass...@wustl.edu>>, > >> "hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: Re: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> Thanks Michael and Matt. > >> I indeed ran it from the terminal, as described on this website: > >> > >>https://github.com/Washington-University/Pipelines/wiki/v3.4.0-Release- > No > >>tes,-Installation,-and-Usage#structural-preprocessing > >> What would be your recommendation now? Rerun the pipelines but add a > >> modification like this? > >> > >> ./FreeSurferPipelineBatch.sh |& tee ~/filename.txt > >> > >> On Mon, Jan 22, 2018 at 7:38 PM, Harms, Michael > >> <mha...@wustl.edu<mailto:mha...@wustl.edu>> wrote: > >> > >> To expand on that slightly, the HCP Pipelines themselves don't > >>automatically > >> generate log files. You need to either capture the output to the > >>terminal > >> (e.g., using the 'tee' command) if running in a terminal, or use > >>appropriate > >> arguments in your compute cluster to capture the stderr and stdout. > >> > >> Cheers, > >> -MH > >> > >> -- > >> Michael Harms, Ph.D. > >> ----------------------------------------------------------- > >> Conte Center for the Neuroscience of Mental Disorders > >> Washington University School of Medicine > >> Department of Psychiatry, Box 8134 > >> 660 South Euclid > >> > >>Ave<https://maps.google.com/?q=660+South+Euclid+Ave&entry=gmail&source=g > > > >>. > >> Tel: 314-747-6173<tel:(314)%20747-6173> > >> St. Louis, MO 63110 Email: > >> mha...@wustl.edu<mailto:mha...@wustl.edu> > >> > >> From: > >> > >><hcp-users-boun...@humanconnectome.org<mailto:hcp > -users-bounces@humanconn > >>ectome.org>> > >> on behalf of "Glasser, Matthew" > >> <glass...@wustl.edu<mailto:glass...@wustl.edu>> > >> Date: Monday, January 22, 2018 at 12:35 PM > >> To: Darko Komnenić <komnen...@gmail.com<mailto:komnen...@gmail.com>> > >> Cc: > >>"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: Re: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> I don't know how you launched the pipelines. If you used SGE you would > >>look > >> in the stderr or stdout log files (e and o). If you just ran this in a > >> terminal and closed the terminal, the logs may no longer be available. > >> > >> Peace, > >> > >> Matt. > >> > >> From: Darko Komnenić <komnen...@gmail.com<mailto:komnen...@gmail.com>> > >> Date: Monday, January 22, 2018 at 12:03 PM > >> To: Matt Glasser <glass...@wustl.edu<mailto:glass...@wustl.edu>> > >> Cc: > >>"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: Re: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> Hi Matt, > >> not sure if I can't find it, or if it's just not there, but I don't see > >>that > >> log anywhere in the subject folder. Attached are the screenshot of the > >>T1w > >> folder for a particular participant, subfolder with the participant's > >>name > >> inside the T1w folder, as well as "scripts" subfolder where recon-all > >>logs > >> are found. Any idea where the complete log for FS Pipeline should be? > >> Thanks! > >> [line image 1][line image 2] > >> [line image 4] > >> > >> On Mon, Jan 22, 2018 at 6:22 PM, Glasser, Matthew > >> <glass...@wustl.edu<mailto:glass...@wustl.edu>> wrote: > >> That is not a complete FreeSurfer run, so you need to look at the log > >>of the > >> FreeSurfer HCP Pipeline itself. > >> > >> Peace, > >> > >> Matt. > >> > >> From: Darko Komnenić <komnen...@gmail.com<mailto:komnen...@gmail.com>> > >> Date: Monday, January 22, 2018 at 11:18 AM > >> > >> To: Matt Glasser <glass...@wustl.edu<mailto:glass...@wustl.edu>> > >> Cc: > >>"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: Re: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> Thanks! > >> In the ${StudyFolder}/${Subject}/T1w/${Subject}/Scripts folder, there > >>are > >> recon-all.log and recon-all-status.log files (which is what I assume I > >> should be looking at since recon-all is the main part of the FS > >>pipeline). > >> The latter just lists the times when certain steps were completed, but > >>the > >> former is really massive, and I'm not sure where to start looking. At > >>the > >> very end it says that it finished without error, but in the middle > >>there are > >> some things that seem like warnings, but I can't really understand > >>them. If > >> someone could take a look, I would be very grateful! > >> Thanks a lot in advance! > >> Darko > >> > >> On Mon, Jan 22, 2018 at 6:11 PM, Glasser, Matthew > >> <glass...@wustl.edu<mailto:glass...@wustl.edu>> wrote: > >> Look inside the FreeSurfer folder in > >> ${StudyFolder}/${Subject}/T1w/${Subject} > >> > >> Peace, > >> > >> Matt. > >> > >> From: Darko Komnenić <komnen...@gmail.com<mailto:komnen...@gmail.com>> > >> Date: Monday, January 22, 2018 at 11:08 AM > >> To: Matt Glasser <glass...@wustl.edu<mailto:glass...@wustl.edu>> > >> Cc: > >>"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: Re: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> Hi Matt, this is most likely a very silly question, but where can I > >>find the > >> logs? > >> Best, > >> Darko > >> > >> On Mon, Jan 22, 2018 at 6:06 PM, Glasser, Matthew > >> <glass...@wustl.edu<mailto:glass...@wustl.edu>> wrote: > >> I would look into the logs for the FreeSurfer pipeline. Perhaps it did > >>not > >> complete correctly. > >> > >> Peace, > >> > >> Matt. > >> > >> From: > >> > >><hcp-users-boun...@humanconnectome.org<mailto:hcp > -users-bounces@humanconn > >>ectome.org>> > >> on behalf of Darko Komnenić > >> <komnen...@gmail.com<mailto:komnen...@gmail.com>> > >> Date: Monday, January 22, 2018 at 10:59 AM > >> To: > >>"hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" > >> <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> > >> Subject: [HCP-Users] An error in Post-FreeSurfer Pipeline > >> > >> Dear HCP experts, > >> I am trying to compare the cortical myelin content between patients and > >> healthy controls. I ran the pre-FreeSurfer and FreeSurfer pipelines, and > >> they finished without reporting an error. When I tried to run the > >> Post-FreeSurfer pipeline, I received the following error note after each > >> individual participant in the subject list (the only thing changing are > >>the > >> file paths): > >> > >> START: FS2CaretConvertRegisterNonlinear > >> FreeSurfer2CaretConvertAndRegisterNonLinear.sh: RegName: FS > >> Cannot open volume > >> /home/lisak/Desktop/NMDA_HCP/controls/VIMS_HC_070/T1w/wmparc_1mm for > >> reading! > >> set -- --path=/home/lisak/Desktop/NMDA_HCP/controls > >> --subject=VIMS_HC_070 > >> > >>--surfatlasdir=/home/lisak/Desktop/Pipelines/global/ > templates/standard_me > >>sh_atlases > >> > >> > >>--grayordinatesdir=/home/lisak/Desktop/Pipelines/ > global/templates/91282_G > >>reyordinates > >> --grayordinatesres=2 --hiresmesh=164 --lowresmesh=32 > >> > >>--subcortgraylabels=/home/lisak/Desktop/Pipelines/ > global/config/FreeSurfe > >>rSubcorticalLabelTableLut.txt > >> > >> > >>--freesurferlabels=/home/lisak/Desktop/Pipelines/ > global/config/FreeSurfer > >>AllLut.txt > >> > >> > >>--refmyelinmaps=/home/lisak/Desktop/Pipelines/global/ > templates/standard_m > >>esh_atlases/Conte69.MyelinMap_BC.164k_fs_LR.dscalar.nii > >> --regname=FS --printcom= > >> . /home/lisak/Desktop/Pipelines/Examples/Scripts/SetUpHCPPipeline.sh > >> So in this particular example, VIMS_HC_070 is a subject codename, and > >>s/he > >> is in the "controls" subfolder. When I navigate to > >> ~/controls/VIMS_HC_070/T1w folder, there is nothing (neither file or > >>folder) > >> called wmparc_1mm in there. So it seems like one of the previous > >>pipelines > >> (maybe the FreeSurfer one) finished in a faulty way? > >> Some maybe relevant background information: I was able to run all the > >> pipelines without issue last summer, but in the meantime a colleague > >>updated > >> python on the computer by downloading the version 3.6, which made FSL > >> scripts not work (since they require python 2). In an attempt to fix > >>it, he > >> uninstalled the entire python3 from the computer, which basically > >> incapacitated the entire system. We have since restored it more or less > >>to > >> its previous functioning, but I cannot be 100% sure that there aren't > >>maybe > >> some residual glitches still left. > >> After running the PreFS pipeline, there were 17 items in the T1w > >>subfolder > >> in each participant's folder, and after the FS pipeline, the number > >>went up > >> to 20. I'm not sure if this is enough to let you know whether something > >>was > >> missing after each step. > >> I'm looking forward to any help or suggestions you can provide! > >> Thanks in advance, > >> Darko > >> > >> _______________________________________________ > >> HCP-Users mailing list > >> HCP-Users@humanconnectome.org<mailto:HCP-Users@humanconnectome.org> > >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > >> > >> > >> > >> > >> _______________________________________________ > >> HCP-Users mailing list > >> HCP-Users@humanconnectome.org<mailto:HCP-Users@humanconnectome.org> > >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > >> > >> ________________________________ > >> The materials in this message are private and may contain Protected > >> Healthcare Information or other information of a sensitive nature. If > >>you > >> are not the intended recipient, be advised that any unauthorized use, > >> disclosure, copying or the taking of any action in reliance on the > >>contents > >> of this information is strictly prohibited. If you have received this > >>email > >> in error, please immediately notify the sender via telephone or return > >> mail. > >> > >> > >> ________________________________ > >> The materials in this message are private and may contain Protected > >> Healthcare Information or other information of a sensitive nature. If > >>you > >> are not the intended recipient, be advised that any unauthorized use, > >> disclosure, copying or the taking of any action in reliance on the > >>contents > >> of this information is strictly prohibited. If you have received this > >>email > >> in error, please immediately notify the sender via telephone or return > >> mail. > >> > >> > >> _______________________________________________ > >> HCP-Users mailing list > >> HCP-Users@humanconnectome.org<mailto:HCP-Users@humanconnectome.org> > >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users > >> > >> > >> > > _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users
