I agree with your review and I agree that there are plenty of problems to be able to get everything working correctly.
I would like to warn you and point to an issue about the use of --depend_format=unix: https://github.com/martine/ninja/issues/794 Also, I've got around the .obj.d problem patching CMake generator adding a flag: set(CMAKE_NINJA_DEPFILE_REMOVE_OBJECT_EXT TRUE) I currently use this: set(CMAKE_DEPFILE_FLAGS_C "--md --no_depend_system_headers --depend_format=unix --depend_single_line --depend=<OBJECT>.d") The sad is that these are simple things to solve but require patches and maintaining a "fork". On Tue, Oct 28, 2014 at 10:55 AM, Olsson Gerhard <[email protected]> wrote: > Update: > > > > 1. Append to logfile > > Just resolving <OBJECT> would be an improvement, but that may require > changes to all generators, no global resolution what I see. > > Should be changed in armcc, dropping for CMake. > > > > 2. Dependfile > > This can be done in armcc: > > SET( CMAKE_DEPFILE_FLAGS_C "--depend=<OBJECT>.d --depend_format=unix" ) > > Similar for ARM, but untested. > > > > WindRiver: > > SET( CMAKE_DEPFILE_FLAGS_C "-Xmake-dependency=6 > -Xmake-dependency-savefile=<OBJECT>.d" ) > > (not possible for asm) > > > > So this should be working as intended > > > > > > 3. Ninja PRE_LINK/POST_BUILD adds cmd.exe > > Need to be changed in cmNormalNinjaTargetGenerator.cxx to not add twice. > > > > vars["PRE_LINK"] = > > this->GetLocalGenerator()->BuildCommandLine(preLinkCmdLines); > > (cmLocalNinjaGenerator::BuildCommandLine) > > > > /Gerhard > > > > *From:* Olsson Gerhard > *Sent:* den 27 oktober 2014 3:02 > *To:* '[email protected]' > *Subject:* CMake, armcc and Ninja workarounds and problems > > > > A summary of limitations with (mostly) armcc and Cmake, primarly using > Ninja generator. > > (Note that this is referring to two emails and add a new problem.) > > The workarounds where applicable are listed, as well as possible > corrections/enhancements in CMake. I will probably open issues later, maybe > patches too, but there may be other solutions. > > > > > > Setup: > > CMake 3.0.2 with Ninja 1.5.1 on Windows. Unix Makefiles was previously > used in Windows and is still used in Linux (Linux is used for speedup > compilation). > > WindRiver Diab (DCC) and ARM DS-5 armcc (RVDS) cross compiling. > > > > > > 1. No append to log file for armcc > > (see original email) > > Workaround is to patch generated files, replacing a token with a temp file > that is later concatenated. > > > > A proper solution could be custom variables. > > For example (set in user part of configuration) > > ADD_DEFINITIONS( --errors= logs/<SOURCE_FILE_NAME>.log) > > <SOURCE_FILE_NAME> would be a new expand variable, i.e. “SourceFileName.c” > in SOURCE path/to/SourceFileName.c > > The configuratione > > > > > > 2. Cannot control name of dependfile set in Ninja > > > > Ninja generator sets the name to objectpath.d > > cmGlobalNinjaGenerator::EncodeDepfileSpace(objectFileName + > ".d"); > > > > WindRiver sets the dep file name to objectDir/SourceName.d, so the “.obj” > in “DEP_FILE = path/BaseName.c.obj.d” must be removed > > Armcc just uses base name in the same directory as the command runs, > “BaseName.d”. –depend_dir= can be used to control the directory for C but > not asm. > > > > The workaround is to patch the generated build.ninja > > > > A proper solution is using new variables similar to above. (Just examples, > not figured out <ARCH> etc) > > > > WindRiver (not added to CMake): > > SET (CMAKE_<LANG>_<COMPILER_ID>_DEPFILE_PATTERN > <OBJECT_DIR>/<SOURCE_FILE_NAME>.d ) > > > > Armcc (RVDS patch exists for CMake): > > SET (CMAKE_DEPFILE_FLAGS_C –depend_dir=<OBJECT_FULL_DIR> > --depend_format=unix) > > SET (CMAKE_C_ARMCC_DEPFILE_PATTERN <OBJECT_FULL_DIR>/<SOURCE_BASE_NAME>.d ) > > SET (CMAKE_DEPFILE_FLAGS_ASM --depend_format=unix) > > SET (CMAKE_ASM_ARMCC_DEPFILE_PATTERN <SOURCE_BASE_NAME>.d ) > > > > Note: <OBJECT> is replaced in CMAKE_C_ARMCC_DEPFILE_PATTERN (but adding a > semicolon), but not <OBJECT_DIR> (as that is evaluated after FLAGS/DEFINES) > so it is a little more than just add new variables. > > The workaround for armcc could therefore not use CMAKE_DEPFILE_FLAGS_C, > the –depend_dir flag was added to normal FLAGS > > > > > > 3. CMake adds CMD /C for both LINKER and prelink/postbuild > > PRE_LINK and POST_BUILD is executed as part of linking, so extra CMD is an > error (at least the quotes) > > CMake adds CMD /C for linker, but also for PRE_LINK and POST_BUILD.CMD /C > is added twice. > > > > rule C_EXECUTABLE_LINKER_RSP_FILE > > command = cmd.exe /C "$PRE_LINK && C:\PROGRA~1\DS-5\bin\armcc.exe > $FLAGS $LINK_FLAGS --via=$RSP_FILE -o $out && $POST_BUILD" > > > > The first workaround was to patch build.ninja. Not really needed after the > second workaround that removed the need for PRE_LINK/POST_BUILD, stll > included in the patches below. > > Setup separate targets for a few ASM files (not compiling in PRE_LINK), > move out POST_BUILD to separate targets included in ALL. > > This works fine here (better if full armcc/WindRiver support was added of > course) > > > > I believe the solution is to remove CMD /C for PRE_LINK/POSTBUILD. There > are command length complications though. > > > > > > 4. Commands can be longer than 8000 bytes in Windows > > a. The Ninja generator may need to use RSP_FILE also when compiling. > > b. PRE_LINK/POSTBUILD can give long lines > > > > The workaround for a. is to decrease the include paths, for b. to not use > PRE_LINK. > > > > The solution for a. is to evaluate using RSP_FILE if needed in Ninja. I > believe the Unix Makefiles generator handles this better. > > For b. I have no good proposal. > > > > > > /Gerhard > > > > -- > > > > Rough workaround > > Build is invoked from make (CMAKE_MAKEFILE is set to path to build.ninja): > > > > ${CMAKE_MAKEFILE}: ${CMAKELISTS} ${CMAKE_TOOLCHAIN} | ${CMAKE_BUILD_DIR} > > @${ECHO} > > $(CMD_V)cd ${CMAKE_BUILD_DIR} && > ${CMAKE_MPC5516E} $(CMAKE_FLAGS_MPC5516E) ${CMAKELISTS_OSDIR} > > > > ifeq ($(CMAKE_GENERATOR),${CMAKE_NINJA}) > > @#CMake Ninja uses cmd, quotes incorrectly > > @perl -pi -e > 's%^(\s*(POST_BUILD|PRE_LINK)\s*=\s*)cmd.exe\s(/C)?\s*"(.*)"%$$1$$4%i; ' $@ > > ifneq ($(COMPILE_DEPEND_DIR),) > > @#no path for armcc, use --depend_dir=depend > to avoid have all in root (asm go there) > > @perl -pi -e > 'BEGIN{$$c="$(COMPILE_DEPEND_DIR)";} > s%^(\s*DEP_FILE\s*=\s*).*/(.*).c\.obj\.d\s*$$%$$1$$c/$$2.d\n%;' $@ > > @$(MKDIR) -p $(dir ${CMAKE_MAKEFILE})depend > > else > > @#unexpected suffix (probably other ways to > do this) > > @perl -pi -e > 's%^(\s*DEP_FILE\s*=\s*.*)\.obj\.d\s*$$%$$1.d\n%;' $@ > > endif > > ifneq ($(COMPILE_LOG_NO_APPEND),) > > @perl -pi -e > 'BEGIN{$$c="$(COMPILE_LOG_NO_APPEND)"; $$n="unknown"; $$i=1;}if(m%^build > .*?([\w\.]+):%){$$n=$$1;} if (s%(--errors)=$$c%$$1=logs/$$n.$$i%){$$i++; > $$n="unknown";}' $@ > > @$(MKDIR) -p $(dir ${CMAKE_MAKEFILE})logs > > endif > > > > compile_target: ${CMAKE_MAKEFILE} | $(INFO_BUILD_DIR) ${BUILD_RESULT_DIR} > > @${ECHO} > > ifneq ($(COMPILE_LOG_NO_APPEND),) > > @$(ECHO) " Note: If compile fails: \"make > copy_log\"" > > endif > > $(CMAKE_MAKE) install > > ifneq ($(COMPILE_LOG_NO_APPEND),) > > @$(MAKE) copy_logs > > endif > > > > copy_logs: > > ifneq ($(COMPILE_LOG_NO_APPEND),) > > $(CMD_V)$(CAT) $(CMAKE_BUILD_DIR)/logs/* > > ${INFO_BUILD_DIR}/error.txt > > $(CMD_V)$(RM) $(CMAKE_BUILD_DIR)/logs/* > > endif > > > > *From:* Olsson Gerhard > *Sent:* den 8 september 2014 7:46 > *To:* '[email protected]' > *Subject:* Compilation logs: Workaround for no appendfile > > > > ARM DS-5 compiler (armcc compiler 5.04) > > Platform: Windows and Linux > > > > Armcc (as well as arasm, armlink etc) has option ‘—errors=file ’to log > compilation messages (warnings and errors) to a logfile instead of stdout. > > There is no possibility from what I have found out to append to a log > file, all output overwrites the log. > > > > Background: > > We cannot change some 3rd party code and cannot suppress all compilation > warnings. The workaround is to parse the logs. > > If printed to the console, the messages are mixed up with progress and > license messages. > > Other compilers has the possibility to append to logs, but not armcc what > I can find. > > > > A workaround would be to CMake to save stdout to a logfile. May be > possible, but I am not sure that is a good solution. > > Another is to log to unique logname, include the compiled file in the > logname. I am not sure how that can be done in CMake. > > > > Any hints how to do this? > > > > /Gerhard > > > > > > > > *From:* Olsson Gerhard > *Sent:* den 1 oktober 2014 7:46 > *To:* '[email protected]' > *Subject:* Ninja: CMD /C, quoting added for PRE_LINK/POST_BUILD > > > > The commands for PRE_LINK/POST_BUILD is not correctly quoted for Ninja in > Windows. ‘CMD /C “<command>”’ is added to the command, just as Ninja itself > does. > > (this was maybe different in earlier Ninja versions, but has been around > for a couple of years) > > (The quoting in the working command is strange, but works.) > > > > A workaround that is not so bad, is to patch build.ninja > > Is there a better way? > > > > Note: The lines LINK_FLAGS ,POST_BUILD , PRE_LINK , TARGET_PDB are > duplicated in build.ninja. Does not make a difference. > > > > ARM DS-5 compiler (armcc compiler 5.04) > > Platform: Windows (Linux used too, but with Unix Makefiles) > > CMake 3.0.2 > > Ninja 1.5.1 > > > > /Gerhard > > > > > > CmakeLists.txt excerpt: > > > > ADD_CUSTOM_COMMAND( > > TARGET ECU > > PRE_LINK > > COMMAND "C:/Program Files/DS-5/bin/armasm.exe" --cpu=Cortex-A9 -g > --cpreproc --apcs=interwork --arm --no_unaligned_access --pd "__EVAL SETA > 1" --pd "__MICROLIB SETA 1" -Ic:/Temp/includes -o > ${CMAKE_CURRENT_BINARY_DIR}/osekasm.o c:/Os/osekasm.s VERBATIM > > ) > > > > #Generate hex, copy to view > > add_custom_command(TARGET ECU POST_BUILD > > COMMAND "C:/Program Files/DS-5/bin/fromelf.exe" --m32combined > --output=${CMAKE_CURRENT_BINARY_DIR}/ECU.hex > ${CMAKE_CURRENT_BINARY_DIR}/ECU.elf VERBATIM > > COMMAND "C:/cygwin/bin/objcopy" -I elf32-big -O srec --srec-len=32 > --srec-forceS3 -j .securityLevel ${CMAKE_CURRENT_BINARY_DIR}/ECU.elf > ${CMAKE_CURRENT_BINARY_DIR}/parameter_section.hex VERBATIM > > ) > > > > > > Generated in build.ninja: > > > > LINK_FLAGS = --cpu=Cortex-A9 c:/Temp/includes/SecModLib.a > -L--list=c:/Temp/build/ECU.map -L--scatter=c:/Zynq7000/scatter.txt -L--map > -L--no_remove -L--symbols > -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers > C:/Temp/build/osekasm.o > > POST_BUILD = cmd.exe /C "cd /D C:\Temp\build && "C:\Program > Files\DS-5\bin\fromelf.exe" --m32combined --output=C:/Temp/build/ECU.hex > C:/Temp/build/ECU.elf && C:\cygwin\bin\objcopy -I elf32-big -O srec > --srec-len=32 --srec-forceS3 -j .securityLevel C:/Temp/build/ECU.elf > C:/Temp/build/parameter_section.hex" > > PRE_LINK = cmd.exe /C "cd /D C:\Temp\build && "C:\Program > Files\DS-5\bin\armasm.exe" --cpu=Cortex-A9 -g --cpreproc --apcs=interwork > --arm --no_unaligned_access --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1" > -Ic:/Temp/includes -o C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd > C:\Temp\build" > > TARGET_PDB = ECU.elf.dbg > > > > > > Console output: > > > > [106/107] Linking C executable ECU.elf > > FAILED: cmd.exe /C "cmd.exe /C "cd /D C:\Temp\build && "C:\Program > Files\DS-5\bin\armasm.exe" --cpu=Cortex-A9 -g --cpreproc --apcs=interwork > --arm --no_unaligned_access --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1" > -Ic:/Temp/includes -o C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd > C:\Temp\build" && C:\PROGRA~1\DS-5\bin\armcc.exe --cpu=Cortex-A9 > c:/Temp/includes/SecModLib.a -L--list=c:/Temp/build/ECU.map > -L--scatter=c:/Zynq7000/scatter.txt -L--map -L--no_remove -L--symbols > -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers > C:/Temp/build/osekasm.o --via=CMakeFiles/ECU.rsp -o ECU.elf && cmd.exe /C > "cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\fromelf.exe" > --m32combined --output=C:/Temp/build/ECU.hex C:/Temp/build/ECU.elf && > C:\cygwin\bin\objcopy -I elf32-big -O srec --srec-len=32 --srec-forceS3 -j > .securityLevel C:/Temp/build/ECU.elf C:/Temp/build/parameter_section.hex"" > > 'cmd.exe' is not recognized as an internal or external command, > > operable program or batch file. > > ninja: build stopped: subcommand failed. > > > > > > Patched (in Makefile): > > > > ${CMAKE_MAKEFILE}: ${CMAKELISTS} ${CMAKE_TOOLCHAIN} | ${CMAKE_BUILD_DIR} > > @#CMake Ninja uses cmd incorrectly > > @perl -pi -e > 's%^(\s*(POST_BUILD|PRE_LINK)\s*=\s*)cmd.exe\s(/C)?\s*"(.*)"%$$1$$4%i; ' $@ > > > > > > > > Rules patched OK: > > > > LINK_FLAGS = --cpu=Cortex-A9 c:/Temp/includes/SecModLib.a > -L--list=c:/Temp/build/ECU.map -L--scatter=c:/Zynq7000/scatter.txt -L--map > -L--no_remove -L--symbols > -L--info=common,summarysizes,summarystack,sizes,totals,unused,unusedsymbols,veneers > C:/Temp/build/osekasm.o > > POST_BUILD = cd /D C:\Temp\build && "C:\Program > Files\DS-5\bin\fromelf.exe" --m32combined --output=C:/Temp/build/ECU.hex > C:/Temp/build/ECU.elf && C:\cygwin\bin\objcopy -I elf32-big -O srec > --srec-len=32 --srec-forceS3 -j .securityLevel C:/Temp/build/ECU.elf > C:/Temp/build/parameter_section.hex > > PRE_LINK = cd /D C:\Temp\build && "C:\Program Files\DS-5\bin\armasm.exe" > --cpu=Cortex-A9 -g --cpreproc --apcs=interwork --arm --no_unaligned_access > --pd "__EVAL SETA 1" --pd "__MICROLIB SETA 1" -Ic:/Temp/includes -o > C:/Temp/build/osekasm.o c:/Temp/Os/osekasm.s && cd C:\Temp\build > > TARGET_PDB = ECU.elf.dbg > > > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake > -- Animal Liberation Front http://www.animal-liberation.com/
-- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
