Re: [CMake] How to get custom commands to build in parallel ?

2014-09-24 Thread Glenn Coombs
I am already using /MP and the equivalent to the /maxcpucount.  It is the
/MP one that is important in this context as I am only concerned with
improving the speed of a single project out of the many in my solution.  I
believe the problem is with cmake rather than with Visual Studio as I can
select any of the other projects in the same solution and build them in
isolation and they will build their object files in parallel.  For some
reason cmake isn't seeing the custom commands as something it can run in
parallel.

--
Glenn

On 23 September 2014 16:20, Robert Maynard robert.mayn...@kitware.com
wrote:

 If you are using VS2013 you should look at the /maxcpucount flag which
 allows msbuild to build multiple projects at the same time. You will
 have to manually balance /MP and /maxcpucount as they cause a P*C
 number of processes to execute.

 On Tue, Sep 23, 2014 at 11:05 AM, Glenn Coombs glenn.coo...@gmail.com
 wrote:
  I have the following code in one of my CMakeLists.txt files:
 
  set(feature_files_h)
  set(feature_files_cpp)
 
  macro(create_feature_files NAME)
  add_custom_command(
  OUTPUT ${FEATURES_OUTPUT_DIR}/${NAME}.cpp
  ${FEATURES_OUTPUT_DIR}/${NAME}.h
  DEPENDS ${FEATURES_INPUT_DIR}/${NAME}.txt
  WORKING_DIRECTORY ${SIM_ROOT}/tools/features
  COMMAND perl ${FEATURES_PERL_PATH} -s${FEATURES_INPUT_DIR}
  -d${FEATURES_OUTPUT_DIR} -f${NAME}
  )
  list(APPEND feature_files_h   ${FEATURES_OUTPUT_DIR}/${NAME}.h)
  list(APPEND feature_files_cpp ${FEATURES_OUTPUT_DIR}/${NAME}.cpp)
  endmacro()
 
  create_feature_files(example_features)
  create_feature_files(fme_core_features)
  create_feature_files(fme_features)
  create_feature_files(front_end_features)
  create_feature_files(inloop_filter_features)
 
  add_library(${PROJECT_NAME} STATIC ${feature_files_cpp}
 ${feature_files_h}
  ${SIM_ROOT}/tools/features/cpp-code/cfeatures.cpp)
 
  which creates a library from an existing source file called cfeatures.cpp
  and from a bunch of auto-generated .cpp and .h files.  The whole thing
 works
  fine and runs the perl script to create each of the auto-generated files
  before creating the library.  The only problem I have with it is that it
 is
  quite slow and I would like to speed it up by having it run the perl
  commands to create the auto-generated files in parallel.  Currently,
 each of
  the auto-generated cpp files is generated sequentially, then they are all
  compiled in parallel and the library created in the final step.
 
  I'm using cmake 3.0.0 with Visual Studio 2013 Express Edition.  And I've
  added the /MP option so that Visual Studio will do parallel builds.  I've
  also set the Projects and Solutions=Build and Run=maximum number of
  parallel project builds option to 4.  This works well with all the
 normal
  cpp files in other projects and I easily see all 4 cpu cores maxxed out.
  But when building this project only 1 cpu core is used.
 
  Can anybody tell me how to change it so that the auto-geneated files are
  created in parallel as well ?
 
  --
  Glenn
 
 
  --
 
  Powered by www.kitware.com
 
  Please keep messages on-topic and check the CMake FAQ at:
  http://www.cmake.org/Wiki/CMake_FAQ
 
  Kitware offers various services to support the CMake community. For more
  information on each offering, please visit:
 
  CMake Support: http://cmake.org/cmake/help/support.html
  CMake Consulting: http://cmake.org/cmake/help/consulting.html
  CMake Training Courses: http://cmake.org/cmake/help/training.html
 
  Visit other Kitware open-source projects at
  http://www.kitware.com/opensource/opensource.html
 
  Follow this link to subscribe/unsubscribe:
  http://public.kitware.com/mailman/listinfo/cmake

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How to get custom commands to build in parallel ?

2014-09-24 Thread Robert Maynard
/MP flag is specifically for compiling object files in parallel so it
won't cause multiple custom commands to be run in parallel as each one
is a msbuild custom build item task. I don't know exactly how msbuild
treats custom tasks such as these, but they might be implicitly
serial.


On Wed, Sep 24, 2014 at 3:27 AM, Glenn Coombs glenn.coo...@gmail.com wrote:
 I am already using /MP and the equivalent to the /maxcpucount.  It is the
 /MP one that is important in this context as I am only concerned with
 improving the speed of a single project out of the many in my solution.  I
 believe the problem is with cmake rather than with Visual Studio as I can
 select any of the other projects in the same solution and build them in
 isolation and they will build their object files in parallel.  For some
 reason cmake isn't seeing the custom commands as something it can run in
 parallel.

 --
 Glenn

 On 23 September 2014 16:20, Robert Maynard robert.mayn...@kitware.com
 wrote:

 If you are using VS2013 you should look at the /maxcpucount flag which
 allows msbuild to build multiple projects at the same time. You will
 have to manually balance /MP and /maxcpucount as they cause a P*C
 number of processes to execute.

 On Tue, Sep 23, 2014 at 11:05 AM, Glenn Coombs glenn.coo...@gmail.com
 wrote:
  I have the following code in one of my CMakeLists.txt files:
 
  set(feature_files_h)
  set(feature_files_cpp)
 
  macro(create_feature_files NAME)
  add_custom_command(
  OUTPUT ${FEATURES_OUTPUT_DIR}/${NAME}.cpp
  ${FEATURES_OUTPUT_DIR}/${NAME}.h
  DEPENDS ${FEATURES_INPUT_DIR}/${NAME}.txt
  WORKING_DIRECTORY ${SIM_ROOT}/tools/features
  COMMAND perl ${FEATURES_PERL_PATH} -s${FEATURES_INPUT_DIR}
  -d${FEATURES_OUTPUT_DIR} -f${NAME}
  )
  list(APPEND feature_files_h   ${FEATURES_OUTPUT_DIR}/${NAME}.h)
  list(APPEND feature_files_cpp ${FEATURES_OUTPUT_DIR}/${NAME}.cpp)
  endmacro()
 
  create_feature_files(example_features)
  create_feature_files(fme_core_features)
  create_feature_files(fme_features)
  create_feature_files(front_end_features)
  create_feature_files(inloop_filter_features)
 
  add_library(${PROJECT_NAME} STATIC ${feature_files_cpp}
  ${feature_files_h}
  ${SIM_ROOT}/tools/features/cpp-code/cfeatures.cpp)
 
  which creates a library from an existing source file called
  cfeatures.cpp
  and from a bunch of auto-generated .cpp and .h files.  The whole thing
  works
  fine and runs the perl script to create each of the auto-generated files
  before creating the library.  The only problem I have with it is that it
  is
  quite slow and I would like to speed it up by having it run the perl
  commands to create the auto-generated files in parallel.  Currently,
  each of
  the auto-generated cpp files is generated sequentially, then they are
  all
  compiled in parallel and the library created in the final step.
 
  I'm using cmake 3.0.0 with Visual Studio 2013 Express Edition.  And I've
  added the /MP option so that Visual Studio will do parallel builds.
  I've
  also set the Projects and Solutions=Build and Run=maximum number of
  parallel project builds option to 4.  This works well with all the
  normal
  cpp files in other projects and I easily see all 4 cpu cores maxxed out.
  But when building this project only 1 cpu core is used.
 
  Can anybody tell me how to change it so that the auto-geneated files are
  created in parallel as well ?
 
  --
  Glenn
 
 
  --
 
  Powered by www.kitware.com
 
  Please keep messages on-topic and check the CMake FAQ at:
  http://www.cmake.org/Wiki/CMake_FAQ
 
  Kitware offers various services to support the CMake community. For more
  information on each offering, please visit:
 
  CMake Support: http://cmake.org/cmake/help/support.html
  CMake Consulting: http://cmake.org/cmake/help/consulting.html
  CMake Training Courses: http://cmake.org/cmake/help/training.html
 
  Visit other Kitware open-source projects at
  http://www.kitware.com/opensource/opensource.html
 
  Follow this link to subscribe/unsubscribe:
  http://public.kitware.com/mailman/listinfo/cmake


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


[CMake] How to get custom commands to build in parallel ?

2014-09-23 Thread Glenn Coombs
I have the following code in one of my CMakeLists.txt files:

set(feature_files_h)
set(feature_files_cpp)

macro(create_feature_files NAME)
add_custom_command(
OUTPUT ${FEATURES_OUTPUT_DIR}/${NAME}.cpp
${FEATURES_OUTPUT_DIR}/${NAME}.h
DEPENDS ${FEATURES_INPUT_DIR}/${NAME}.txt
WORKING_DIRECTORY ${SIM_ROOT}/tools/features
COMMAND perl ${FEATURES_PERL_PATH} -s${FEATURES_INPUT_DIR}
-d${FEATURES_OUTPUT_DIR} -f${NAME}
)
list(APPEND feature_files_h   ${FEATURES_OUTPUT_DIR}/${NAME}.h)
list(APPEND feature_files_cpp ${FEATURES_OUTPUT_DIR}/${NAME}.cpp)
endmacro()

create_feature_files(example_features)
create_feature_files(fme_core_features)
create_feature_files(fme_features)
create_feature_files(front_end_features)
create_feature_files(inloop_filter_features)

add_library(${PROJECT_NAME} STATIC ${feature_files_cpp} ${feature_files_h}
${SIM_ROOT}/tools/features/cpp-code/cfeatures.cpp)

which creates a library from an existing source file called cfeatures.cpp
and from a bunch of auto-generated .cpp and .h files.  The whole thing
works fine and runs the perl script to create each of the auto-generated
files before creating the library.  The only problem I have with it is that
it is quite slow and I would like to speed it up by having it run the perl
commands to create the auto-generated files in parallel.  Currently, each
of the auto-generated cpp files is generated sequentially, then they are
all compiled in parallel and the library created in the final step.

I'm using cmake 3.0.0 with Visual Studio 2013 Express Edition.  And I've
added the /MP option so that Visual Studio will do parallel builds.  I've
also set the Projects and Solutions=Build and Run=maximum number of
parallel project builds option to 4.  This works well with all the normal
cpp files in other projects and I easily see all 4 cpu cores maxxed out.
But when building this project only 1 cpu core is used.

Can anybody tell me how to change it so that the auto-geneated files are
created in parallel as well ?

--
Glenn
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How to get custom commands to build in parallel ?

2014-09-23 Thread Robert Maynard
If you are using VS2013 you should look at the /maxcpucount flag which
allows msbuild to build multiple projects at the same time. You will
have to manually balance /MP and /maxcpucount as they cause a P*C
number of processes to execute.

On Tue, Sep 23, 2014 at 11:05 AM, Glenn Coombs glenn.coo...@gmail.com wrote:
 I have the following code in one of my CMakeLists.txt files:

 set(feature_files_h)
 set(feature_files_cpp)

 macro(create_feature_files NAME)
 add_custom_command(
 OUTPUT ${FEATURES_OUTPUT_DIR}/${NAME}.cpp
 ${FEATURES_OUTPUT_DIR}/${NAME}.h
 DEPENDS ${FEATURES_INPUT_DIR}/${NAME}.txt
 WORKING_DIRECTORY ${SIM_ROOT}/tools/features
 COMMAND perl ${FEATURES_PERL_PATH} -s${FEATURES_INPUT_DIR}
 -d${FEATURES_OUTPUT_DIR} -f${NAME}
 )
 list(APPEND feature_files_h   ${FEATURES_OUTPUT_DIR}/${NAME}.h)
 list(APPEND feature_files_cpp ${FEATURES_OUTPUT_DIR}/${NAME}.cpp)
 endmacro()

 create_feature_files(example_features)
 create_feature_files(fme_core_features)
 create_feature_files(fme_features)
 create_feature_files(front_end_features)
 create_feature_files(inloop_filter_features)

 add_library(${PROJECT_NAME} STATIC ${feature_files_cpp} ${feature_files_h}
 ${SIM_ROOT}/tools/features/cpp-code/cfeatures.cpp)

 which creates a library from an existing source file called cfeatures.cpp
 and from a bunch of auto-generated .cpp and .h files.  The whole thing works
 fine and runs the perl script to create each of the auto-generated files
 before creating the library.  The only problem I have with it is that it is
 quite slow and I would like to speed it up by having it run the perl
 commands to create the auto-generated files in parallel.  Currently, each of
 the auto-generated cpp files is generated sequentially, then they are all
 compiled in parallel and the library created in the final step.

 I'm using cmake 3.0.0 with Visual Studio 2013 Express Edition.  And I've
 added the /MP option so that Visual Studio will do parallel builds.  I've
 also set the Projects and Solutions=Build and Run=maximum number of
 parallel project builds option to 4.  This works well with all the normal
 cpp files in other projects and I easily see all 4 cpu cores maxxed out.
 But when building this project only 1 cpu core is used.

 Can anybody tell me how to change it so that the auto-geneated files are
 created in parallel as well ?

 --
 Glenn


 --

 Powered by www.kitware.com

 Please keep messages on-topic and check the CMake FAQ at:
 http://www.cmake.org/Wiki/CMake_FAQ

 Kitware offers various services to support the CMake community. For more
 information on each offering, please visit:

 CMake Support: http://cmake.org/cmake/help/support.html
 CMake Consulting: http://cmake.org/cmake/help/consulting.html
 CMake Training Courses: http://cmake.org/cmake/help/training.html

 Visit other Kitware open-source projects at
 http://www.kitware.com/opensource/opensource.html

 Follow this link to subscribe/unsubscribe:
 http://public.kitware.com/mailman/listinfo/cmake
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake