Re: Make cmake default
Here [1] you can find a work in progress PR regarding BLAS libraries handling with cmake in MXNet. -- Anton [1] https://github.com/apache/incubator-mxnet/pull/11148 2018-06-04 11:43 GMT+02:00 Anton Chernov : > +1 > > > > Cmake build scripts have currently some limitations (CUDA, lapack, F16 > etc) especially for cross compilations. > > I am currently working on those [1]. lapack and BLAS cmake module coming > soon. > > Once this is done all ci builds can be ported to use cmake builds. > > > > Regarding amalgamation: > > It would certainly be beneficial to remove amalgamation ASAP since it's > misleading customers. > > > > -- Anton > > [1] CUDA, F16 https://github.com/apache/incubator-mxnet/pull/10564 > > > 2018-06-04 10:17 GMT+02:00 Chen HY : > >> glad to hear mxnet.js is back again. >> >> 2018-06-04 8:43 GMT+01:00 Asmus Hetzel : >> >> > +1 >> > >> > I have dealt with the make/cmake stuff when integrating lapack/cusolver. >> > Having a single cmake would have made things far easier. >> > >> > Asmus >> > >> > >> > >> > Am Freitag, 1. Juni 2018, 23:58:17 MESZ hat Alex Zai < >> aza...@gmail.com> >> > Folgendes geschrieben: >> > >> > Just realized that the email lists strips aways all hyperlinks. >> Attached >> > is a >> > copy of my previous email with links pasted in. >> > >> > What are peoples' thought on requiring cmake when building from source? >> > Currently we have to maintain two independent build files (CMakeLists >> and >> > Makefile) which makes it more difficult to develop (each are 600+ >> lines). >> > Also, >> > our current build system (in Makefile) requires that 3rdparty >> dependencies >> > have >> > binaries present (or a Makefile to generate binaries) in the repo, which >> > is not >> > always the case. >> > Generating a makefile with cmake will make our Makefile very simple like >> > PyTorch'sMakefile (20 lines of code - >> > https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not all >> > 3rdparty >> > dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up >> > calling >> > cmake >> > (https://github.com/apache/incubator-mxnet/blob/master/ >> > prepare_mkldnn.sh#L96) >> > to generate binaries (this does not violate our 'no cmake dependency' as >> > USE_MKLDNN is OFF by default). If we encounter any library in the future >> > that >> > requires us to generate artifacts with cmake, it would be better to make >> > the >> > switch now. Lastly, we already require cmake as a dependency forwindows' >> > developers >> > (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot% >> > 202018-06-01%2013.43.08.png?dl=0) >> > so this would only affect linux / mac developers who do not have cmake >> > already. >> > I currently have a pendingPR >> > (https://github.com/apache/incubator-mxnet/pull/8/) that depends >> on >> > this >> > change. The library does not have a Makefile or binaries present. Unlike >> > mkldnn, >> > we would want this library included by default so I cannot generate >> > artifacts >> > with cmake. The alternative would be to strip out only the relevant >> parts >> > of the >> > code we need from the library. I did this in a previous version of myPR >> > (https://github.com/apache/incubator-mxnet/compare/ >> > dfdfd1ad15de8bb1b899effb0860a4e834093cfc...a4267eb80488804a7 >> f74ff01f5627c >> > 47dd46bd78) >> > but it is incredible messy. >> > Please let me know your thoughts. >> > Best, >> > Alex >> > >> > >> > >> > >> > >> > On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: >> > What are peoples' thought on requiring cmake when building from source? >> > Currently we have to maintain two independent build files (CMakeLists >> and >> > Makefile) which makes it more difficult to develop (each are 600+ >> lines). >> > Also, >> > our current build system (in Makefile) requires that 3rdparty >> dependencies >> > have >> > binaries present (or a Makefile to generate binaries) in the repo, which >> > is not >> > always the case. >> > Generating a makefile with cmake will make our Makefile very simple like >> > PyTorch's Makefile (20 lines of code). Also, not all 3rdparty >> dependencies >> > have >> > binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to >> > generate >> > binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN >> is >> > OFF >> > by default). If we encounter any library in the future that requires us >> to >> > generate artifacts with cmake, it would be better to make the switch >> now. >> > Lastly, we already require cmake as a dependency for windows' >> > developers so this >> > would only affect linux / mac developers who do not have cmake already. >> > I currently have a pending PR that depends on this change. The library >> > does not >> > have a Makefile or binaries present. Unlike mkldnn, we would want this >> > library >> > included by default so I cannot generate artifacts with cmake. The >> > alternative >> > would be to strip out only the relevant parts of the code we need
Re: Make cmake default
+1 Cmake build scripts have currently some limitations (CUDA, lapack, F16 etc) especially for cross compilations. I am currently working on those [1]. lapack and BLAS cmake module coming soon. Once this is done all ci builds can be ported to use cmake builds. Regarding amalgamation: It would certainly be beneficial to remove amalgamation ASAP since it's misleading customers. -- Anton [1] CUDA, F16 https://github.com/apache/incubator-mxnet/pull/10564 2018-06-04 10:17 GMT+02:00 Chen HY : > glad to hear mxnet.js is back again. > > 2018-06-04 8:43 GMT+01:00 Asmus Hetzel : > > > +1 > > > > I have dealt with the make/cmake stuff when integrating lapack/cusolver. > > Having a single cmake would have made things far easier. > > > > Asmus > > > > > > > > Am Freitag, 1. Juni 2018, 23:58:17 MESZ hat Alex Zai < > aza...@gmail.com> > > Folgendes geschrieben: > > > > Just realized that the email lists strips aways all hyperlinks. Attached > > is a > > copy of my previous email with links pasted in. > > > > What are peoples' thought on requiring cmake when building from source? > > Currently we have to maintain two independent build files (CMakeLists and > > Makefile) which makes it more difficult to develop (each are 600+ lines). > > Also, > > our current build system (in Makefile) requires that 3rdparty > dependencies > > have > > binaries present (or a Makefile to generate binaries) in the repo, which > > is not > > always the case. > > Generating a makefile with cmake will make our Makefile very simple like > > PyTorch'sMakefile (20 lines of code - > > https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not all > > 3rdparty > > dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up > > calling > > cmake > > (https://github.com/apache/incubator-mxnet/blob/master/ > > prepare_mkldnn.sh#L96) > > to generate binaries (this does not violate our 'no cmake dependency' as > > USE_MKLDNN is OFF by default). If we encounter any library in the future > > that > > requires us to generate artifacts with cmake, it would be better to make > > the > > switch now. Lastly, we already require cmake as a dependency forwindows' > > developers > > (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot% > > 202018-06-01%2013.43.08.png?dl=0) > > so this would only affect linux / mac developers who do not have cmake > > already. > > I currently have a pendingPR > > (https://github.com/apache/incubator-mxnet/pull/8/) that depends on > > this > > change. The library does not have a Makefile or binaries present. Unlike > > mkldnn, > > we would want this library included by default so I cannot generate > > artifacts > > with cmake. The alternative would be to strip out only the relevant parts > > of the > > code we need from the library. I did this in a previous version of myPR > > (https://github.com/apache/incubator-mxnet/compare/ > > dfdfd1ad15de8bb1b899effb0860a4e834093cfc... > a4267eb80488804a7f74ff01f5627c > > 47dd46bd78) > > but it is incredible messy. > > Please let me know your thoughts. > > Best, > > Alex > > > > > > > > > > > > On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: > > What are peoples' thought on requiring cmake when building from source? > > Currently we have to maintain two independent build files (CMakeLists and > > Makefile) which makes it more difficult to develop (each are 600+ lines). > > Also, > > our current build system (in Makefile) requires that 3rdparty > dependencies > > have > > binaries present (or a Makefile to generate binaries) in the repo, which > > is not > > always the case. > > Generating a makefile with cmake will make our Makefile very simple like > > PyTorch's Makefile (20 lines of code). Also, not all 3rdparty > dependencies > > have > > binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to > > generate > > binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN > is > > OFF > > by default). If we encounter any library in the future that requires us > to > > generate artifacts with cmake, it would be better to make the switch now. > > Lastly, we already require cmake as a dependency for windows' > > developers so this > > would only affect linux / mac developers who do not have cmake already. > > I currently have a pending PR that depends on this change. The library > > does not > > have a Makefile or binaries present. Unlike mkldnn, we would want this > > library > > included by default so I cannot generate artifacts with cmake. The > > alternative > > would be to strip out only the relevant parts of the code we need from > the > > library. I did this in a previous version of my PR but it is incredible > > messy. > > Please let me know your thoughts. > > Best, > > Alex > > > > > > -- > Chen Hanyang 陈涵洋 > Software School Fudan University > +86-138-1881-7745 >
Re: Make cmake default
glad to hear mxnet.js is back again. 2018-06-04 8:43 GMT+01:00 Asmus Hetzel : > +1 > > I have dealt with the make/cmake stuff when integrating lapack/cusolver. > Having a single cmake would have made things far easier. > > Asmus > > > > Am Freitag, 1. Juni 2018, 23:58:17 MESZ hat Alex Zai > Folgendes geschrieben: > > Just realized that the email lists strips aways all hyperlinks. Attached > is a > copy of my previous email with links pasted in. > > What are peoples' thought on requiring cmake when building from source? > Currently we have to maintain two independent build files (CMakeLists and > Makefile) which makes it more difficult to develop (each are 600+ lines). > Also, > our current build system (in Makefile) requires that 3rdparty dependencies > have > binaries present (or a Makefile to generate binaries) in the repo, which > is not > always the case. > Generating a makefile with cmake will make our Makefile very simple like > PyTorch'sMakefile (20 lines of code - > https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not all > 3rdparty > dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up > calling > cmake > (https://github.com/apache/incubator-mxnet/blob/master/ > prepare_mkldnn.sh#L96) > to generate binaries (this does not violate our 'no cmake dependency' as > USE_MKLDNN is OFF by default). If we encounter any library in the future > that > requires us to generate artifacts with cmake, it would be better to make > the > switch now. Lastly, we already require cmake as a dependency forwindows' > developers > (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot% > 202018-06-01%2013.43.08.png?dl=0) > so this would only affect linux / mac developers who do not have cmake > already. > I currently have a pendingPR > (https://github.com/apache/incubator-mxnet/pull/8/) that depends on > this > change. The library does not have a Makefile or binaries present. Unlike > mkldnn, > we would want this library included by default so I cannot generate > artifacts > with cmake. The alternative would be to strip out only the relevant parts > of the > code we need from the library. I did this in a previous version of myPR > (https://github.com/apache/incubator-mxnet/compare/ > dfdfd1ad15de8bb1b899effb0860a4e834093cfc...a4267eb80488804a7f74ff01f5627c > 47dd46bd78) > but it is incredible messy. > Please let me know your thoughts. > Best, > Alex > > > > > > On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: > What are peoples' thought on requiring cmake when building from source? > Currently we have to maintain two independent build files (CMakeLists and > Makefile) which makes it more difficult to develop (each are 600+ lines). > Also, > our current build system (in Makefile) requires that 3rdparty dependencies > have > binaries present (or a Makefile to generate binaries) in the repo, which > is not > always the case. > Generating a makefile with cmake will make our Makefile very simple like > PyTorch's Makefile (20 lines of code). Also, not all 3rdparty dependencies > have > binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to > generate > binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN is > OFF > by default). If we encounter any library in the future that requires us to > generate artifacts with cmake, it would be better to make the switch now. > Lastly, we already require cmake as a dependency for windows' > developers so this > would only affect linux / mac developers who do not have cmake already. > I currently have a pending PR that depends on this change. The library > does not > have a Makefile or binaries present. Unlike mkldnn, we would want this > library > included by default so I cannot generate artifacts with cmake. The > alternative > would be to strip out only the relevant parts of the code we need from the > library. I did this in a previous version of my PR but it is incredible > messy. > Please let me know your thoughts. > Best, > Alex > -- Chen Hanyang 陈涵洋 Software School Fudan University +86-138-1881-7745
Re: Make cmake default
In light of this, maybe it is time to deprecate amalgamation? GIven there is already support for Pi, and we could use TVM to compile to javascript with WebGL Tianqi On Fri, Jun 1, 2018 at 3:24 PM, Rahul Huilgol wrote: > +1 > > Let's move to CMake. It has much better support, and it's not worth > maintaining two build systems. > If we really want we could maintain a make file which manages the > installation of cmake and calls cmake internally! It seems easy to install > cmake. There is a shell script with binary for all Linux x86_64, Windows, > Mac. For other systems as well, it's just a couple of steps. > > Regards, > Rahul > > > > On Fri, 1 Jun 2018 at 15:12 Chen HY wrote: > > > building for rpi doesn't mean you should build on a rpi... that takes > > forever. > > > > 2018-06-01 23:06 GMT+01:00 Anirudh : > > > > > +1 to using cmake and deprecating Makefile. I was able to find a > previous > > > discussion on this: > > > https://github.com/apache/incubator-mxnet/issues/8702 > > > > > > The concerns raised were > > > 1. Building on devices like raspberry pi where cmake is non existent or > > > old. > > > 2. Adding an additional dependency. > > > > > > As mentioned in the thread, if we provide good instructions on how to > > > install cmake/build cmake from source, > > > these concerns will be addressed. > > > > > > Anirudh > > > > > > On Fri, Jun 1, 2018 at 2:58 PM, Alex Zai wrote: > > > > > > > Just realized that the email lists strips aways all hyperlinks. > > Attached > > > > is a > > > > copy of my previous email with links pasted in. > > > > > > > > What are peoples' thought on requiring cmake when building from > source? > > > > Currently we have to maintain two independent build files (CMakeLists > > and > > > > Makefile) which makes it more difficult to develop (each are 600+ > > lines). > > > > Also, > > > > our current build system (in Makefile) requires that 3rdparty > > > dependencies > > > > have > > > > binaries present (or a Makefile to generate binaries) in the repo, > > which > > > > is not > > > > always the case. > > > > Generating a makefile with cmake will make our Makefile very simple > > like > > > > PyTorch'sMakefile (20 lines of code - > > > > https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not > > all > > > > 3rdparty > > > > dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end > up > > > > calling > > > > cmake > > > > (https://github.com/apache/incubator-mxnet/blob/master/ > > > > prepare_mkldnn.sh#L96) > > > > to generate binaries (this does not violate our 'no cmake dependency' > > as > > > > USE_MKLDNN is OFF by default). If we encounter any library in the > > future > > > > that > > > > requires us to generate artifacts with cmake, it would be better to > > make > > > > the > > > > switch now. Lastly, we already require cmake as a dependency > > forwindows' > > > > developers > > > > (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot% > > > > 202018-06-01%2013.43.08.png?dl=0) > > > > so this would only affect linux / mac developers who do not have > cmake > > > > already. > > > > I currently have a pendingPR > > > > (https://github.com/apache/incubator-mxnet/pull/8/) that > depends > > on > > > > this > > > > change. The library does not have a Makefile or binaries present. > > Unlike > > > > mkldnn, > > > > we would want this library included by default so I cannot generate > > > > artifacts > > > > with cmake. The alternative would be to strip out only the relevant > > parts > > > > of the > > > > code we need from the library. I did this in a previous version of > myPR > > > > (https://github.com/apache/incubator-mxnet/compare/ > > > > dfdfd1ad15de8bb1b899effb0860a4e834093cfc... > > > a4267eb80488804a7f74ff01f5627c > > > > 47dd46bd78) > > > > but it is incredible messy. > > > > Please let me know your thoughts. > > > > Best, > > > > Alex > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: > > > > What are peoples' thought on requiring cmake when building from > source? > > > > Currently we have to maintain two independent build files (CMakeLists > > and > > > > Makefile) which makes it more difficult to develop (each are 600+ > > lines). > > > > Also, > > > > our current build system (in Makefile) requires that 3rdparty > > > dependencies > > > > have > > > > binaries present (or a Makefile to generate binaries) in the repo, > > which > > > > is not > > > > always the case. > > > > Generating a makefile with cmake will make our Makefile very simple > > like > > > > PyTorch's Makefile (20 lines of code). Also, not all 3rdparty > > > dependencies > > > > have > > > > binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to > > > > generate > > > > binaries (this does not violate our 'no cmake dependency' as > USE_MKLDNN > > > is > > > > OFF > > > > by default). If we encounter any library in the future that requires > us > > > to > > > > generate
Re: Make cmake default
+1 Let's move to CMake. It has much better support, and it's not worth maintaining two build systems. If we really want we could maintain a make file which manages the installation of cmake and calls cmake internally! It seems easy to install cmake. There is a shell script with binary for all Linux x86_64, Windows, Mac. For other systems as well, it's just a couple of steps. Regards, Rahul On Fri, 1 Jun 2018 at 15:12 Chen HY wrote: > building for rpi doesn't mean you should build on a rpi... that takes > forever. > > 2018-06-01 23:06 GMT+01:00 Anirudh : > > > +1 to using cmake and deprecating Makefile. I was able to find a previous > > discussion on this: > > https://github.com/apache/incubator-mxnet/issues/8702 > > > > The concerns raised were > > 1. Building on devices like raspberry pi where cmake is non existent or > > old. > > 2. Adding an additional dependency. > > > > As mentioned in the thread, if we provide good instructions on how to > > install cmake/build cmake from source, > > these concerns will be addressed. > > > > Anirudh > > > > On Fri, Jun 1, 2018 at 2:58 PM, Alex Zai wrote: > > > > > Just realized that the email lists strips aways all hyperlinks. > Attached > > > is a > > > copy of my previous email with links pasted in. > > > > > > What are peoples' thought on requiring cmake when building from source? > > > Currently we have to maintain two independent build files (CMakeLists > and > > > Makefile) which makes it more difficult to develop (each are 600+ > lines). > > > Also, > > > our current build system (in Makefile) requires that 3rdparty > > dependencies > > > have > > > binaries present (or a Makefile to generate binaries) in the repo, > which > > > is not > > > always the case. > > > Generating a makefile with cmake will make our Makefile very simple > like > > > PyTorch'sMakefile (20 lines of code - > > > https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not > all > > > 3rdparty > > > dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up > > > calling > > > cmake > > > (https://github.com/apache/incubator-mxnet/blob/master/ > > > prepare_mkldnn.sh#L96) > > > to generate binaries (this does not violate our 'no cmake dependency' > as > > > USE_MKLDNN is OFF by default). If we encounter any library in the > future > > > that > > > requires us to generate artifacts with cmake, it would be better to > make > > > the > > > switch now. Lastly, we already require cmake as a dependency > forwindows' > > > developers > > > (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot% > > > 202018-06-01%2013.43.08.png?dl=0) > > > so this would only affect linux / mac developers who do not have cmake > > > already. > > > I currently have a pendingPR > > > (https://github.com/apache/incubator-mxnet/pull/8/) that depends > on > > > this > > > change. The library does not have a Makefile or binaries present. > Unlike > > > mkldnn, > > > we would want this library included by default so I cannot generate > > > artifacts > > > with cmake. The alternative would be to strip out only the relevant > parts > > > of the > > > code we need from the library. I did this in a previous version of myPR > > > (https://github.com/apache/incubator-mxnet/compare/ > > > dfdfd1ad15de8bb1b899effb0860a4e834093cfc... > > a4267eb80488804a7f74ff01f5627c > > > 47dd46bd78) > > > but it is incredible messy. > > > Please let me know your thoughts. > > > Best, > > > Alex > > > > > > > > > > > > > > > > > > On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: > > > What are peoples' thought on requiring cmake when building from source? > > > Currently we have to maintain two independent build files (CMakeLists > and > > > Makefile) which makes it more difficult to develop (each are 600+ > lines). > > > Also, > > > our current build system (in Makefile) requires that 3rdparty > > dependencies > > > have > > > binaries present (or a Makefile to generate binaries) in the repo, > which > > > is not > > > always the case. > > > Generating a makefile with cmake will make our Makefile very simple > like > > > PyTorch's Makefile (20 lines of code). Also, not all 3rdparty > > dependencies > > > have > > > binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to > > > generate > > > binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN > > is > > > OFF > > > by default). If we encounter any library in the future that requires us > > to > > > generate artifacts with cmake, it would be better to make the switch > now. > > > Lastly, we already require cmake as a dependency for windows' > > > developers so this > > > would only affect linux / mac developers who do not have cmake already. > > > I currently have a pending PR that depends on this change. The library > > > does not > > > have a Makefile or binaries present. Unlike mkldnn, we would want this > > > library > > > included by default so I cannot generate artifacts with cmake. The > > > alternative
Re: Make cmake default
building for rpi doesn't mean you should build on a rpi... that takes forever. 2018-06-01 23:06 GMT+01:00 Anirudh : > +1 to using cmake and deprecating Makefile. I was able to find a previous > discussion on this: > https://github.com/apache/incubator-mxnet/issues/8702 > > The concerns raised were > 1. Building on devices like raspberry pi where cmake is non existent or > old. > 2. Adding an additional dependency. > > As mentioned in the thread, if we provide good instructions on how to > install cmake/build cmake from source, > these concerns will be addressed. > > Anirudh > > On Fri, Jun 1, 2018 at 2:58 PM, Alex Zai wrote: > > > Just realized that the email lists strips aways all hyperlinks. Attached > > is a > > copy of my previous email with links pasted in. > > > > What are peoples' thought on requiring cmake when building from source? > > Currently we have to maintain two independent build files (CMakeLists and > > Makefile) which makes it more difficult to develop (each are 600+ lines). > > Also, > > our current build system (in Makefile) requires that 3rdparty > dependencies > > have > > binaries present (or a Makefile to generate binaries) in the repo, which > > is not > > always the case. > > Generating a makefile with cmake will make our Makefile very simple like > > PyTorch'sMakefile (20 lines of code - > > https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not all > > 3rdparty > > dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up > > calling > > cmake > > (https://github.com/apache/incubator-mxnet/blob/master/ > > prepare_mkldnn.sh#L96) > > to generate binaries (this does not violate our 'no cmake dependency' as > > USE_MKLDNN is OFF by default). If we encounter any library in the future > > that > > requires us to generate artifacts with cmake, it would be better to make > > the > > switch now. Lastly, we already require cmake as a dependency forwindows' > > developers > > (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot% > > 202018-06-01%2013.43.08.png?dl=0) > > so this would only affect linux / mac developers who do not have cmake > > already. > > I currently have a pendingPR > > (https://github.com/apache/incubator-mxnet/pull/8/) that depends on > > this > > change. The library does not have a Makefile or binaries present. Unlike > > mkldnn, > > we would want this library included by default so I cannot generate > > artifacts > > with cmake. The alternative would be to strip out only the relevant parts > > of the > > code we need from the library. I did this in a previous version of myPR > > (https://github.com/apache/incubator-mxnet/compare/ > > dfdfd1ad15de8bb1b899effb0860a4e834093cfc... > a4267eb80488804a7f74ff01f5627c > > 47dd46bd78) > > but it is incredible messy. > > Please let me know your thoughts. > > Best, > > Alex > > > > > > > > > > > > On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: > > What are peoples' thought on requiring cmake when building from source? > > Currently we have to maintain two independent build files (CMakeLists and > > Makefile) which makes it more difficult to develop (each are 600+ lines). > > Also, > > our current build system (in Makefile) requires that 3rdparty > dependencies > > have > > binaries present (or a Makefile to generate binaries) in the repo, which > > is not > > always the case. > > Generating a makefile with cmake will make our Makefile very simple like > > PyTorch's Makefile (20 lines of code). Also, not all 3rdparty > dependencies > > have > > binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to > > generate > > binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN > is > > OFF > > by default). If we encounter any library in the future that requires us > to > > generate artifacts with cmake, it would be better to make the switch now. > > Lastly, we already require cmake as a dependency for windows' > > developers so this > > would only affect linux / mac developers who do not have cmake already. > > I currently have a pending PR that depends on this change. The library > > does not > > have a Makefile or binaries present. Unlike mkldnn, we would want this > > library > > included by default so I cannot generate artifacts with cmake. The > > alternative > > would be to strip out only the relevant parts of the code we need from > the > > library. I did this in a previous version of my PR but it is incredible > > messy. > > Please let me know your thoughts. > > Best, > > Alex > > > -- Chen Hanyang 陈涵洋 Software School Fudan University +86-138-1881-7745
Re: Make cmake default
+1 Thanks for bringing this up. Maintaining two build systems is a pain. If we decide to make cmake default, please make sure all installation documentations are updated correspondingly. They're currently all using "make" if installed from source. Best, Haibin On Fri, Jun 1, 2018 at 3:06 PM, Anirudh wrote: > +1 to using cmake and deprecating Makefile. I was able to find a previous > discussion on this: > https://github.com/apache/incubator-mxnet/issues/8702 > > The concerns raised were > 1. Building on devices like raspberry pi where cmake is non existent or > old. > 2. Adding an additional dependency. > > As mentioned in the thread, if we provide good instructions on how to > install cmake/build cmake from source, > these concerns will be addressed. > > Anirudh > > On Fri, Jun 1, 2018 at 2:58 PM, Alex Zai wrote: > > > Just realized that the email lists strips aways all hyperlinks. Attached > > is a > > copy of my previous email with links pasted in. > > > > What are peoples' thought on requiring cmake when building from source? > > Currently we have to maintain two independent build files (CMakeLists and > > Makefile) which makes it more difficult to develop (each are 600+ lines). > > Also, > > our current build system (in Makefile) requires that 3rdparty > dependencies > > have > > binaries present (or a Makefile to generate binaries) in the repo, which > > is not > > always the case. > > Generating a makefile with cmake will make our Makefile very simple like > > PyTorch'sMakefile (20 lines of code - > > https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not all > > 3rdparty > > dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up > > calling > > cmake > > (https://github.com/apache/incubator-mxnet/blob/master/ > > prepare_mkldnn.sh#L96) > > to generate binaries (this does not violate our 'no cmake dependency' as > > USE_MKLDNN is OFF by default). If we encounter any library in the future > > that > > requires us to generate artifacts with cmake, it would be better to make > > the > > switch now. Lastly, we already require cmake as a dependency forwindows' > > developers > > (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot% > > 202018-06-01%2013.43.08.png?dl=0) > > so this would only affect linux / mac developers who do not have cmake > > already. > > I currently have a pendingPR > > (https://github.com/apache/incubator-mxnet/pull/8/) that depends on > > this > > change. The library does not have a Makefile or binaries present. Unlike > > mkldnn, > > we would want this library included by default so I cannot generate > > artifacts > > with cmake. The alternative would be to strip out only the relevant parts > > of the > > code we need from the library. I did this in a previous version of myPR > > (https://github.com/apache/incubator-mxnet/compare/ > > dfdfd1ad15de8bb1b899effb0860a4e834093cfc... > a4267eb80488804a7f74ff01f5627c > > 47dd46bd78) > > but it is incredible messy. > > Please let me know your thoughts. > > Best, > > Alex > > > > > > > > > > > > On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: > > What are peoples' thought on requiring cmake when building from source? > > Currently we have to maintain two independent build files (CMakeLists and > > Makefile) which makes it more difficult to develop (each are 600+ lines). > > Also, > > our current build system (in Makefile) requires that 3rdparty > dependencies > > have > > binaries present (or a Makefile to generate binaries) in the repo, which > > is not > > always the case. > > Generating a makefile with cmake will make our Makefile very simple like > > PyTorch's Makefile (20 lines of code). Also, not all 3rdparty > dependencies > > have > > binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to > > generate > > binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN > is > > OFF > > by default). If we encounter any library in the future that requires us > to > > generate artifacts with cmake, it would be better to make the switch now. > > Lastly, we already require cmake as a dependency for windows' > > developers so this > > would only affect linux / mac developers who do not have cmake already. > > I currently have a pending PR that depends on this change. The library > > does not > > have a Makefile or binaries present. Unlike mkldnn, we would want this > > library > > included by default so I cannot generate artifacts with cmake. The > > alternative > > would be to strip out only the relevant parts of the code we need from > the > > library. I did this in a previous version of my PR but it is incredible > > messy. > > Please let me know your thoughts. > > Best, > > Alex > > >
Re: Make cmake default
+1 to using cmake and deprecating Makefile. I was able to find a previous discussion on this: https://github.com/apache/incubator-mxnet/issues/8702 The concerns raised were 1. Building on devices like raspberry pi where cmake is non existent or old. 2. Adding an additional dependency. As mentioned in the thread, if we provide good instructions on how to install cmake/build cmake from source, these concerns will be addressed. Anirudh On Fri, Jun 1, 2018 at 2:58 PM, Alex Zai wrote: > Just realized that the email lists strips aways all hyperlinks. Attached > is a > copy of my previous email with links pasted in. > > What are peoples' thought on requiring cmake when building from source? > Currently we have to maintain two independent build files (CMakeLists and > Makefile) which makes it more difficult to develop (each are 600+ lines). > Also, > our current build system (in Makefile) requires that 3rdparty dependencies > have > binaries present (or a Makefile to generate binaries) in the repo, which > is not > always the case. > Generating a makefile with cmake will make our Makefile very simple like > PyTorch'sMakefile (20 lines of code - > https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not all > 3rdparty > dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up > calling > cmake > (https://github.com/apache/incubator-mxnet/blob/master/ > prepare_mkldnn.sh#L96) > to generate binaries (this does not violate our 'no cmake dependency' as > USE_MKLDNN is OFF by default). If we encounter any library in the future > that > requires us to generate artifacts with cmake, it would be better to make > the > switch now. Lastly, we already require cmake as a dependency forwindows' > developers > (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot% > 202018-06-01%2013.43.08.png?dl=0) > so this would only affect linux / mac developers who do not have cmake > already. > I currently have a pendingPR > (https://github.com/apache/incubator-mxnet/pull/8/) that depends on > this > change. The library does not have a Makefile or binaries present. Unlike > mkldnn, > we would want this library included by default so I cannot generate > artifacts > with cmake. The alternative would be to strip out only the relevant parts > of the > code we need from the library. I did this in a previous version of myPR > (https://github.com/apache/incubator-mxnet/compare/ > dfdfd1ad15de8bb1b899effb0860a4e834093cfc...a4267eb80488804a7f74ff01f5627c > 47dd46bd78) > but it is incredible messy. > Please let me know your thoughts. > Best, > Alex > > > > > > On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: > What are peoples' thought on requiring cmake when building from source? > Currently we have to maintain two independent build files (CMakeLists and > Makefile) which makes it more difficult to develop (each are 600+ lines). > Also, > our current build system (in Makefile) requires that 3rdparty dependencies > have > binaries present (or a Makefile to generate binaries) in the repo, which > is not > always the case. > Generating a makefile with cmake will make our Makefile very simple like > PyTorch's Makefile (20 lines of code). Also, not all 3rdparty dependencies > have > binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to > generate > binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN is > OFF > by default). If we encounter any library in the future that requires us to > generate artifacts with cmake, it would be better to make the switch now. > Lastly, we already require cmake as a dependency for windows' > developers so this > would only affect linux / mac developers who do not have cmake already. > I currently have a pending PR that depends on this change. The library > does not > have a Makefile or binaries present. Unlike mkldnn, we would want this > library > included by default so I cannot generate artifacts with cmake. The > alternative > would be to strip out only the relevant parts of the code we need from the > library. I did this in a previous version of my PR but it is incredible > messy. > Please let me know your thoughts. > Best, > Alex >
Re: Make cmake default
Just realized that the email lists strips aways all hyperlinks. Attached is a copy of my previous email with links pasted in. What are peoples' thought on requiring cmake when building from source? Currently we have to maintain two independent build files (CMakeLists and Makefile) which makes it more difficult to develop (each are 600+ lines). Also, our current build system (in Makefile) requires that 3rdparty dependencies have binaries present (or a Makefile to generate binaries) in the repo, which is not always the case. Generating a makefile with cmake will make our Makefile very simple like PyTorch'sMakefile (20 lines of code - https://github.com/pytorch/pytorch/blob/master/Makefile). Also, not all 3rdparty dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake (https://github.com/apache/incubator-mxnet/blob/master/prepare_mkldnn.sh#L96) to generate binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN is OFF by default). If we encounter any library in the future that requires us to generate artifacts with cmake, it would be better to make the switch now. Lastly, we already require cmake as a dependency forwindows' developers (https://www.dropbox.com/s/9sfnderg58z4j1l/Screenshot%202018-06-01%2013.43.08.png?dl=0) so this would only affect linux / mac developers who do not have cmake already. I currently have a pendingPR (https://github.com/apache/incubator-mxnet/pull/8/) that depends on this change. The library does not have a Makefile or binaries present. Unlike mkldnn, we would want this library included by default so I cannot generate artifacts with cmake. The alternative would be to strip out only the relevant parts of the code we need from the library. I did this in a previous version of myPR (https://github.com/apache/incubator-mxnet/compare/dfdfd1ad15de8bb1b899effb0860a4e834093cfc...a4267eb80488804a7f74ff01f5627c47dd46bd78) but it is incredible messy. Please let me know your thoughts. Best, Alex On Fri, Jun 1, 2018 2:51 PM, Alex Zai aza...@gmail.com wrote: What are peoples' thought on requiring cmake when building from source? Currently we have to maintain two independent build files (CMakeLists and Makefile) which makes it more difficult to develop (each are 600+ lines). Also, our current build system (in Makefile) requires that 3rdparty dependencies have binaries present (or a Makefile to generate binaries) in the repo, which is not always the case. Generating a makefile with cmake will make our Makefile very simple like PyTorch's Makefile (20 lines of code). Also, not all 3rdparty dependencies have binaries or Makefiles. For 3rdparty/mkldnn we end up calling cmake to generate binaries (this does not violate our 'no cmake dependency' as USE_MKLDNN is OFF by default). If we encounter any library in the future that requires us to generate artifacts with cmake, it would be better to make the switch now. Lastly, we already require cmake as a dependency for windows' developers so this would only affect linux / mac developers who do not have cmake already. I currently have a pending PR that depends on this change. The library does not have a Makefile or binaries present. Unlike mkldnn, we would want this library included by default so I cannot generate artifacts with cmake. The alternative would be to strip out only the relevant parts of the code we need from the library. I did this in a previous version of my PR but it is incredible messy. Please let me know your thoughts. Best, Alex
