Re: move entirely to CMakefiles for building MXNet, drop Makefiles

[Zha, Sheng]

You can set rpath to $${ORIGIN} to make the path where the libmxnet.so resides 
a path to look up.

-sz



- Sent by my thumb
> On Mar 7, 2018, at 7:45 AM, Pedro Larroy  wrote:
> 
> About libomp.so
> 
> This is giving me some problems when creating a pip package for installing
> on Jetson. I'm thinking that in this case would be better to compile with
> -fopenmp. I tried adding libomp.so to the pip package next to libmxnet.so
> and still I couldn't load the library...  Any ideas?
> 
> Pedro
> 
>> On Wed, Mar 7, 2018 at 6:31 AM, Eric Xie  wrote:
>> 
>> We want as few dependencies as possible.
>> CMake alone is enough trouble for our users. We don't want to burden them
>> with other stuff.
>> 
>> On 2018/03/06 17:21:15, kellen sunderland 
>> wrote:
>>> Short term solution sounds good to me Chris.  Converting the CI should be
>>> pretty easy.  One thing we should keep in mind is that there's going to
>> be
>>> a bunch of doc's we'll have to update.
>>> 
>>> Warning, slight thread hijack ahead:
>>> As a more long term change I was wondering if we had considered using
>>> hunter for third party packages?  It seems like a good system, and while
>> it
>>> likely won't have support for all our projects, we can contribute back
>>> support for the ones we care about.
>>> 
>>> For me the primary benefit would be that it would conditionally fetch
>>> source at build time based on your cmake configuration.  This would mean
>> it
>>> could say, detect you want opencv/mp/protobuf (if you're using onnx) and
>>> then it'd check out the pinned version we specify and build for your
>>> platform.
>>> 
>>> 
>>> On Tue, Mar 6, 2018 at 6:19 PM, Chris Olivier 
>> wrote:
>>> 
 Here is discussion:
 
 https://github.com/apache/incubator-mxnet/issues/8702
 
 On Tue, Mar 6, 2018 at 9:14 AM, Chris Olivier 
 wrote:
 
> This was agreed upon some time ago in a github issue thread, unless
>> there
> are new objections to it.
> 
> As far as I know, it's just a matter of someone putting in the work
>> to
 add
> more functionality to cmake or to fuse the two builds.
> 
> One solution for the short term might include having the Makefile
>> launch
> cmake for most of the build and fall back to Makefile for some of the
> remaining stuff, like scalapkg, rpkg, etc.
> 
> btw, cmake uses the openmp in 3rdparty
> 
> 
> 
> On Tue, Mar 6, 2018 at 8:51 AM, Pedro Larroy <
 pedro.larroy.li...@gmail.com
>> wrote:
> 
>> Hi
>> 
>> I would like to raise the issue that maintaining two overlapping
>> build
>> systems is too much overhead. It adds unnecessary complexity and
>> differences on how the project is built.
>> 
>> For example, openmp is used differently from CMake and Make, in the
 former
>> the one provided by gcc is used and in the later is compiled from
>> the
>> 3rdparty folder.
>> 
>> I think this situation is not sustainable for the project, and
>> specially
>> if
>> we add that we want to support compilation and cross compilation on
>> devices.
>> 
>> My proposal would be to identify any gaps that are not covered by
>> the
>> CMake
>> build system, cover them and make CMake the single build system for
 MXNet,
>> well tested and fully supported.
>> 
>> Pedro.
>> 
> 
> 
 
>>> 
>> 

Re: move entirely to CMakefiles for building MXNet, drop Makefiles

[Pedro Larroy]

About libomp.so

This is giving me some problems when creating a pip package for installing
on Jetson. I'm thinking that in this case would be better to compile with
-fopenmp. I tried adding libomp.so to the pip package next to libmxnet.so
and still I couldn't load the library...  Any ideas?

Pedro

On Wed, Mar 7, 2018 at 6:31 AM, Eric Xie  wrote:

> We want as few dependencies as possible.
> CMake alone is enough trouble for our users. We don't want to burden them
> with other stuff.
>
> On 2018/03/06 17:21:15, kellen sunderland 
> wrote:
> > Short term solution sounds good to me Chris.  Converting the CI should be
> > pretty easy.  One thing we should keep in mind is that there's going to
> be
> > a bunch of doc's we'll have to update.
> >
> > Warning, slight thread hijack ahead:
> > As a more long term change I was wondering if we had considered using
> > hunter for third party packages?  It seems like a good system, and while
> it
> > likely won't have support for all our projects, we can contribute back
> > support for the ones we care about.
> >
> > For me the primary benefit would be that it would conditionally fetch
> > source at build time based on your cmake configuration.  This would mean
> it
> > could say, detect you want opencv/mp/protobuf (if you're using onnx) and
> > then it'd check out the pinned version we specify and build for your
> > platform.
> >
> >
> > On Tue, Mar 6, 2018 at 6:19 PM, Chris Olivier 
> wrote:
> >
> > > Here is discussion:
> > >
> > > https://github.com/apache/incubator-mxnet/issues/8702
> > >
> > > On Tue, Mar 6, 2018 at 9:14 AM, Chris Olivier 
> > > wrote:
> > >
> > > > This was agreed upon some time ago in a github issue thread, unless
> there
> > > > are new objections to it.
> > > >
> > > > As far as I know, it's just a matter of someone putting in the work
> to
> > > add
> > > > more functionality to cmake or to fuse the two builds.
> > > >
> > > > One solution for the short term might include having the Makefile
> launch
> > > > cmake for most of the build and fall back to Makefile for some of the
> > > > remaining stuff, like scalapkg, rpkg, etc.
> > > >
> > > > btw, cmake uses the openmp in 3rdparty
> > > >
> > > >
> > > >
> > > > On Tue, Mar 6, 2018 at 8:51 AM, Pedro Larroy <
> > > pedro.larroy.li...@gmail.com
> > > > > wrote:
> > > >
> > > >> Hi
> > > >>
> > > >> I would like to raise the issue that maintaining two overlapping
> build
> > > >> systems is too much overhead. It adds unnecessary complexity and
> > > >> differences on how the project is built.
> > > >>
> > > >> For example, openmp is used differently from CMake and Make, in the
> > > former
> > > >> the one provided by gcc is used and in the later is compiled from
> the
> > > >> 3rdparty folder.
> > > >>
> > > >> I think this situation is not sustainable for the project, and
> specially
> > > >> if
> > > >> we add that we want to support compilation and cross compilation on
> > > >> devices.
> > > >>
> > > >> My proposal would be to identify any gaps that are not covered by
> the
> > > >> CMake
> > > >> build system, cover them and make CMake the single build system for
> > > MXNet,
> > > >> well tested and fully supported.
> > > >>
> > > >> Pedro.
> > > >>
> > > >
> > > >
> > >
> >
>

Re: move entirely to CMakefiles for building MXNet, drop Makefiles

[Eric Xie]

We want as few dependencies as possible.
CMake alone is enough trouble for our users. We don't want to burden them with 
other stuff.

On 2018/03/06 17:21:15, kellen sunderland  wrote: 
> Short term solution sounds good to me Chris.  Converting the CI should be
> pretty easy.  One thing we should keep in mind is that there's going to be
> a bunch of doc's we'll have to update.
> 
> Warning, slight thread hijack ahead:
> As a more long term change I was wondering if we had considered using
> hunter for third party packages?  It seems like a good system, and while it
> likely won't have support for all our projects, we can contribute back
> support for the ones we care about.
> 
> For me the primary benefit would be that it would conditionally fetch
> source at build time based on your cmake configuration.  This would mean it
> could say, detect you want opencv/mp/protobuf (if you're using onnx) and
> then it'd check out the pinned version we specify and build for your
> platform.
> 
> 
> On Tue, Mar 6, 2018 at 6:19 PM, Chris Olivier  wrote:
> 
> > Here is discussion:
> >
> > https://github.com/apache/incubator-mxnet/issues/8702
> >
> > On Tue, Mar 6, 2018 at 9:14 AM, Chris Olivier 
> > wrote:
> >
> > > This was agreed upon some time ago in a github issue thread, unless there
> > > are new objections to it.
> > >
> > > As far as I know, it's just a matter of someone putting in the work to
> > add
> > > more functionality to cmake or to fuse the two builds.
> > >
> > > One solution for the short term might include having the Makefile launch
> > > cmake for most of the build and fall back to Makefile for some of the
> > > remaining stuff, like scalapkg, rpkg, etc.
> > >
> > > btw, cmake uses the openmp in 3rdparty
> > >
> > >
> > >
> > > On Tue, Mar 6, 2018 at 8:51 AM, Pedro Larroy <
> > pedro.larroy.li...@gmail.com
> > > > wrote:
> > >
> > >> Hi
> > >>
> > >> I would like to raise the issue that maintaining two overlapping build
> > >> systems is too much overhead. It adds unnecessary complexity and
> > >> differences on how the project is built.
> > >>
> > >> For example, openmp is used differently from CMake and Make, in the
> > former
> > >> the one provided by gcc is used and in the later is compiled from the
> > >> 3rdparty folder.
> > >>
> > >> I think this situation is not sustainable for the project, and specially
> > >> if
> > >> we add that we want to support compilation and cross compilation on
> > >> devices.
> > >>
> > >> My proposal would be to identify any gaps that are not covered by the
> > >> CMake
> > >> build system, cover them and make CMake the single build system for
> > MXNet,
> > >> well tested and fully supported.
> > >>
> > >> Pedro.
> > >>
> > >
> > >
> >
> 

Re: move entirely to CMakefiles for building MXNet, drop Makefiles

[kellen sunderland]

Short term solution sounds good to me Chris.  Converting the CI should be
pretty easy.  One thing we should keep in mind is that there's going to be
a bunch of doc's we'll have to update.

Warning, slight thread hijack ahead:
As a more long term change I was wondering if we had considered using
hunter for third party packages?  It seems like a good system, and while it
likely won't have support for all our projects, we can contribute back
support for the ones we care about.

For me the primary benefit would be that it would conditionally fetch
source at build time based on your cmake configuration.  This would mean it
could say, detect you want opencv/mp/protobuf (if you're using onnx) and
then it'd check out the pinned version we specify and build for your
platform.


On Tue, Mar 6, 2018 at 6:19 PM, Chris Olivier  wrote:

> Here is discussion:
>
> https://github.com/apache/incubator-mxnet/issues/8702
>
> On Tue, Mar 6, 2018 at 9:14 AM, Chris Olivier 
> wrote:
>
> > This was agreed upon some time ago in a github issue thread, unless there
> > are new objections to it.
> >
> > As far as I know, it's just a matter of someone putting in the work to
> add
> > more functionality to cmake or to fuse the two builds.
> >
> > One solution for the short term might include having the Makefile launch
> > cmake for most of the build and fall back to Makefile for some of the
> > remaining stuff, like scalapkg, rpkg, etc.
> >
> > btw, cmake uses the openmp in 3rdparty
> >
> >
> >
> > On Tue, Mar 6, 2018 at 8:51 AM, Pedro Larroy <
> pedro.larroy.li...@gmail.com
> > > wrote:
> >
> >> Hi
> >>
> >> I would like to raise the issue that maintaining two overlapping build
> >> systems is too much overhead. It adds unnecessary complexity and
> >> differences on how the project is built.
> >>
> >> For example, openmp is used differently from CMake and Make, in the
> former
> >> the one provided by gcc is used and in the later is compiled from the
> >> 3rdparty folder.
> >>
> >> I think this situation is not sustainable for the project, and specially
> >> if
> >> we add that we want to support compilation and cross compilation on
> >> devices.
> >>
> >> My proposal would be to identify any gaps that are not covered by the
> >> CMake
> >> build system, cover them and make CMake the single build system for
> MXNet,
> >> well tested and fully supported.
> >>
> >> Pedro.
> >>
> >
> >
>

Re: move entirely to CMakefiles for building MXNet, drop Makefiles

[Chris Olivier]

Here is discussion:

https://github.com/apache/incubator-mxnet/issues/8702

On Tue, Mar 6, 2018 at 9:14 AM, Chris Olivier  wrote:

> This was agreed upon some time ago in a github issue thread, unless there
> are new objections to it.
>
> As far as I know, it's just a matter of someone putting in the work to add
> more functionality to cmake or to fuse the two builds.
>
> One solution for the short term might include having the Makefile launch
> cmake for most of the build and fall back to Makefile for some of the
> remaining stuff, like scalapkg, rpkg, etc.
>
> btw, cmake uses the openmp in 3rdparty
>
>
>
> On Tue, Mar 6, 2018 at 8:51 AM, Pedro Larroy  > wrote:
>
>> Hi
>>
>> I would like to raise the issue that maintaining two overlapping build
>> systems is too much overhead. It adds unnecessary complexity and
>> differences on how the project is built.
>>
>> For example, openmp is used differently from CMake and Make, in the former
>> the one provided by gcc is used and in the later is compiled from the
>> 3rdparty folder.
>>
>> I think this situation is not sustainable for the project, and specially
>> if
>> we add that we want to support compilation and cross compilation on
>> devices.
>>
>> My proposal would be to identify any gaps that are not covered by the
>> CMake
>> build system, cover them and make CMake the single build system for MXNet,
>> well tested and fully supported.
>>
>> Pedro.
>>
>
>

Re: move entirely to CMakefiles for building MXNet, drop Makefiles

[Chris Olivier]

This was agreed upon some time ago in a github issue thread, unless there
are new objections to it.

As far as I know, it's just a matter of someone putting in the work to add
more functionality to cmake or to fuse the two builds.

One solution for the short term might include having the Makefile launch
cmake for most of the build and fall back to Makefile for some of the
remaining stuff, like scalapkg, rpkg, etc.

btw, cmake uses the openmp in 3rdparty



On Tue, Mar 6, 2018 at 8:51 AM, Pedro Larroy 
wrote:

> Hi
>
> I would like to raise the issue that maintaining two overlapping build
> systems is too much overhead. It adds unnecessary complexity and
> differences on how the project is built.
>
> For example, openmp is used differently from CMake and Make, in the former
> the one provided by gcc is used and in the later is compiled from the
> 3rdparty folder.
>
> I think this situation is not sustainable for the project, and specially if
> we add that we want to support compilation and cross compilation on
> devices.
>
> My proposal would be to identify any gaps that are not covered by the CMake
> build system, cover them and make CMake the single build system for MXNet,
> well tested and fully supported.
>
> Pedro.
>