Hi Junru, > I am actually a bit concerned about the security issues. We are asked to download binaries from third-party websites, which are not controlled or validated by Apache it's possible to run conan server inside apache network and download binaries and sources only from this remote
> CMake does support download artifacts from a pre-defined website very well same for conan, it also supports downloading of artifacts from pre-defined web-sites > rather than broken links to nonsense please let me know which links are broken. I've checked them all, and they are all available for me. may be your ISP is blocking something? > most of the dependencies you mentioned adopts CMake, rather than Conan I don't think it's relevant to compare CMake (meta-build-system) and Conan (package manager), they have different purposes, and they might be used simultaneously, or only one of them might be used, they aren't mutually exclusive. > Would you mind at least mentioning the benefits somewhere in this thread okay, let me list some benefits (not all of them, but something I have on mind) - cross-platform, runs on Windows, Linux, OSX, FreeBSD, etc., also supports cross-compiling for various platforms, like iOS, Android, Emscripten, Windows Phone, etc. - open-source (MIT licensed) and freeware - supports both build from sources and pre-built artifacts (for instance, if compare to usage of submodules, you probably wouldn't use pre-built binaries in that case, as repo size will grow) - supports multiple versions of libraries (some package managers provide only "latest") - supports options of libraries (whatever you need is configurable per library, e.g. shared vs static, with or without assembler, multi-threaded vs single-threaded, etc) - supports installation of build tools as well (something like bison, flex, nasm, yasm, etc.) - has concept of profiles and their management (include options, tools and environment variables to be used for build) - supports various build systems (CMake, Meson, MSBuild, boost build, etc.), pretty much build system agnostic - decentralized, you may use in-house server (e.g. conan server or artifactory), or bintray - extensible via hooks there are some comparisons of package managers on reddit (e.g. https://www.reddit.com/r/cpp/comments/9m4l0p/conan_vcpkg_or_build2/ and https://www.reddit.com/r/cpp/comments/8ldhu0/opinions_on_the_conan_package_manager/ ). > and eagerly asking for avote? I believe that reasonable discussion would keep us within a *healthy* discussion. I am asking for a vote because I was told to do this on GitHub discussion by MXNet developers. I've already started discussions using all suggested channels (GitHub, JIRA and this mail list). all questions were answered, and no further questions appeared until today. I was thinking we already have passed the discussion stage, as all discussion have stalled with no objections (nobody clearly said something like "we're going to adopt conan" or "we will not use conan definetely"). so my impression was discussion stage was already passed, and now it's time for the decision. sorry if I had wrong impression, I am not really very familiar with your processes. > why not I simply specify versions in git submidule, which everyone understands how it behaves I haven't said usage of submodules itself is something bad, if you're fine with submodules, it's okay, but it seems like for MXNet they don't fit all use-cases, as some dependencies are downloaded via other ways (as mentioned above, via conda, cmake or just downloaded from GitHub archives in CI scripts). this causes fragmentation and includes complexity. so, it's clear that submodules somehow didn't satisfy all MXNet needs, as they aren't used for everything. > Everyone know how to include a sub-directory in cmake in one line probably, because not all of your dependencies are using CMake to build, so you can't simply include them into cmake in one line. yours sincerely, Konstantin пт, 3 мая 2019 г. в 23:10, Junru Shao <junrushao1...@gmail.com>: > I am actually a bit concerned about the security issues. We are asked to > download binaries from third-party websites, which are not controlled or > validated by Apache. Although it is claimed to be “decentralized”, I am > really not convinced where the security comes from. > > In the meantime, sacrificing security doesn’t really bring us tangible > benefits. CMake does support download artifacts from a pre-defined website > very well. We may also have pre-built binaries stored in our CI docker > without having to download it them from the internet. > > Another point is that I am not convinced of any advantage of Conan over > other package managers for C++. Would you mind at least mentioning the > benefits somewhere in this thread, rather than carelessly includes tons of > irrelevant links (some of which are even wrong) and eagerly asking for a > vote? I believe that reasonable discussion would keep us within a *healthy* > discussion. > > Last but not least, as we all know, most of the dependencies you mentioned > adopts CMake, rather than Conan, the meta-generator which generates CMake. > I didn’t see your logic stands like “oh you have tons of dependencies so > you must use Conan”, why not I simply specify versions in git submidule, > which everyone understands how it behaves? Everyone know how to include a > sub-directory in cmake in one line, so why we have to write python to make > it less understandable and more complicated? > > In conclusion, we need to be reasonable in healthy discussion. I don’t > particularly want to rudely +1 or -1 for a thing that is unclear to me, but > I really want to see pros and cons, discussion about issues and concerns, > rather than broken links to nonsense. > > Looking forward to your reply! > > Thanks, > Junru > > On Fri, May 3, 2019 at 08:05 kellen sunderland < > kellen.sunderl...@gmail.com> > wrote: > > > Hey Konstantin. Thanks for starting an email thread and sorry for the > > confusion. I think the ides is that we should discuss and agree on > > Conan.io adoption first on the dev list, then start merging PRs. Release > > 1.4.1 is already in testing and the 1.5 code freeze deadline is also near > > so I think it could be difficult to make such a large change on one of > > those releases. I've looked into package management solutions for the > > project before. I was in favour of hunter, but I think Conan's adoption > > rate now makes it the best option. It's simple to use and is becoming > > industry standard, with a minor downside of requiring Python (which has > > meanwhile become the most popular dev language). I'd personally be -1 > for > > 1.4.1 or 1.5, +1 for using Conan in 1.6 or 2.0. > > > > -Kellen > > > > On Fri, May 3, 2019 at 12:59 AM Konstantin Ivlev <tomsks...@gmail.com> > > wrote: > > > > > hi Sheng Zha, > > > > > > on pull request review I was told by Anirudh anirudhacharya and Roshani > > > Nagmote to start discussion/vote on the mxnet dev list. it seems to be > a > > > vicious circle now - on GitHub I am told to use vote, and on vote I am > > told > > > to use GitHub, this doesn't help much. > > > FYI GitHub review stuck, it's already opened since November 2018, and > > it's > > > still not approved (however, there were no objections during the > review). > > > Previous discussion in e-mail thread also didn't encounter any > > objections, > > > and all questions were answered. > > > JIRA ticket has no discussion at all (except it has duplicates of > > comments > > > made on GitHub). > > > so let's process with 3-day vote for now, as other communication > channels > > > were already tried with no success. > > > > > > yours sincerely, Konstantin > > > > > > пт, 3 мая 2019 г. в 14:17, Sheng Zha <zhash...@apache.org>: > > > > > > > Hi Konstantin, > > > > > > > > While conan looks like an option that's worth exploring, given that > > your > > > > request is to merge the pull request, I'd suggest that the request > > should > > > > go through the regular pull request review and it doesn't really > need a > > > > vote (as it doesn't substitute reviews anyway) > > > > > > > > If you would like to gather more attention to it, feel free to ping > in > > a > > > > discussion thread. > > > > > > > > -sz > > > > > > > > On 2019/05/03 06:29:55, Konstantin Ivlev <tomsks...@gmail.com> > wrote: > > > > > Dear MXNet community, > > > > > > > > > > This is the 3-day vote to add conan support for Apache MXNet > > > (incubating) > > > > > version v1.4.1. > > > > > The voting on dev@ list will start May 03 23:59:59 (PST) and close > > on > > > > May > > > > > 06 23:59:59. > > > > > > > > > > Background: conan is open-source, freeware, cross-platform package > > > > manager > > > > > for C and C++ projects, written in python. it provides integration > > with > > > > > various build systems, include CMake. conan may use bintray as a > > server > > > > to > > > > > store and download pre-built packages, or packages might be always > > > built > > > > > from sources. > > > > > > > > > > Problem: currently (as for v1.4.1), Apache MXNet (incubating) is > > using > > > > > several ways to fetch 3rd-party dependencies simultaneously, for > > > > instance: > > > > > 1. download GitHub archives during the build > > > > > - OpenBLAS > > > > > - OpenCV > > > > > 2. conda (alternative way to GitHub archives) > > > > > 3. download from CMake > > > > > - Intel Math Kernel Library (MKL) > > > > > 4. Git submodules > > > > > - cub > > > > > - dlpack > > > > > - dmlc-core > > > > > - googletest > > > > > - mkldnn > > > > > - mshadow > > > > > - onnx-tensorrt > > > > > - openmp > > > > > - ps-lite > > > > > - tvm > > > > > therefore, there are multiple places to look for 3rd parties, and > its > > > > hard > > > > > to update them, as you need to remember or figure it out how to > > update > > > a > > > > > particular dependency to newer version, for instance. > > > > > current Apache MXNet (incubating) build instructions differ very > much > > > per > > > > > platform, require to download and unzip some archives manually, > > > > specifying > > > > > variables with paths to this archives, in conjunction of updating > git > > > > > submodules, > > > > > > > > > > Action: merge pull request providing an initial conan support for > > > Apache > > > > > MXNet (incubating). support conan as an alternate approach to fetch > > > > various > > > > > 3rd-party dependencies. old approaches will be still available, > > > supported > > > > > and left intact. > > > > > > > > > > Below are links to > > > > > 1) conan web-site: https://conan.io/ > > > > > 2) conan GitHub repository: https://github.com/conan-io/conan > > > > > 3) conan documentation: https://docs.conan.io/en/latest/ > > > > > 4) bintray: https://bintray.com > > > > > 5) pull request adding conan support to Apache MXNet (incubating): > > > > > https://github.com/apache/incubator-mxnet/pull/13400 > > > > > 6) JIRA issue: https://issues.apache.org/jira/browse/MXNET-1229 > > > > > 7) previous email discussion: > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/301a46a637f7e3c249c475713f701bef7530c32bc92d8834c0882897@%3Cdev.mxnet.apache.org%3E > > > > > 8) MXNet build instructions: > > > > > https://mxnet-tqchen.readthedocs.io/en/latest/how_to/build.html > > > > > 9) MXNet build instructions (Windows): > > > > > > > > > > > > > > > https://mxnet.incubator.apache.org/versions/master/install/windows_setup.html > > > > > 10) MXNet build instructions (OSX): > > > > > > > > > http://mxnet.incubator.apache.org/versions/master/install/osx_setup.html > > > > > 11) MXNet build instructions (Linux): > > > > > > > > > > > > > > > http://mxnet.incubator.apache.org/versions/master/install/ubuntu_setup.html > > > > > 12) MXNet development setup (OSX): > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+Developer+Setup+on+Mac > > > > > > > > > > Please remember to TEST first before voting accordingly: > > > > > +1 = approve > > > > > +0 = no opinion > > > > > -1 = disapprove (provide reason) > > > > > > > > > > Best regards, > > > > > Konstantin Ivlev > > > > > > > > > > > > > > >
