Re: Apache MXNet Scala nightly-build now available on Apache Nexus
Thanks for attention. Here's an example demonstrates how to configure pom.xml to use the nightly build in your own project: https://github.com/yzhliu/mxnet-scala-example/blob/master/pom.xml#L152-L163 On Mon, Oct 29, 2018 at 2:53 PM Carin Meier wrote: > > Fantastic! This is going to be great for the Clojure MXNet users as well. > > On Mon, Oct 29, 2018 at 1:39 PM YiZhi Liu wrote: > > > Hi, > > > > > > > > I am happy to announce that the availability of the experimental > > nightly-build Scala package on Maven in Nexus. The nightly-builds, > > currently published as 1.3.1-SNAPSHOT version, include the latest changes > > from the master branch from apache/incubator-mxnet GitHub repo and refresh > > every day. Furthermore, we completely overhauled the MXNet library shipped > > inside and adopted the binary build logic used for Python pip, so now Scala > > users can also enjoy the portable MXNet with fully loaded features. > > Currently, the supported artifacts are: > > > > - mxnet-full_2.11-linux-x86_64-cpu (Linux-CPU) > > > > - mxnet-full_2.11-linux-x86_64-gpu (Linux-GPU, CUDA 9.0) > > > > - mxnet-full_2.11-osx-x86_64-cpu (OSX-CPU) > > > > > > > > FAQ: > > > > - What's new in the nightly Scala packages compared to the previous Scala > > Maven artifacts? > > > > Most notably, better usability, more supported features, and better > > platform compatibility. The new packages now support Ubuntu 14.04 and > > Amazon Linux and support the same broad set of features as the existing pip > > packages. Besides, users are no longer required to install all the > > dependencies to use the nightly Scala packages as the included MXNet > > libraries are fully loaded. For the list of supported platforms and > > features, see [1]. > > > > > > > > - What's different in the build approach compared to what's in the previous > > Scala Maven packages? > > > > The previous Scala Maven packages (1.2, 1.2.1, 1.3) were built with the > > same build commands that regular users would use when building from source > > [2]. This naive approach limits the compatibility to only similar > > environments as where the packages were built (which was Ubuntu 16.04 for > > Linux, and OSX 10.13 for OSX). As a result, users cannot use it on more > > CentOS or Amazon Linux and may suffer the problem of mismatching of > > dependency versions due to the difference in their environment. > > > > On the other hand, Python users have long been enjoying portable and fully > > loaded MXNet packages from pip. The MXNet libraries in pip packages are > > built in more controlled environments with compatibility in mind. > > Dependencies for extended features are statically linked and masked so that > > no additional steps are needed. > > > > Now, in the new Scala nightly packages, the same approach as pip is used > > for building the MXNet library so that the Scala users can benefit from the > > same great features. > > > > > > > > - How do I get the new nightly packages? > > > > You can find the packages on Apache (Snapshots) repository [3] including > > pom files and jars, which you can download and add to your projects. > > Alternatively, you can configure your pom.xml to use the Maven repository. > > Remember to add repository config: > > > > > > > > > > > > Apache Snapshot > > > > https://repository.apache.org/content/groups/snapshots > > > > > > > > > > > > > > > > - What's the support level of the Scala nightly packages? > > > > This package is currently experimental, so use at your own risk. While the > > first nightly packages are already manually verified on several platforms, > > and the automated publish pipeline runs unit tests and integration tests on > > OSX and Ubuntu 14.04 before publishing, the testing is currently limited. > > > > > > > > - How can I help? > > > > We call on all MXNet Scala community to test out the new nightly packages. > > > > > > > > - I found a problem in using the package. What should I do? > > > > Please report this in the Github issue in [1]. > > > > > > > > - What's next? > > > > There are several more things to be done: > > > > 1. We are adding more comprehensive support for GPU versions for different > > CUDA versions. > > > > 2. We are adding more tests on more platforms. We are currently examining > > the MXNet's Jenkins CI solution for supporting more platforms. > > > > 3. As we recognize that the build logic benefits the community, we plan to > > open source the build scripts in the mxnet main repo. See progress at [5]. > > > > > > > > Qing Lan, Zach Kimberg, Sheng Zha, and I worked together to make this > > happen. Thanks to Sheng Zha for sharing the pip build logic, and for > > building the automated publishing pipelines. Special thanks to Qing and > > Zach for sharing their expertise in Scala with Sheng and guiding and > > supporting him locally for this milestone. > > > > > > > > [1] https://github.com/apache/incubator-mxnet/issues/8671 > > > > [2] > > > > https://github.com/apache/incubator
Re: Apache MXNet Scala nightly-build now available on Apache Nexus
Fantastic! This is going to be great for the Clojure MXNet users as well. On Mon, Oct 29, 2018 at 1:39 PM YiZhi Liu wrote: > Hi, > > > > I am happy to announce that the availability of the experimental > nightly-build Scala package on Maven in Nexus. The nightly-builds, > currently published as 1.3.1-SNAPSHOT version, include the latest changes > from the master branch from apache/incubator-mxnet GitHub repo and refresh > every day. Furthermore, we completely overhauled the MXNet library shipped > inside and adopted the binary build logic used for Python pip, so now Scala > users can also enjoy the portable MXNet with fully loaded features. > Currently, the supported artifacts are: > > - mxnet-full_2.11-linux-x86_64-cpu (Linux-CPU) > > - mxnet-full_2.11-linux-x86_64-gpu (Linux-GPU, CUDA 9.0) > > - mxnet-full_2.11-osx-x86_64-cpu (OSX-CPU) > > > > FAQ: > > - What's new in the nightly Scala packages compared to the previous Scala > Maven artifacts? > > Most notably, better usability, more supported features, and better > platform compatibility. The new packages now support Ubuntu 14.04 and > Amazon Linux and support the same broad set of features as the existing pip > packages. Besides, users are no longer required to install all the > dependencies to use the nightly Scala packages as the included MXNet > libraries are fully loaded. For the list of supported platforms and > features, see [1]. > > > > - What's different in the build approach compared to what's in the previous > Scala Maven packages? > > The previous Scala Maven packages (1.2, 1.2.1, 1.3) were built with the > same build commands that regular users would use when building from source > [2]. This naive approach limits the compatibility to only similar > environments as where the packages were built (which was Ubuntu 16.04 for > Linux, and OSX 10.13 for OSX). As a result, users cannot use it on more > CentOS or Amazon Linux and may suffer the problem of mismatching of > dependency versions due to the difference in their environment. > > On the other hand, Python users have long been enjoying portable and fully > loaded MXNet packages from pip. The MXNet libraries in pip packages are > built in more controlled environments with compatibility in mind. > Dependencies for extended features are statically linked and masked so that > no additional steps are needed. > > Now, in the new Scala nightly packages, the same approach as pip is used > for building the MXNet library so that the Scala users can benefit from the > same great features. > > > > - How do I get the new nightly packages? > > You can find the packages on Apache (Snapshots) repository [3] including > pom files and jars, which you can download and add to your projects. > Alternatively, you can configure your pom.xml to use the Maven repository. > Remember to add repository config: > > > > > > Apache Snapshot > > https://repository.apache.org/content/groups/snapshots > > > > > > > > - What's the support level of the Scala nightly packages? > > This package is currently experimental, so use at your own risk. While the > first nightly packages are already manually verified on several platforms, > and the automated publish pipeline runs unit tests and integration tests on > OSX and Ubuntu 14.04 before publishing, the testing is currently limited. > > > > - How can I help? > > We call on all MXNet Scala community to test out the new nightly packages. > > > > - I found a problem in using the package. What should I do? > > Please report this in the Github issue in [1]. > > > > - What's next? > > There are several more things to be done: > > 1. We are adding more comprehensive support for GPU versions for different > CUDA versions. > > 2. We are adding more tests on more platforms. We are currently examining > the MXNet's Jenkins CI solution for supporting more platforms. > > 3. As we recognize that the build logic benefits the community, we plan to > open source the build scripts in the mxnet main repo. See progress at [5]. > > > > Qing Lan, Zach Kimberg, Sheng Zha, and I worked together to make this > happen. Thanks to Sheng Zha for sharing the pip build logic, and for > building the automated publishing pipelines. Special thanks to Qing and > Zach for sharing their expertise in Scala with Sheng and guiding and > supporting him locally for this milestone. > > > > [1] https://github.com/apache/incubator-mxnet/issues/8671 > > [2] > > https://github.com/apache/incubator-mxnet/blob/master/scala-package/dev/compile-mxnet-backend.sh#L59-L105 > > [3] > > https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.3.1-SNAPSHOT~~ > > [4] https://maven.apache.org/repository-management.html > > [5] https://github.com/apache/incubator-mxnet/projects/13 >
Re: Apache MXNet Scala nightly-build now available on Apache Nexus
This is a great step for the community Tianqi On Mon, Oct 29, 2018 at 10:39 AM YiZhi Liu wrote: > Hi, > > > > I am happy to announce that the availability of the experimental > nightly-build Scala package on Maven in Nexus. The nightly-builds, > currently published as 1.3.1-SNAPSHOT version, include the latest changes > from the master branch from apache/incubator-mxnet GitHub repo and refresh > every day. Furthermore, we completely overhauled the MXNet library shipped > inside and adopted the binary build logic used for Python pip, so now Scala > users can also enjoy the portable MXNet with fully loaded features. > Currently, the supported artifacts are: > > - mxnet-full_2.11-linux-x86_64-cpu (Linux-CPU) > > - mxnet-full_2.11-linux-x86_64-gpu (Linux-GPU, CUDA 9.0) > > - mxnet-full_2.11-osx-x86_64-cpu (OSX-CPU) > > > > FAQ: > > - What's new in the nightly Scala packages compared to the previous Scala > Maven artifacts? > > Most notably, better usability, more supported features, and better > platform compatibility. The new packages now support Ubuntu 14.04 and > Amazon Linux and support the same broad set of features as the existing pip > packages. Besides, users are no longer required to install all the > dependencies to use the nightly Scala packages as the included MXNet > libraries are fully loaded. For the list of supported platforms and > features, see [1]. > > > > - What's different in the build approach compared to what's in the previous > Scala Maven packages? > > The previous Scala Maven packages (1.2, 1.2.1, 1.3) were built with the > same build commands that regular users would use when building from source > [2]. This naive approach limits the compatibility to only similar > environments as where the packages were built (which was Ubuntu 16.04 for > Linux, and OSX 10.13 for OSX). As a result, users cannot use it on more > CentOS or Amazon Linux and may suffer the problem of mismatching of > dependency versions due to the difference in their environment. > > On the other hand, Python users have long been enjoying portable and fully > loaded MXNet packages from pip. The MXNet libraries in pip packages are > built in more controlled environments with compatibility in mind. > Dependencies for extended features are statically linked and masked so that > no additional steps are needed. > > Now, in the new Scala nightly packages, the same approach as pip is used > for building the MXNet library so that the Scala users can benefit from the > same great features. > > > > - How do I get the new nightly packages? > > You can find the packages on Apache (Snapshots) repository [3] including > pom files and jars, which you can download and add to your projects. > Alternatively, you can configure your pom.xml to use the Maven repository. > Remember to add repository config: > > > > > > Apache Snapshot > > https://repository.apache.org/content/groups/snapshots > > > > > > > > - What's the support level of the Scala nightly packages? > > This package is currently experimental, so use at your own risk. While the > first nightly packages are already manually verified on several platforms, > and the automated publish pipeline runs unit tests and integration tests on > OSX and Ubuntu 14.04 before publishing, the testing is currently limited. > > > > - How can I help? > > We call on all MXNet Scala community to test out the new nightly packages. > > > > - I found a problem in using the package. What should I do? > > Please report this in the Github issue in [1]. > > > > - What's next? > > There are several more things to be done: > > 1. We are adding more comprehensive support for GPU versions for different > CUDA versions. > > 2. We are adding more tests on more platforms. We are currently examining > the MXNet's Jenkins CI solution for supporting more platforms. > > 3. As we recognize that the build logic benefits the community, we plan to > open source the build scripts in the mxnet main repo. See progress at [5]. > > > > Qing Lan, Zach Kimberg, Sheng Zha, and I worked together to make this > happen. Thanks to Sheng Zha for sharing the pip build logic, and for > building the automated publishing pipelines. Special thanks to Qing and > Zach for sharing their expertise in Scala with Sheng and guiding and > supporting him locally for this milestone. > > > > [1] https://github.com/apache/incubator-mxnet/issues/8671 > > [2] > > https://github.com/apache/incubator-mxnet/blob/master/scala-package/dev/compile-mxnet-backend.sh#L59-L105 > > [3] > > https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.3.1-SNAPSHOT~~ > > [4] https://maven.apache.org/repository-management.html > > [5] https://github.com/apache/incubator-mxnet/projects/13 >
Re: Apache MXNet Scala nightly-build now available on Apache Nexus
Nice work! It is quite helpful for deployment of MXNet Scala! On Mon, Oct 29, 2018 at 10:39 AM YiZhi Liu wrote: > Hi, > > > > I am happy to announce that the availability of the experimental > nightly-build Scala package on Maven in Nexus. The nightly-builds, > currently published as 1.3.1-SNAPSHOT version, include the latest changes > from the master branch from apache/incubator-mxnet GitHub repo and refresh > every day. Furthermore, we completely overhauled the MXNet library shipped > inside and adopted the binary build logic used for Python pip, so now Scala > users can also enjoy the portable MXNet with fully loaded features. > Currently, the supported artifacts are: > > - mxnet-full_2.11-linux-x86_64-cpu (Linux-CPU) > > - mxnet-full_2.11-linux-x86_64-gpu (Linux-GPU, CUDA 9.0) > > - mxnet-full_2.11-osx-x86_64-cpu (OSX-CPU) > > > > FAQ: > > - What's new in the nightly Scala packages compared to the previous Scala > Maven artifacts? > > Most notably, better usability, more supported features, and better > platform compatibility. The new packages now support Ubuntu 14.04 and > Amazon Linux and support the same broad set of features as the existing pip > packages. Besides, users are no longer required to install all the > dependencies to use the nightly Scala packages as the included MXNet > libraries are fully loaded. For the list of supported platforms and > features, see [1]. > > > > - What's different in the build approach compared to what's in the previous > Scala Maven packages? > > The previous Scala Maven packages (1.2, 1.2.1, 1.3) were built with the > same build commands that regular users would use when building from source > [2]. This naive approach limits the compatibility to only similar > environments as where the packages were built (which was Ubuntu 16.04 for > Linux, and OSX 10.13 for OSX). As a result, users cannot use it on more > CentOS or Amazon Linux and may suffer the problem of mismatching of > dependency versions due to the difference in their environment. > > On the other hand, Python users have long been enjoying portable and fully > loaded MXNet packages from pip. The MXNet libraries in pip packages are > built in more controlled environments with compatibility in mind. > Dependencies for extended features are statically linked and masked so that > no additional steps are needed. > > Now, in the new Scala nightly packages, the same approach as pip is used > for building the MXNet library so that the Scala users can benefit from the > same great features. > > > > - How do I get the new nightly packages? > > You can find the packages on Apache (Snapshots) repository [3] including > pom files and jars, which you can download and add to your projects. > Alternatively, you can configure your pom.xml to use the Maven repository. > Remember to add repository config: > > > > > > Apache Snapshot > > https://repository.apache.org/content/groups/snapshots > > > > > > > > - What's the support level of the Scala nightly packages? > > This package is currently experimental, so use at your own risk. While the > first nightly packages are already manually verified on several platforms, > and the automated publish pipeline runs unit tests and integration tests on > OSX and Ubuntu 14.04 before publishing, the testing is currently limited. > > > > - How can I help? > > We call on all MXNet Scala community to test out the new nightly packages. > > > > - I found a problem in using the package. What should I do? > > Please report this in the Github issue in [1]. > > > > - What's next? > > There are several more things to be done: > > 1. We are adding more comprehensive support for GPU versions for different > CUDA versions. > > 2. We are adding more tests on more platforms. We are currently examining > the MXNet's Jenkins CI solution for supporting more platforms. > > 3. As we recognize that the build logic benefits the community, we plan to > open source the build scripts in the mxnet main repo. See progress at [5]. > > > > Qing Lan, Zach Kimberg, Sheng Zha, and I worked together to make this > happen. Thanks to Sheng Zha for sharing the pip build logic, and for > building the automated publishing pipelines. Special thanks to Qing and > Zach for sharing their expertise in Scala with Sheng and guiding and > supporting him locally for this milestone. > > > > [1] https://github.com/apache/incubator-mxnet/issues/8671 > > [2] > > https://github.com/apache/incubator-mxnet/blob/master/scala-package/dev/compile-mxnet-backend.sh#L59-L105 > > [3] > > https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.3.1-SNAPSHOT~~ > > [4] https://maven.apache.org/repository-management.html > > [5] https://github.com/apache/incubator-mxnet/projects/13 > -- Ziheng Jiang Fudan University | Computer Science
Apache MXNet Scala nightly-build now available on Apache Nexus
Hi, I am happy to announce that the availability of the experimental nightly-build Scala package on Maven in Nexus. The nightly-builds, currently published as 1.3.1-SNAPSHOT version, include the latest changes from the master branch from apache/incubator-mxnet GitHub repo and refresh every day. Furthermore, we completely overhauled the MXNet library shipped inside and adopted the binary build logic used for Python pip, so now Scala users can also enjoy the portable MXNet with fully loaded features. Currently, the supported artifacts are: - mxnet-full_2.11-linux-x86_64-cpu (Linux-CPU) - mxnet-full_2.11-linux-x86_64-gpu (Linux-GPU, CUDA 9.0) - mxnet-full_2.11-osx-x86_64-cpu (OSX-CPU) FAQ: - What's new in the nightly Scala packages compared to the previous Scala Maven artifacts? Most notably, better usability, more supported features, and better platform compatibility. The new packages now support Ubuntu 14.04 and Amazon Linux and support the same broad set of features as the existing pip packages. Besides, users are no longer required to install all the dependencies to use the nightly Scala packages as the included MXNet libraries are fully loaded. For the list of supported platforms and features, see [1]. - What's different in the build approach compared to what's in the previous Scala Maven packages? The previous Scala Maven packages (1.2, 1.2.1, 1.3) were built with the same build commands that regular users would use when building from source [2]. This naive approach limits the compatibility to only similar environments as where the packages were built (which was Ubuntu 16.04 for Linux, and OSX 10.13 for OSX). As a result, users cannot use it on more CentOS or Amazon Linux and may suffer the problem of mismatching of dependency versions due to the difference in their environment. On the other hand, Python users have long been enjoying portable and fully loaded MXNet packages from pip. The MXNet libraries in pip packages are built in more controlled environments with compatibility in mind. Dependencies for extended features are statically linked and masked so that no additional steps are needed. Now, in the new Scala nightly packages, the same approach as pip is used for building the MXNet library so that the Scala users can benefit from the same great features. - How do I get the new nightly packages? You can find the packages on Apache (Snapshots) repository [3] including pom files and jars, which you can download and add to your projects. Alternatively, you can configure your pom.xml to use the Maven repository. Remember to add repository config: Apache Snapshot https://repository.apache.org/content/groups/snapshots - What's the support level of the Scala nightly packages? This package is currently experimental, so use at your own risk. While the first nightly packages are already manually verified on several platforms, and the automated publish pipeline runs unit tests and integration tests on OSX and Ubuntu 14.04 before publishing, the testing is currently limited. - How can I help? We call on all MXNet Scala community to test out the new nightly packages. - I found a problem in using the package. What should I do? Please report this in the Github issue in [1]. - What's next? There are several more things to be done: 1. We are adding more comprehensive support for GPU versions for different CUDA versions. 2. We are adding more tests on more platforms. We are currently examining the MXNet's Jenkins CI solution for supporting more platforms. 3. As we recognize that the build logic benefits the community, we plan to open source the build scripts in the mxnet main repo. See progress at [5]. Qing Lan, Zach Kimberg, Sheng Zha, and I worked together to make this happen. Thanks to Sheng Zha for sharing the pip build logic, and for building the automated publishing pipelines. Special thanks to Qing and Zach for sharing their expertise in Scala with Sheng and guiding and supporting him locally for this milestone. [1] https://github.com/apache/incubator-mxnet/issues/8671 [2] https://github.com/apache/incubator-mxnet/blob/master/scala-package/dev/compile-mxnet-backend.sh#L59-L105 [3] https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.3.1-SNAPSHOT~~ [4] https://maven.apache.org/repository-management.html [5] https://github.com/apache/incubator-mxnet/projects/13