Hi, Yizhi, You mean users have 2.10 env while we assemble 2.11 in it?
Best, Nan On Wed, Aug 16, 2017 at 9:08 AM, YiZhi Liu <[email protected]> wrote: > Hi Joern, > > The point is that, the front is not a simple wrapper of c_api.h, as > you mentioned, which can be easily achieved by JavaCPP. > > I have noticed the potential conflicts between the assembled scala > library and the one in users' environment. Can we remove the scala > library from the assembly jar? @Nan It wouldn't be a problem since the > scala libraries with same major version are compatible. > > 2017-08-16 23:49 GMT+08:00 Joern Kottmann <[email protected]>: > > Hello, > > > > I personally had quite some issues with Scala dependencies in > > different versions and Spark, where one version is not compatible with > > the other version. Then you need to debug the dependency tree to find > > the places where the versions don't match. Every project which would > > like to use MXnet then has to depend on Scala and might also get > > conflicts if other dependencies depend on different Scala versions. > > Probably something which will cause issues for some of your users. > > Users who want to use Java might not be familiar with Scala dependency > > problems and have a hard time resolving them by getting strange error > > messages. > > > > The JNI layer could be generated with JavaCPP, then we would not need > > to write/maintain the C and the jvm side for that our self. > > A good example of JavaCPP and Scala usage is Apache Mahout [1]. > > > > Even if we don't use JavaCPP, the JNI layer should be easy to get into > > a state where both can share it, the current Scala JNI layers LibInfo > > classes could be converted to Java classes and would in most cases > > require only minor changes in the Scala code. > > > > Jörn > > > > [1] https://github.com/apache/mahout/tree/master/viennacl/src/main > > > > On Wed, Aug 16, 2017 at 5:30 PM, Nan Zhu <[email protected]> wrote: > >> I agree with Yizhi > >> > >> My major concern is the duplicate implementations, which are usually > one of > >> the major sources of bugs, especially with two languages which are > >> naturally interactive (OK, Calling Scala from Java might need some more > >> efforts). It is just like we provide C++ & C APIs of MxNet in two > separated > >> packages. > >> > >> About dependency problem, when you say "As far as I see this has the > great > >> disadvantage that the Java API would force Scala as a dependency onto > the > >> java users.", would you please give a concrete example causing critical > >> issues? > >> > >> Best, > >> > >> Nan > >> > >> > >> > >> On Wed, Aug 16, 2017 at 8:19 AM, YiZhi Liu <[email protected]> wrote: > >> > >>> Hi, > >>> > >>> If we build the Java API from the very beginning, i.e. the JNI part, > >>> we have to rewrite the codes for training, predict, inferShape, etc. > >>> It would be too heavy to maintain a totally new front language. > >>> > >>> As far as I see, I don't think Scala library dependency would be a big > >>> problem in most cases, unless we are going to use it in embedded > >>> devices. Could you illustrate some use-cases where you cannot involve > >>> Scala dependencies? > >>> > >>> 2017-08-16 22:13 GMT+08:00 Joern Kottmann <[email protected]>: > >>> > Hello, > >>> > > >>> > the approach which is taken by Spark is described here [1]. > >>> > > >>> > As far as I see this has the great disadvantage that the Java API > >>> > would force Scala as a dependency onto the java users. > >>> > For a library it is always a great advantage if it doesn't have many > >>> > dependencies, or zero dependencies. In our case it could be quite > >>> > realistic to have a thin wrapper around the C API without needing any > >>> > other dependencies (or only dependencies which can't be avoided). > >>> > > >>> > The JNI layer could easily be shared between the Java and Scala API. > >>> > As far as I understand is the JNI layer in the Scala API anyway > >>> > private and a change to it wouldn't require that the public part of > >>> > the Scala API is changed. > >>> > > >>> > What do you think? > >>> > > >>> > Jörn > >>> > > >>> > [1] https://cwiki.apache.org/confluence/display/SPARK/Java+ > API+Internals > >>> > > >>> > On Wed, Aug 16, 2017 at 3:39 PM, YiZhi Liu <[email protected]> > wrote: > >>> >> Hi Joern, > >>> >> > >>> >> I suggest to build Java API as a wrapper of Scala API, re-use most > of > >>> >> the procedures. Referring to the Java API in Apache Spark. > >>> >> > >>> >> 2017-08-16 18:21 GMT+08:00 Joern Kottmann <[email protected]>: > >>> >>> Hello all, > >>> >>> > >>> >>> I would like to propose the addition of a Java API to MXNet. > >>> >>> > >>> >>> There has been some previous work done for the Scala API, and it > makes > >>> >>> sense to at least share the JNI layer between the two. > >>> >>> > >>> >>> The Java API probably should be aligned with the Python API (and > >>> >>> others which exist already) with a few changes to give it a native > >>> >>> Java feel. > >>> >>> > >>> >>> As far as I understand there are multiple people interested to > work on > >>> >>> this and it would be good to maybe come up with a written proposal > on > >>> >>> how things should be. > >>> >>> > >>> >>> My motivation is to get a Java API which can be used by Apache > OpenNLP > >>> >>> to solve various NLP tasks using Deep Learning based approaches > and I > >>> >>> am also interested to work on MXNet. > >>> >>> > >>> >>> Jörn > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Yizhi Liu > >>> >> DMLC member > >>> >> Technical Manager > >>> >> Qihoo 360 Inc, Shanghai, China > >>> > >>> > >>> > >>> -- > >>> Yizhi Liu > >>> DMLC member > >>> Technical Manager > >>> Qihoo 360 Inc, Shanghai, China > >>> > > > > -- > Yizhi Liu > DMLC member > Technical Manager > Qihoo 360 Inc, Shanghai, China >
