No need to be disappointed, Scala generated APIs by code generation for ndarray and symbol and if clojure depends on Scala it get those as well.
> On Jun 10, 2018, at 3:40 PM, Chen HY <[email protected]> wrote: > > Yep. An example is the cpp port used following code to generate op.h > automatically. > > https://github.com/apache/incubator-mxnet/blob/master/cpp-package/scripts/OpWrapperGenerator.py > > It makes easier for everyone to follow up with the latest APIs. > It is disappointing if the python version supports the latest API but the > version of your favorite language doesn't. > > 2018-06-10 14:19 GMT+01:00 Naveen Swamy <[email protected]>: > >> this is fantastic, thanks for being a champ. I am glad to have you join us >> :). Yes, we would love to make it work and be available for our users, you >> can make features available iteratively. I understand you alone can't have >> everything in one go, I am hoping more contributors will join the effort. >> >> My guess is Chen is recommending generation of Clojure code from Scala. >> >>> On Sun, Jun 10, 2018 at 2:55 PM, Carin Meier <[email protected]> wrote: >>> >>> Naveen, >>> >>> Thanks. I appreciate your help. I acknowledge there are some challenges >> in >>> trying to bring in a new language package, but I think if we can make it >>> work, it will be the best way for it to grow along with the MXNet project >>> and be a good fit with its goals and ecosystem long term. >>> >>> I added a confluence page of the general state of what is working and >> needs >>> help. >>> https://cwiki.apache.org/confluence/display/MXNET/ >>> Clojure+Package+Contribution+Needs. >>> Feel free to edit. >>> >>> Chen, >>> >>> I added the code generator use on the page as well. I don't fully >>> understand what you are proposing, but I would like to. If you could >>> provide some more details to help me, that would be great. >>> >>> - Carin >>> >>> >>>> On Sun, Jun 10, 2018 at 8:00 AM, Chen HY <[email protected]> wrote: >>>> >>>> I personally think that is the reason the clojure package should use >> more >>>> code generator. >>>> It makes easier to build everything align with scala package. >>>> >>>> 2018-06-10 12:45 GMT+01:00 Naveen Swamy <[email protected]>: >>>> >>>>> I have called for contribution within my organization to help with >> the >>>>> review, some of them have shown interest and hope to see them help >> with >>>> the >>>>> review. Carin, I learnt you are popular to have given great talks on >>>>> Clojure. >>>>> >>>>> Could I please ask you document what functionality is working and >> what >>>>> parts of it need contribution. >>>>> >>>>> In general I want to request a few things: >>>>> Good API documentation, >>>>> Examples, turn examples into integration tests. >>>>> Tests, more tests >>>>> Everything testable on CI. >>>>> >>>>> In the long term, We have to think how to maintain API compatibility >>> and >>>>> tests across language bindings. >>>>> >>>>> Thanks, Naveen >>>>> >>>>> >>>>> On Sat, Jun 9, 2018 at 2:38 AM, YiZhi Liu <[email protected]> >> wrote: >>>>> >>>>>> Carin, >>>>>> >>>>>> Thank you for the great work. I'll do the review. As I have no >>>>>> expertise in Clojure, it will really help to have people from >> Clojure >>>>>> community reviewing it as well :) >>>>>> >>>>>> 2018-06-08 14:22 GMT-07:00 Carin Meier <[email protected]>: >>>>>>> A couple of questions came up in regard to the PR and the current >>>> test >>>>>>> suite state as well as the best way to review the PR since it is >> a >>>> new >>>>>>> language binding. >>>>>>> >>>>>>> In regards to the Clojure test suite, most of the Scala test >> suite >>>> has >>>>>> been >>>>>>> ported over with the goal of having comparable coverage. I can go >>>>> ahead a >>>>>>> put in a coverage tool to make that it a bit more transparent. >>>>>>> >>>>>>> For reviewing, I have a couple people in the Clojure community >> that >>>> are >>>>>>> interested in collaborating in this project and I can ask them >> for >>>> help >>>>>> in >>>>>>> reviewing the PR in some way if that is helpful too. >>>>>>> >>>>>>> I'm also open to other suggestions. >>>>>>> >>>>>>> Thanks, >>>>>>> Carin >>>>>>> >>>>>>> On Fri, Jun 8, 2018 at 4:06 PM, Carin Meier < >> [email protected]> >>>>>> wrote: >>>>>>> >>>>>>>> Here is the PR https://github.com/apache/ >>> incubator-mxnet/pull/11205 >>>>>>>> >>>>>>>> I've ported in the content from the external github repo ( >>>>>>>> https://github.com/gigasquid/clojure-mxnet), that has been >> tested >>>> by >>>>>> some >>>>>>>> of the Clojure community, into the contrib directory. >>>>>>>> >>>>>>>> There is still lots more to do in relation to adding tests, >>>>> benchmarks, >>>>>>>> and increasing stability, but I thought this might be a good >> point >>>> to >>>>>> bring >>>>>>>> it in initially so that the other work can be reviewed in >> smaller >>>>>> chunks. >>>>>>>> I also would like to get other people involved in making it >>> better, >>>>> so I >>>>>>>> thought that having the base package in there would be a good >>>> starting >>>>>>>> point for collaboration. >>>>>>>> >>>>>>>> Feedback welcome. >>>>>>>> >>>>>>>> - Carin >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jun 5, 2018 at 11:40 AM, Carin Meier < >>> [email protected]> >>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks everyone. I'll work on getting together a PR with your >>>>> feedback >>>>>>>>> and post it here. >>>>>>>>> >>>>>>>>> On Tue, Jun 5, 2018 at 4:05 AM, Chen HY <[email protected] >>> >>>>> wrote: >>>>>>>>> >>>>>>>>>> I would suggest using code generators in case upstream library >>>>> adding >>>>>> the >>>>>>>>>> functions for arrays. >>>>>>>>>> It seems that cpp binding is using a code generator and works >>>> fine. >>>>>>>>>> >>>>>>>>>> 2018-06-05 7:59 GMT+01:00 Naveen Swamy <[email protected]>: >>>>>>>>>> >>>>>>>>>>> ~/mxnet/contrib/clojure-package good place for the code. >>>>>>>>>>> >>>>>>>>>>> the package name org.apache.mxnet.contrib.clojure ? do you >>> need >>>>>> mxnet >>>>>>>>>>> again? >>>>>>>>>>> >>>>>>>>>>> I forgot to request to run some benchmarks and document. One >>> of >>>>> the >>>>>>>>>> reasons >>>>>>>>>>> users use MXNet is because of its performance and we want to >>>>> ensure >>>>>>>>>> that we >>>>>>>>>>> maintain it across language bindings. >>>>>>>>>>> >>>>>>>>>>> Also invite your other clojure programmer buddies to the >> party >>>> :) >>>>>>>>>>> >>>>>>>>>>> Thanks, Naveen >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Mon, Jun 4, 2018 at 1:55 PM, Carin Meier < >>>> [email protected] >>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Oh right. That's not a problem, I wonder if something like >>>>>>>>>>>> >>>>>>>>>>>> org.apache.mxnet.contrib/clojure-mxnet >>>>>>>>>>>> >>>>>>>>>>>> would work? >>>>>>>>>>>> >>>>>>>>>>>> If this seems like it is the right direction, we could >> work >>>> out >>>>>> the >>>>>>>>>>> details >>>>>>>>>>>> in a PR. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jun 4, 2018 at 4:44 PM, Naveen Swamy < >>>>> [email protected]> >>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I agree with your assessment that we shouldn't need the >>> user >>>>> to >>>>>>>>>> change >>>>>>>>>>>>> their code. I am not sure if we can release under >>>>>>>>>>>> org.apache.clojure-mxnet >>>>>>>>>>>>> we might have to stick with our primary group id >>>>>> org.apache.mxnet >>>>>>>>>> and >>>>>>>>>>> may >>>>>>>>>>>>> be create a sub-package under it? any creative ideas? >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Jun 4, 2018 at 1:29 PM, Carin Meier < >>>>>> [email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for the feedback everyone. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I agree on the contrib option. I think it's a great >> path >>>>>> forward >>>>>>>>>> and >>>>>>>>>>>>> would >>>>>>>>>>>>>> allow it time for feedback, contribution by others, >> and >>>>>>>>>>> stabilization. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If I'm understanding correctly, that would mean >> putting >>>> the >>>>>>>>>> source >>>>>>>>>>> code >>>>>>>>>>>>> in: >>>>>>>>>>>>>> ~/mxnet/contrib/clojure-package >>>>>>>>>>>>>> >>>>>>>>>>>>>> and having the artifact jar named >>>>>>>>>>>>>> `org.apache.contrib.clojure-mxnet/clojure-mxnet` >>>>>>>>>>>>>> >>>>>>>>>>>>>> I would recommend not having the individual namespaces >>> of >>>>> the >>>>>>>>>> files >>>>>>>>>>>> have >>>>>>>>>>>>>> contrib embedded in them, so that if it graduates, >> users >>>>> won't >>>>>>>>>> have >>>>>>>>>>> to >>>>>>>>>>>>>> change their code, only the dependency. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please let me know if this is correct or if there are >>> any >>>>>> other >>>>>>>>>>> ideas. >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Carin >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Jun 4, 2018 at 4:03 PM, Naveen Swamy < >>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think that's a great idea to bring in under >> contrib >>>> and >>>>> we >>>>>>>>>> can >>>>>>>>>>> also >>>>>>>>>>>>> get >>>>>>>>>>>>>>> user feedback >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Jun 4, 2018, at 12:44 PM, sandeep >> krishnamurthy < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Carin, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This is a commendable work. Thanks a lot for all >> the >>>>> hard >>>>>> and >>>>>>>>>>> smart >>>>>>>>>>>>>> work >>>>>>>>>>>>>>>> you have put behind this :-) I think this will be >> a >>>>> great >>>>>>>>>> value >>>>>>>>>>>>>> addition. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If people are not sure about usage, can I suggest >>> this >>>>>>>>>> awesome >>>>>>>>>>> work >>>>>>>>>>>>> to >>>>>>>>>>>>>> be >>>>>>>>>>>>>>>> brought in "contrib" package? Invite and build the >>>>>> community >>>>>>>>>>> around >>>>>>>>>>>>>>>> Clojure, stabilize and increase the coverage, and >>>> based >>>>> on >>>>>>>>>> usage >>>>>>>>>>>> and >>>>>>>>>>>>>>>> development, graduate it to main stable support >> from >>>>>> contrib. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Suggestions and thoughts? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>> Sandeep >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Jun 4, 2018 at 12:27 PM, Ivan Serdyuk < >>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hello everyone. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> A small comment, about Scala API: main commiters >>> are >>>>>> hardly >>>>>>>>>>>>> available, >>>>>>>>>>>>>>> as >>>>>>>>>>>>>>>>> for today. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> As for Clojure - I might suggest that it might be >>>>>> possible >>>>>>>>>> to >>>>>>>>>>>>> enlight >>>>>>>>>>>>>>>>> future work, for that package, for Clojure >>>> developers. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Mon, Jun 4, 2018 at 8:50 PM, Naveen Swamy < >>>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Carin, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This is great effort and very nicely documented. >>>> When >>>>> I >>>>>>>>>>> surveyed >>>>>>>>>>>> at >>>>>>>>>>>>>> my >>>>>>>>>>>>>>>>> day >>>>>>>>>>>>>>>>>> job on the interest for Scala packages there >> were >>> a >>>>> few >>>>>>>>>> people >>>>>>>>>>>> who >>>>>>>>>>>>>>>>>> specifically were interested in the Clojure >>>> packages, >>>>> I >>>>>>>>>> think >>>>>>>>>>>> those >>>>>>>>>>>>>> are >>>>>>>>>>>>>>>>>> people who might have tried and understand the >>>>>> complexity >>>>>>>>>> of >>>>>>>>>>>> making >>>>>>>>>>>>>>> Scala >>>>>>>>>>>>>>>>>> work in Clojure. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I am in support of a MXNet-Clojure package, I >>>> request >>>>>> you >>>>>>>>>> to >>>>>>>>>>> add >>>>>>>>>>>>> more >>>>>>>>>>>>>>>>> unit >>>>>>>>>>>>>>>>>> tests and integration tests that can be ported >> to >>>> CI. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks for your efforts. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -Naveen >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Fri, Jun 1, 2018 at 6:16 PM, Carin Meier < >>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> It is always a good thing to consider the cost >>> with >>>>> the >>>>>>>>>>> benefit. >>>>>>>>>>>>>> I'll >>>>>>>>>>>>>>>>> do >>>>>>>>>>>>>>>>>> my >>>>>>>>>>>>>>>>>>> best to explain what I see the tradeoffs to be. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> First, I wanted to clarify that it took >>> significant >>>>>>>>>>> development >>>>>>>>>>>>>> effort >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>> get the Clojure package and the interop working >>>>>> properly >>>>>>>>>>> despite >>>>>>>>>>>>> my >>>>>>>>>>>>>>>>>> simple >>>>>>>>>>>>>>>>>>> looking design on the confluence page :) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> One of the advantages of MXNet over its >>> competitors >>>>> is >>>>>> its >>>>>>>>>>> many >>>>>>>>>>>>>>>>> language >>>>>>>>>>>>>>>>>>> support. The Clojure package would only >> increase >>>> the >>>>>>>>>> value of >>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>> proposition and bring new users and growth into >>> the >>>>>>>>>> community. >>>>>>>>>>>>>>>>>>> However, there is a cost associated with adding >>>> this >>>>>>>>>> language >>>>>>>>>>>>>> support >>>>>>>>>>>>>>>>> as >>>>>>>>>>>>>>>>>>> you pointed out. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Since the Clojure package right now is only >>> reliant >>>>> on >>>>>> the >>>>>>>>>>> Scala >>>>>>>>>>>>>> jars >>>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>>>> Maven, it can exist outside the main project as >>> an >>>>>>>>>> independent >>>>>>>>>>>>> repo >>>>>>>>>>>>>>>>> but I >>>>>>>>>>>>>>>>>>> think that would lessen the growth benefit both >>> to >>>>> the >>>>>>>>>> Clojure >>>>>>>>>>>>>>>>> community >>>>>>>>>>>>>>>>>>> and to the MXNet community to not be included >> as >>> a >>>>>> first >>>>>>>>>> class >>>>>>>>>>>>>>>>> language. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I believe having first class Clojure support in >>>> MXNet >>>>>> is >>>>>>>>>>>> valuable, >>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> cost of that support is up to the community to >>>>> decide. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Is there a process for considering a new >> package >>> in >>>>>> MXNet? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> - Carin >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Fri, Jun 1, 2018 at 5:51 PM, Chen HY < >>>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Have checked the issue and the confluence >> page, >>>> but >>>>>> still >>>>>>>>>>>>> curious. >>>>>>>>>>>>>>>>>>>> Clojure and Scala are both JVM based >> languages. >>>>>>>>>>>>>>>>>>>> They, as well as many JVM based languages, can >>>> share >>>>>>>>>> their >>>>>>>>>>>> class >>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>> method >>>>>>>>>>>>>>>>>>>> at a certain level. >>>>>>>>>>>>>>>>>>>> Why should the community maintain two APIs for >>> two >>>>>>>>>> languages >>>>>>>>>>>> with >>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>> share >>>>>>>>>>>>>>>>>>>> their packages with almost zero effort? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 2018-06-01 21:58 GMT+01:00 Carin Meier < >>>>>>>>>> [email protected] >>>>>>>>>>>> : >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I've been working on a Clojure package for >>> MXNet. >>>>>> Since >>>>>>>>>>>> Clojure >>>>>>>>>>>>> is >>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>> JVM >>>>>>>>>>>>>>>>>>>>> language, the package leverages the great >> work >>> of >>>>> the >>>>>>>>>>> existing >>>>>>>>>>>>>>>>> Scala >>>>>>>>>>>>>>>>>>>>> package. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I would appreciate any feedback and testing. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Here is the original issue: >>>>>>>>>>>>>>>>>>>>> https://github.com/apache/ >>>>>> incubator-mxnet/issues/8971 >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Architecture & Design: >>>>>>>>>>>>>>>>>>>>> https://cwiki.apache.org/ >>>> confluence/display/MXNET/ >>>>>>>>>>>> MXNet+Clojure >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> and the github repo for rapid testing and >> issue >>>>>> fixing >>>>>>>>>>> before >>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>> opening >>>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>> official PR https://github.com/gigasquid/ >>>>>> clojure-mxnet >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I'm also active in the slack channel so feel >>> free >>>>> to >>>>>>>>>> ping me >>>>>>>>>>>>>> there. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> Carin Meier >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> Chen Hanyang 陈涵洋 >>>>>>>>>>>>>>>>>>>> Software School Fudan University >>>>>>>>>>>>>>>>>>>> +86-138-1881-7745 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Sandeep Krishnamurthy >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Chen Hanyang 陈涵洋 >>>>>>>>>> Software School Fudan University >>>>>>>>>> +86-138-1881-7745 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Yizhi Liu >>>>>> DMLC member >>>>>> Amazon Web Services >>>>>> Vancouver, Canada >>>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Chen Hanyang 陈涵洋 >>>> Software School Fudan University >>>> +86-138-1881-7745 >>>> >>> >> > > > > -- > Chen Hanyang 陈涵洋 > Software School Fudan University > +86-138-1881-7745
