+1 I think that’s a good idea. It would also help to have some wording regarding how one requests a project be added to that page.
-Taylor > On Apr 2, 2018, at 10:03 AM, Stig Rohde Døssing <stigdoess...@gmail.com> > wrote: > > Sorry about necro'ing this thread, but I think this is relevant again, due > to https://github.com/apache/storm/pull/2613. > > I think it makes more sense to have adapters like this in separate > repositories from Storm, as they can then release fixes when they need to, > without being coupled to the Storm release schedule. For example, > https://github.com/Parsely/streamparse is released this way. Releasing > separately also makes it easier for users to tell whether the adapter is > still being actively developed. > > Maybe we should add an "Ecosystem" section to the web page, similar to what > Kafka has here https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem. > It would help users find projects related to Storm, which is one of the few > advantages I can see to adding the adapter to the Storm repo (visibility). > > 2018-02-02 5:23 GMT+01:00 Jungtaek Lim <kabh...@gmail.com>: > >> CORRECTION: I don't work on multilang for now. "had some works" on >> multilang and introduced small changes on multilang protocol. >> >> 2018년 2월 2일 (금) 오전 9:12, Jungtaek Lim <kabh...@gmail.com>님이 작성: >> >>> Please remove or bcc. for company mail/mailing list which don't allow >>> sending mail outside of MS. My previous mail bounced. I just removed CCC >>> from recipient in current mail. >>> >>> 2018년 2월 2일 (금) 오전 9:08, Jungtaek Lim <kabh...@gmail.com>님이 작성: >>> >>>> Mauro, >>>> >>>> First of all, thanks for proposing contribution of valuable effort on >>>> Apache Storm! Really appreciated. >>>> >>>> I don't know about C#/.net but I have been working on the change of >>>> multilang, so adding my voice here. >>>> >>>> 1. >>>> Major concern from my side for code donation is sustainability. Not sure >>>> but according to my experience, most of Storm PMCs and contributors >> don't >>>> look using Windows at their dev. environment. It doesn't strictly mean >> they >>>> don't have experience with C#, but relatively higher chance. I'm sorry >> to >>>> the Azure team, but I recently found a forgotten major pull request on >>>> storm-eventhubs, which doesn't look like no one could review and >> maintain. >>>> It might become real pain if we receive the code which we can't/don't >>>> maintain, hence I'd rather not vote +1 unless at least two PMCs know C# >>>> well, understand the code completely, and willing to volunteer to >> maintain. >>>> >>>> 2. >>>> As you may know, all of multilang adapters in Storm repo are actually >>>> close to "example" of implementations. They're just implementations of >> the >>>> protocol, and don't provide any language specific optimization as well >> as >>>> language-standard code style. Most of Python users in Storm community >> would >>>> rather use StreamParse, and it is not uncommon to see StreamParse >> question >>>> in user group (whereas they have their own Github repo and issue as >> well). >>>> I would like to see adapter (and more) projects really looking >> attractive >>>> for other languages as well. >>>> >>>> 3. >>>> How it affect releasing Storm? We don't publish them as package in its >>>> language package environment. If NuGet is one of them, we need to add >> the >>>> sequence of release phase for NuGet while releasing Storm, which was not >>>> happened yet, and will become another pain. Moreover, if it's the case, >> I >>>> don't feel needs for having strict coupled between Storm and .net >> adapter >>>> package. For user side it's not a "battery-included" and there's no >>>> difference between maintaining inside/outside. You could freely use >>>> user/dev list to announce new .net adapter release and such >> announcements >>>> are happening from other projects as well. >>>> >>>> 4. >>>> It's completely a thought on my own, but I feel more needs of having >> more >>>> language-native way of supporting language instead of keep improving >>>> multilang. (Not meant to discontinue.) We will introduce streams API in >>>> Storm 2.0.0, a higher-level API like Trident but typed and >> record-to-record >>>> processing. We haven't supported Trident in multilang, but I'd like to >> see >>>> support streams API in non-JVM language, not only defining protocol, but >>>> also have it as first-class support (users should be able to construct >>>> their own topology with only using the language. there's thrift but not >>>> that convenient.). IMHO, according to Spark's "Lesson learned" >> (Databricks >>>> had a poll and published it), I think it's really clear that Python >> should >>>> be first (and only, R might be another good to have). >>>> >>>> Thanks for reading a wall of text. Regardless of whether we could >> include >>>> .net adapter into Storm core or not, thanks again for crafting .net >> adapter >>>> and proposing for donation. >>>> >>>> Jungtaek Lim (HeartSaVioR) >>>> >>>> 2018년 2월 2일 (금) 오전 2:03, Mauro Giusti <mau...@microsoft.com.invalid>님이 >>>> 작성: >>>> >>>>> Hello Storm devs - >>>>> >>>>> We are working on a project that uses Storm and C# / .net core >>>>> components. >>>>> >>>>> As part of that, we would love to contribute to the Storm project with >> a >>>>> multilang protocol sample that uses our C# component/adapter. >>>>> The adapter will be a NuGet package, and we intend to publish the >> source >>>>> code of this component as well. >>>>> >>>>> With that in mind, I have two questions: >>>>> >>>>> * Would you prefer the code for the .net adapter (not the sample: >>>>> that will be on the Storm repo, just the adapter) to be hosted inside >> the >>>>> Storm repo or in a separate GitHub repo? We see pros and cons: we might >>>>> need to introduce .net core compilation in the Storm repo if we host >> the >>>>> adapter there, the code will be a bit more scattered and harder to >> test if >>>>> we host the adapter outside. >>>>> * Can you help us review / answer design questions about our >> adapter >>>>> and the multilang protocol? Is there a specific set of people that is >> more >>>>> knowledgeable about this and/or wants to help in this specific area? >>>>> >>>>> Thank you, >>>>> Mauro Giusti >>>>> Azure Team, Microsoft. >>>>> >>>> >>
