Lambdas can also lead to clean code reuse. It is part of evolving to adopt new
JDK functionalities, and I think that we should do it.
It is expected that contributors starting with a new technology/paradigm may
not always implement code that follows perfect practices. However, we should
look at that as an opportunity to have more experienced community members give
constructive feedback through code reviews, and sharing good resources where
one can master such best practices.
I don’t think that a few less than ideal implementations should lead us not to
use this powerful language feature.
Hugo
> On Jan 3, 2017, at 10:19 AM, S G <sg.online.em...@gmail.com> wrote:
>
> Yes, I agree.
>
> +1s on both these lines:
>
> *Badly designed APIs can get more complicated when combined with lambdas*
> *Lambdas and functional style can greatly improve the readability if used
> properly*.
>
>
>
> On Mon, Jan 2, 2017 at 9:34 PM, Arun Mahadevan <ar...@apache.org> wrote:
>
>> Its more about getting used to and judiciously deciding when to use
>> functional v/s procedural approach. Badly designed APIs can get more
>> complicated when combined with lambdas, but overall what I observe is
>> lambdas and functional style can greatly improve the readability (with many
>> other benefits) if used properly.
>>
>> A simple word count pipeline like below can get too verbose when expressed
>> without lambdas.
>>
>> Stream<String> stream = …
>> stream
>> .flatMap(s -> Arrays.asList(s.split(" ")))
>> .mapToPair(w -> Pair.of(w, 1))
>> .countByKey()
>> .filter(x -> x.getSecond() >= 5)
>> .print();
>>
>> *Without lambda*
>>
>> stream
>> .flatMap(new FlatMapFunction<String, String>() {
>> @Override
>> public Iterable<String> apply(String s) {
>> return Arrays.asList(s.split(" "));
>> }
>> })
>> .mapToPair(new PairFunction<String, String, Integer>() {
>> @Override
>> public Pair<String, Integer> apply(String w) {
>> return Pair.of(w, 1);
>> }
>> })
>> .countByKey()
>> .filter(new Predicate<Pair<String, Long>>() {
>> @Override
>> public boolean test(Pair<String, Long> x) {
>> return x.getSecond() >= 5;
>> }
>> })
>> .print();
>>
>>
>> Thanks,
>> Arun
>>
>> On 1/3/17, 10:21 AM, "S G" <sg.online.em...@gmail.com> wrote:
>>
>>> One bad usage I have found is missing types for lambda arguments.
>>> When the types for arguments are not provided in the code itself, github
>> or
>>> vi-editor browsing of such code becomes impossible.
>>>
>>> You must have an IDE like Eclipse to infer the types for such cases and
>>> that becomes a big limitation to browse such code. I have not looked into
>>> the storm code-base to find such occurrences but the above is just a
>>> general observation from the lambda code usages.
>>>
>>>
>>> Secondly, lambdas tend to generate bad stack-traces on exceptions.
>>>
>>>
>>> And lastly, functional programming is not natural to many Java developers.
>>> https://zeroturnaround.com/rebellabs/how-to-avoid-
>> ruining-your-world-with-lambdas-in-java-8/
>>> hints at some of these problems. Java code is usually very easy to
>>> understand. But once you throw in more than a single level of lambdas,
>>> things tend to become more and more confusing for several developers.
>> (This
>>> last point reflects the thoughts from some of my fellow developers, not
>>> representative of all the devs)
>>>
>>>
>>> Thanks
>>> SG
>>>
>>>
>>> On Mon, Jan 2, 2017 at 8:26 PM, Jungtaek Lim <kabh...@gmail.com> wrote:
>>>
>>>> Hi S G,
>>>>
>>>> I'd be happy if you could elaborate your opinion. Did you found bad
>> usages
>>>> from master branch of Storm code?
>>>>
>>>> Regarding comfortable of lambda, IMHO, I don't think many users are
>>>> unfamiliar with lambda, since they should have been used it with various
>>>> languages. We might not be comfortable with Java 8 lambda (since
>> transition
>>>> to Java 8 is going slowly), but it's just a matter of familiarizing.
>>>>
>>>> Are there kind of best practices for Java 8 lambda? We can refer these
>> to
>>>> construct some guides / restrictions for Storm project.
>>>>
>>>> Thanks,
>>>> Jungtaek Lim (HeartSaVioR)
>>>>
>>>> 2017년 1월 3일 (화) 오후 12:26, S G <sg.online.em...@gmail.com>님이 작성:
>>>>
>>>>> I have found several bad usages of Java 8 lambdas and many developers
>> are
>>>>> not comfortable using them.
>>>>>
>>>>> So we should use them only if it really makes the code beautiful and
>>>> easier
>>>>> to understand.
>>>>>
>>>>> My 2c,
>>>>> Thanks
>>>>> SG
>>>>>
>>>>>
>>>>> On Mon, Dec 26, 2016 at 5:59 AM, Arun Mahadevan <ar...@apache.org>
>>>> wrote:
>>>>>
>>>>>> The streams API implementation has limited usage of 1.8 features and
>>>> can
>>>>>> be easily ported to 1.7 if required. The examples are written in
>> 1.8,
>>>> the
>>>>>> thought being users would stick to the Java 8 style usage (lambdas)
>>>> from
>>>>>> the beginning. If there is consensus we could also consider moving
>> the
>>>>> 1.x
>>>>>> branch to JDK 8.
>>>>>>
>>>>>> Anyways would like interested folks to start reviewing the changes
>> so
>>>>> that
>>>>>> we can take it forward.
>>>>>>
>>>>>> Thanks,
>>>>>> Arun
>>>>>>
>>>>>>
>>>>>> On 12/23/16, 10:09 AM, "Jungtaek Lim" <kabh...@gmail.com> wrote:
>>>>>>
>>>>>>> FYI, I've realized that internal of Stream API (pull request)
>> relies
>>>> on
>>>>>> JDK
>>>>>>> 8 (what I've found is 'static method in interface' and maybe more)
>> so
>>>>> for
>>>>>>> now Stream API is expected to be included for at least Storm 2.0.0
>>>>> unless
>>>>>>> the PR is modified to fit to JDK 7.
>>>>>>>
>>>>>>> - Jungtaek Lim (HeartSaVioR)
>>>>>>>
>>>>>>> 2016년 12월 21일 (수) 오전 9:40, Jungtaek Lim <kabh...@gmail.com>님이 작성:
>>>>>>>
>>>>>>>> Thanks Manu and Taylor for giving your opinions.
>>>>>>>>
>>>>>>>> - Storm SQL improvement
>>>>>>>>
>>>>>>>> There're some huge PRs available but there're all about
>> improvement
>>>>>> which
>>>>>>>> shouldn't be blocker for releasing 1.1.0. (I'd like to also
>> include
>>>>>> them to
>>>>>>>> 1.1.0 but not sure it can be happen really soon.)
>>>>>>>> I'll send a request for reviewing about pending Storm SQL PRs.
>>>>>>>>
>>>>>>>> Only one issue (STORM-2200) is linked to release 1.1.0 epic
>> which is
>>>>>>>> blocker for me.
>>>>>>>>
>>>>>>>> - Java port
>>>>>>>>
>>>>>>>> I also had some developers saying 'If core of Storm were written
>> by
>>>>>> Java,
>>>>>>>> I could experiment and even contribute on something'. I was one
>> of
>>>>> them,
>>>>>>>> and to be honest, I'm still a beginner of Clojure. Moving to
>> Java 8
>>>>> also
>>>>>>>> gives great functionalities for us, so Java port is what I think
>> the
>>>>>> most
>>>>>>>> important thing among the huge works now in progress. Ideally,
>> and
>>>>>>>> hopefully, I'd like to see us focus on this and make this happen
>> at
>>>>> the
>>>>>>>> very early next year.
>>>>>>>> (Yes we should do some manual tests and maybe some refactoring
>> too.)
>>>>>>>>
>>>>>>>> - Metrics V2
>>>>>>>>
>>>>>>>> I'm not sure when we plan to release Storm 1.2.0, but given that
>>>>>> there're
>>>>>>>> only two things left (logviewer / ui) for completing port work
>>>> (except
>>>>>>>> tests) I guess Storm 2.0.0 might be happen earlier.
>>>>>>>> Taylor, when do you expect metrics V2 will be available for
>>>> reviewing?
>>>>>>>>
>>>>>>>> - Stream API
>>>>>>>>
>>>>>>>> With labeling as experiment or annotating with evolving, we could
>>>>>> include
>>>>>>>> the first version to next minor excluding 1.1.0. (We could even
>>>>> include
>>>>>>>> this to 1.1.0 if we start reviewing this very soon.)
>>>>>>>>
>>>>>>>> I'd like to hear others' opinions as well.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>
>>>>>>>> 2016년 12월 21일 (수) 오전 7:33, P. Taylor Goetz <ptgo...@gmail.com>님이
>>>> 작성:
>>>>>>>>
>>>>>>>> Hi Jungtaek,
>>>>>>>>
>>>>>>>>> - Beam runner
>>>>>>>>
>>>>>>>> There’s not been much activity around this, and I haven’t had
>> much
>>>>> time
>>>>>> to
>>>>>>>> work on it recently, but there’s a decent foundation to build
>> upon.
>>>> So
>>>>>> it
>>>>>>>> would be fairly easy for others to start contributing to that
>>>> effort.
>>>>>>>> There’s also interest from the Beam community in that runner, so
>> one
>>>>>>>> possibility is to move that effort to the Apache Beam project.
>>>>>>>>
>>>>>>>> This is very preliminary work, so I don’t have a good handle on
>> what
>>>>> the
>>>>>>>> target release would be.
>>>>>>>>
>>>>>>>>> - Metrics renewal
>>>>>>>>
>>>>>>>>
>>>>>>>> This is what I’ve been referring to as “metrics_v2”. This is
>>>>> progressing
>>>>>>>> fairly well with support for multiple reporters (e.g. Graphite,
>>>>> Ganglia,
>>>>>>>> console, etc.), worker metrics, disruptor metrics, etc.
>>>>>>>>
>>>>>>>> I would like to target this work for 1.2.0.
>>>>>>>>
>>>>>>>>> - Java port
>>>>>>>>
>>>>>>>> This effort seems to have picked up (for example Bobby’s
>> conversion
>>>> of
>>>>>>>> Nimbus, etc.) and is progressing steadily. It’s taken a lot
>> longer
>>>>> than
>>>>>>>> initially thought, but a lot of that can be attributed to the ebb
>>>> and
>>>>>> flow
>>>>>>>> of people’s availability to do the work.
>>>>>>>>
>>>>>>>>> - Storm SQL improvement (Streaming SQL in future)
>>>>>>>>
>>>>>>>> You’ve been spearheading most of the work here, so I’d delegate
>> to
>>>> you
>>>>>> for
>>>>>>>> your opinion on where it stands. If you need additional reviews,
>>>> just
>>>>>> ask
>>>>>>>> on list or via GitHub (e.g. “[REVIEW REQUEST]” in the subject
>> line
>>>>> might
>>>>>>>> help get attention).
>>>>>>>>
>>>>>>>> My thinking has been that this could be included in the 1.1.0
>>>> release.
>>>>>> Is
>>>>>>>> there a set of JIRA issues you would like to include in order to
>>>> make
>>>>>> that
>>>>>>>> happen?
>>>>>>>>
>>>>>>>>> - Stream API
>>>>>>>>
>>>>>>>> This seems to have stalled a bit, though there seems to be a lot
>> of
>>>>>>>> interest around it. I think we all would agree that when
>>>> introducing a
>>>>>> new
>>>>>>>> API for building topologies, it’s important that we get right
>> from
>>>> the
>>>>>>>> start and have strong buy-in from the development community. I
>> would
>>>>>>>> encourage anyone interested in the Streams API to review the
>>>> proposal
>>>>>> and
>>>>>>>> initial code.
>>>>>>>>
>>>>>>>> I think it is close, but I’m not sure what release to target.
>>>> Possibly
>>>>>> the
>>>>>>>> 2.0 release?
>>>>>>>>
>>>>>>>> Re: 1.1.0 Release
>>>>>>>>
>>>>>>>> STORM-2176 is a fairly big concern of mine since the feature it
>>>>> involves
>>>>>>>> was introduced in 1.0.0 and did not work then nor in any
>> subsequent
>>>> or
>>>>>>>> future releases (may not be a problem in 2.0). Unfortunately, as
>>>>> you’ve
>>>>>>>> seen, finding the root cause is elusive. That issue could
>> definitely
>>>>> use
>>>>>>>> more eyes.
>>>>>>>>
>>>>>>>> -Taylor
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Dec 20, 2016, at 2:19 AM, Jungtaek Lim <kabh...@gmail.com>
>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi devs,
>>>>>>>>>
>>>>>>>>> I'm seeing lots of huge works in parallel, and we individual
>> are
>>>>> busy
>>>>>>>>> regarding each work so common works (review, release,
>>>> documentation,
>>>>>>>> etc.)
>>>>>>>>> have been not made in progress for several months.
>>>>>>>>>
>>>>>>>>> - Beam runner
>>>>>>>>> - Metrics renewal
>>>>>>>>> - Java port
>>>>>>>>> - Storm SQL improvement (Streaming SQL in future)
>>>>>>>>> - Stream API
>>>>>>>>>
>>>>>>>>> IMHO, it would be better to set target versions for them, and
>> set
>>>> a
>>>>>>>> roadmap
>>>>>>>>> (per version), and prioritize based on roadmap.
>>>>>>>>>
>>>>>>>>> Stream API (very first version), and Storm SQL improvement are
>>>>> waiting
>>>>>>>> for
>>>>>>>>> review, and personally I would like to publish them soon.
>>>>>>>>>
>>>>>>>>> If we're OK to have 2.0.0 without adding much features, I'm in
>>>> favor
>>>>>> of
>>>>>>>>> concentrating Java port work (postponing other things except
>>>>> releasing
>>>>>>>> 1.x
>>>>>>>>> version line) and moving to Apache Storm 2.0.0 really soon.
>>>>>>>>> (I'm even OK we decide to postpone some clojure files to be
>>>>> addressed
>>>>>>>> after
>>>>>>>>> 2.0.0.)
>>>>>>>>> Actually we're suffering other annoying issue: JDK 7 (1.x) vs 8
>>>>> (2.x)
>>>>>>>> which
>>>>>>>>> is other reason to move to 2.x quickly.
>>>>>>>>>
>>>>>>>>> I'd be really happy if we have metrics renewal and beam runner,
>>>> but
>>>>>> I'm
>>>>>>>> not
>>>>>>>>> sure when they're available to be published. Do we have any
>>>> updates
>>>>>> here?
>>>>>>>>>
>>>>>>>>> What do you think? It might be ideal, and/or broader discussion
>>>> but
>>>>> we
>>>>>>>>> haven't discussed our plan / vision for a long time so better
>> to
>>>>> give
>>>>>> it
>>>>>>>> a
>>>>>>>>> try.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>
>>
