Re: [VOTE] [DISCUSSION] Remove support for Java 7

[Rune Fevang]

+1

On Tue, Oct 17, 2017 at 6:52 PM, Henning Rohde  wrote:

> +1
>
> On Tue, Oct 17, 2017 at 9:47 AM, Jean-Baptiste Onofré 
> wrote:
>
>> However, it's good to target this for Beam 3.0.0 as it can have an impact
>> especially for runners.
>>
>> Regards
>> JB
>>
>>
>> On 10/17/2017 06:45 PM, Jean-Baptiste Onofré wrote:
>>
>>> +1 from a general purpose.
>>>
>>> +0 from a runner perspective (as it depends of the execution engine).
>>>
>>> Regards
>>> JB
>>>
>>> On 10/17/2017 06:16 PM, Ismaël Mejía wrote:
>>>
 We have discussed recently in the developer mailing list about the
 idea of removing support for Java 7 on Beam. There are multiple
 reasons for this:

 - Java 7 has not received public updates for almost two years and most
 companies are moving / have already moved to Java 8.
 - A good amount of the systems Beam users rely on have decided to drop
 Java 7 support, e.g. Spark, Flink, Elasticsearch, even Hadoop plans to
 do it on version 3.
 - Most Big data distributions and Cloud managed Spark/Hadoop services
 have already moved to Java 8.
 - Recent versions of core libraries Beam uses are moving to be Java 8
 only (or mostly), e.g. Guava, Google Auto, etc.
 - Java 8 has some nice features that can make Beam code nicer e.g.
 lambdas, streams.

 Considering that Beam is a ‘recent’ project we expect users to be
 already using Java 8. However we wanted first to ask the opinion of
 the Beam users on this subject. It could be the case that some of the
 users are still dealing with some old cluster running on Java 7 or
 have another argument to keep the Java 7 compatibility.

 So, please vote:
 +1 Yes, go ahead and move Beam support to Java 8.
   0 Do whatever you want. I don’t have a preference.
 -1 Please keep Java 7 compatibility (if possible add your argument to
 keep supporting for Java 7).


>>>
>> --
>> Jean-Baptiste Onofré
>> jbono...@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
>

Modify default command line options

[Rune Fevang]

I'd like to modify my pipeline to default to run in Dataflow, but still be
settable to a different runner on the command line. As far as I can tell
there's no way of doing this outside of parsing the command line myself:

- Creating my own extended PipelineOptions class with a different @Default
annotation doesn't work, it fails with a message saying @Default must match
between PipelineOptions classes.
- Calling options.setRunner() doesn't work, as it will override whatever is
on the command line, and as far as I can tell there's no way of doing this
before parsing the command line.
- Calling options.getRunner() and only setting the runner when it's the
default makes it so you can no longer use the DirectRunner, as there's no
way of telling the difference between an unset option and one set to
DirectRunner explicitly.

Am I missing something? And if not is this a use-case it makes sense to
accommodate?

Thanks,
Rune