[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159386#comment-15159386
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/1581


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
> Fix For: 1.0.0
>
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159091#comment-15159091
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-187764378
  
@rmetzger Thanks, it's been a nice starter, thanks to both you and @fhueske 
for the help!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159056#comment-15159056
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-187752847
  
+1 to merge the changes.

I rebased the code already locally. Right now, I'm running a final travis 
test. If it passes, I'll merge the change.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157611#comment-15157611
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-187364918
  
Hi, the error is not related to your changes. Its an issue of our build 
system (an inconsistent file has been cached).
From a CI point of view, the changes look good to be merged. I try to 
review your changes tomorrow.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157102#comment-15157102
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-187221864
  
Was this a hiccup in the build or an issue related to my changes? I really 
can't tell from the error message.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152442#comment-15152442
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-185772362
  
Three build configurations passes, one is currently running, the other 
fails and I've spotted this stacktrace:
```
[ERROR] error: error while loading , Error accessing 
/home/travis/.m2/repository/org/scalanlp/breeze_2.11/0.11.2/breeze_2.11-0.11.2.jar
[ERROR] error: scala.reflect.internal.MissingRequirementError: object 
java.lang.Object in compiler mirror not found.
[ERROR] at 
scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:17)
[ERROR] at 
scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:18)
```


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150496#comment-15150496
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-185204685
  
You can rebase your PR's branch to our current master and then force push 
it again. This will update the PR.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149186#comment-15149186
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-184850520
  
Thanks for the tip, I'll try to replicate the S3 setup on our fork as well, 
it'll probably come in handy for similar cases.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148863#comment-15148863
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-184754971
  
To get to the logs, I executed your branch in my travis account. I have a 
S3 bucket configured where we upload the full logs to (travis has a log limit 
of ~5 mb): https://travis-ci.org/rmetzger/flink/builds/109557167

Then, in the log of one build: 
https://s3.amazonaws.com/archive.travis-ci.org/jobs/109557168/log.txt
you'll find the link to the s3 bucket: 
https://s3.amazonaws.com/flink-logs-us/travis-artifacts/rmetzger/flink/1419/1419.1.tar.gz

unpacking that archive, you'll find the output in the `1.log` file. The 
YARN test is executing the WordCount example, and I guess the arguments passed 
are wrong.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148860#comment-15148860
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-184753539
  
@rmetzger I'm looking at the logs as well but I couldn't find that 
stacktrace in particular. I've looked for `ParameterTool.fromArgs` (that is 
present in the stacktrace you extracted) in the raw logs of the three failing 
builds but found nothing...


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148843#comment-15148843
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-184747590
  
Thanks for digging up the tests, I missed that exception in particular, 
I'll fix it right away.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148448#comment-15148448
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-184624521
  
The `testDetachedPerJobYarnCluster()` YARN test is failing with


```
10:33:26,969 INFO  org.apache.flink.client.CliFrontend  
 - Starting execution of program
10:33:26,969 INFO  org.apache.flink.client.program.Client   
 - Starting program in interactive mode
10:33:26,971 ERROR org.apache.flink.client.CliFrontend  
 - Error while running the command.
org.apache.flink.client.program.ProgramInvocationException: The main method 
caused an error.
at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:520)
at 
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:403)
at org.apache.flink.client.program.Client.runDetached(Client.java:277)
at 
org.apache.flink.client.CliFrontend.executeProgramDetached(CliFrontend.java:774)
at org.apache.flink.client.CliFrontend.run(CliFrontend.java:320)
at 
org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1112)
at org.apache.flink.yarn.YarnTestBase$Runner.run(YarnTestBase.java:565)
Caused by: java.lang.RuntimeException: Error parsing arguments 
'[/tmp/junit2827618452146338513/junit4708065236621010716.tmp, 
/tmp/junit2827618452146338513/junit2411517217475739253]' on 
'/tmp/junit2827618452146338513/junit4708065236621010716.tmp'. Unexpected value. 
Please prefix values with -- or -.
at 
org.apache.flink.api.java.utils.ParameterTool.fromArgs(ParameterTool.java:107)
at 
org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:58)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:505)
... 6 more
10:33:26,974 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator
 - Shutting down remote daemon.
```


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148316#comment-15148316
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-184594465
  
Looks like the YARN tests are failing. I'll look into it.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148303#comment-15148303
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-184589232
  
I think this should do it.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144281#comment-15144281
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-183240236
  
Hi @stefanobaghino, that looks good! Maybe print the usage string (`"Usage: 
KMeans --points  --centroids  --output  --iterations "`) 
always at the beginning of the program such that all parameters are shown? 
Otherwise, the iterations parameter will never be shown.
Thanks!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144053#comment-15144053
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-183181049
  
@fhueske If this is what you had in mind I can go on and apply it to the 
rest of the examples.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141409#comment-15141409
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-182520553
  
That's absolutely not a problem @fhueske, your remarks are reasonable and I 
do appreciate the care you all take in making sure Flink turns out great. It's 
my first non-trivial PR on Flink (or any major open source project, for that 
matter) and the guidance you are both providing is extremely precious.
I agree with your remarks and will make sure to provide the required fixes 
ASAP. Feel free to feed any more suggestions asynchronously, it's in the best 
interest for the project to have a solid set of examples for newcomers and I'd 
like to give a significant contribution.
One final question: when we feel that the refactoring is complete, should I 
squash the commits coming from the PR review as well? Thanks in advance.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141302#comment-15141302
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/1581#discussion_r52495987
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -61,42 +62,46 @@
  * Cluster centers are represented by an integer id and a point 
value.
  * For example "1 6.2 3.2\n2 2.9 5.7\n" gives two centers 
(id=1, x=6.2, y=3.2) and (id=2, x=2.9, y=5.7).
  * 
- * 
+ *
  * 
- * Usage: KMeans points path centers path result 
path num iterations
- * If no parameters are provided, the program is run with default data 
from {@link org.apache.flink.examples.java.clustering.util.KMeansData} and 10 
iterations. 
- * 
+ * Usage: KMeans --points path --centroids path 
--output path --iterations n
+ * If no parameters are provided, the program is run with default data 
from {@link org.apache.flink.examples.java.clustering.util.KMeansData} and 10 
iterations.
+ *
  * 
  * This example shows how to use:
  * 
  * Bulk iterations
  * Broadcast variables in bulk iterations
- * Custom Java objects (PoJos)
+ * Custom Java objects (POJOs)
  * 
  */
 @SuppressWarnings("serial")
 public class KMeans {
-   
-   // 
*
-   // PROGRAM
-   // 
*
-   
+
public static void main(String[] args) throws Exception {
-   
-   if(!parseParameters(args)) {
-   return;
+
+   // Checking input parameters
+   final ParameterTool params = ParameterTool.fromArgs(args);
+   if (params.getNumberOfParameters() < 4) {
--- End diff --

We used this pattern to check unnamed parameters. I think we can do better 
now and a shorter message into the `else` clause of the named parameter check. 
For this example print something like "Executing K-Means example with 
default point data set. Use --points to specify file input." in the `else` case 
of the `getPointDataSet()` method.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141305#comment-15141305
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/1581#discussion_r52496167
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -61,42 +62,46 @@
  * Cluster centers are represented by an integer id and a point 
value.
  * For example "1 6.2 3.2\n2 2.9 5.7\n" gives two centers 
(id=1, x=6.2, y=3.2) and (id=2, x=2.9, y=5.7).
  * 
- * 
+ *
  * 
- * Usage: KMeans points path centers path result 
path num iterations
- * If no parameters are provided, the program is run with default data 
from {@link org.apache.flink.examples.java.clustering.util.KMeansData} and 10 
iterations. 
- * 
+ * Usage: KMeans --points path --centroids path 
--output path --iterations n
+ * If no parameters are provided, the program is run with default data 
from {@link org.apache.flink.examples.java.clustering.util.KMeansData} and 10 
iterations.
+ *
  * 
  * This example shows how to use:
  * 
  * Bulk iterations
  * Broadcast variables in bulk iterations
- * Custom Java objects (PoJos)
+ * Custom Java objects (POJOs)
  * 
  */
 @SuppressWarnings("serial")
 public class KMeans {
-   
-   // 
*
-   // PROGRAM
-   // 
*
-   
+
public static void main(String[] args) throws Exception {
-   
-   if(!parseParameters(args)) {
-   return;
+
+   // Checking input parameters
+   final ParameterTool params = ParameterTool.fromArgs(args);
+   if (params.getNumberOfParameters() < 4) {
--- End diff --

Otherwise we will use the file input if a user provides only the --points 
parameter although the printed message says differently.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141564#comment-15141564
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-182555170
  
Thanks a lot for updating the PR @stefanobaghino! 
It's up to you if you want to squash the commits. We can also do it before 
we merge the PR. 


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141319#comment-15141319
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-182505103
  
Hi @stefanobaghino, sorry to bug you again. :-(
I understand if you don't want to refactor this PR again. Please drop a 
note if you update it one last time (*promise*) or not. I can also take over 
and do the final changes before merging it.
Thanks, Fabian


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136221#comment-15136221
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-180995095
  
Ok, thanks! Should I squash the changes coming from the PR feedback now?


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136216#comment-15136216
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-180994099
  
The first test failed because some files on travis were corrupted, the two 
other tests failed probably because the YARN tests became unstable recently.

I think you can consider your changes stable


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Assignee: Stefano Baghino
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15134383#comment-15134383
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-180422247
  
Awesome, thank you!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133923#comment-15133923
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-180279675
  
Hi Stefano, thanks for the update!

- you are right, it is good practice to not squash commits the address PR 
feedback. That make following the changes much easier. These commits will be 
squashed before merging to the master branch.
- I am not sure if we are talking about the same thing. Robert suggested to 
move the code for reading inputs into the main method for examples that have 
only a single input such as TwitterStream or WordCount. This is the point we 
discussed before. Usually, there is only one parameter to check, i.e., whether 
an input path was provided or not. We do not need to move that into a 
`checkParameters` method, IMO.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15134081#comment-15134081
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-180329011
  
Oh, ok, I obviously got it wrong, thanks for the explanation. Fine with me, 
I'll fix it along the other issues of the PR.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132369#comment-15132369
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-179859441
  
I just checked out your pull request locally and looked through it.

I think the following things need to be fixed before we can merge it:
- Squash the 60+ commits in the PR into one
- Fix the failing tests
- Adopt also the "http://localhost:4000/apis/batch/examples.html; page.
- If @fhueske agrees, do not use a `get*` (for example 
`TwitterStream.getTextDataStream`) method if there is only one input. I'm okay 
with using those methods for examples with input >= 2. 


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132358#comment-15132358
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on a diff in the pull request:

https://github.com/apache/flink/pull/1581#discussion_r51875652
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/distcp/DistCp.java
 ---
@@ -62,18 +63,32 @@
public static final String FILES_COPIED_CNT_NAME = "FILES_COPIED";
 
public static void main(String[] args) throws Exception {
-   if (args.length != 3) {
-   printHelp();
+
+   // set up the execution environment
+   final ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();
+
+   ParameterTool params = ParameterTool.fromArgs(args);
+   if (!params.has("input") || !params.has("output")) {
+   System.err.println("Usage: --input  --output 
 [--parallelism ]");
return;
}
 
-   final Path sourcePath = new Path(args[0]);
-   final Path targetPath = new Path(args[1]);
-   int parallelism = Integer.valueOf(args[2], 10);
+   final Path sourcePath = new Path(params.get("input"));
+   final Path targetPath = new Path(params.get("output"));
+   if (!isLocal(env) && !(isOnDistributedFS(sourcePath) && 
isOnDistributedFS(targetPath))) {
+   System.out.println("In a distributed mode only HDFS 
input/output paths are supported");
+   return;
+   }
+
+   final int parallelism = 
Integer.valueOf(params.getInt("parallelism", 10));
--- End diff --

Right, fixing it right away, thanks!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132379#comment-15132379
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-179867266
  
OK, let's do it. 
+1 for @rmetzger's suggestion.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132348#comment-15132348
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on a diff in the pull request:

https://github.com/apache/flink/pull/1581#discussion_r51874764
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/distcp/DistCp.java
 ---
@@ -62,18 +63,32 @@
public static final String FILES_COPIED_CNT_NAME = "FILES_COPIED";
 
public static void main(String[] args) throws Exception {
-   if (args.length != 3) {
-   printHelp();
+
+   // set up the execution environment
+   final ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();
+
+   ParameterTool params = ParameterTool.fromArgs(args);
+   if (!params.has("input") || !params.has("output")) {
+   System.err.println("Usage: --input  --output 
 [--parallelism ]");
return;
}
 
-   final Path sourcePath = new Path(args[0]);
-   final Path targetPath = new Path(args[1]);
-   int parallelism = Integer.valueOf(args[2], 10);
+   final Path sourcePath = new Path(params.get("input"));
+   final Path targetPath = new Path(params.get("output"));
+   if (!isLocal(env) && !(isOnDistributedFS(sourcePath) && 
isOnDistributedFS(targetPath))) {
+   System.out.println("In a distributed mode only HDFS 
input/output paths are supported");
+   return;
+   }
+
+   final int parallelism = 
Integer.valueOf(params.getInt("parallelism", 10));
--- End diff --

The Integer.valueOf() is not needed.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132602#comment-15132602
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1581#issuecomment-179949096
  
Following your suggestions, thanks.
* squash: should I squash the commits coming from the review as well?
* tests: I thought it was about some Travis configuration on our fork, I'll 
have to look into it
* docs: thanks for pointing out the documentation, I missed it!
* get: I'm not sure about it, I think it would be cleaner logically but it 
would make the `params` checking a little bit heavier on the reader — I can 
work on that but I'd wrap the parameter checking in a `checkParameters` method 
that is used for the initial checking and the one in the `get*` methods, would 
that work for you?


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130367#comment-15130367
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

GitHub user stefanobaghino opened a pull request:

https://github.com/apache/flink/pull/1581

[FLINK-2021] Rework examples to use ParameterTool

This should fix [FLINK-2021]. I was unable to run a full build because of 
permission issues but it should be ready to merged, after any fixes that may be 
highlighted by a review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/radicalbit/flink 2021

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1581.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1581


commit f68cb95af1c27dd8bf99e1a2374969915dd567f9
Author: Stefano Baghino 
Date:   2016-01-21T11:20:49Z

[FLINK-2021] Rework the K-Means examples to use built-in argument parsing 
facilities instead of ad-hoc functions

commit a432cc204741dd0a2d505886b987c4113e781c84
Author: Stefano Baghino 
Date:   2016-01-21T11:33:15Z

[FLINK-2021] Renamed user-facing names for options

commit a6d7f38a0a41f5e7e1290afd267eb6b260f3ae06
Author: Stefano Baghino 
Date:   2016-01-21T12:07:39Z

[FLINK-2021] Updated documentation to be in sync with reworked examples

commit 31a6a1e4d1bae900866fc74aff9402b2cd3cee94
Author: Stefano Baghino 
Date:   2016-01-21T13:40:39Z

[FLINK-2021] Removed return statement in Scala, updated usage message to 
reflect the changes

commit 2c4978dce0d4a8a8cd11c2e32319a71b2b57427a
Author: Stefano Baghino 
Date:   2016-01-21T14:01:45Z

[FLINK-2021] Delegating the usage line to the the  object

commit 65b64746a006c01b92b02f4549d744776ed00980
Author: Stefano Baghino 
Date:   2016-01-21T14:05:04Z

[FLINK-2021] Removed unused imports (checkstyle compliance)

commit 47c0d26f3fa7b3d9913114e45e8895b70c80e538
Author: Stefano Baghino 
Date:   2016-01-21T14:47:00Z

[FLINK-2021] Removed the parseParameters and printRunWithDefaultParams 
methods

commit 63bb710babcd1b0fb8d7a5253e442612ef3a86e6
Author: Stefano Baghino 
Date:   2016-01-21T17:08:47Z

[FLINK-2021] Removing static members, ditching RequiredParameters to 
improve readability

commit 857e625d29490112bfd63d0595aa6bcfe4ab8890
Author: Stefano Baghino 
Date:   2016-01-22T09:11:46Z

[FLINK-2021] Reworked the Scala example to keep it up to date with the Java 
one

commit fbd5f8ee0a06238d7f1d7b10be2b12f2c1428599
Author: Stefano Baghino 
Date:   2016-01-26T11:44:13Z

[FLINK-2021] Restored input source reading in separate methods

commit dc2480c8fe97c698a31918c825c63e193fe84302
Author: Stefano Baghino 
Date:   2016-01-26T13:46:09Z

[FLINK-2021] Using POJO types instead of tuple converters

commit 32b51c4e4b9ec43e3ea54bc7ae2573e96a961e58
Author: Stefano Baghino 
Date:   2016-01-29T11:35:44Z

[FLINK-2021] Restored source reading methods in Scala examples, introduced 
case classes

commit 567a3e01d18919ab4f570398de0ef7a0002d1a2d
Author: Stefano Baghino 
Date:   2016-01-29T11:39:54Z

[FLINK-2021] Documentation updated to reflect changes in the example

commit 179ca81b4f9d41a83c130d5c9edd436a1fdf3a03
Author: Stefano Baghino 
Date:   2016-01-29T13:32:19Z

[FLINK-2021] Indentation problem in docs fixed

commit ef04e9aa13bf85f8d0d8018f7f6db4239ba5a01d
Author: Stefano Baghino 
Date:   2016-01-29T13:33:41Z

[FLINK-2021] ScalaDoc comment re-indented to comply with coding style

commit d163105d0b04fa085853d4f577f55a3ffcdc4e3e
Author: Stefano Baghino 
Date:   2016-01-29T14:50:42Z

[FLINK-2021] Tests updated to reflect usage of ParameterTools in KMeans

commit a715cb150eae18881c106610f02f8ea80f553f92
Author: Stefano Baghino 
Date:   2016-01-29T16:44:20Z

[FLINK-2021] Boy Scout rule: fixed typo in method name

commit 2083bc66efb98e33f07707b5f69d6fa69d8db273
Author: Stefano Baghino 
Date:   2016-02-01T13:06:04Z

[FLINK-2021] Reworked Java distcp example to use ParameterTool

commit 844f2dc050c2ac28122b2c306fed637d9dd0e678
Author: Stefano Baghino 
Date:   2016-02-01T13:49:55Z

[FLINK-2021] Parameter validation simplified and brought into main method

commit 92dcf3da9f7e79c5f65b23256f1861cb6299d5d1
Author: 

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130372#comment-15130372
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-179225452
  
Branch recreated to remove merge commits, now opened as #1581.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126192#comment-15126192
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino closed the pull request at:

https://github.com/apache/flink/pull/1536


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126191#comment-15126191
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-177954808
  
Since it looks like I've gathered enough feedback, I'm closing the PR so 
that I can keep working on it and not burden on the CI system. Sorry for the 
WIP PR, I'll reopen it as it's presumably ready for merge.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123359#comment-15123359
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-176700143
  
It works, I'll be pushing the code in a few minutes. If everything looks 
good I can move on to the other examples. Thank you all the precious feedback 
provided so far.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123289#comment-15123289
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-176675459
  
Thanks for the update! 
+1 using case classes in the Scala example. Can you also check if we can 
directly read the CSV file into the case classes (not sure if that works).

Looks good otherwise.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117046#comment-15117046
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174950384
  
Good, I'll keep on working on the PR with the other examples, thank you for 
the feedback and for the guidance provided so far. :smiley: 


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117048#comment-15117048
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174950736
  
I think its important that users also see how to read data from sources.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117070#comment-15117070
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174960226
  
I'll edit the code so that we can all read the result and maybe discuss 
about the pros and cons of this approach. Again, thank you both for the 
feedback.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117052#comment-15117052
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174951427
  
Maybe I can rework the example so that secondary needs (like parameter 
parsing and source reading) are encapsulated in functions kept at the bottom, 
while eventually adding one further example with a spotlights on these 
secondary (but important) tasks. What do you think?


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117069#comment-15117069
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174959098
  
The examples handle inputs differently than most other DataSet programs 
because they use default input data if no input paths are specified. This adds 
complexity which is not present in common programs. 

I agree that reading input is an important aspect of a DataSet program, but 
adding 15 lines for each input to each example seems unnecessary to me.

Extracting code into methods can be good practice, even if a method is only 
called once. Reading code can become a lot easier if it is clear what a method 
does (good method name + comments) because it can hide all little nasty details 
and helps to focus on the relevant parts. 
The only question is how do we define relevance ;-)

@stefanobaghino, I think this is a good idea. I would keep the parameter 
parsing in the main function of the examples, but only move the code to either 
read a CSV file or get a collection DataSource to a method.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117042#comment-15117042
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174946246
  
Okay, since nobody objected on the mailing list, we can start changing all 
examples.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117072#comment-15117072
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174962013
  
Thanks @stefanobaghino!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117050#comment-15117050
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174950907
  
Right, but this is boilerplate code would still be accessible at the bottom 
of the file. It is not gone.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117057#comment-15117057
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174953115
  
I don't really see the points of encapsulating parts of the examples into 
methods: Usually, methods are used when functionality is used in different 
places. In this case, every method is only called once (there is no generic 
abstraction). Its adding an unnecessary level of indirection, making it harder 
to read the code.
I don't see why reading the initial dataset is less important than the rest 
of the application. 

Reading the file is just 5 lines of code (relevant code, I woudn't call 
that boilerplate).


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117047#comment-15117047
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174950409
  
Sorry for the late notice, but why did we move the code to fetch the data 
to the top of the main function? 
IMO, it distracts from the main part of the example code, no?


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117138#comment-15117138
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user fhueske commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-174991021
  
Good point! When the example was added, only Tuples were supported by 
CsvInputFormats. In the meantime, we Pojo support was added for 
CsvInputFormats. IMO, it makes sense to use this feature for this example.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15112288#comment-15112288
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-173876414
  
I'll take some time. I've spotted another possible issue in the docs, I'll 
look into it as we gain more feedback from the mailing list. Thank you!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15112257#comment-15112257
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-173871987
  
It seems like your improvements have had a positive feedback from the ML. 
Shall I proceed replicating those on other examples as well?


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15112281#comment-15112281
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-173874750
  
mh, I understand that you are motivated to move on with this task. I think 
most likely other Flink committers agree to the change, but it might also 
happen that the discussion is not over yet.
Worst case: You'll have to redo some of the changes because the discussion 
goes into a different direction.
If you are confident enough and you are aware of the risks, you can also do 
it ... its up to you :)


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110519#comment-15110519
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user chiwanpark commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50394888
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/scala/org/apache/flink/examples/scala/clustering/KMeans.scala
 ---
@@ -104,36 +105,52 @@ object KMeans {
 
   }
 
-  private def parseParameters(programArguments: Array[String]): Boolean = {
-if (programArguments.length > 0) {
+  private val POINTS_PATH_OPTION: Option =
+new Option("points").alt("P").help("The path to the input points")
+  private val CENTERS_PATH_OPTION: Option =
+new Option("centroids").alt("C").help("The path to the input 
centroids")
+  private val OUTPUT_PATH_OPTION: Option =
+new Option("output").alt("O").help("The path where the output will be 
written")
+  private val NUM_ITERATIONS_OPTION: Option =
+new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm")
+
+  @throws(classOf[RequiredParametersException])
+  private def parseParameters(params: ParameterTool): Boolean = {
+val requiredParameters: RequiredParameters = new RequiredParameters
+var parseStatus: Boolean = false
+requiredParameters.add(POINTS_PATH_OPTION)
+requiredParameters.add(CENTERS_PATH_OPTION)
+requiredParameters.add(OUTPUT_PATH_OPTION)
+requiredParameters.add(NUM_ITERATIONS_OPTION)
+try {
+  requiredParameters.applyTo(params)
+  pointsPath = params.get(POINTS_PATH_OPTION.getName)
+  centersPath = params.get(CENTERS_PATH_OPTION.getName)
+  outputPath = params.get(OUTPUT_PATH_OPTION.getName)
+  numIterations = params.getInt(NUM_ITERATIONS_OPTION.getName)
   fileOutput = true
-  if (programArguments.length == 4) {
-pointsPath = programArguments(0)
-centersPath = programArguments(1)
-outputPath = programArguments(2)
-numIterations = Integer.parseInt(programArguments(3))
-
-true
-  }
-  else {
-System.err.println("Usage: KMeans   
 ")
-
-false
+  parseStatus = true
+}
+catch {
+  case e: RequiredParametersException => {
+if (params.getNumberOfParameters == 0) {
+  printRunWithDefaultParams()
+  parseStatus = true
+}
+else {
+  println(requiredParameters.getHelp(e.getMissingArguments))
+}
   }
 }
-else {
-  System.out.println("Executing K-Means example with default 
parameters and built-in default " +
-"data.")
-  System.out.println("  Provide parameters to read input data from 
files.")
-  System.out.println("  See the documentation for the correct format 
of input files.")
-  System.out.println("  We provide a data generator to create 
synthetic input files for this " +
-"program.")
-  System.out.println("  Usage: KMeans   
 ")
+return parseStatus
--- End diff --

In Scala, we don't need `return` keyword. 


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110520#comment-15110520
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user chiwanpark commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50395017
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -291,31 +297,54 @@ public Centroid map(Tuple3 
value) {
private static String centersPath = null;
private static String outputPath = null;
private static int numIterations = 10;
-   
-   private static boolean parseParameters(String[] programArguments) {
-   
-   if(programArguments.length > 0) {
-   // parse input arguments
+
+   private static final Option POINTS_PATH_OPTION =
+   new Option("points").alt("P").help("The path to the input 
points");
+   private static final Option CENTERS_PATH_OPTION =
+   new Option("centroids").alt("C").help("The path to the input 
centroids");
+   private static final Option OUTPUT_PATH_OPTION =
+   new Option("output").alt("O").help("The path where the output 
will be written");
+   private static final Option NUM_ITERATIONS_OPTION =
+   new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm");
+
+   private static boolean parseParameters(final ParameterTool params) 
throws RequiredParametersException {
--- End diff --

Does this method throw `RequiredParametersException`? The exception seems 
caught in `catch` statement.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110570#comment-15110570
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50399418
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -291,31 +297,54 @@ public Centroid map(Tuple3 
value) {
private static String centersPath = null;
private static String outputPath = null;
private static int numIterations = 10;
-   
-   private static boolean parseParameters(String[] programArguments) {
-   
-   if(programArguments.length > 0) {
-   // parse input arguments
+
+   private static final Option POINTS_PATH_OPTION =
+   new Option("points").alt("P").help("The path to the input 
points");
+   private static final Option CENTERS_PATH_OPTION =
+   new Option("centroids").alt("C").help("The path to the input 
centroids");
+   private static final Option OUTPUT_PATH_OPTION =
+   new Option("output").alt("O").help("The path where the output 
will be written");
+   private static final Option NUM_ITERATIONS_OPTION =
+   new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm");
+
+   private static boolean parseParameters(final ParameterTool params) 
throws RequiredParametersException {
+
+   final RequiredParameters requiredParameters = new 
RequiredParameters();
+   boolean parseStatus = false;
+
+   requiredParameters.add(POINTS_PATH_OPTION);
+   requiredParameters.add(CENTERS_PATH_OPTION);
+   requiredParameters.add(OUTPUT_PATH_OPTION);
+   requiredParameters.add(NUM_ITERATIONS_OPTION);
+
+   try {
+   requiredParameters.applyTo(params);
+   pointsPath = params.get(POINTS_PATH_OPTION.getName());
+   centersPath = params.get(CENTERS_PATH_OPTION.getName());
+   outputPath = params.get(OUTPUT_PATH_OPTION.getName());
+   numIterations = 
params.getInt(NUM_ITERATIONS_OPTION.getName());
fileOutput = true;
-   if(programArguments.length == 4) {
-   pointsPath = programArguments[0];
-   centersPath = programArguments[1];
-   outputPath = programArguments[2];
-   numIterations = 
Integer.parseInt(programArguments[3]);
+   parseStatus = true;
+   } catch (RequiredParametersException e) {
+   if (params.getNumberOfParameters() == 0) {
+   printRunWithDefaultParams();
+   parseStatus = true;
} else {
-   System.err.println("Usage: KMeans  
  ");
-   return false;
+   
System.out.println(requiredParameters.getHelp(e.getMissingArguments()));
}
-   } else {
-   System.out.println("Executing K-Means example with 
default parameters and built-in default data.");
-   System.out.println("  Provide parameters to read input 
data from files.");
-   System.out.println("  See the documentation for the 
correct format of input files.");
-   System.out.println("  We provide a data generator to 
create synthetic input files for this program.");
-   System.out.println("  Usage: KMeans  
  ");
}
-   return true;
+
+   return parseStatus;
}
-   
+
+   private static void printRunWithDefaultParams() {
+   System.out.println("Executing K-Means example with default 
parameters and built-in default data.");
+   System.out.println("  Provide parameters to read input data 
from files.");
+   System.out.println("  See the documentation for the correct 
format of input files.");
+   System.out.println("  We provide a data generator to create 
synthetic input files for this program.");
+   System.out.println("  Usage: KMeans");
--- End diff --

Thanks for making me notice it, I'll fix it right away!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: 

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110521#comment-15110521
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user chiwanpark commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50395062
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/scala/org/apache/flink/examples/scala/clustering/KMeans.scala
 ---
@@ -104,36 +105,52 @@ object KMeans {
 
   }
 
-  private def parseParameters(programArguments: Array[String]): Boolean = {
-if (programArguments.length > 0) {
+  private val POINTS_PATH_OPTION: Option =
+new Option("points").alt("P").help("The path to the input points")
+  private val CENTERS_PATH_OPTION: Option =
+new Option("centroids").alt("C").help("The path to the input 
centroids")
+  private val OUTPUT_PATH_OPTION: Option =
+new Option("output").alt("O").help("The path where the output will be 
written")
+  private val NUM_ITERATIONS_OPTION: Option =
+new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm")
+
+  @throws(classOf[RequiredParametersException])
--- End diff --

This statement doesn't need also. We catch `RequiredParametersException` in 
`catch` statement.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110609#comment-15110609
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50402286
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -291,31 +297,54 @@ public Centroid map(Tuple3 
value) {
private static String centersPath = null;
private static String outputPath = null;
private static int numIterations = 10;
-   
-   private static boolean parseParameters(String[] programArguments) {
-   
-   if(programArguments.length > 0) {
-   // parse input arguments
+
+   private static final Option POINTS_PATH_OPTION =
+   new Option("points").alt("P").help("The path to the input 
points");
+   private static final Option CENTERS_PATH_OPTION =
+   new Option("centroids").alt("C").help("The path to the input 
centroids");
+   private static final Option OUTPUT_PATH_OPTION =
+   new Option("output").alt("O").help("The path where the output 
will be written");
+   private static final Option NUM_ITERATIONS_OPTION =
+   new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm");
+
+   private static boolean parseParameters(final ParameterTool params) 
throws RequiredParametersException {
+
+   final RequiredParameters requiredParameters = new 
RequiredParameters();
+   boolean parseStatus = false;
+
+   requiredParameters.add(POINTS_PATH_OPTION);
+   requiredParameters.add(CENTERS_PATH_OPTION);
+   requiredParameters.add(OUTPUT_PATH_OPTION);
+   requiredParameters.add(NUM_ITERATIONS_OPTION);
+
+   try {
+   requiredParameters.applyTo(params);
+   pointsPath = params.get(POINTS_PATH_OPTION.getName());
+   centersPath = params.get(CENTERS_PATH_OPTION.getName());
+   outputPath = params.get(OUTPUT_PATH_OPTION.getName());
+   numIterations = 
params.getInt(NUM_ITERATIONS_OPTION.getName());
fileOutput = true;
-   if(programArguments.length == 4) {
-   pointsPath = programArguments[0];
-   centersPath = programArguments[1];
-   outputPath = programArguments[2];
-   numIterations = 
Integer.parseInt(programArguments[3]);
+   parseStatus = true;
+   } catch (RequiredParametersException e) {
+   if (params.getNumberOfParameters() == 0) {
+   printRunWithDefaultParams();
+   parseStatus = true;
} else {
-   System.err.println("Usage: KMeans  
  ");
-   return false;
+   
System.out.println(requiredParameters.getHelp(e.getMissingArguments()));
}
-   } else {
-   System.out.println("Executing K-Means example with 
default parameters and built-in default data.");
-   System.out.println("  Provide parameters to read input 
data from files.");
-   System.out.println("  See the documentation for the 
correct format of input files.");
-   System.out.println("  We provide a data generator to 
create synthetic input files for this program.");
-   System.out.println("  Usage: KMeans  
  ");
}
-   return true;
+
+   return parseStatus;
}
-   
+
+   private static void printRunWithDefaultParams() {
+   System.out.println("Executing K-Means example with default 
parameters and built-in default data.");
+   System.out.println("  Provide parameters to read input data 
from files.");
+   System.out.println("  See the documentation for the correct 
format of input files.");
+   System.out.println("  We provide a data generator to create 
synthetic input files for this program.");
+   System.out.println("  Usage: KMeans --points  
--centroids  --output  --iterations ");
--- End diff --

Clever! I'll do it, thanks!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110574#comment-15110574
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50399628
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/scala/org/apache/flink/examples/scala/clustering/KMeans.scala
 ---
@@ -104,36 +105,52 @@ object KMeans {
 
   }
 
-  private def parseParameters(programArguments: Array[String]): Boolean = {
-if (programArguments.length > 0) {
+  private val POINTS_PATH_OPTION: Option =
+new Option("points").alt("P").help("The path to the input points")
+  private val CENTERS_PATH_OPTION: Option =
+new Option("centroids").alt("C").help("The path to the input 
centroids")
+  private val OUTPUT_PATH_OPTION: Option =
+new Option("output").alt("O").help("The path where the output will be 
written")
+  private val NUM_ITERATIONS_OPTION: Option =
+new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm")
+
+  @throws(classOf[RequiredParametersException])
+  private def parseParameters(params: ParameterTool): Boolean = {
+val requiredParameters: RequiredParameters = new RequiredParameters
+var parseStatus: Boolean = false
+requiredParameters.add(POINTS_PATH_OPTION)
+requiredParameters.add(CENTERS_PATH_OPTION)
+requiredParameters.add(OUTPUT_PATH_OPTION)
+requiredParameters.add(NUM_ITERATIONS_OPTION)
+try {
+  requiredParameters.applyTo(params)
+  pointsPath = params.get(POINTS_PATH_OPTION.getName)
+  centersPath = params.get(CENTERS_PATH_OPTION.getName)
+  outputPath = params.get(OUTPUT_PATH_OPTION.getName)
+  numIterations = params.getInt(NUM_ITERATIONS_OPTION.getName)
   fileOutput = true
-  if (programArguments.length == 4) {
-pointsPath = programArguments(0)
-centersPath = programArguments(1)
-outputPath = programArguments(2)
-numIterations = Integer.parseInt(programArguments(3))
-
-true
-  }
-  else {
-System.err.println("Usage: KMeans   
 ")
-
-false
+  parseStatus = true
+}
+catch {
+  case e: RequiredParametersException => {
+if (params.getNumberOfParameters == 0) {
+  printRunWithDefaultParams()
+  parseStatus = true
+}
+else {
+  println(requiredParameters.getHelp(e.getMissingArguments))
+}
   }
 }
-else {
-  System.out.println("Executing K-Means example with default 
parameters and built-in default " +
-"data.")
-  System.out.println("  Provide parameters to read input data from 
files.")
-  System.out.println("  See the documentation for the correct format 
of input files.")
-  System.out.println("  We provide a data generator to create 
synthetic input files for this " +
-"program.")
-  System.out.println("  Usage: KMeans   
 ")
+return parseStatus
--- End diff --

Ok, I kept because I tend to do it on "Java-like" code, but I'll fix this 
as well. Thanks!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110534#comment-15110534
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user chiwanpark commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50395674
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -291,31 +297,54 @@ public Centroid map(Tuple3 
value) {
private static String centersPath = null;
private static String outputPath = null;
private static int numIterations = 10;
-   
-   private static boolean parseParameters(String[] programArguments) {
-   
-   if(programArguments.length > 0) {
-   // parse input arguments
+
+   private static final Option POINTS_PATH_OPTION =
+   new Option("points").alt("P").help("The path to the input 
points");
+   private static final Option CENTERS_PATH_OPTION =
+   new Option("centroids").alt("C").help("The path to the input 
centroids");
+   private static final Option OUTPUT_PATH_OPTION =
+   new Option("output").alt("O").help("The path where the output 
will be written");
+   private static final Option NUM_ITERATIONS_OPTION =
+   new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm");
+
+   private static boolean parseParameters(final ParameterTool params) 
throws RequiredParametersException {
--- End diff --

Oh, `add` method in `RequiredParameters` can throw 
`RequireParametersException`. Ignore my previous comment.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110514#comment-15110514
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user chiwanpark commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50394801
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -291,31 +297,54 @@ public Centroid map(Tuple3 
value) {
private static String centersPath = null;
private static String outputPath = null;
private static int numIterations = 10;
-   
-   private static boolean parseParameters(String[] programArguments) {
-   
-   if(programArguments.length > 0) {
-   // parse input arguments
+
+   private static final Option POINTS_PATH_OPTION =
+   new Option("points").alt("P").help("The path to the input 
points");
+   private static final Option CENTERS_PATH_OPTION =
+   new Option("centroids").alt("C").help("The path to the input 
centroids");
+   private static final Option OUTPUT_PATH_OPTION =
+   new Option("output").alt("O").help("The path where the output 
will be written");
+   private static final Option NUM_ITERATIONS_OPTION =
+   new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm");
+
+   private static boolean parseParameters(final ParameterTool params) 
throws RequiredParametersException {
+
+   final RequiredParameters requiredParameters = new 
RequiredParameters();
+   boolean parseStatus = false;
+
+   requiredParameters.add(POINTS_PATH_OPTION);
+   requiredParameters.add(CENTERS_PATH_OPTION);
+   requiredParameters.add(OUTPUT_PATH_OPTION);
+   requiredParameters.add(NUM_ITERATIONS_OPTION);
+
+   try {
+   requiredParameters.applyTo(params);
+   pointsPath = params.get(POINTS_PATH_OPTION.getName());
+   centersPath = params.get(CENTERS_PATH_OPTION.getName());
+   outputPath = params.get(OUTPUT_PATH_OPTION.getName());
+   numIterations = 
params.getInt(NUM_ITERATIONS_OPTION.getName());
fileOutput = true;
-   if(programArguments.length == 4) {
-   pointsPath = programArguments[0];
-   centersPath = programArguments[1];
-   outputPath = programArguments[2];
-   numIterations = 
Integer.parseInt(programArguments[3]);
+   parseStatus = true;
+   } catch (RequiredParametersException e) {
+   if (params.getNumberOfParameters() == 0) {
+   printRunWithDefaultParams();
+   parseStatus = true;
} else {
-   System.err.println("Usage: KMeans  
  ");
-   return false;
+   
System.out.println(requiredParameters.getHelp(e.getMissingArguments()));
}
-   } else {
-   System.out.println("Executing K-Means example with 
default parameters and built-in default data.");
-   System.out.println("  Provide parameters to read input 
data from files.");
-   System.out.println("  See the documentation for the 
correct format of input files.");
-   System.out.println("  We provide a data generator to 
create synthetic input files for this program.");
-   System.out.println("  Usage: KMeans  
  ");
}
-   return true;
+
+   return parseStatus;
}
-   
+
+   private static void printRunWithDefaultParams() {
+   System.out.println("Executing K-Means example with default 
parameters and built-in default data.");
+   System.out.println("  Provide parameters to read input data 
from files.");
+   System.out.println("  See the documentation for the correct 
format of input files.");
+   System.out.println("  We provide a data generator to create 
synthetic input files for this program.");
+   System.out.println("  Usage: KMeans");
--- End diff --

This parameter description should be updated.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
> 

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110469#comment-15110469
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

GitHub user stefanobaghino opened a pull request:

https://github.com/apache/flink/pull/1536

[FLINK-2021] Rework examples to use ParameterTool (K-Means)

I've reworked the K-Means examples in Java and Scala to get a feeling of 
what it would mean on the whole examples code base. Just a remark: I've caught 
the `RequiredParametersException` on the requirements application but not on 
their definition because I've felt the errors are different in nature (the 
application depends on user input while the definition should be somehow caught 
sooner — it would be great to have a way to catch it at compile-time).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/radicalbit/flink 2021

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1536.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1536


commit f68cb95af1c27dd8bf99e1a2374969915dd567f9
Author: Stefano Baghino 
Date:   2016-01-21T11:20:49Z

[FLINK-2021] Rework the K-Means examples to use built-in argument parsing 
facilities instead of ad-hoc functions

commit a432cc204741dd0a2d505886b987c4113e781c84
Author: Stefano Baghino 
Date:   2016-01-21T11:33:15Z

[FLINK-2021] Renamed user-facing names for options




> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110516#comment-15110516
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user chiwanpark commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50394812
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/scala/org/apache/flink/examples/scala/clustering/KMeans.scala
 ---
@@ -104,36 +105,52 @@ object KMeans {
 
   }
 
-  private def parseParameters(programArguments: Array[String]): Boolean = {
-if (programArguments.length > 0) {
+  private val POINTS_PATH_OPTION: Option =
+new Option("points").alt("P").help("The path to the input points")
+  private val CENTERS_PATH_OPTION: Option =
+new Option("centroids").alt("C").help("The path to the input 
centroids")
+  private val OUTPUT_PATH_OPTION: Option =
+new Option("output").alt("O").help("The path where the output will be 
written")
+  private val NUM_ITERATIONS_OPTION: Option =
+new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm")
+
+  @throws(classOf[RequiredParametersException])
+  private def parseParameters(params: ParameterTool): Boolean = {
+val requiredParameters: RequiredParameters = new RequiredParameters
+var parseStatus: Boolean = false
+requiredParameters.add(POINTS_PATH_OPTION)
+requiredParameters.add(CENTERS_PATH_OPTION)
+requiredParameters.add(OUTPUT_PATH_OPTION)
+requiredParameters.add(NUM_ITERATIONS_OPTION)
+try {
+  requiredParameters.applyTo(params)
+  pointsPath = params.get(POINTS_PATH_OPTION.getName)
+  centersPath = params.get(CENTERS_PATH_OPTION.getName)
+  outputPath = params.get(OUTPUT_PATH_OPTION.getName)
+  numIterations = params.getInt(NUM_ITERATIONS_OPTION.getName)
   fileOutput = true
-  if (programArguments.length == 4) {
-pointsPath = programArguments(0)
-centersPath = programArguments(1)
-outputPath = programArguments(2)
-numIterations = Integer.parseInt(programArguments(3))
-
-true
-  }
-  else {
-System.err.println("Usage: KMeans   
 ")
-
-false
+  parseStatus = true
+}
+catch {
+  case e: RequiredParametersException => {
+if (params.getNumberOfParameters == 0) {
+  printRunWithDefaultParams()
+  parseStatus = true
+}
+else {
+  println(requiredParameters.getHelp(e.getMissingArguments))
+}
   }
 }
-else {
-  System.out.println("Executing K-Means example with default 
parameters and built-in default " +
-"data.")
-  System.out.println("  Provide parameters to read input data from 
files.")
-  System.out.println("  See the documentation for the correct format 
of input files.")
-  System.out.println("  We provide a data generator to create 
synthetic input files for this " +
-"program.")
-  System.out.println("  Usage: KMeans   
 ")
+return parseStatus
+  }
 
-  true
-}
+  private def printRunWithDefaultParams() {
+println("Executing K-Means example with default parameters and 
built-in default data.")
+println("  Provide parameters to read input data from files.")
+println("  See the documentation for the correct format of input 
files.")
+println("  We provide a data generator to create synthetic input files 
for this program.")
+println("  Usage: KMeans
")
--- End diff --

This parameter description should be updated also.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment 

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110577#comment-15110577
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50400122
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/scala/org/apache/flink/examples/scala/clustering/KMeans.scala
 ---
@@ -104,36 +105,52 @@ object KMeans {
 
   }
 
-  private def parseParameters(programArguments: Array[String]): Boolean = {
-if (programArguments.length > 0) {
+  private val POINTS_PATH_OPTION: Option =
+new Option("points").alt("P").help("The path to the input points")
+  private val CENTERS_PATH_OPTION: Option =
+new Option("centroids").alt("C").help("The path to the input 
centroids")
+  private val OUTPUT_PATH_OPTION: Option =
+new Option("output").alt("O").help("The path where the output will be 
written")
+  private val NUM_ITERATIONS_OPTION: Option =
+new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm")
+
+  @throws(classOf[RequiredParametersException])
--- End diff --

Should I ignore this remark, too? You can find a more detailed explanation 
of why I kept the checked exceptions for the `add` calls in the PR description.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110599#comment-15110599
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50401470
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -291,31 +297,54 @@ public Centroid map(Tuple3 
value) {
private static String centersPath = null;
private static String outputPath = null;
private static int numIterations = 10;
-   
-   private static boolean parseParameters(String[] programArguments) {
-   
-   if(programArguments.length > 0) {
-   // parse input arguments
+
+   private static final Option POINTS_PATH_OPTION =
+   new Option("points").alt("P").help("The path to the input 
points");
+   private static final Option CENTERS_PATH_OPTION =
+   new Option("centroids").alt("C").help("The path to the input 
centroids");
+   private static final Option OUTPUT_PATH_OPTION =
+   new Option("output").alt("O").help("The path where the output 
will be written");
+   private static final Option NUM_ITERATIONS_OPTION =
+   new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm");
+
+   private static boolean parseParameters(final ParameterTool params) 
throws RequiredParametersException {
+
+   final RequiredParameters requiredParameters = new 
RequiredParameters();
+   boolean parseStatus = false;
+
+   requiredParameters.add(POINTS_PATH_OPTION);
+   requiredParameters.add(CENTERS_PATH_OPTION);
+   requiredParameters.add(OUTPUT_PATH_OPTION);
+   requiredParameters.add(NUM_ITERATIONS_OPTION);
+
+   try {
+   requiredParameters.applyTo(params);
+   pointsPath = params.get(POINTS_PATH_OPTION.getName());
+   centersPath = params.get(CENTERS_PATH_OPTION.getName());
+   outputPath = params.get(OUTPUT_PATH_OPTION.getName());
+   numIterations = 
params.getInt(NUM_ITERATIONS_OPTION.getName());
fileOutput = true;
-   if(programArguments.length == 4) {
-   pointsPath = programArguments[0];
-   centersPath = programArguments[1];
-   outputPath = programArguments[2];
-   numIterations = 
Integer.parseInt(programArguments[3]);
+   parseStatus = true;
+   } catch (RequiredParametersException e) {
+   if (params.getNumberOfParameters() == 0) {
+   printRunWithDefaultParams();
+   parseStatus = true;
} else {
-   System.err.println("Usage: KMeans  
  ");
-   return false;
+   
System.out.println(requiredParameters.getHelp(e.getMissingArguments()));
}
-   } else {
-   System.out.println("Executing K-Means example with 
default parameters and built-in default data.");
-   System.out.println("  Provide parameters to read input 
data from files.");
-   System.out.println("  See the documentation for the 
correct format of input files.");
-   System.out.println("  We provide a data generator to 
create synthetic input files for this program.");
-   System.out.println("  Usage: KMeans  
  ");
}
-   return true;
+
+   return parseStatus;
}
-   
+
+   private static void printRunWithDefaultParams() {
+   System.out.println("Executing K-Means example with default 
parameters and built-in default data.");
+   System.out.println("  Provide parameters to read input data 
from files.");
+   System.out.println("  See the documentation for the correct 
format of input files.");
+   System.out.println("  We provide a data generator to create 
synthetic input files for this program.");
+   System.out.println("  Usage: KMeans --points  
--centroids  --output  --iterations ");
--- End diff --

I think you can generate the usage information from the required parameters


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> 

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110804#comment-15110804
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user uce commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-173612372
  
Nice to see it reworked. Thanks for the contribution. I think Robert raises 
a good point with the readability improvements though. I would be in favour of 
Robert's take on the ParameterTool. ;)


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110776#comment-15110776
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user rmetzger commented on the pull request:

https://github.com/apache/flink/pull/1536#issuecomment-173607167
  
I have quickly checked out your branch and came up with another variant for 
reworking the KMeans example with the `ParameterTool`: 
https://github.com/rmetzger/flink/blob/4c3b569b522172043fe1f49da2858bae37519eef/flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java

I didn't use the required parameters, because they make the code too clumsy.
I also think that getting rid of all these static variables and methods is 
a great improvement in readability. Users new to Flink should be able to focus 
on understanding the Flink's APIs and abstractions and not be distracted by our 
examples.


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110666#comment-15110666
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user stefanobaghino commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50406845
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/clustering/KMeans.java
 ---
@@ -291,31 +297,54 @@ public Centroid map(Tuple3 
value) {
private static String centersPath = null;
private static String outputPath = null;
private static int numIterations = 10;
-   
-   private static boolean parseParameters(String[] programArguments) {
-   
-   if(programArguments.length > 0) {
-   // parse input arguments
+
+   private static final Option POINTS_PATH_OPTION =
+   new Option("points").alt("P").help("The path to the input 
points");
+   private static final Option CENTERS_PATH_OPTION =
+   new Option("centroids").alt("C").help("The path to the input 
centroids");
+   private static final Option OUTPUT_PATH_OPTION =
+   new Option("output").alt("O").help("The path where the output 
will be written");
+   private static final Option NUM_ITERATIONS_OPTION =
+   new Option("iterations").alt("I").help("The number of iteration 
performed by the K-Means algorithm");
+
+   private static boolean parseParameters(final ParameterTool params) 
throws RequiredParametersException {
--- End diff --

I thought about it but was afraid of polluting the ´main´ method with 
"secondary" concerns. I'm going to fix this right now. I'll keep this in mind 
for other examples as well, thanks!


> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111848#comment-15111848
 ] 

ASF GitHub Bot commented on FLINK-2021:
---

Github user chiwanpark commented on a diff in the pull request:

https://github.com/apache/flink/pull/1536#discussion_r50498429
  
--- Diff: 
flink-examples/flink-examples-batch/src/main/scala/org/apache/flink/examples/scala/clustering/KMeans.scala
 ---
@@ -26,53 +27,84 @@ import 
org.apache.flink.examples.java.clustering.util.KMeansData
 import scala.collection.JavaConverters._
 
 /**
- * This example implements a basic K-Means clustering algorithm.
- *
- * K-Means is an iterative clustering algorithm and works as follows:
- * K-Means is given a set of data points to be clustered and an initial 
set of ''K'' cluster
- * centers.
- * In each iteration, the algorithm computes the distance of each data 
point to each cluster center.
- * Each point is assigned to the cluster center which is closest to it.
- * Subsequently, each cluster center is moved to the center (''mean'') of 
all points that have
- * been assigned to it.
- * The moved cluster centers are fed into the next iteration. 
- * The algorithm terminates after a fixed number of iterations (as in this 
implementation) 
- * or if cluster centers do not (significantly) move in an iteration.
- * This is the Wikipedia entry for the [[http://en.wikipedia
- * .org/wiki/K-means_clustering K-Means Clustering algorithm]].
- *
- * This implementation works on two-dimensional data points.
- * It computes an assignment of data points to cluster centers, i.e., 
- * each data point is annotated with the id of the final cluster (center) 
it belongs to.
- *
- * Input files are plain text files and must be formatted as follows:
- *
- *  - Data points are represented as two double values separated by a 
blank character.
- *Data points are separated by newline characters.
- *For example `"1.2 2.3\n5.3 7.2\n"` gives two data points (x=1.2, 
y=2.3) and (x=5.3,
- *y=7.2).
- *  - Cluster centers are represented by an integer id and a point value.
- *For example `"1 6.2 3.2\n2 2.9 5.7\n"` gives two centers (id=1, 
x=6.2,
- *y=3.2) and (id=2, x=2.9, y=5.7).
- *
- * Usage:
- * {{{
- *   KMeans
- * }}}
- * If no parameters are provided, the program is run with default data from
- * [[org.apache.flink.examples.java.clustering.util.KMeansData]]
- * and 10 iterations.
- *
- * This example shows how to use:
- *
- *  - Bulk iterations
- *  - Broadcast variables in bulk iterations
- *  - Custom Java objects (PoJos)
- */
+  * This example implements a basic K-Means clustering algorithm.
+  *
+  * K-Means is an iterative clustering algorithm and works as follows:
+  * K-Means is given a set of data points to be clustered and an initial 
set of ''K'' cluster
+  * centers.
+  * In each iteration, the algorithm computes the distance of each data 
point to each cluster center.
+  * Each point is assigned to the cluster center which is closest to it.
+  * Subsequently, each cluster center is moved to the center (''mean'') of 
all points that have
+  * been assigned to it.
+  * The moved cluster centers are fed into the next iteration.
+  * The algorithm terminates after a fixed number of iterations (as in 
this implementation)
+  * or if cluster centers do not (significantly) move in an iteration.
+  * This is the Wikipedia entry for the [[http://en.wikipedia
+  * .org/wiki/K-means_clustering K-Means Clustering algorithm]].
+  *
+  * This implementation works on two-dimensional data points.
+  * It computes an assignment of data points to cluster centers, i.e.,
+  * each data point is annotated with the id of the final cluster (center) 
it belongs to.
+  *
+  * Input files are plain text files and must be formatted as follows:
+  *
+  * - Data points are represented as two double values separated by a 
blank character.
+  * Data points are separated by newline characters.
+  * For example `"1.2 2.3\n5.3 7.2\n"` gives two data points (x=1.2, 
y=2.3) and (x=5.3,
+  * y=7.2).
+  * - Cluster centers are represented by an integer id and a point value.
+  * For example `"1 6.2 3.2\n2 2.9 5.7\n"` gives two centers (id=1, x=6.2,
+  * y=3.2) and (id=2, x=2.9, y=5.7).
+  *
+  * Usage:
+  * {{{
+  *   KMeans
+  * }}}
+  * If no parameters are provided, the program is run with default data 
from
+  * [[org.apache.flink.examples.java.clustering.util.KMeansData]]
+  * and 10 iterations.
+  *
+  * This example shows how to use:
+  *
+  * - Bulk iterations
+  * - Broadcast variables in 

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[Robert Metzger (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15108398#comment-15108398
 ] 

Robert Metzger commented on FLINK-2021:
---

Hi Stefano,

cool. I think nobody is currently working on this issue.
Some thoughts on this: 
I think that using the ParameterTool will improve the readability of the 
examples. In the beginning I was a bit hesitant with pushing the ParameterTool 
into all examples because I wasn't sure if users would like it. But I've seen 
many happy users and a wide adoption of it.

We recently added RequiredParameters into Flink as well. They'll also allow you 
to print a help text. Maybe it makes sense to use the RequiredParameters in the 
examples.

How about you introduce the tool into one or two examples, we review the change 
together and then you apply it to all examples?

Also note, that we need to carefully check and rework the examples in the 
documentation to be in sync after the change.

> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLINK-2021) Rework examples to use ParameterTool

[Stefano Baghino (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15108537#comment-15108537
 ] 

Stefano Baghino commented on FLINK-2021:


Sounds fine, thanks for the tips, I'll start working on it.

> Rework examples to use ParameterTool
> 
>
> Key: FLINK-2021
> URL: https://issues.apache.org/jira/browse/FLINK-2021
> Project: Flink
>  Issue Type: Improvement
>  Components: Examples
>Affects Versions: 0.9
>Reporter: Robert Metzger
>Priority: Minor
>  Labels: starter
>
> In FLINK-1525, we introduced the {{ParameterTool}}.
> We should port the examples to use the tool.
> The examples could look like this (we should maybe discuss it first on the 
> mailing lists):
> {code}
> public static void main(String[] args) throws Exception {
> ParameterTool pt = ParameterTool.fromArgs(args);
> boolean fileOutput = pt.getNumberOfParameters() == 2;
> String textPath = null;
> String outputPath = null;
> if(fileOutput) {
> textPath = pt.getRequired("input");
> outputPath = pt.getRequired("output");
> }
> // set up the execution environment
> final ExecutionEnvironment env = 
> ExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().setUserConfig(pt);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)