[FLINK-3852] [quickstart] Add skeleton StreamingJob - move Job to BatchJob - comment out transformers for the mainClass setting - tidy up SocketTextStreamWordCount - update docs
This closes #1982 Project: http://git-wip-us.apache.org/repos/asf/flink/repo Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/3080ea48 Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/3080ea48 Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/3080ea48 Branch: refs/heads/master Commit: 3080ea48cf687893b0823f34b1e1716343d72167 Parents: 6c0c0a5 Author: markreddy <[email protected]> Authored: Wed May 11 09:55:19 2016 +0100 Committer: Stephan Ewen <[email protected]> Committed: Tue May 17 20:47:14 2016 +0200 ---------------------------------------------------------------------- docs/quickstart/java_api_quickstart.md | 4 +- docs/quickstart/scala_api_quickstart.md | 4 +- .../main/resources/archetype-resources/pom.xml | 7 +- .../src/main/java/BatchJob.java | 76 +++++++++++++++++++ .../archetype-resources/src/main/java/Job.java | 71 ------------------ .../main/java/SocketTextStreamWordCount.java | 9 ++- .../src/main/java/StreamingJob.java | 73 +++++++++++++++++++ .../main/resources/archetype-resources/pom.xml | 7 +- .../src/main/scala/BatchJob.scala | 77 ++++++++++++++++++++ .../src/main/scala/Job.scala | 71 ------------------ .../main/scala/SocketTextStreamWordCount.scala | 10 +-- .../src/main/scala/StreamingJob.scala | 73 +++++++++++++++++++ 12 files changed, 323 insertions(+), 159 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/docs/quickstart/java_api_quickstart.md ---------------------------------------------------------------------- diff --git a/docs/quickstart/java_api_quickstart.md b/docs/quickstart/java_api_quickstart.md index 8b582bd..69a177b 100644 --- a/docs/quickstart/java_api_quickstart.md +++ b/docs/quickstart/java_api_quickstart.md @@ -63,9 +63,9 @@ Use one of the following commands to __create a project__: There will be a new directory in your working directory. If you've used the _curl_ approach, the directory is called `quickstart`. Otherwise, it has the name of your artifactId. -The sample project is a __Maven project__, which contains two classes. _Job_ is a basic skeleton program and _WordCountJob_ a working example. Please note that the _main_ method of both classes allow you to start Flink in a development/testing mode. +The sample project is a __Maven project__, which contains four classes. _StreamingJob_ and _BatchJob_ are basic skeleton programs, _SocketTextStreamWordCount_ is a working streaming example and _WordCountJob_ is a working batch example. Please note that the _main_ method of all classes allow you to start Flink in a development/testing mode. -We recommend to __import this project into your IDE__ to develop and test it. If you use Eclipse, the [m2e plugin](http://www.eclipse.org/m2e/) allows to [import Maven projects](http://books.sonatype.com/m2eclipse-book/reference/creating-sect-importing-projects.html#fig-creating-import). Some Eclipse bundles include that plugin by default, others require you to install it manually. The IntelliJ IDE also supports Maven projects out of the box. +We recommend you __import this project into your IDE__ to develop and test it. If you use Eclipse, the [m2e plugin](http://www.eclipse.org/m2e/) allows to [import Maven projects](http://books.sonatype.com/m2eclipse-book/reference/creating-sect-importing-projects.html#fig-creating-import). Some Eclipse bundles include that plugin by default, others require you to install it manually. The IntelliJ IDE also supports Maven projects out of the box. A note to Mac OS X users: The default JVM heapsize for Java is too small for Flink. You have to manually increase it. Choose "Run Configurations" -> Arguments and write into the "VM Arguments" box: "-Xmx800m" in Eclipse. http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/docs/quickstart/scala_api_quickstart.md ---------------------------------------------------------------------- diff --git a/docs/quickstart/scala_api_quickstart.md b/docs/quickstart/scala_api_quickstart.md index 423409a..b04922f 100644 --- a/docs/quickstart/scala_api_quickstart.md +++ b/docs/quickstart/scala_api_quickstart.md @@ -154,9 +154,9 @@ $ curl https://flink.apache.org/q/quickstart-scala.sh | bash There will be a new directory in your working directory. If you've used the _curl_ approach, the directory is called `quickstart`. Otherwise, it has the name of your artifactId. -The sample project is a __Maven project__, which contains two classes. _Job_ is a basic skeleton program and _WordCountJob_ a working example. Please note that the _main_ method of both classes allow you to start Flink in a development/testing mode. +The sample project is a __Maven project__, which contains four classes. _StreamingJob_ and _BatchJob_ are basic skeleton programs, _SocketTextStreamWordCount_ is a working streaming example and _WordCountJob_ is a working batch example. Please note that the _main_ method of all classes allow you to start Flink in a development/testing mode. -We recommend to __import this project into your IDE__. For Eclipse, you need the following plugins, which you can install from the provided Eclipse Update Sites: +We recommend you __import this project into your IDE__. For Eclipse, you need the following plugins, which you can install from the provided Eclipse Update Sites: * _Eclipse 4.x_ * [Scala IDE](http://download.scala-ide.org/sdk/e38/scala210/stable/site) http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml index c9fb2ab..5837c41 100644 --- a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml +++ b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml @@ -256,12 +256,15 @@ under the License. </excludes> </filter> </filters> + <!-- If you want to use ./bin/flink run <quickstart jar> uncomment the following lines. + This will add a Main-Class entry to the manifest file --> + <!-- <transformers> - <!-- add Main-Class to manifest file --> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> - <mainClass>${package}.Job</mainClass> + <mainClass>${package}.StreamingJob</mainClass> </transformer> </transformers> + --> <createDependencyReducedPom>false</createDependencyReducedPom> </configuration> </execution> http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/BatchJob.java ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/BatchJob.java b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/BatchJob.java new file mode 100644 index 0000000..325ebae --- /dev/null +++ b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/BatchJob.java @@ -0,0 +1,76 @@ +package ${package}; + +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.flink.api.java.ExecutionEnvironment; + +/** + * Skeleton for a Flink Batch Job. + * + * For a full example of a Flink Batch Job, see the WordCountJob.java file in the + * same package/directory or have a look at the website. + * + * You can also generate a .jar file that you can submit on your Flink + * cluster. + * Just type + * mvn clean package + * in the projects root directory. + * You will find the jar in + * target/flink-quickstart-${version}.jar + * From the CLI you can then run + * ./bin/flink run -c ${package}.BatchJob target/flink-quickstart-${version}.jar + * + * For more information on the CLI see: + * + * http://flink.apache.org/docs/latest/apis/cli.html + */ +public class BatchJob { + + public static void main(String[] args) throws Exception { + // set up the batch execution environment + final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); + + /** + * Here, you can start creating your execution plan for Flink. + * + * Start with getting some data from the environment, like + * env.readTextFile(textPath); + * + * then, transform the resulting DataSet<String> using operations + * like + * .filter() + * .flatMap() + * .join() + * .coGroup() + * + * and many more. + * Have a look at the programming guide for the Java API: + * + * http://flink.apache.org/docs/latest/apis/batch/index.html + * + * and the examples + * + * http://flink.apache.org/docs/latest/apis/batch/examples.html + * + */ + + // execute program + env.execute("Flink Batch Java API Skeleton"); + } +} http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/Job.java ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/Job.java b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/Job.java deleted file mode 100644 index 984dc2f..0000000 --- a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/Job.java +++ /dev/null @@ -1,71 +0,0 @@ -package ${package}; - -/** - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -import org.apache.flink.api.java.ExecutionEnvironment; - -/** - * Skeleton for a Flink Job. - * - * For a full example of a Flink Job, see the WordCountJob.java file in the - * same package/directory or have a look at the website. - * - * You can also generate a .jar file that you can submit on your Flink - * cluster. - * Just type - * mvn clean package - * in the projects root directory. - * You will find the jar in - * target/flink-quickstart-0.1-SNAPSHOT-Sample.jar - * - */ -public class Job { - - public static void main(String[] args) throws Exception { - // set up the execution environment - final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); - - - /** - * Here, you can start creating your execution plan for Flink. - * - * Start with getting some data from the environment, like - * env.readTextFile(textPath); - * - * then, transform the resulting DataSet<String> using operations - * like - * .filter() - * .flatMap() - * .join() - * .coGroup() - * and many more. - * Have a look at the programming guide for the Java API: - * - * http://flink.apache.org/docs/latest/apis/batch/index.html - * - * and the examples - * - * http://flink.apache.org/docs/latest/apis/batch/examples.html - * - */ - - // execute program - env.execute("Flink Java API Skeleton"); - } -} http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/SocketTextStreamWordCount.java ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/SocketTextStreamWordCount.java b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/SocketTextStreamWordCount.java index 6a3383c..abd62bb 100644 --- a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/SocketTextStreamWordCount.java +++ b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/SocketTextStreamWordCount.java @@ -36,15 +36,15 @@ import org.apache.flink.util.Collector; * * <p> * Usage: - * <code>SocketTextStreamWordCount <hostname> <port> <result path></code> + * <code>SocketTextStreamWordCount <hostname> <port></code> * <br> * * <p> * This example shows how to: * <ul> * <li>use StreamExecutionEnvironment.socketTextStream - * <li>write a simple Flink program, - * <li>write and use user-defined functions. + * <li>write a simple Flink program + * <li>write and use user-defined functions * </ul> * * @see <a href="www.openbsd.org/cgi-bin/man.cgi?query=nc">netcat</a> @@ -82,8 +82,9 @@ public class SocketTextStreamWordCount { counts.print(); // execute program - env.execute("WordCount from SocketTextStream Example"); + env.execute("Java WordCount from SocketTextStream Example"); } + // // User Functions // http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/StreamingJob.java ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/StreamingJob.java b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/StreamingJob.java new file mode 100644 index 0000000..0d56fb6 --- /dev/null +++ b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/StreamingJob.java @@ -0,0 +1,73 @@ +package ${package}; + +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; + + +/** + * Skeleton for a Flink Streaming Job. + * + * For a full example of a Flink Streaming Job, see the SocketTextStreamWordCount.java + * file in the same package/directory or have a look at the website. + * + * You can also generate a .jar file that you can submit on your Flink + * cluster. + * Just type + * mvn clean package + * in the projects root directory. + * You will find the jar in + * target/flink-quickstart-${version}.jar + * From the CLI you can then run + * ./bin/flink run -c ${package}.StreamingJob target/flink-quickstart-${version}.jar + * + * For more information on the CLI see: + * + * http://flink.apache.org/docs/latest/apis/cli.html + */ +public class StreamingJob { + + public static void main(String[] args) throws Exception { + // set up the streaming execution environment + final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); + + /** + * Here, you can start creating your execution plan for Flink. + * + * Start with getting some data from the environment, like + * env.readTextFile(textPath); + * + * then, transform the resulting DataStream<String> using operations + * like + * .filter() + * .flatMap() + * .join() + * .coGroup() + * + * and many more. + * Have a look at the programming guide for the Java API: + * + * http://flink.apache.org/docs/latest/apis/streaming/index.html + * + */ + + // execute program + env.execute("Flink Streaming Java API Skeleton"); + } +} http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml index 4752e6f..d64cc12 100644 --- a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml +++ b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml @@ -259,12 +259,15 @@ under the License. </excludes> </filter> </filters> + <!-- If you want to use ./bin/flink run <quickstart jar> uncomment the following lines. + This will add a Main-Class entry to the manifest file --> + <!-- <transformers> - <!-- add Main-Class to manifest file --> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> - <mainClass>${package}.Job</mainClass> + <mainClass>${package}.StreamingJob</mainClass> </transformer> </transformers> + --> <createDependencyReducedPom>false</createDependencyReducedPom> </configuration> </execution> http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/BatchJob.scala ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/BatchJob.scala b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/BatchJob.scala new file mode 100644 index 0000000..d0d4d26 --- /dev/null +++ b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/BatchJob.scala @@ -0,0 +1,77 @@ +package ${package} + +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.flink.api.scala._ + +/** + * Skeleton for a Flink Batch Job. + * + * For a full example of a Flink Batch Job, see the WordCountJob.scala file in the + * same package/directory or have a look at the website. + * + * You can also generate a .jar file that you can submit on your Flink + * cluster. Just type + * {{{ + * mvn clean package + * }}} + * in the projects root directory. You will find the jar in + * target/flink-quickstart-${version}.jar + * From the CLI you can then run + * {{{ + * ./bin/flink run -c ${package}.BatchJob target/flink-quickstart-${version}.jar + * }}} + * + * For more information on the CLI see: + * + * http://flink.apache.org/docs/latest/apis/cli.html + */ +object BatchJob { + def main(args: Array[String]) { + // set up the batch execution environment + val env = ExecutionEnvironment.getExecutionEnvironment + + /** + * Here, you can start creating your execution plan for Flink. + * + * Start with getting some data from the environment, like + * env.readTextFile(textPath); + * + * then, transform the resulting DataSet[String] using operations + * like + * .filter() + * .flatMap() + * .join() + * .group() + * + * and many more. + * Have a look at the programming guide: + * + * http://flink.apache.org/docs/latest/apis/batch/index.html + * + * and the examples + * + * http://flink.apache.org/docs/latest/apis/batch/examples.html + * + */ + + // execute program + env.execute("Flink Batch Scala API Skeleton") + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/Job.scala ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/Job.scala b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/Job.scala deleted file mode 100644 index 45fcbcb..0000000 --- a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/Job.scala +++ /dev/null @@ -1,71 +0,0 @@ -package ${package} - -/** - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -import org.apache.flink.api.scala._ - -/** - * Skeleton for a Flink Job. - * - * For a full example of a Flink Job, see the WordCountJob.scala file in the - * same package/directory or have a look at the website. - * - * You can also generate a .jar file that you can submit on your Flink - * cluster. Just type - * {{{ - * mvn clean package - * }}} - * in the projects root directory. You will find the jar in - * target/flink-quickstart-0.1-SNAPSHOT-Sample.jar - * - */ -object Job { - def main(args: Array[String]) { - // set up the execution environment - val env = ExecutionEnvironment.getExecutionEnvironment - - /** - * Here, you can start creating your execution plan for Flink. - * - * Start with getting some data from the environment, like - * env.readTextFile(textPath); - * - * then, transform the resulting DataSet[String] using operations - * like: - * .filter() - * .flatMap() - * .join() - * .group() - * - * and many more. - * Have a look at the programming guide: - * - * http://flink.apache.org/docs/latest/apis/batch/index.html - * - * and the examples - * - * http://flink.apache.org/docs/latest/apis/batch/examples.html - * - */ - - - // execute program - env.execute("Flink Scala API Skeleton") - } -} http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/SocketTextStreamWordCount.scala ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/SocketTextStreamWordCount.scala b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/SocketTextStreamWordCount.scala index d08974d..a6987ac 100644 --- a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/SocketTextStreamWordCount.scala +++ b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/SocketTextStreamWordCount.scala @@ -33,14 +33,14 @@ import org.apache.flink.streaming.api.scala._ * * Usage: * {{{ - * SocketTextStreamWordCount <hostname> <port> <output path> + * SocketTextStreamWordCount <hostname> <port> * }}} * * This example shows how to: * * - use StreamExecutionEnvironment.socketTextStream - * - write a simple Flink Streaming program in scala. - * - write and use user-defined functions. + * - write a simple Flink Streaming program in scala + * - write and use user-defined functions */ object SocketTextStreamWordCount { @@ -55,7 +55,7 @@ object SocketTextStreamWordCount { val env = StreamExecutionEnvironment.getExecutionEnvironment - //Create streams for names and ages by mapping the inputs to the corresponding objects + // create streams for names and ages by mapping the inputs to the corresponding objects val text = env.socketTextStream(hostName, port) val counts = text.flatMap { _.toLowerCase.split("\\W+") filter { _.nonEmpty } } .map { (_, 1) } @@ -64,6 +64,6 @@ object SocketTextStreamWordCount { counts print - env.execute("Scala SocketTextStreamWordCount Example") + env.execute("Scala WordCount from SocketTextStream Example") } } http://git-wip-us.apache.org/repos/asf/flink/blob/3080ea48/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/StreamingJob.scala ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/StreamingJob.scala b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/StreamingJob.scala new file mode 100644 index 0000000..cb1ad3c --- /dev/null +++ b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/StreamingJob.scala @@ -0,0 +1,73 @@ +package ${package} + +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.flink.api.scala._ + +/** + * Skeleton for a Flink Streaming Job. + * + * For a full example of a Flink Streaming Job, see the SocketTextStreamWordCount.java + * file in the same package/directory or have a look at the website. + * + * You can also generate a .jar file that you can submit on your Flink + * cluster. Just type + * {{{ + * mvn clean package + * }}} + * in the projects root directory. You will find the jar in + * target/flink-quickstart-${version}.jar + * From the CLI you can then run + * {{{ + * ./bin/flink run -c ${package}.StreamingJob target/flink-quickstart-${version}.jar + * }}} + * + * For more information on the CLI see: + * + * http://flink.apache.org/docs/latest/apis/cli.html + */ +object StreamingJob { + def main(args: Array[String]) { + // set up the streaming execution environment + val env = StreamExecutionEnvironment.getExecutionEnvironment + + /** + * Here, you can start creating your execution plan for Flink. + * + * Start with getting some data from the environment, like + * env.readTextFile(textPath); + * + * then, transform the resulting DataStream[String] using operations + * like + * .filter() + * .flatMap() + * .join() + * .group() + * + * and many more. + * Have a look at the programming guide: + * + * http://flink.apache.org/docs/latest/apis/streaming/index.html + * + */ + + // execute program + env.execute("Flink Streaming Scala API Skeleton") + } +} \ No newline at end of file
