RE: Is Flink easy to deploy ?

[Esa Heikkinen]

I’d want to write simple Scala code that: 1) reads data of csv-file (time 
series data, where one column is timestamp) 2) converts data of csv-file 
compatible for CEP 3) sets pattern for CEP 4) Runs CEP  5) writes results. (I 
would very much like to find a complete example of this.)

What artifacts and “imports” do I need ?

Whether there is a list of all artifacts ?

And what “imports” is included in one artifacts ?

And what classes and “functions” is included in one “import” ?

Is it problem to include (almost) all possible artifacts and “imports” to 
project and code ?

By the way, is it right way to edit directly XML-code in pom.xml to change 
artifacts ? pom.xml looks like a bit of cryptic and I am not sure I edit  right 
places in pom.xml because ’s ja ’s are in many places.

Best,
Esa

From: Niclas Hedhman [mailto:nic...@apache.org]
Sent: Saturday, February 24, 2018 4:25 AM
To: user@flink.apache.org
Cc: Esa Heikkinen 
Subject: Re: Is Flink easy to deploy ?

I think you simply missing a bunch of the Flink artifacts.

Flink is broken into dozens of pieces, and you need to select from a large set 
of artifacts what you need to depend on.

Typically, there is one Flink artifact per "extension".
I ended up with
   flink-core
   flink-core
   flink-connector-cassandra_2.11
   flink-connector-kafka_2.11
   flink-queryable-state-runtime_2.11
   flink-streaming-java_2.11
   flink-streaming-scala_2.11

With transitive dependencies enabled, meaning whatever Flink depends on is also 
included

Cheers

On Sat, Feb 24, 2018 at 3:22 AM, Esa Heikkinen 
mailto:heikk...@student.tut.fi>> wrote:



Yes i have looked. For example, if i want to compile and run 
StreamTableExample.scala from:

https://github.com/apache/flink/blob/master/flink-examples/flink-examples-table/src/main/scala/org/apache/flink/table/examples/scala/StreamTableExample.scala

I have taken all examples (and also latest Flink at same time) to my Linux from 
git.

Where directory should i be in for compiling and running StreamTableExample in 
command line ? flink-examples-table ?

What is the command for compiling ? mvn clean install -Pbuild-jar ?

What is the command running the StreamTableExample ? //flink 
run -c org.apache.flink.table.examples.scala.StreamTableExample 
target/flink-examples-table_2.1-1.5-SNAPSHOT.jar ?

This does not work because of error: java.lang.NoClassDefFoundError:...

BR Esa

Fabian Hueske kirjoitti 23.2.2018 klo 15:07:
Have you had a look at the examples? [1]
They can be run out of the IDE.
Fabian

[1] 
https://github.com/apache/flink/tree/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples

2018-02-23 13:30 GMT+01:00 Esa Heikkinen 
mailto:esa.heikki...@student.tut.fi>>:
I have lot of difficulties to deploy Flink. That is maybe because I am new with 
Flink and its (Java and Maven) development environment, but I would hear the 
opinions of others. I would like to use Scala.

There are many examples, but often there are missing “imports” and settings in 
pom.xml. It seem to be very hard to job to find correct ones. Maybe use of IDE 
(IntelliJ IDEA) is almost mandatory and it helps to find “imports”, but it does 
not find all of them.

Generally you have to do and study a lot of basic work before you get into the 
actual thing ?

If there is a ready example (with source code) that is enough close to what you 
want, it is much easier to deploy. But if not, it can be a surprisingly 
difficult and time-consuming task. Because the documentation seem to be 
partially incomplete, it is often necessary to “google” and query the mailing 
list.

Or have I misunderstand something or I can not use Flink correctly (yet) ?

Features of Flink are so good that I would want to learn to use it.

Best Regards
Esa





--
Niclas Hedhman, Software Developer
http://zest.apache.org - New Energy for Java

RE: Is Flink easy to deploy ?

[Esa Heikkinen]

Hi

Whether these instructions of IDE are only for Java, but no or Scala ?

Best,
Esa

From: xingcan [mailto:xc...@foxmail.com]
Sent: Saturday, February 24, 2018 3:25 AM
To: Esa Heikkinen 
Cc: user@flink.apache.org
Subject: Re: Is Flink easy to deploy ?

Hi Esa,

maybe you can follow the 
instructions<https://ci.apache.org/projects/flink/flink-docs-release-1.4/internals/ide_setup.html>
 to import Flink to your IDE and run the examples in it. That’s a good 
beginning I think.

Best,
Xingcan


On 24 Feb 2018, at 3:22 AM, Esa Heikkinen 
mailto:heikk...@student.tut.fi>> wrote:


Yes i have looked. For example, if i want to compile and run 
StreamTableExample.scala from:
https://github.com/apache/flink/blob/master/flink-examples/flink-examples-table/src/main/scala/org/apache/flink/table/examples/scala/StreamTableExample.scala
I have taken all examples (and also latest Flink at same time) to my Linux from 
git.
Where directory should i be in for compiling and running StreamTableExample in 
command line ? flink-examples-table ?
What is the command for compiling ? mvn clean install -Pbuild-jar ?
What is the command running the StreamTableExample ? //flink 
run -c org.apache.flink.table.examples.scala.StreamTableExample 
target/flink-examples-table_2.1-1.5-SNAPSHOT.jar ?
This does not work because of error: java.lang.NoClassDefFoundError:...
BR Esa

Fabian Hueske kirjoitti 23.2.2018 klo 15:07:
Have you had a look at the examples? [1]
They can be run out of the IDE.
Fabian

[1] 
https://github.com/apache/flink/tree/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples

2018-02-23 13:30 GMT+01:00 Esa Heikkinen 
mailto:esa.heikki...@student.tut.fi>>:
I have lot of difficulties to deploy Flink. That is maybe because I am new with 
Flink and its (Java and Maven) development environment, but I would hear the 
opinions of others. I would like to use Scala.

There are many examples, but often there are missing “imports” and settings in 
pom.xml. It seem to be very hard to job to find correct ones. Maybe use of IDE 
(IntelliJ IDEA) is almost mandatory and it helps to find “imports”, but it does 
not find all of them.

Generally you have to do and study a lot of basic work before you get into the 
actual thing ?

If there is a ready example (with source code) that is enough close to what you 
want, it is much easier to deploy. But if not, it can be a surprisingly 
difficult and time-consuming task. Because the documentation seem to be 
partially incomplete, it is often necessary to “google” and query the mailing 
list.

Or have I misunderstand something or I can not use Flink correctly (yet) ?

Features of Flink are so good that I would want to learn to use it.

Best Regards
Esa

Re: Is Flink easy to deploy ?

[Esa Heikkinen]

Hi

I have done like that, but no "checkstyle" settings. Checkstyleonly 
affects how the code looks ?


But should it be possible to compile and run without IDE ?

Best,
Esa

xingcan kirjoitti 24.2.2018 klo 3:24:

Hi Esa,

maybe you can follow the instructions 
 to 
import Flink to your IDE and run the examples in it. That’s a good 
beginning I think.


Best,
Xingcan

On 24 Feb 2018, at 3:22 AM, Esa Heikkinen > wrote:



Yes i have looked. For example, if i want to compile and run 
StreamTableExample.scala from:


https://github.com/apache/flink/blob/master/flink-examples/flink-examples-table/src/main/scala/org/apache/flink/table/examples/scala/StreamTableExample.scala

I have taken all examples (and also latest Flink at same time) to my 
Linux from git.


Where directory should i be in for compiling and running 
StreamTableExample in command line ? flink-examples-table ?


What is the command for compiling ? mvn clean install -Pbuild-jar ?

What is the command running the StreamTableExample ? /bin>/flink run -c 
org.apache.flink.table.examples.scala.StreamTableExample 
target/flink-examples-table_2.1-1.5-SNAPSHOT.jar ?


This does not work because of error: java.lang.NoClassDefFoundError:...

BR Esa


Fabian Hueske kirjoitti 23.2.2018 klo 15:07:

Have you had a look at the examples? [1]
They can be run out of the IDE.

Fabian

[1] 
https://github.com/apache/flink/tree/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples


2018-02-23 13:30 GMT+01:00 Esa Heikkinen 
mailto:esa.heikki...@student.tut.fi>>:


I have lot of difficulties to deploy Flink. That is maybe
because I am new with Flink and its (Java and Maven) development
environment, but Iwould hear the opinions of others. I would
like to use Scala.


There are many examples, but often there are missing “imports”
and settings in pom.xml. It seem to be very hard to job to find
correct ones. Maybe use of IDE (IntelliJ IDEA) is almost
mandatory and it helps to find “imports”, but it does not find
all of them.


Generally you have to do and study a lot of basic work before
you get into the actual thing ?


If there is a ready example (with source code) that is enough
close to what you want, it is much easier to deploy. But if not,
it can be a surprisingly difficult and time-consuming task.
Because the documentation seem to be partially incomplete, it is
often necessary to “google” and query the mailing list.


Or have I misunderstand something or I can not use Flink
correctly (yet) ?


Features of Flink are so good that I would want to learn to use it.


Best Regards

Esa

Re: Is Flink easy to deploy ?

[Niclas Hedhman]

I think you simply missing a bunch of the Flink artifacts.

Flink is broken into dozens of pieces, and you need to select from a large
set of artifacts what you need to depend on.

Typically, there is one Flink artifact per "extension".
I ended up with
   flink-core
   flink-core
   flink-connector-cassandra_2.11
   flink-connector-kafka_2.11
   flink-queryable-state-runtime_2.11
   flink-streaming-java_2.11
   flink-streaming-scala_2.11

With transitive dependencies enabled, meaning whatever Flink depends on is
also included

Cheers

On Sat, Feb 24, 2018 at 3:22 AM, Esa Heikkinen 
wrote:

>
> Yes i have looked. For example, if i want to compile and run
> StreamTableExample.scala from:
>
> https://github.com/apache/flink/blob/master/flink-
> examples/flink-examples-table/src/main/scala/org/apache/
> flink/table/examples/scala/StreamTableExample.scala
>
> I have taken all examples (and also latest Flink at same time) to my Linux
> from git.
>
> Where directory should i be in for compiling and running
> StreamTableExample in command line ? flink-examples-table ?
>
> What is the command for compiling ? mvn clean install -Pbuild-jar ?
>
> What is the command running the StreamTableExample ? / bin>/flink run -c org.apache.flink.table.examples.scala.StreamTableExample
> target/flink-examples-table_2.1-1.5-SNAPSHOT.jar ?
>
> This does not work because of error: java.lang.NoClassDefFoundError:...
>
> BR Esa
>
> Fabian Hueske kirjoitti 23.2.2018 klo 15:07:
>
> Have you had a look at the examples? [1]
> They can be run out of the IDE.
>
> Fabian
>
> [1] https://github.com/apache/flink/tree/master/flink-
> examples/flink-examples-streaming/src/main/scala/org/
> apache/flink/streaming/scala/examples
>
> 2018-02-23 13:30 GMT+01:00 Esa Heikkinen :
>
>> I have lot of difficulties to deploy Flink. That is maybe because I am
>> new with Flink and its (Java and Maven) development environment, but I
>> would hear the opinions of others. I would like to use Scala.
>>
>>
>>
>> There are many examples, but often there are missing “imports” and
>> settings in pom.xml. It seem to be very hard to job to find correct ones.
>> Maybe use of IDE (IntelliJ IDEA) is almost mandatory and it helps to find
>> “imports”, but it does not find all of them.
>>
>>
>>
>> Generally you have to do and study a lot of basic work before you get
>> into the actual thing ?
>>
>>
>>
>> If there is a ready example (with source code) that is enough close to
>> what you want, it is much easier to deploy. But if not, it can be a
>> surprisingly difficult and time-consuming task. Because the
>> documentation seem to be partially incomplete, it is often necessary to
>> “google” and query the mailing list.
>>
>>
>>
>> Or have I misunderstand something or I can not use Flink correctly (yet) ?
>>
>>
>>
>> Features of Flink are so good that I would want to learn to use it.
>>
>>
>>
>> Best Regards
>>
>> Esa
>>
>
>
>


-- 
Niclas Hedhman, Software Developer
http://zest.apache.org - New Energy for Java

Re: Is Flink easy to deploy ?

[Esa Heikkinen]

Yes i have looked. For example, if i want to compile and run 
StreamTableExample.scala from:


https://github.com/apache/flink/blob/master/flink-examples/flink-examples-table/src/main/scala/org/apache/flink/table/examples/scala/StreamTableExample.scala

I have taken all examples (and also latest Flink at same time) to my 
Linux from git.


Where directory should i be in for compiling and running 
StreamTableExample in command line ? flink-examples-table ?


What is the command for compiling ? mvn clean install -Pbuild-jar ?

What is the command running the StreamTableExample ? /bin>/flink run -c 
org.apache.flink.table.examples.scala.StreamTableExample 
target/flink-examples-table_2.1-1.5-SNAPSHOT.jar ?


This does not work because of error: java.lang.NoClassDefFoundError:...

BR Esa


Fabian Hueske kirjoitti 23.2.2018 klo 15:07:

Have you had a look at the examples? [1]
They can be run out of the IDE.

Fabian

[1] 
https://github.com/apache/flink/tree/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples


2018-02-23 13:30 GMT+01:00 Esa Heikkinen >:


I have lot of difficulties to deploy Flink. That is maybe because
I am new with Flink and its (Java and Maven) development
environment, but Iwould hear the opinions of others. I would like
to use Scala.

There are many examples, but often there are missing “imports” and
settings in pom.xml. It seem to be very hard to job to find
correct ones. Maybe use of IDE (IntelliJ IDEA) is almost mandatory
and it helps to find “imports”, but it does not find all of them.

Generally you have to do and study a lot of basic work before you
get into the actual thing ?

If there is a ready example (with source code) that is enough
close to what you want, it is much easier to deploy. But if not,
it can be a surprisingly difficult and time-consuming task.
Because the documentation seem to be partially incomplete, it is
often necessary to “google” and query the mailing list.

Or have I misunderstand something or I can not use Flink correctly
(yet) ?

Features of Flink are so good that I would want to learn to use it.

Best Regards

Esa

Re: Is Flink easy to deploy ?

[Fabian Hueske]

Have you had a look at the examples? [1]
They can be run out of the IDE.

Fabian

[1]
https://github.com/apache/flink/tree/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples

2018-02-23 13:30 GMT+01:00 Esa Heikkinen :

> I have lot of difficulties to deploy Flink. That is maybe because I am new
> with Flink and its (Java and Maven) development environment, but I would
> hear the opinions of others. I would like to use Scala.
>
>
>
> There are many examples, but often there are missing “imports” and
> settings in pom.xml. It seem to be very hard to job to find correct ones.
> Maybe use of IDE (IntelliJ IDEA) is almost mandatory and it helps to find
> “imports”, but it does not find all of them.
>
>
>
> Generally you have to do and study a lot of basic work before you get
> into the actual thing ?
>
>
>
> If there is a ready example (with source code) that is enough close to
> what you want, it is much easier to deploy. But if not, it can be a
> surprisingly difficult and time-consuming task. Because the documentation
> seem to be partially incomplete, it is often necessary to “google” and
> query the mailing list.
>
>
>
> Or have I misunderstand something or I can not use Flink correctly (yet) ?
>
>
>
> Features of Flink are so good that I would want to learn to use it.
>
>
>
> Best Regards
>
> Esa
>

Is Flink easy to deploy ?

[Esa Heikkinen]

I have lot of difficulties to deploy Flink. That is maybe because I am new with 
Flink and its (Java and Maven) development environment, but I would hear the 
opinions of others. I would like to use Scala.

There are many examples, but often there are missing "imports" and settings in 
pom.xml. It seem to be very hard to job to find correct ones. Maybe use of IDE 
(IntelliJ IDEA) is almost mandatory and it helps to find "imports", but it does 
not find all of them.

Generally you have to do and study a lot of basic work before you get into the 
actual thing ?

If there is a ready example (with source code) that is enough close to what you 
want, it is much easier to deploy. But if not, it can be a surprisingly 
difficult and time-consuming task. Because the documentation seem to be 
partially incomplete, it is often necessary to "google" and query the mailing 
list.

Or have I misunderstand something or I can not use Flink correctly (yet) ?

Features of Flink are so good that I would want to learn to use it.

Best Regards
Esa