Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Gary Yao]

Congratulations Andrey, well deserved!

Best,
Gary

On Thu, Aug 15, 2019 at 7:50 AM Bowen Li  wrote:

> Congratulations Andrey!
>
> On Wed, Aug 14, 2019 at 10:18 PM Rong Rong  wrote:
>
>> Congratulations Andrey!
>>
>> On Wed, Aug 14, 2019 at 10:14 PM chaojianok  wrote:
>>
>> > Congratulations Andrey!
>> > At 2019-08-14 21:26:37, "Till Rohrmann"  wrote:
>> > >Hi everyone,
>> > >
>> > >I'm very happy to announce that Andrey Zagrebin accepted the offer of
>> the
>> > >Flink PMC to become a committer of the Flink project.
>> > >
>> > >Andrey has been an active community member for more than 15 months. He
>> has
>> > >helped shaping numerous features such as State TTL, FRocksDB release,
>> > >Shuffle service abstraction, FLIP-1, result partition management and
>> > >various fixes/improvements. He's also frequently helping out on the
>> > >user@f.a.o mailing lists.
>> > >
>> > >Congratulations Andrey!
>> > >
>> > >Best, Till
>> > >(on behalf of the Flink PMC)
>> >
>>
>

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Bowen Li]

Congratulations Andrey!

On Wed, Aug 14, 2019 at 10:18 PM Rong Rong  wrote:

> Congratulations Andrey!
>
> On Wed, Aug 14, 2019 at 10:14 PM chaojianok  wrote:
>
> > Congratulations Andrey!
> > At 2019-08-14 21:26:37, "Till Rohrmann"  wrote:
> > >Hi everyone,
> > >
> > >I'm very happy to announce that Andrey Zagrebin accepted the offer of
> the
> > >Flink PMC to become a committer of the Flink project.
> > >
> > >Andrey has been an active community member for more than 15 months. He
> has
> > >helped shaping numerous features such as State TTL, FRocksDB release,
> > >Shuffle service abstraction, FLIP-1, result partition management and
> > >various fixes/improvements. He's also frequently helping out on the
> > >user@f.a.o mailing lists.
> > >
> > >Congratulations Andrey!
> > >
> > >Best, Till
> > >(on behalf of the Flink PMC)
> >
>

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Rong Rong]

Congratulations Andrey!

On Wed, Aug 14, 2019 at 10:14 PM chaojianok  wrote:

> Congratulations Andrey!
> At 2019-08-14 21:26:37, "Till Rohrmann"  wrote:
> >Hi everyone,
> >
> >I'm very happy to announce that Andrey Zagrebin accepted the offer of the
> >Flink PMC to become a committer of the Flink project.
> >
> >Andrey has been an active community member for more than 15 months. He has
> >helped shaping numerous features such as State TTL, FRocksDB release,
> >Shuffle service abstraction, FLIP-1, result partition management and
> >various fixes/improvements. He's also frequently helping out on the
> >user@f.a.o mailing lists.
> >
> >Congratulations Andrey!
> >
> >Best, Till
> >(on behalf of the Flink PMC)
>

Re:[ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[chaojianok]

Congratulations Andrey!

At 2019-08-14 21:26:37, "Till Rohrmann"  wrote:

Hi everyone,


I'm very happy to announce that Andrey Zagrebin accepted the offer of the Flink 
PMC to become a committer of the Flink project.


Andrey has been an active community member for more than 15 months. He has 
helped shaping numerous features such as State TTL, FRocksDB release, Shuffle 
service abstraction, FLIP-1, result partition management and various 
fixes/improvements. He's also frequently helping out on the user@f.a.o mailing 
lists.


Congratulations Andrey!


Best, Till 

(on behalf of the Flink PMC)

Re:[ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[chaojianok]

Congratulations Andrey!
At 2019-08-14 21:26:37, "Till Rohrmann"  wrote:
>Hi everyone,
>
>I'm very happy to announce that Andrey Zagrebin accepted the offer of the
>Flink PMC to become a committer of the Flink project.
>
>Andrey has been an active community member for more than 15 months. He has
>helped shaping numerous features such as State TTL, FRocksDB release,
>Shuffle service abstraction, FLIP-1, result partition management and
>various fixes/improvements. He's also frequently helping out on the
>user@f.a.o mailing lists.
>
>Congratulations Andrey!
>
>Best, Till
>(on behalf of the Flink PMC)

Re: Flink metrics scope for YARN single job

[Biao Liu]

Hi Vasily,

> Is there any way to distinguish logs from different JobManager running on
same host?

Do you mean "distinguish metrics from different JobManager running on same
host"?
I guess there is no other variable you could use for now.

But I think it's reasonable to support this requirement. I would like to
discuss with the devs to hear their opinions. Will give you a feedback if
there is a conclusion.

Thanks,
Biao /'bɪ.aʊ/



On Wed, 14 Aug 2019 at 19:46, Vasily Melnik <
vasily.mel...@glowbyteconsulting.com> wrote:

> Hi,
> I want to run Flink apps on YARN in single job mode and keep metrics in
> Graphite. But as i see, the only variable i can use for JobManager scope
> customization is :
> https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#list-of-all-variables
>
> Is there any way to distinguish logs from different JobManager running on
> same host?
>
>
> Thanks in advance.
>

Re: Customize file assignments logic in flink application

[Zhu Zhu]

Hi Lu,

Implementing your own *InputFormat* and *InputSplitAssigner*(which has the
interface "InputSplit getNextInputSplit(String host, int taskId)") created
by it should work if you want to assign InputSplit to tasks according to
the task index and file name patterns.
To assign 2 *InputSplit*s in one request, you can implement a new
*InputSplit* which wraps multiple *FileInputSplit*s. And you may need to
define in your *InputFormat* on how to process the new *InputSplit*.

Thanks,
Zhu Zhu

Lu Niu  于2019年8月15日周四 上午12:26写道：

> Hi,
>
> I have a data set backed by a directory of files in which file names are
> meaningful.
>
> folder1
>+-file01
>+-file02
>+-file03
>+-file04
>
> I want to control the file assignments in my flink application. For
> example, when parallelism is 2, worker 1 get file01 and file02 to read and
> worker2 get 3 and 4. Also each worker get 2 files all at once because
> reading requires jumping back and forth between those two files.
>
> What's the best way to do this? It seems like FileInputFormat is not
> extensible in this case.
>
> Best
> Lu
>
>
>

Re: External classpath

[Biao Liu]

Hi Vishwas,

> Does my external class path have to be on NFS share ? Can I not have the
config directory on each machine in the same location ?

It can be local files on each machine.

> -C file://home/was/classpathconfig/

I guess the problem is your URI format of local file is not correct. Try to
use "-C file:///home/was/classpathconfig/" instead.
BTW, if it's a directory, it must end with '/' [1].

> Also I tried to put my files in s3 and tried to run flink run -C s3://
flink.dev/config -c com.visa.flink.cli.Main
flink-job-assembly-0.1-SNAPSHOT.jar flink druid

Currently it doesn't support s3 or other distributed file system as the
parameter of -C.

1. https://docs.oracle.com/javase/8/docs/api/java/net/URLClassLoader.html

Thanks,
Biao /'bɪ.aʊ/



On Thu, 15 Aug 2019 at 07:51, Vishwas Siravara  wrote:

> Hi guys,
> I m very close to deploying my application in production so I am trying to
> externalize some of the config files which has to be available on the
> classpath when I run my application via flink command line interface.
>
> From the flink doc I can add to class path by
>
> -C,--classpath  Adds a URL to each user code
>   classloader  on all nodes in the
>   cluster. The paths must specify a
>   protocol (e.g. file://) and be
>   accessible on all nodes (e.g. by 
> means
>   of a NFS share). You can use this
>   option multiple times for specifying
>   more than one URL. The protocol must
>   be supported by the {@link
>   java.net.URLClassLoader}.
>
> So my job has 8 task managers( 8 different nodes) with 8 slots in each .
> Does my external class path have to be on NFS share ? Can I not have the
> config directory on each machine in the same location ? For instance on
> Node 1 the config files are in  the directory is
> /home/was/classpathconfig/  and the same on every node. Does it have to
> be on an NFS ?
> My command looks like this flink run -C file://home/was/classpathconfig/
> -c com.visa.flink.cli.Main flink-job-assembly-0.1-SNAPSHOT.jar flink druid
>
> Also I tried to put my files in s3 and tried to run flink run -C s3://
> flink.dev/config -c com.visa.flink.cli.Main
> flink-job-assembly-0.1-SNAPSHOT.jar flink druid
>
> Bad syntax for classpath: s3://flink.dev/config
>
>
> s3 does support URLClassLoader but I get the error saying bad syntax.
>
>
> Please let me know your thoughts. Thanks a lot to this community , I was able 
> to write my code in a week.
>
>
> Thanks,
>
> Vishwas
>
>

Re: How can I pass jvm options to flink when started from command line

[Zhu Zhu]

Hi Vishwas,

If what you want is to set JVM options for Flink client JVM when running
jobs with "flink run", I think export the variable 'JVM_ARGS' does help.

Thanks,
Zhu Zhu

Vishwas Siravara  于2019年8月15日周四 上午4:03写道：

> I understand that when I run a flink job from command line it forks a jvm
> and runs the main method and the flink related code run in the task
> manager. So when I say "flink run " the main does not run on JobManager
> hence it does not take env.java.options set in the flink-conf.yaml as this
> applies to the job manager and task manager. Now how can I pass jvm options
> like -Dconfig.resource=qa.conf from command line ?
>
> Thanks,
> Vishwas
>

Re: Implementing a low level join

[Hequn Cheng]

Hi Felipe,

If I understand correctly, you also have to decide whether to broadcast the
datastream from the right side before performing the function?

One option is you can add a Util method to join dynamically, e.g.,
Util.joinDynamically(ds1, ds2). In the util method, you can implement your
own strategy logic and decide whether to broadcast or use CoProcessFunction.

Best, Hequn

On Wed, Aug 14, 2019 at 3:07 PM Felipe Gutierrez <
felipe.o.gutier...@gmail.com> wrote:

> Hi Hequn,
>
> I am implementing the broadcast and the regular join. As you said I need
> different functions. My question is more about if I can have an operator
> which decides beteween broadcast and regular join dynamically. I suppose I
> will have to extend the generic TwoInputStreamOperator in Flink. Do you
> have any suggestion?
>
> Thanks
>
> On Wed, 14 Aug 2019, 03:59 Hequn Cheng,  wrote:
>
>> Hi Felipe,
>>
>> > I want to implement a join operator which can use different strategies
>> for joining tuples.
>> Not all kinds of join strategies can be applied to streaming jobs. Take
>> sort-merge join as an example, it's impossible to sort an unbounded data.
>> However, you can perform a window join and use the sort-merge strategy to
>> join the data within a window. Even though, I'm not sure it's worth to do
>> it considering the performance.
>>
>> > Therefore, I am not sure if I will need to implement my own operator to
>> do this or if it is still possible to do with CoProcessFunction.
>> You can't implement broadcast join with CoProcessFunction. But you can
>> implement it with BroadcastProcessFunction or
>> KeyedBroadcastProcessFunction, more details here[1].
>>
>> Furthermore, you can take a look at the implementation of both window
>> join and non-window join in Table API & SQL[2]. The code can be found
>> here[3].
>> Hope this helps.
>>
>> Best, Hequn
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html
>> [2]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/table/sql.html#joins
>> [3]
>> https://github.com/apache/flink/tree/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/runtime/join
>>
>>
>> On Tue, Aug 13, 2019 at 11:30 PM Felipe Gutierrez <
>> felipe.o.gutier...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I want to implement a join operator which can use different strategies
>>> for joining tuples. I saw that with CoProcessFunction I am able to
>>> implement low-level joins [1]. However, I do know how to decide between
>>> different algorithms to join my tuples.
>>>
>>> On the other hand, to do a broadcast join I will need to use the
>>> broadcast operator [2] which yields a BroadcastStream. Therefore, I am not
>>> sure if I will need to implement my own operator to do this or if it is
>>> still possible to do with CoProcessFunction.
>>>
>>> Does anyone have some clues for this matter?
>>> Thanks
>>>
>>> [1]
>>> https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/stream/operators/process_function.html#low-level-joins
>>> [2]
>>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html
>>> *--*
>>> *-- Felipe Gutierrez*
>>>
>>> *-- skype: felipe.o.gutierrez*
>>> *--* *https://felipeogutierrez.blogspot.com
>>> *
>>>
>>

Update tables env after have been register them

[Andres Angel]

Hello everyone,

My use case assume that we execute a job where we load from Redis few data
and turn it into DS to register them as tables.

But, it's possible that after have completed this step the data might
change and we may need to read again the data to keep the tables content up
to date. Here is where our doubt comes up. We need to know whether or not
is possible within a job reload the tables information after certain time
frame frequently without restart the job and have to register again the
tables from the beginning.

I appreciate your comments and the way we could approach this use case.

Thanks so much

External classpath

[Vishwas Siravara]

Hi guys,
I m very close to deploying my application in production so I am trying to
externalize some of the config files which has to be available on the
classpath when I run my application via flink command line interface.

>From the flink doc I can add to class path by

-C,--classpath  Adds a URL to each user code
  classloader  on all nodes in the
  cluster. The paths must specify a
  protocol (e.g. file://) and be
  accessible on all nodes (e.g. by means
  of a NFS share). You can use this
  option multiple times for specifying
  more than one URL. The protocol must
  be supported by the {@link
  java.net.URLClassLoader}.

So my job has 8 task managers( 8 different nodes) with 8 slots in each .
Does my external class path have to be on NFS share ? Can I not have the
config directory on each machine in the same location ? For instance on
Node 1 the config files are in  the directory is
/home/was/classpathconfig/  and
the same on every node. Does it have to be on an NFS ?
My command looks like this flink run -C file://home/was/classpathconfig/ -c
com.visa.flink.cli.Main flink-job-assembly-0.1-SNAPSHOT.jar flink druid

Also I tried to put my files in s3 and tried to run flink run -C s3://
flink.dev/config -c com.visa.flink.cli.Main
flink-job-assembly-0.1-SNAPSHOT.jar flink druid

Bad syntax for classpath: s3://flink.dev/config


s3 does support URLClassLoader but I get the error saying bad syntax.


Please let me know your thoughts. Thanks a lot to this community , I
was able to write my code in a week.


Thanks,

Vishwas

How can I pass jvm options to flink when started from command line

[Vishwas Siravara]

I understand that when I run a flink job from command line it forks a jvm
and runs the main method and the flink related code run in the task
manager. So when I say "flink run " the main does not run on JobManager
hence it does not take env.java.options set in the flink-conf.yaml as this
applies to the job manager and task manager. Now how can I pass jvm options
like -Dconfig.resource=qa.conf from command line ?

Thanks,
Vishwas

Re: How can I pass multiple java options in standalone mode ?

[Vishwas Siravara]

Thanks a lot, I fixed that, so now this works when I submit my job with the
flink UI but when I submit it via flink run(command line) it does not take
this env.java.opts: -Dconfig.resource=qa.conf property . How can I pass the
jvm property to flink run which is running standalone without resource
manager.

Thanks,
Vishwas

On Wed, Aug 14, 2019 at 2:35 PM Aleksandar Mastilovic <
amastilo...@sightmachine.com> wrote:

> It’s a YAML file, so I think you need to do something like
>
>  env.java.opts: -Dconfig.resource=qa.conf
>
> On Aug 14, 2019, at 11:58 AM, Vishwas Siravara 
> wrote:
>
> When I add env.java.opts like this
>  env.java.opts:"-Dconfig.resource=qa.conf"
>
> I see an error in the log file saying :  - Error while trying to split
> key and value in configuration file
> /data/flink-1.7.2/conf/flink-conf.yaml:248:
> "env.java.opts:"-Dconfig.resource=qa.conf""
>
> This is really confusing and I cant find any document on how I should pass
> this option.
>
> Thanks,
> Vishwas
>
> On Wed, Aug 14, 2019 at 12:40 PM Vishwas Siravara 
> wrote:
>
>> Is it possible for me to pass these arguments along with the job when I
>> do flink run and then pass the jvm options. For example if I want to pass
>> this parameter -Dconfig.resource=qa.conf and qa.conf is packaged in the
>> job's fat jar then flink will not find this file if I pass
>> -Dconfig.resource=qa.conf and qa.conf in env.java.opts.
>>
>> Thanks,
>> Vishwas
>>
>> On Mon, Aug 12, 2019 at 6:00 PM Zili Chen  wrote:
>>
>>> Hi Vishwas,
>>>
>>> Replace ',' with ' '(space) should work.
>>>
>>> Best,
>>> tison.
>>>
>>>
>>> Vishwas Siravara  于2019年8月13日周二 上午6:50写道：
>>>
 Hi guys,
 I have this entry in flink-conf.yaml file for jvm options.
 env.java.opts: "-Djava.security.auth.login.config={{
 flink_installed_dir }}/kafka-jaas.conf,-Djava.security.krb5.conf={{
 flink_installed_dir }}/krb5.conf"

 Is this supposed to be a , separated list ? I get a parse exception
 when the cluster starts.

 Thanks,
 Vishwas

>>>
>

Re: How can I pass multiple java options in standalone mode ?

[Aleksandar Mastilovic]

It’s a YAML file, so I think you need to do something like

 env.java.opts: -Dconfig.resource=qa.conf

> On Aug 14, 2019, at 11:58 AM, Vishwas Siravara  wrote:
> 
> When I add env.java.opts like this
>  env.java.opts:"-Dconfig.resource=qa.conf"
> 
> I see an error in the log file saying :  - Error while trying to split key 
> and value in configuration file /data/flink-1.7.2/conf/flink-conf.yaml:248: 
> "env.java.opts:"-Dconfig.resource=qa.conf""
> 
> This is really confusing and I cant find any document on how I should pass 
> this option. 
> 
> Thanks,
> Vishwas
> 
> On Wed, Aug 14, 2019 at 12:40 PM Vishwas Siravara  > wrote:
> Is it possible for me to pass these arguments along with the job when I do 
> flink run and then pass the jvm options. For example if I want to pass this 
> parameter -Dconfig.resource=qa.conf and qa.conf is packaged in the job's fat 
> jar then flink will not find this file if I pass -Dconfig.resource=qa.conf 
> and qa.conf in env.java.opts. 
> 
> Thanks,
> Vishwas 
> 
> On Mon, Aug 12, 2019 at 6:00 PM Zili Chen  > wrote:
> Hi Vishwas,
> 
> Replace ',' with ' '(space) should work.
> 
> Best,
> tison.
> 
> 
> Vishwas Siravara mailto:vsirav...@gmail.com>> 
> 于2019年8月13日周二 上午6:50写道：
> Hi guys,
> I have this entry in flink-conf.yaml file for jvm options. 
> env.java.opts: "-Djava.security.auth.login.config={{ flink_installed_dir 
> }}/kafka-jaas.conf,-Djava.security.krb5.conf={{ flink_installed_dir 
> }}/krb5.conf"
> 
> Is this supposed to be a , separated list ? I get a parse exception when the 
> cluster starts. 
> 
> Thanks,
> Vishwas 

Customize file assignments logic in flink application

[Lu Niu]

Hi,

I have a data set backed by a directory of files in which file names are
meaningful.

folder1
   +-file01
   +-file02
   +-file03
   +-file04

I want to control the file assignments in my flink application. For
example, when parallelism is 2, worker 1 get file01 and file02 to read and
worker2 get 3 and 4. Also each worker get 2 files all at once because
reading requires jumping back and forth between those two files.

What's the best way to do this? It seems like FileInputFormat is not
extensible in this case.

Best
Lu

Re: Flink SQL: How to tag a column as 'rowtime' from a Avro based DataStream?

[Timo Walther]

Hi Niels,

if you are coming from DataStream API, all you need to do is to write a 
timestamp extractor.


When you call:

tableEnv.registerDataStream("TestStream", letterStream, 
"EventTime.rowtime, letter, counter");


The ".rowtime" means that the framework will extract the rowtime from 
the stream record timestamp. You don't need to name all fields again but 
could simply construct a string from 
letterStream.getTypeInfo().getFieldNames(). I hope we can improve this 
further in the future as part of FLIP-37.


Regards,
Timo

Am 14.08.19 um 17:00 schrieb Niels Basjes:

Hi,

Experimenting with the StreamTableEnvironment I build something like this:

DataStream> letterStream = ...
tableEnv.registerDataStream("TestStream", letterStream,
"EventTime.rowtime, letter, counter");


Because the "EventTime" was tagged with ".rowtime" it is now being 
used as the rowtime and has the DATETIME so I can do this


TUMBLE_START(eventTime, INTERVAL '1' MINUTE)


So far so good.

Working towards a more realistic scenario I have a source that 
produces a stream of records that have been defined using Apache Avro.


So I have a Measurement.avdl that (among other things) contains 
something like this:


record Measurement {
   /** The time (epoch in milliseconds since 1970-01-01 UTC) when
the event occurred */
    long  timestamp;
    string  letter;
    long  pageviews;
}


Now because the registerDataStream call can also derive the schema 
from the provided data I can do this:


DataStream inputStream = ...
tableEnv.registerDataStream("DataStream", inputStream);


This is very nice because any real schema is big (few hundred columns) 
and changes over time.


Now In the SQL the timestamp is a BIGINT and not a DATETIME and as a 
consequence I get this error


Cannot apply 'TUMBLE' to arguments of type 'TUMBLE(,
)'. Supported form(s): 'TUMBLE(,
)'


So far I have now yet figured how to make the system understand that 
the timestamp column show be treated as the rowtime.

How do I do that?

--
Best regards / Met vriendelijke groeten,

Niels Basjes

Flink SQL: How to tag a column as 'rowtime' from a Avro based DataStream?

[Niels Basjes]

Hi,

Experimenting with the StreamTableEnvironment I build something like this:

DataStream> letterStream = ...
tableEnv.registerDataStream("TestStream", letterStream, "EventTime.rowtime,
letter, counter");


Because the "EventTime" was tagged with ".rowtime" it is now being used as
the rowtime and has the DATETIME so I can do this

TUMBLE_START(eventTime, INTERVAL '1' MINUTE)


So far so good.

Working towards a more realistic scenario I have a source that produces a
stream of records that have been defined using Apache Avro.

So I have a Measurement.avdl that (among other things) contains something
like this:

record Measurement {
   /** The time (epoch in milliseconds since 1970-01-01 UTC) when the event
occurred */
longtimestamp;
string  letter;
longpageviews;
}


Now because the registerDataStream call can also derive the schema from the
provided data I can do this:

DataStream inputStream = ...
tableEnv.registerDataStream("DataStream", inputStream);


This is very nice because any real schema is big (few hundred columns) and
changes over time.

Now In the SQL the timestamp is a BIGINT and not a DATETIME and as a
consequence I get this error

Cannot apply 'TUMBLE' to arguments of type 'TUMBLE(, )'. Supported form(s): 'TUMBLE(, )'


So far I have now yet figured how to make the system understand that the
timestamp column show be treated as the rowtime.
How do I do that?

-- 
Best regards / Met vriendelijke groeten,

Niels Basjes

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Dawid Wysakowicz]

Congratulations Andrey! Pleasure to have you in the community!

Best,

Dawid

On 14/08/2019 16:06, Kostas Kloudas wrote:
> Congratulations Andrey!
> Well deserved!
>
> Kostas
>
> On Wed, Aug 14, 2019 at 4:04 PM Yun Tang  wrote:
>> Congratulations Andrey.
>>
>> Best
>> Yun Tang
>> 
>> From: Xintong Song 
>> Sent: Wednesday, August 14, 2019 21:40
>> To: Oytun Tez 
>> Cc: Zili Chen ; Till Rohrmann ; 
>> dev ; user 
>> Subject: Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer
>>
>> Congratulations Andery~!
>>
>> Thank you~
>>
>> Xintong Song
>>
>>
>>
>> On Wed, Aug 14, 2019 at 3:31 PM Oytun Tez  wrote:
>>
>> Congratulations Andrey!
>>
>> I am glad the Flink committer team is growing at such a pace!
>>
>> ---
>> Oytun Tez
>>
>> M O T A W O R D
>> The World's Fastest Human Translation Platform.
>> oy...@motaword.com — www.motaword.com
>>
>>
>> On Wed, Aug 14, 2019 at 9:29 AM Zili Chen  wrote:
>>
>> Congratulations Andrey!
>>
>> Best,
>> tison.
>>
>>
>> Till Rohrmann  于2019年8月14日周三 下午9:26写道：
>>
>> Hi everyone,
>>
>> I'm very happy to announce that Andrey Zagrebin accepted the offer of the 
>> Flink PMC to become a committer of the Flink project.
>>
>> Andrey has been an active community member for more than 15 months. He has 
>> helped shaping numerous features such as State TTL, FRocksDB release, 
>> Shuffle service abstraction, FLIP-1, result partition management and various 
>> fixes/improvements. He's also frequently helping out on the user@f.a.o 
>> mailing lists.
>>
>> Congratulations Andrey!
>>
>> Best, Till
>> (on behalf of the Flink PMC)



signature.asc
Description: OpenPGP digital signature

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Robert Metzger]

Congratulations! Very happy to have you onboard :)

On Wed, Aug 14, 2019 at 4:06 PM Kostas Kloudas  wrote:

> Congratulations Andrey!
> Well deserved!
>
> Kostas
>
> On Wed, Aug 14, 2019 at 4:04 PM Yun Tang  wrote:
> >
> > Congratulations Andrey.
> >
> > Best
> > Yun Tang
> > 
> > From: Xintong Song 
> > Sent: Wednesday, August 14, 2019 21:40
> > To: Oytun Tez 
> > Cc: Zili Chen ; Till Rohrmann <
> trohrm...@apache.org>; dev ; user <
> user@flink.apache.org>
> > Subject: Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer
> >
> > Congratulations Andery~!
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Wed, Aug 14, 2019 at 3:31 PM Oytun Tez  wrote:
> >
> > Congratulations Andrey!
> >
> > I am glad the Flink committer team is growing at such a pace!
> >
> > ---
> > Oytun Tez
> >
> > M O T A W O R D
> > The World's Fastest Human Translation Platform.
> > oy...@motaword.com — www.motaword.com
> >
> >
> > On Wed, Aug 14, 2019 at 9:29 AM Zili Chen  wrote:
> >
> > Congratulations Andrey!
> >
> > Best,
> > tison.
> >
> >
> > Till Rohrmann  于2019年8月14日周三 下午9:26写道：
> >
> > Hi everyone,
> >
> > I'm very happy to announce that Andrey Zagrebin accepted the offer of
> the Flink PMC to become a committer of the Flink project.
> >
> > Andrey has been an active community member for more than 15 months. He
> has helped shaping numerous features such as State TTL, FRocksDB release,
> Shuffle service abstraction, FLIP-1, result partition management and
> various fixes/improvements. He's also frequently helping out on the
> user@f.a.o mailing lists.
> >
> > Congratulations Andrey!
> >
> > Best, Till
> > (on behalf of the Flink PMC)
>

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Kostas Kloudas]

Congratulations Andrey!
Well deserved!

Kostas

On Wed, Aug 14, 2019 at 4:04 PM Yun Tang  wrote:
>
> Congratulations Andrey.
>
> Best
> Yun Tang
> 
> From: Xintong Song 
> Sent: Wednesday, August 14, 2019 21:40
> To: Oytun Tez 
> Cc: Zili Chen ; Till Rohrmann ; 
> dev ; user 
> Subject: Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer
>
> Congratulations Andery~!
>
> Thank you~
>
> Xintong Song
>
>
>
> On Wed, Aug 14, 2019 at 3:31 PM Oytun Tez  wrote:
>
> Congratulations Andrey!
>
> I am glad the Flink committer team is growing at such a pace!
>
> ---
> Oytun Tez
>
> M O T A W O R D
> The World's Fastest Human Translation Platform.
> oy...@motaword.com — www.motaword.com
>
>
> On Wed, Aug 14, 2019 at 9:29 AM Zili Chen  wrote:
>
> Congratulations Andrey!
>
> Best,
> tison.
>
>
> Till Rohrmann  于2019年8月14日周三 下午9:26写道：
>
> Hi everyone,
>
> I'm very happy to announce that Andrey Zagrebin accepted the offer of the 
> Flink PMC to become a committer of the Flink project.
>
> Andrey has been an active community member for more than 15 months. He has 
> helped shaping numerous features such as State TTL, FRocksDB release, Shuffle 
> service abstraction, FLIP-1, result partition management and various 
> fixes/improvements. He's also frequently helping out on the user@f.a.o 
> mailing lists.
>
> Congratulations Andrey!
>
> Best, Till
> (on behalf of the Flink PMC)

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Yun Tang]

Congratulations Andrey.

Best
Yun Tang

From: Xintong Song 
Sent: Wednesday, August 14, 2019 21:40
To: Oytun Tez 
Cc: Zili Chen ; Till Rohrmann ; dev 
; user 
Subject: Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

Congratulations Andery~!


Thank you~

Xintong Song


On Wed, Aug 14, 2019 at 3:31 PM Oytun Tez 
mailto:oy...@motaword.com>> wrote:
Congratulations Andrey!

I am glad the Flink committer team is growing at such a pace!

---
Oytun Tez

M O T A W O R D
The World's Fastest Human Translation Platform.
oy...@motaword.com ― 
www.motaword.com


On Wed, Aug 14, 2019 at 9:29 AM Zili Chen 
mailto:wander4...@gmail.com>> wrote:
Congratulations Andrey!

Best,
tison.


Till Rohrmann mailto:trohrm...@apache.org>> 于2019年8月14日周三 
下午9:26写道：
Hi everyone,

I'm very happy to announce that Andrey Zagrebin accepted the offer of the Flink 
PMC to become a committer of the Flink project.

Andrey has been an active community member for more than 15 months. He has 
helped shaping numerous features such as State TTL, FRocksDB release, Shuffle 
service abstraction, FLIP-1, result partition management and various 
fixes/improvements. He's also frequently helping out on the user@f.a.o mailing 
lists.

Congratulations Andrey!

Best, Till
(on behalf of the Flink PMC)

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Xintong Song]

Congratulations Andery~!

Thank you~

Xintong Song



On Wed, Aug 14, 2019 at 3:31 PM Oytun Tez  wrote:

> Congratulations Andrey!
>
> I am glad the Flink committer team is growing at such a pace!
>
> ---
> Oytun Tez
>
> *M O T A W O R D*
> The World's Fastest Human Translation Platform.
> oy...@motaword.com — www.motaword.com
>
>
> On Wed, Aug 14, 2019 at 9:29 AM Zili Chen  wrote:
>
>> Congratulations Andrey!
>>
>> Best,
>> tison.
>>
>>
>> Till Rohrmann  于2019年8月14日周三 下午9:26写道：
>>
>>> Hi everyone,
>>>
>>> I'm very happy to announce that Andrey Zagrebin accepted the offer of
>>> the Flink PMC to become a committer of the Flink project.
>>>
>>> Andrey has been an active community member for more than 15 months. He
>>> has helped shaping numerous features such as State TTL, FRocksDB release,
>>> Shuffle service abstraction, FLIP-1, result partition management and
>>> various fixes/improvements. He's also frequently helping out on the
>>> user@f.a.o mailing lists.
>>>
>>> Congratulations Andrey!
>>>
>>> Best, Till
>>> (on behalf of the Flink PMC)
>>>
>>

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Oytun Tez]

Congratulations Andrey!

I am glad the Flink committer team is growing at such a pace!

---
Oytun Tez

*M O T A W O R D*
The World's Fastest Human Translation Platform.
oy...@motaword.com — www.motaword.com


On Wed, Aug 14, 2019 at 9:29 AM Zili Chen  wrote:

> Congratulations Andrey!
>
> Best,
> tison.
>
>
> Till Rohrmann  于2019年8月14日周三 下午9:26写道：
>
>> Hi everyone,
>>
>> I'm very happy to announce that Andrey Zagrebin accepted the offer of
>> the Flink PMC to become a committer of the Flink project.
>>
>> Andrey has been an active community member for more than 15 months. He
>> has helped shaping numerous features such as State TTL, FRocksDB release,
>> Shuffle service abstraction, FLIP-1, result partition management and
>> various fixes/improvements. He's also frequently helping out on the
>> user@f.a.o mailing lists.
>>
>> Congratulations Andrey!
>>
>> Best, Till
>> (on behalf of the Flink PMC)
>>
>

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Zili Chen]

Congratulations Andrey!

Best,
tison.


Till Rohrmann  于2019年8月14日周三 下午9:26写道：

> Hi everyone,
>
> I'm very happy to announce that Andrey Zagrebin accepted the offer of the
> Flink PMC to become a committer of the Flink project.
>
> Andrey has been an active community member for more than 15 months. He
> has helped shaping numerous features such as State TTL, FRocksDB release,
> Shuffle service abstraction, FLIP-1, result partition management and
> various fixes/improvements. He's also frequently helping out on the
> user@f.a.o mailing lists.
>
> Congratulations Andrey!
>
> Best, Till
> (on behalf of the Flink PMC)
>

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[zhijiang]

Congratulations Andrey, great work and well deserved!

Best,
Zhijiang
--
From:Till Rohrmann 
Send Time:2019年8月14日(星期三) 15:26
To:dev ; user 
Subject:[ANNOUNCE] Andrey Zagrebin becomes a Flink committer

Hi everyone,

I'm very happy to announce that Andrey Zagrebin accepted the offer of the Flink 
PMC to become a committer of the Flink project.

Andrey has been an active community member for more than 15 months. He has 
helped shaping numerous features such as State TTL, FRocksDB release, Shuffle 
service abstraction, FLIP-1, result partition management and various 
fixes/improvements. He's also frequently helping out on the user@f.a.o mailing 
lists.

Congratulations Andrey!

Best, Till 
(on behalf of the Flink PMC)

[ANNOUNCE] Andrey Zagrebin becomes a Flink committer

[Till Rohrmann]

Hi everyone,

I'm very happy to announce that Andrey Zagrebin accepted the offer of the
Flink PMC to become a committer of the Flink project.

Andrey has been an active community member for more than 15 months. He has
helped shaping numerous features such as State TTL, FRocksDB release,
Shuffle service abstraction, FLIP-1, result partition management and
various fixes/improvements. He's also frequently helping out on the
user@f.a.o mailing lists.

Congratulations Andrey!

Best, Till
(on behalf of the Flink PMC)

Re: Flink 1.8: Using the RocksDB state backend causes "NoSuchMethodError" when trying to stop a pipeline

[Kaymak, Tobias]

You are right, my bad. We had a company internal java dependency that was
referring to an older version of RocksDB. I've spotted it via running

mvn dependency:tree

while investigating with a colleague.
Thank you!

On Tue, Aug 13, 2019 at 8:01 PM Yun Tang  wrote:

> Hi Tobias
>
> First of all, I think you would not need to ADD the
> flink-statebackend-rocksdb jar package into your docker image's lib folder,
> as the flink-dist jar package within lib folder already include all classes
> of flink-statebackend-rocksdb.
>
> I think the root cause is that you might assemble the rocksdbjni jar
> package in your user application jar which was rocksdbjni-5.7.5.jar in
> Flink-1.7. As Flink would load classes first from the user code jar [1],
> however, method org.rocksdb.ColumnFamilyHandle.getDescriptor() is not
> existed in rocksdbjni-5.7.5.jar but in rocksdbjni-5.17.2 (or we can say
> frocksdbjni-5.17.2-artisans-1.0 in Flink-1.8). That's why you come across
> this NoSuchMethodError exception.
>
> If no necessary, please do not assemble rocksdbjni package in your user
> code jar as flink-dist already provide all needed classes. Moreover, adding
> dependency of flink-statebackend-rocksdb_2.11 in your pom.xml should be
> enough as it already includes the dependency of rocksdbjni.
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#classloader-resolve-order
>
> Best
> Yun Tang
>
> --
> *From:* Kaymak, Tobias 
> *Sent:* Tuesday, August 13, 2019 21:20
> *To:* user@flink.apache.org 
> *Subject:* Flink 1.8: Using the RocksDB state backend causes
> "NoSuchMethodError" when trying to stop a pipeline
>
> Hi,
>
> I am using Apache Beam 2.14.0 with Flink 1.8.0 and I have included the
> RocksDb dependency in my projects pom.xml as well as baked it into the
> Dockerfile like this:
>
> FROM flink:1.8.0-scala_2.11
>
> ADD --chown=flink:flink
> http://central.maven.org/maven2/org/apache/flink/flink-statebackend-rocksdb_2.11/1.8.0/flink-statebackend-rocksdb_2.11-1.8.0.jar
> /opt/flink/lib/flink-statebackend-rocksdb_2.11-1.8.0.jar
>
>
> Everything seems to be normal up to the point when I try to stop and
> cleanly shutdown my pipeline. I get the following error:
>
> java.lang.NoSuchMethodError:
> org.rocksdb.ColumnFamilyHandle.getDescriptor()Lorg/rocksdb/ColumnFamilyDescriptor;
> at
> org.apache.flink.contrib.streaming.state.RocksDBOperationUtils.addColumnFamilyOptionsToCloseLater(RocksDBOperationUtils.java:160)
> at
> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.dispose(RocksDBKeyedStateBackend.java:331)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.dispose(AbstractStreamOperator.java:362)
> at
> org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator.dispose(DoFnOperator.java:470)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.tryDisposeAllOperators(StreamTask.java:454)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:337)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> at java.lang.Thread.run(Thread.java:748)
>
> I can cancel my pipeline and snapshotting in general works, however. Flink
> 1.7.2 with Beam 2.12.0 did not have any problem, could it be that this is
> caused by the switch to FRocksDb?[0]
>
> Best,
> Tobias
>
> [0]
> https://ci.apache.org/projects/flink/flink-docs-stable/release-notes/flink-1.8.html#rocksdb-version-bump-and-switch-to-frocksdb-flink-10471
>

Re: Changing the way keys are defined breaks savepoints

[Congxian Qiu]

Hi,

>From what you described, does the key have the same type before and after
the change? and for using Lambda expression, maybe the doc[1] can be
helpful(especially the limitation)

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/java_lambdas.html#examples-and-limitations
Best,
Congxian


Andrea Gallina  于2019年8月12日周一 下午7:45写道：

> Hi everyone,
>
> I have a job running in production whose structure is approximately this;
>
> stream
>  ?? .filter(inboundData -> inboundData.hasToBeFiltered())
>  ?? .keyBy("myKey")
>  ?? .process(doSomething());
>
> I've recently decided to test the extent to which I can change a job's
> structure without breaking backward compatibility; more specifically,
> I've tried to change the way the key is defined in the keyBy() operator
> by defining it as a lambda function rather than by field expression. The
> modified structure would therefore look like this:
>
> stream
>  ?? .filter(inboundData -> inboundData.hasToBeFiltered())
>  ?? .keyBy(inboundData -> inboundData.getMyKey())
>  ?? .process(doSomething());
>
> I then tried to run the new job by restoring the savepoint taken with
> the old structure, but I get a state migration exception:
>
> org.apache.flink.util.StateMigrationException: The new key serializer
> must be compatible
>
> Now this was a bit unexpected since changing the way a key is defined
> does not seem like a breaking change (unlike changing the actual key
> used for partitioning).
>
> Is this an expected behavior or am I missing something?
>
> Thanks
>
>
> ---
> Questa e-mail ? stata controllata per individuare virus con Avast
> antivirus.
> https://www.avast.com/antivirus
>
>

Flink metrics scope for YARN single job

[Vasily Melnik]

Hi,
I want to run Flink apps on YARN in single job mode and keep metrics in
Graphite. But as i see, the only variable i can use for JobManager scope
customization is :
https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#list-of-all-variables

Is there any way to distinguish logs from different JobManager running on
same host?


Thanks in advance.

Re: Making broadcast state queryable?

[Yu Li]

Good to know your thoughts and the coming talk in Flink Forward Berlin
Oytun, interesting topic and great job! And it's great to hear the voice
from application perspective.

Could you share, if possible, the reason why you need to open the state to
outside instead of writing the result to sink and directly query there? In
another thread there's a case that sink writes to different kafka topics so
state is the only place to get a global view, is this the same case you're
facing? Or some different requirements? I believe more attention will be
drawn to QS if more and more user requirements emerge (smile).

Thanks.

Best Regards,
Yu


On Wed, 14 Aug 2019 at 00:50, Oytun Tez  wrote:

> Thank you for the honest response, Yu!
>
> There is so much that comes to mind when we look at Flink as a
> "application framework" (my talk
> 
> in Flink Forward in Berlin will be about this). QS is one of them
> (need-wise, not QS itself necessarily). I opened the design doc Vino Yang
> created, I'll try to contribute to that.
>
> Meanwhile, for the sake of opening our state to outside, we will put a
> stupid simple operator in between to keep a *duplicate* of the state...
>
> Thanks again!
>
>
>
>
>
> ---
> Oytun Tez
>
> *M O T A W O R D*
> The World's Fastest Human Translation Platform.
> oy...@motaword.com — www.motaword.com
>
>
> On Tue, Aug 13, 2019 at 6:29 PM Yu Li  wrote:
>
>> Hi Oytun,
>>
>> Sorry but TBH such support will probably not be added in the foreseeable
>> future due to lack of committer bandwidth (not only support queryable
>> broadcast state but all about QueryableState module) as pointed out in
>> other threads [1] [2].
>>
>> However, I think you could open a JIRA for this so when things changed it
>> could be taken care of. Thanks.
>>
>> [1] https://s.apache.org/MaOl
>> [2] https://s.apache.org/r8k8a
>>
>> Best Regards,
>> Yu
>>
>>
>> On Tue, 13 Aug 2019 at 20:34, Oytun Tez  wrote:
>>
>>> Hi there,
>>>
>>> Can we set a broadcast state as queryable? I've looked around, not much
>>> to find about it. I am receiving UnknownKvStateLocation when I try to query
>>> with the descriptor/state name I give to the broadcast state.
>>>
>>> If it doesn't work, what could be the alternative? My mind goes around
>>> ctx.getBroadcastState and making it queryable in the operator level (I'd
>>> rather not).
>>>
>>> ---
>>> Oytun Tez
>>>
>>> *M O T A W O R D*
>>> The World's Fastest Human Translation Platform.
>>> oy...@motaword.com — www.motaword.com
>>>
>>

Re: Implementing a low level join

[Felipe Gutierrez]

Hi Hequn,

I am implementing the broadcast and the regular join. As you said I need
different functions. My question is more about if I can have an operator
which decides beteween broadcast and regular join dynamically. I suppose I
will have to extend the generic TwoInputStreamOperator in Flink. Do you
have any suggestion?

Thanks

On Wed, 14 Aug 2019, 03:59 Hequn Cheng,  wrote:

> Hi Felipe,
>
> > I want to implement a join operator which can use different strategies
> for joining tuples.
> Not all kinds of join strategies can be applied to streaming jobs. Take
> sort-merge join as an example, it's impossible to sort an unbounded data.
> However, you can perform a window join and use the sort-merge strategy to
> join the data within a window. Even though, I'm not sure it's worth to do
> it considering the performance.
>
> > Therefore, I am not sure if I will need to implement my own operator to
> do this or if it is still possible to do with CoProcessFunction.
> You can't implement broadcast join with CoProcessFunction. But you can
> implement it with BroadcastProcessFunction or
> KeyedBroadcastProcessFunction, more details here[1].
>
> Furthermore, you can take a look at the implementation of both window join
> and non-window join in Table API & SQL[2]. The code can be found here[3].
> Hope this helps.
>
> Best, Hequn
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html
> [2]
> https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/table/sql.html#joins
> [3]
> https://github.com/apache/flink/tree/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/runtime/join
>
>
> On Tue, Aug 13, 2019 at 11:30 PM Felipe Gutierrez <
> felipe.o.gutier...@gmail.com> wrote:
>
>> Hi all,
>>
>> I want to implement a join operator which can use different strategies
>> for joining tuples. I saw that with CoProcessFunction I am able to
>> implement low-level joins [1]. However, I do know how to decide between
>> different algorithms to join my tuples.
>>
>> On the other hand, to do a broadcast join I will need to use the
>> broadcast operator [2] which yields a BroadcastStream. Therefore, I am not
>> sure if I will need to implement my own operator to do this or if it is
>> still possible to do with CoProcessFunction.
>>
>> Does anyone have some clues for this matter?
>> Thanks
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/stream/operators/process_function.html#low-level-joins
>> [2]
>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html
>> *--*
>> *-- Felipe Gutierrez*
>>
>> *-- skype: felipe.o.gutierrez*
>> *--* *https://felipeogutierrez.blogspot.com
>> *
>>
>

Re: How many task managers can Flink efficiently scale to?

[Chad Dombrova]

Thanks for the info!  It's very helpful.

-chad


On Sun, Aug 11, 2019 at 4:21 AM Zhu Zhu  wrote:

> Hi Chad,
>
> We have (Blink) jobs each running with over 10 thousands of TMs.
> In our experience, the main regression caused by large scale TMs is the in
> TM allocation stage in ResourceManager, that some times it fails to
> allocate enough TMs before the allocation timeout.
> It does not deteriorate much once the Flink cluster has reached a stable
> state.
>
> The main loads, In my mind, increases with the task scale and edge scale
> of a submitted job.
> JM can be overwhelmed by frequent and slow GCs caused by task deployment
> if the JM memory is not fine tuned.
> The JM can also be slower due to more PRCs to JM main thread and increased
> computation complexity of each RPC handling.
>
> Thanks,
> Zhu Zhu
>
> qi luo  于2019年8月11日周日 下午6:17写道：
>
>> Hi Chad,
>>
>> In our cases, 1~2k TMs with up to ~10k TM slots are used in one job. In
>> general, the CPU/memory of Job Manager should be increased with more TMs.
>>
>> Regards,
>> Qi
>>
>> > On Aug 11, 2019, at 2:03 AM, Chad Dombrova  wrote:
>> >
>> > Hi,
>> > I'm still on my task management investigation, and I'm curious to know
>> how many task managers people are reliably using with Flink.  We're
>> currently using AWS | Thinkbox Deadline, and we're able to easily utilize
>> over 300 workers, and I've heard from other customers who use several
>> thousand, so I'm curious how Flink compares in this regard.  Also, what
>> aspects of the system begin to deteriorate at higher scales?
>> >
>> > thanks in advance!
>> >
>> > -chad
>> >
>>
>>

Path Style Access for S3 compliant object stores

[Achyuth Narayan Samudrala]

Hi user group,

I am trying to use Flink to write to an S3 object store. I am using the
flink-s3-fs-hadoop as a filesystem implementation to interact with this
store. How do I provide the s3 property to enable path style access instead
of virtual host addressing?

I tried looking around in the documentation but was unable to find anything
about it. Can someone help with this?

Best regards,
Achyuth