Re: Ratelimiting in the Flink Kafka connector

[Becket Qin]

Hi Thomas,

Good point about counting bytes. It would be difficult to throttle the byte
rate with the existing API. And it seems that for sinks we have to do that
rate limiting in the sink implementation anyways. There are a few ways to
do some abstraction, but maybe adding a RateLimiter is trivial enough so we
don't need to worry about reusing the throttling logic.

But in any case, let's make sure the throttling threshold configuration
names are the same for all the Source and Sinks. So the config parsing
logic should probably still be put together in place. That is probably some
implementation details we can discuss when review the patch.

I am not sure about adding another throttling operator. How would that
operator get the serialized size if it is downstream of a source. And how
would that work on the sink side?

Thanks,

Jiangjie (Becket) Qin








On Fri, Feb 1, 2019 at 11:33 AM Thomas Weise  wrote:

> I initially thought of an approach similar to the collector idea, by
> overriding emitRecord in the fetcher. That makes counting the bytes
> difficult, because it's downstream of decoding.
>
> Another idea of solving this in a reusable way was to have a separate rate
> limiting operator chained downstream of the consumer, which would develop
> back pressure and slow down the consumer. However, that would interfere
> with checkpoint barrier alignment (AFAIK, currently checkpoint barrier will
> also be stuck in the backlog)?
>
> Thomas
>
>
>
> On Thu, Jan 31, 2019 at 7:13 PM Ken Krugler 
> wrote:
>
> > +1, and something I was planning to comment on in the Jira issue.
> >
> > Also, if rate limiting could effectively stop the stream, then this could
> > be used solve a common data enrichment issue.
> >
> > Logically you want to pause one stream (typically the time series data
> > being processed) while another stream (typically the broadcast) is
> > broadcasting an update to enrichment data.
> >
> > Currently you have to buffer the time series data in your enrichment
> > function, but if the rate limiter was pluggable, it could detect when
> this
> > enrichment update was happening.
> >
> >  — Ken
> >
> > > On Jan 31, 2019, at 6:10 PM, Becket Qin  wrote:
> > >
> > > Hi Jamie,
> > >
> > > Thanks for the explanation. That makes sense to me. I am wondering if
> > there
> > > is a more general way to add a rate limiter to all the connecters
> rather
> > > than doing that for each individual one. For example, maybe we can have
> > the
> > > rate limiting logic in the Collector / Output, thus all the connectors
> > > (even operators?) could be rate limited.
> > >
> > > Thanks,
> > >
> > > Jiangjie (Becket) Qin
> > >
> > > On Fri, Feb 1, 2019 at 7:23 AM Lakshmi Gururaja Rao
> > 
> > > wrote:
> > >
> > >> Thanks for adding more context @Jamie Grier  .
> > >>
> > >> JIRA for this feature:
> > https://issues.apache.org/jira/browse/FLINK-11501.
> > >>
> > >> Thanks
> > >> Lakshmi
> > >>
> > >> On Thu, Jan 31, 2019 at 3:20 PM Thomas Weise  wrote:
> > >>
> > >>> I think it would be reasonable to have a rate limiter option in the
> > >>> consumer, given that others have also looked to solve this.
> > >>>
> > >>> I think for this and other optional features, it would be good to
> > >> implement
> > >>> in a way that overrides are possible. Someone else may want to do the
> > >>> limiting differently, taking into account more/other factors.
> > >>>
> > >>> Both, adding the limiter and making the consumer code more adoptable
> > >> could
> > >>> be split into separate work also.
> > >>>
> > >>> BTW is there a JIRA for this?
> > >>>
> > >>> Thomas
> > >>>
> > >>
> > >>
> > >> --
> > >> *Lakshmi Gururaja Rao*
> > >> SWE
> > >> 217.778.7218 <+12177787218>
> > >> [image: Lyft] 
> > >>
> >
> > --
> > Ken Krugler
> > +1 530-210-6378
> > http://www.scaleunlimited.com
> > Custom big data solutions & training
> > Flink, Solr, Hadoop, Cascading & Cassandra
> >
> >
>

[jira] [Created] (FLINK-11507) Remove invalid test JobClientActorTest

[TisonKun (JIRA)]

TisonKun created FLINK-11507:


 Summary: Remove invalid test JobClientActorTest
 Key: FLINK-11507
 URL: https://issues.apache.org/jira/browse/FLINK-11507
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Affects Versions: 1.8.0
Reporter: TisonKun
Assignee: TisonKun
 Fix For: 1.8.0


As discussed in FLINK-11146, {{JobClient}} and {{JobClientActor}} is a static 
helper class as the bridge between the non-actor code and the {{JobManager}}. 
In FLIP-6 codebase we will finally converge to a {{NewClusterClient}}. Thus 
this test is invalid.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11509) Remove invalid test ClientConnectionTest#testJobManagerRetrievalWithHAServices

[TisonKun (JIRA)]

TisonKun created FLINK-11509:


 Summary: Remove invalid test 
ClientConnectionTest#testJobManagerRetrievalWithHAServices
 Key: FLINK-11509
 URL: https://issues.apache.org/jira/browse/FLINK-11509
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Affects Versions: 1.8.0
Reporter: TisonKun
Assignee: TisonKun
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11508) Remove invalid test AkkaJobManagerRetrieverTest

[TisonKun (JIRA)]

TisonKun created FLINK-11508:


 Summary: Remove invalid test AkkaJobManagerRetrieverTest
 Key: FLINK-11508
 URL: https://issues.apache.org/jira/browse/FLINK-11508
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Affects Versions: 1.8.0
Reporter: TisonKun
Assignee: TisonKun
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11506) 通过sql-client.sh登录后，执行sql语句报IO异常，然后客户端中断退出

[Chen Zun (JIRA)]

Chen Zun created FLINK-11506:


 Summary: 通过sql-client.sh登录后，执行sql语句报IO异常，然后客户端中断退出
 Key: FLINK-11506
 URL: https://issues.apache.org/jira/browse/FLINK-11506
 Project: Flink
  Issue Type: Bug
  Components: Table API  SQL
Affects Versions: 1.7.1
 Environment: windows7
Reporter: Chen Zun


{code:java}

Exception in thread "main" org.apache.flink.table.client.SqlClientException: 
Unexpected exception. This is a bug. Please consider filing an issue.
at org.apache.flink.table.client.SqlClient.main(SqlClient.java:199)
Caused by: java.io.IOError: java.io.InterruptedIOException: Command interrupted
at 
org.jline.terminal.impl.AbstractPosixTerminal.setAttributes(AbstractPosixTerminal.java:54)
at org.apache.flink.table.client.cli.CliView.restoreTerminal(CliView.java:398)
at org.apache.flink.table.client.cli.CliView.open(CliView.java:143)
at org.apache.flink.table.client.cli.CliClient.callSelect(CliClient.java:401)
at org.apache.flink.table.client.cli.CliClient.callCommand(CliClient.java:261)
at java.util.Optional.ifPresent(Optional.java:159)
at org.apache.flink.table.client.cli.CliClient.open(CliClient.java:193)
at org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:121)
at org.apache.flink.table.client.SqlClient.start(SqlClient.java:105)
at org.apache.flink.table.client.SqlClient.main(SqlClient.java:187)
Caused by: java.io.InterruptedIOException: Command interrupted
at org.jline.utils.ExecHelper.exec(ExecHelper.java:46)
at org.jline.terminal.impl.ExecPty.doGetConfig(ExecPty.java:175)
at org.jline.terminal.impl.ExecPty.getAttr(ExecPty.java:87)
at org.jline.terminal.impl.ExecPty.doSetAttr(ExecPty.java:93)
at org.jline.terminal.impl.AbstractPty.setAttr(AbstractPty.java:21)


{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11505) Remove invalid test JobManagerRegistrationTest

[TisonKun (JIRA)]

TisonKun created FLINK-11505:


 Summary: Remove invalid test JobManagerRegistrationTest
 Key: FLINK-11505
 URL: https://issues.apache.org/jira/browse/FLINK-11505
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Affects Versions: 1.8.0
Reporter: TisonKun
Assignee: TisonKun
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11504) Remove invalid test JobManagerConnectionTest

[TisonKun (JIRA)]

TisonKun created FLINK-11504:


 Summary: Remove invalid test JobManagerConnectionTest
 Key: FLINK-11504
 URL: https://issues.apache.org/jira/browse/FLINK-11504
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Affects Versions: 1.8.0
Reporter: TisonKun
Assignee: TisonKun
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11503) Remove invalid test TaskManagerLossFailsTasksTest

[TisonKun (JIRA)]

TisonKun created FLINK-11503:


 Summary: Remove invalid test TaskManagerLossFailsTasksTest
 Key: FLINK-11503
 URL: https://issues.apache.org/jira/browse/FLINK-11503
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Affects Versions: 1.8.0
Reporter: TisonKun
Assignee: TisonKun
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11502) Remove invalid test FlinkActorTest

[TisonKun (JIRA)]

TisonKun created FLINK-11502:


 Summary: Remove invalid test FlinkActorTest
 Key: FLINK-11502
 URL: https://issues.apache.org/jira/browse/FLINK-11502
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Affects Versions: 1.8.0
Reporter: TisonKun
Assignee: TisonKun
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ratelimiting in the Flink Kafka connector

[Thomas Weise]

I initially thought of an approach similar to the collector idea, by
overriding emitRecord in the fetcher. That makes counting the bytes
difficult, because it's downstream of decoding.

Another idea of solving this in a reusable way was to have a separate rate
limiting operator chained downstream of the consumer, which would develop
back pressure and slow down the consumer. However, that would interfere
with checkpoint barrier alignment (AFAIK, currently checkpoint barrier will
also be stuck in the backlog)?

Thomas



On Thu, Jan 31, 2019 at 7:13 PM Ken Krugler 
wrote:

> +1, and something I was planning to comment on in the Jira issue.
>
> Also, if rate limiting could effectively stop the stream, then this could
> be used solve a common data enrichment issue.
>
> Logically you want to pause one stream (typically the time series data
> being processed) while another stream (typically the broadcast) is
> broadcasting an update to enrichment data.
>
> Currently you have to buffer the time series data in your enrichment
> function, but if the rate limiter was pluggable, it could detect when this
> enrichment update was happening.
>
>  — Ken
>
> > On Jan 31, 2019, at 6:10 PM, Becket Qin  wrote:
> >
> > Hi Jamie,
> >
> > Thanks for the explanation. That makes sense to me. I am wondering if
> there
> > is a more general way to add a rate limiter to all the connecters rather
> > than doing that for each individual one. For example, maybe we can have
> the
> > rate limiting logic in the Collector / Output, thus all the connectors
> > (even operators?) could be rate limited.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On Fri, Feb 1, 2019 at 7:23 AM Lakshmi Gururaja Rao
> 
> > wrote:
> >
> >> Thanks for adding more context @Jamie Grier  .
> >>
> >> JIRA for this feature:
> https://issues.apache.org/jira/browse/FLINK-11501.
> >>
> >> Thanks
> >> Lakshmi
> >>
> >> On Thu, Jan 31, 2019 at 3:20 PM Thomas Weise  wrote:
> >>
> >>> I think it would be reasonable to have a rate limiter option in the
> >>> consumer, given that others have also looked to solve this.
> >>>
> >>> I think for this and other optional features, it would be good to
> >> implement
> >>> in a way that overrides are possible. Someone else may want to do the
> >>> limiting differently, taking into account more/other factors.
> >>>
> >>> Both, adding the limiter and making the consumer code more adoptable
> >> could
> >>> be split into separate work also.
> >>>
> >>> BTW is there a JIRA for this?
> >>>
> >>> Thomas
> >>>
> >>
> >>
> >> --
> >> *Lakshmi Gururaja Rao*
> >> SWE
> >> 217.778.7218 <+12177787218>
> >> [image: Lyft] 
> >>
>
> --
> Ken Krugler
> +1 530-210-6378
> http://www.scaleunlimited.com
> Custom big data solutions & training
> Flink, Solr, Hadoop, Cascading & Cassandra
>
>

Re: Ratelimiting in the Flink Kafka connector

[Ken Krugler]

+1, and something I was planning to comment on in the Jira issue.

Also, if rate limiting could effectively stop the stream, then this could be 
used solve a common data enrichment issue.

Logically you want to pause one stream (typically the time series data being 
processed) while another stream (typically the broadcast) is broadcasting an 
update to enrichment data.

Currently you have to buffer the time series data in your enrichment function, 
but if the rate limiter was pluggable, it could detect when this enrichment 
update was happening.

 — Ken

> On Jan 31, 2019, at 6:10 PM, Becket Qin  wrote:
> 
> Hi Jamie,
> 
> Thanks for the explanation. That makes sense to me. I am wondering if there
> is a more general way to add a rate limiter to all the connecters rather
> than doing that for each individual one. For example, maybe we can have the
> rate limiting logic in the Collector / Output, thus all the connectors
> (even operators?) could be rate limited.
> 
> Thanks,
> 
> Jiangjie (Becket) Qin
> 
> On Fri, Feb 1, 2019 at 7:23 AM Lakshmi Gururaja Rao 
> wrote:
> 
>> Thanks for adding more context @Jamie Grier  .
>> 
>> JIRA for this feature: https://issues.apache.org/jira/browse/FLINK-11501.
>> 
>> Thanks
>> Lakshmi
>> 
>> On Thu, Jan 31, 2019 at 3:20 PM Thomas Weise  wrote:
>> 
>>> I think it would be reasonable to have a rate limiter option in the
>>> consumer, given that others have also looked to solve this.
>>> 
>>> I think for this and other optional features, it would be good to
>> implement
>>> in a way that overrides are possible. Someone else may want to do the
>>> limiting differently, taking into account more/other factors.
>>> 
>>> Both, adding the limiter and making the consumer code more adoptable
>> could
>>> be split into separate work also.
>>> 
>>> BTW is there a JIRA for this?
>>> 
>>> Thomas
>>> 
>> 
>> 
>> --
>> *Lakshmi Gururaja Rao*
>> SWE
>> 217.778.7218 <+12177787218>
>> [image: Lyft] 
>> 

--
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
Custom big data solutions & training
Flink, Solr, Hadoop, Cascading & Cassandra

Re: [DISCUSS] Standard / Convention for common connector metrics

[Becket Qin]

Thanks for the connector metric url,  Chesnay :)

@Robert, as you can see, the metrics from different connectors are quite
different. And there are different names for similar metrics, which is a
little frustrating when users want to do monitoring / alerting.

Thanks,

Jiangjie (Becket) Qin

On Thu, Jan 31, 2019 at 6:41 PM Chesnay Schepler  wrote:

> @Robert:
>
> https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html#connectors
>
> On 31.01.2019 11:03, Robert Metzger wrote:
> > Hey Becket,
> > thanks a lot for your proposal!
> >
> > Do you have an overview over the current situation of the metrics in the
> > connectors?
> > Which connectors expose metrics at all?
> > Are they different?
> >
> > On Thu, Jan 31, 2019 at 8:44 AM Becket Qin  wrote:
> >
> >> Hi folks,
> >>
> >> I was trying to add some metrics to Kafka connectors and realized that
> >> right now Flink does not have a common metric definition for the
> >> connectors. This complicates the monitoring and operation because the
> >> monitoring / alerts need to be set case by case.
> >>
> >> To address this issue, I would like to see if is possible to have a set
> of
> >> standardized common metrics for all sources and sinks. The following doc
> >> describes the proposal. Feedback is very welcome.
> >>
> >>
> >>
> https://docs.google.com/document/d/1q86bgj_3T6WFbSUoxLDJJXmUcBOUcvWfh2RZvHG-nPU/edit#
> >>
> >> Thanks,
> >>
> >> Jiangjie (Becket) Qin
> >>
>
>

Re: Request contributor permission

[Becket Qin]

Thanks Robert.

Sorry I forgot to mention this, but yes, that is my JIRA id :)

And my Apache confluence ID is also becket_ qin (becket@gmail.com).
Could you also help me with the FLIP permissions?

Thanks,

Jiangjie (Becket) Qin

On Thu, Jan 31, 2019 at 5:56 PM Robert Metzger  wrote:

> Hey Becket,
>
> I gave you contributor permissions in our JIRA. (Assuming your JIRA id is
> "becket_qin").
>
> On Thu, Jan 31, 2019 at 10:37 AM Becket Qin  wrote:
>
> > Hi,
> >
> > Could someone help me with getting the contributor's permission so I can
> > create FLIPs and assign JIRA tickets?
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
>

Re: Ratelimiting in the Flink Kafka connector

[Becket Qin]

Hi Jamie,

Thanks for the explanation. That makes sense to me. I am wondering if there
is a more general way to add a rate limiter to all the connecters rather
than doing that for each individual one. For example, maybe we can have the
rate limiting logic in the Collector / Output, thus all the connectors
(even operators?) could be rate limited.

Thanks,

Jiangjie (Becket) Qin

On Fri, Feb 1, 2019 at 7:23 AM Lakshmi Gururaja Rao 
wrote:

> Thanks for adding more context @Jamie Grier  .
>
> JIRA for this feature: https://issues.apache.org/jira/browse/FLINK-11501.
>
> Thanks
> Lakshmi
>
> On Thu, Jan 31, 2019 at 3:20 PM Thomas Weise  wrote:
>
> > I think it would be reasonable to have a rate limiter option in the
> > consumer, given that others have also looked to solve this.
> >
> > I think for this and other optional features, it would be good to
> implement
> > in a way that overrides are possible. Someone else may want to do the
> > limiting differently, taking into account more/other factors.
> >
> > Both, adding the limiter and making the consumer code more adoptable
> could
> > be split into separate work also.
> >
> > BTW is there a JIRA for this?
> >
> > Thomas
> >
>
>
> --
> *Lakshmi Gururaja Rao*
> SWE
> 217.778.7218 <+12177787218>
> [image: Lyft] 
>

[jira] [Created] (FLINK-11501) Add a ratelimiting feature to the FlinkKafkaConsumer

[Lakshmi Rao (JIRA)]

Lakshmi Rao created FLINK-11501:
---

 Summary: Add a ratelimiting feature to the FlinkKafkaConsumer
 Key: FLINK-11501
 URL: https://issues.apache.org/jira/browse/FLINK-11501
 Project: Flink
  Issue Type: Improvement
  Components: Kafka Connector
Reporter: Lakshmi Rao
Assignee: Lakshmi Rao


There are instances when a Flink job that reads from Kafka can read at a 
significantly high throughput (particularly while processing a backlog) and 
degrade the underlying Kafka cluster.

While Kafka quotas are perhaps the best way to enforce this ratelimiting, there 
are cases where such a setup is not available or easily enabled. 

The approach is essentially to use Guava's rate limiter to ratelimit the bytes 
read from Kafka.

More discussion here: 
https://lists.apache.org/thread.html/8140b759ba83f33a22d809887fd2d711f5ffe7069c888eb9b1142272@%3Cdev.flink.apache.org%3E
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ratelimiting in the Flink Kafka connector

[Lakshmi Gururaja Rao]

Thanks for adding more context @Jamie Grier  .

JIRA for this feature: https://issues.apache.org/jira/browse/FLINK-11501.

Thanks
Lakshmi

On Thu, Jan 31, 2019 at 3:20 PM Thomas Weise  wrote:

> I think it would be reasonable to have a rate limiter option in the
> consumer, given that others have also looked to solve this.
>
> I think for this and other optional features, it would be good to implement
> in a way that overrides are possible. Someone else may want to do the
> limiting differently, taking into account more/other factors.
>
> Both, adding the limiter and making the consumer code more adoptable could
> be split into separate work also.
>
> BTW is there a JIRA for this?
>
> Thomas
>


-- 
*Lakshmi Gururaja Rao*
SWE
217.778.7218 <+12177787218>
[image: Lyft] 

Re: Ratelimiting in the Flink Kafka connector

[Thomas Weise]

I think it would be reasonable to have a rate limiter option in the
consumer, given that others have also looked to solve this.

I think for this and other optional features, it would be good to implement
in a way that overrides are possible. Someone else may want to do the
limiting differently, taking into account more/other factors.

Both, adding the limiter and making the consumer code more adoptable could
be split into separate work also.

BTW is there a JIRA for this?

Thomas

Re: Ratelimiting in the Flink Kafka connector

[Jamie Grier]

I had the same reaction initially as some of the others on this thread --
which is "Use Kafka quotas"..  I agree that in general a service should
protect itself with it's own rate limiting rather than building it into
clients like the FlinkKafkaConsumer.

However, there are a few reasons we need to do this in our company
currently:
  - We can't use Kafka quotas right now because the Kafka vendor we're
using doesn't support them
  - Flink jobs that also make calls to RPC services are frequently DDOS'd
by Flink apps and we simply need to slow them down when processing a
backlog to protect external services.  You could argue those services
should protect themselve, and I agree, but for various technical reasons
that's not possible ATM.
  - If you are going to artificially rate limit a Flink job the best place
to do it is definitely in the source -- otherwise you end up with issues
with backpressure and checkpointing.

So, that said I suspect other users have the same issue so I think it's a
good general feature to add to the Kafka consumer.  It already exists in
the Kinesis consumer as well.

In terms of code bloat -- well the code is dead simple.  It's just adding a
Guava RateLimiter to the poll() loop and it's opt-in.  The code has already
been implemented for this.

@Lakshmi Gururaja Rao   Can you put up a apache/flink PR for
this since it's already finished internally?

Anyway, I'm not opposed to making the KafkaConsumer a little more
customizable via adding some hooks if that's what others prefer -- however,
let's also make the rate limited KafkaConsumer available in the Flink
project at large rather than keeping it internal at Lyft.  I think it's
generally useful.

-Jamie


On Tue, Jan 29, 2019 at 8:57 PM Thomas Weise  wrote:

> It is preferred for the service to rate limit. The problem is that not all
> Kafka setups have that control enabled / support for it.
>
> Even when rate limiting was enabled, it may still be *nice* for the client
> to gracefully handle it.
>
> There was discussion in the past that we should not bloat the Kafka
> consumer further and I agree with that.
>
> On the other hand it would be good if the consumer can be augmented a bit
> to provide hooks for customization (we had done that for the Kinesis
> consumer also).
>
> Thanks,
> Thomas
>
>
> On Mon, Jan 28, 2019 at 3:14 AM Becket Qin  wrote:
>
> > Hi Lakshmi，
> >
> > As Nagajun mentioned, you might want to configure quota on the Kafka
> broker
> > side for your Flink connector client.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On Sat, Jan 26, 2019 at 10:44 AM Ning Shi  wrote:
> >
> > > > We have a Flink job reading from Kafka (specifically it uses
> > > > FlinkKafkaConsumer011). There are instances when the job is
> processing
> > a
> > > > backlog and it ends up reading at a significantly high throughput and
> > > > degrades the underlying Kafka cluster. If there was a way to rate
> limit
> > > the
> > > > calls to Kafka (by controlling how often the *consumer*.poll() is
> > > called),
> > > > it would be a useful feature for our use case.
> > > >
> > > > Has anyone has run into a similar issue? Are there are any
> > > efforts/thoughts
> > > > on implementing a rate-limiting feature in the Flink Kafka connector?
> > >
> > > We has similar problem and ended up putting a Guava rate limiter
> > > inside the Kafka consumer to limit the consumption rate. Since we use
> > > POJO, this is easily done by putting the rate limiter inside the POJO
> > > deserializer, which runs in the Kafka source.
> > >
> > > This has the benefit of not slowing down checkpoints because the
> > > source doesn't have to do alignment. If you don't care about
> > > checkpoint alignment, you can also add a map function with a Guava
> > > rate limiter immediately after the Kafka source. When it throttles,
> > > back pressure should eventually cause the Kafka source to slowdown
> > > consumption.
> > >
> > > Ning
> > >
> >
>

[jira] [Created] (FLINK-11500) 1.8

[Chesnay Schepler (JIRA)]

Chesnay Schepler created FLINK-11500:


 Summary: 1.8
 Key: FLINK-11500
 URL: https://issues.apache.org/jira/browse/FLINK-11500
 Project: Flink
  Issue Type: Bug
Reporter: Chesnay Schepler






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11499) Extend StreamingFileSink BulkFormats to support arbitrary roll policy's

[Seth Wiesman (JIRA)]

Seth Wiesman created FLINK-11499:


 Summary: Extend StreamingFileSink BulkFormats to support arbitrary 
roll policy's 
 Key: FLINK-11499
 URL: https://issues.apache.org/jira/browse/FLINK-11499
 Project: Flink
  Issue Type: Improvement
Reporter: Seth Wiesman


Currently when using the StreamingFilleSink Bulk-encoding formats can only be 
combined with the `OnCheckpointRollingPolicy`, which rolls the in-progress part 
file on every checkpoint.

However, many bulk formats such as parquet are most efficient when written as 
large files; this is not possible when frequent checkpointing is enabled. 
Currently the only work-around is to have long checkpoint intervals which is 
not ideal.

 

The StreamingFileSink should be enhanced to support arbitrary roll policy's so 
users may write large bulk files while retaining frequent checkpoints.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11498) Getting java.lang.InstantiationError when trying to install flink 1.7.1 version through ansible

[sambi reddy (JIRA)]

sambi reddy created FLINK-11498:
---

 Summary: Getting java.lang.InstantiationError when trying to 
install flink 1.7.1 version through ansible
 Key: FLINK-11498
 URL: https://issues.apache.org/jira/browse/FLINK-11498
 Project: Flink
  Issue Type: Bug
Affects Versions: 1.7.1
 Environment: Openstack
Reporter: sambi reddy
 Fix For: 1.7.1


We trying to install flink on our cloud with the help ansible automation. we 
were running 1.3.2 since past two years. using the similar kind of automation 
with few config changes and trying to start the cluster but failing to do so.

Error we are seeing in the logs.
{noformat}
Starting the SlotManager.

Fatal error occurred in the cluster entrypoint.

java.lang.InstantiationError: org.apache.flink.runtime.blob.BlobKey

at sun.reflect.GeneratedSerializationConstructorAccessor43.newInstance(Unknown 
Source)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:1006)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2011)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)

at java.util.ArrayList.readObject(ArrayList.java:791)

at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)

at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2136)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)

at 
org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:524)

at 
org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:503)

at 
org.apache.flink.runtime.state.RetrievableStreamStateHandle.retrieveState(RetrievableStreamStateHandle.java:58)

at 
org.apache.flink.runtime.jobmanager.ZooKeeperSubmittedJobGraphStore.recoverJobGraph(ZooKeeperSubmittedJobGraphStore.java:202)

at 
org.apache.flink.runtime.dispatcher.Dispatcher.recoverJob(Dispatcher.java:696)

at 
org.apache.flink.runtime.dispatcher.Dispatcher.recoverJobGraphs(Dispatcher.java:681)

at 
org.apache.flink.runtime.dispatcher.Dispatcher.recoverJobs(Dispatcher.java:662)

at 
org.apache.flink.runtime.dispatcher.Dispatcher.lambda$null$26(Dispatcher.java:821)

at 
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:72)

at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)

at 
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)

at 
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)

at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)

at 
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)

at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at 
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at 
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Shutting down BLOB cache

Stopped BLOB server at 0.0.0.0:44036{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11497) Remove invalid test JobManagerLeaderElectionTest

[TisonKun (JIRA)]

TisonKun created FLINK-11497:


 Summary: Remove invalid test JobManagerLeaderElectionTest
 Key: FLINK-11497
 URL: https://issues.apache.org/jira/browse/FLINK-11497
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Affects Versions: 1.8.0
Reporter: TisonKun
Assignee: TisonKun
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] Start new Review Process

[Robert Metzger]

Sorry, I haven't been active in the business of merging pull requests
recently. But you are both totally right: There is now a shiny big button
for merging pull requests.

Regarding permissions for the bot: I'm going to ask Infra if the bot can
get permissions to label and merge pull requests.

I have addressed most of Fabian's feature requests. I will now update the
website PR with the review guide and then activate the bot!


On Tue, Jan 29, 2019 at 1:13 PM Chesnay Schepler  wrote:

> I assume he means that bots cannot do it, as they'd require committer
> permissions. Same with assigning reviewers and such.
>
> On 29.01.2019 13:09, Aljoscha Krettek wrote:
> > What do you mean by “merging cannot happen through the GitHub user
> interface”? You can in fact merge PRs by clicking on the merge button, or
> “rebase and merge”.
> >
> > Aljoscha
> >
> >> On 29. Jan 2019, at 11:58, Robert Metzger  wrote:
> >>
> >> @Fabian: Thank you for your suggestions. Multiple approvals in one
> comment
> >> are already possible.
> >> I will see if I can easily add multiple approvals in one line as well.
> >> I will also address 2) and 3).
> >>
> >> Regarding usage of the bot: Anyone can use it! It is up to the committer
> >> who's merging in the end whether they are happy with the approver.
> >> One of the feature ideas I have is to indicate whether somebody is PMC
> or
> >> committer.
> >>
> >> I'm against enforcing the order of approvals for now. I fear that this
> will
> >> make the tool too restrictive. I like Ufuk's idea of putting a note into
> >> the tracking comment for now.
> >> Once it's active and we are using it day to day, we'll probably learn
> what
> >> features we need the most.
> >>
> >>
> >> @Ufuk: I think we should put it into a Apache repo at some point. But
> I'm
> >> not sure if it's worth going through the effort of setting up a new repo
> >> now, given that the bot is not even active yet, and we are not sure if
> it's
> >> going to be useful.
> >> Once it is active for a month or two, I will move it.
> >>
> >> Regarding the bots in general: I don't see a problem with having
> multiple
> >> bots in place, as long as they get along with each other well.
> >> We should try not to reinvent the wheel, if there's already a good bot
> >> implementation, I don't see a reason to not use it.
> >> The problem in our case is that we have limited access to our GitHub
> page,
> >> and merging can not happen through the GitHub user interface -- so I
> guess
> >> many "off the shelf" bots will not work in our environment.
> >> I'm thinking already about approaches how to automatically merge pull
> >> requests with the bot. But this will be a separate mailing list thread
> :)
> >>
> >> Thanks for the feedback!
> >>
> >>
> >>
> >>
> >> On Mon, Jan 28, 2019 at 5:20 PM Ufuk Celebi  wrote:
> >>
> >>> Thanks for the clarification. I agree that it only makes sense to
> >>> check the points in order. +1 to add this if we can think of a nice
> >>> way to do it. I'm not sure how we would enforce the order with the bot
> >>> since there is only indirect feedback to a bot command. The only thing
> >>> I can think of at the moment is to add a note to a check in case
> >>> earlier steps where not executed. Just ignoring a bot command because
> >>> other commands have not been executed before feels not helpful to me
> >>> (since we can't prevent reviewers to jump to later steps if they wish
> >>> to do so).
> >>>
> >>> I'd rather add a bold note to the bot template that makes clear that
> >>> all points should be followed in order to avoid potentially redundant
> >>> work.
> >>>
> >>> – Ufuk
> >>>
> >>> On Mon, Jan 28, 2019 at 5:01 PM Fabian Hueske 
> wrote:
>  The points in the review template are in the order in which they
> should
> >>> be
>  checked, i.e., first checking the description, then consensus and
> finally
>  checking the code.
>  Currently, it is possible to tick off the code box before checking the
>  description.
>  One motivation for the process was to do the steps in exactly the
> >>> proposed
>  order for example to to avoid detailed code reviews before there was
>  consensus whether the contribution was welcome or not.
> 
>  Am Mo., 28. Jan. 2019 um 16:54 Uhr schrieb Ufuk Celebi <
> u...@apache.org>:
> 
> > I played around with the bot and it works pretty well. :-) @Robert:
> > Are there any plans to contribute the code for the bot to Apache
> > (potentially in another repository)?
> >
> > I like Fabians suggestions. Regarding the questions:
> > 1) I would make that dependent on whether you expected the review
> > guideline to only apply to committers or also regular contributors.
> > Since the bot is not merging PRs, I don't see a down side in having
> it
> > open for all contributors (except potential noise).
> > 2) What do you mean with "order of approvals"?
> >
> > Since there is another lively discussion going on 

[jira] [Created] (FLINK-11496) FlinkS3 FileSysten is not handling multiple local temp directories

[Elango Ganesan (JIRA)]

Elango Ganesan created FLINK-11496:
--

 Summary: FlinkS3 FileSysten is not handling multiple local temp 
directories
 Key: FLINK-11496
 URL: https://issues.apache.org/jira/browse/FLINK-11496
 Project: Flink
  Issue Type: Bug
Reporter: Elango Ganesan
 Fix For: 1.7.1


S3 Flink Filesystem when creating localTemp directory is not splitting and 
handling the availability of  multiple local temp directories . As a result we 
are seeing exception mentioned below any time we run in a EC2 instance type 
with more than one ephemeral drive or EBS volume.

 

https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/main/java/org/apache/flink/fs/s3/common/FlinkS3FileSystem.java#L101
 
Timestamp: 2019-01-29, 12:42:39
java.nio.file.NoSuchFileException: 
/mnt/yarn/usercache/hadoop/appcache/application_1548598173158_0004,/mnt1/yarn/usercache/hadoop/appcache/application_1548598173158_0004/.tmp_072167ee-6432-412c-809a-bd0599961cf0
at 
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at 
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at 
java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
at java.nio.file.Files.newOutputStream(Files.java:216)
at 
org.apache.flink.fs.s3.common.utils.RefCountedTmpFileCreator.apply(RefCountedTmpFileCreator.java:80)
at 
org.apache.flink.fs.s3.common.utils.RefCountedTmpFileCreator.apply(RefCountedTmpFileCreator.java:39)
at 
org.apache.flink.fs.s3.common.utils.RefCountedBufferingFileStream.openNew(RefCountedBufferingFileStream.java:174)
at 
org.apache.flink.fs.s3.common.writer.S3RecoverableFsDataOutputStream.boundedBufferingFileStream(S3RecoverableFsDataOutputStream.java:271)
at 
org.apache.flink.fs.s3.common.writer.S3RecoverableFsDataOutputStream.newStream(S3RecoverableFsDataOutputStream.java:236)
at 
org.apache.flink.fs.s3.common.writer.S3RecoverableWriter.open(S3RecoverableWriter.java:78)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11495) Remove legacy job archiving paths

[Chesnay Schepler (JIRA)]

Chesnay Schepler created FLINK-11495:


 Summary: Remove legacy job archiving paths
 Key: FLINK-11495
 URL: https://issues.apache.org/jira/browse/FLINK-11495
 Project: Flink
  Issue Type: Sub-task
  Components: History Server
Reporter: Chesnay Schepler
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11494) Remove legacy WebRuntimeMonitor

[Chesnay Schepler (JIRA)]

Chesnay Schepler created FLINK-11494:


 Summary: Remove legacy WebRuntimeMonitor
 Key: FLINK-11494
 URL: https://issues.apache.org/jira/browse/FLINK-11494
 Project: Flink
  Issue Type: Sub-task
Reporter: Chesnay Schepler






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11493) Finalize the Blink SQL merging efforts

[Timo Walther (JIRA)]

Timo Walther created FLINK-11493:


 Summary: Finalize the Blink SQL merging efforts
 Key: FLINK-11493
 URL: https://issues.apache.org/jira/browse/FLINK-11493
 Project: Flink
  Issue Type: Sub-task
  Components: Table API  SQL
Reporter: Timo Walther


A more detailed description can be found in 
[FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions].

This issue is an umbrella issue for tasks related to the finalization of the 
Blink SQL merging efforts. The goal is to make the separate Blink branch 
obsolete (at least from a Blink SQL perspective).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11492) Support the full Blink SQL runtime

[Timo Walther (JIRA)]

Timo Walther created FLINK-11492:


 Summary: Support the full Blink SQL runtime
 Key: FLINK-11492
 URL: https://issues.apache.org/jira/browse/FLINK-11492
 Project: Flink
  Issue Type: Sub-task
  Components: Table API  SQL
Reporter: Timo Walther


A more detailed description can be found in 
[FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions].

This issue is an umbrella issue for tasks related to the execution of batch and 
streaming operations with a performance that is comparable to the performance 
of Blink's SQL runtime.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11491) Support all TPC-DS queries

[Timo Walther (JIRA)]

Timo Walther created FLINK-11491:


 Summary: Support all TPC-DS queries
 Key: FLINK-11491
 URL: https://issues.apache.org/jira/browse/FLINK-11491
 Project: Flink
  Issue Type: Sub-task
  Components: Table API  SQL
Reporter: Timo Walther


A more detailed description can be found in 
[FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions].

This issue describes the goal of supporting all TPC-DS queries on a unified 
runtime for batch and streaming.

Operations might not be executed with the full performance until changes in 
other Flink core components have taken place.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11490) Add an initial batch runtime

[Timo Walther (JIRA)]

Timo Walther created FLINK-11490:


 Summary: Add an initial batch runtime
 Key: FLINK-11490
 URL: https://issues.apache.org/jira/browse/FLINK-11490
 Project: Flink
  Issue Type: Sub-task
  Components: Table API  SQL
Reporter: Timo Walther


A more detailed description can be found in 
[FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions].

This issue is an umbrella issue for tasks related to the merging of Blink batch 
runtime features. The goal is to provide minimum viable product (MVP) to batch 
users.

An exact list of batch features, their properties, and dependencies needs to be 
defined.

The type system might not have been reworked at this stage. Operations might 
not be executed with the full performance until changes in other Flink core 
components have taken place.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11489) Add an initial Blink streaming runtime

[Timo Walther (JIRA)]

Timo Walther created FLINK-11489:


 Summary: Add an initial Blink streaming runtime
 Key: FLINK-11489
 URL: https://issues.apache.org/jira/browse/FLINK-11489
 Project: Flink
  Issue Type: Sub-task
  Components: Table API  SQL
Reporter: Timo Walther


A more detailed description can be found in 
[FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions].

This issue is an umbrella issue for tasks related to the merging of Blink 
streaming runtime features. The goal is to provide minimum viable product (MVP) 
to streaming users.

An exact list of streaming features, their properties, and dependencies needs 
to be defined.

The type system might not have been reworked at this stage. Operations might 
not be executed with the full performance until changes in other Flink core 
components have taken place.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-11488) Merge a basic Blink planner framework

[Timo Walther (JIRA)]

Timo Walther created FLINK-11488:


 Summary: Merge a basic Blink planner framework
 Key: FLINK-11488
 URL: https://issues.apache.org/jira/browse/FLINK-11488
 Project: Flink
  Issue Type: Sub-task
  Components: Table API  SQL
Reporter: Timo Walther


A more detailed description can be found in 
[FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions].

This issue is an umbrella issue for tasks related to the setup of a basic, 
unified planner framework for introducing new Blink features (in both batch and 
streaming).

It includes tasks such as:
 - Introduction of {{flink-table-planner-blink}} and 
{{flink-table-runtime-blink}} modules
 - Addition of a planner implementation that matches the interfaces introduced 
in FLINK-11452
 - Addition of basic planner components such as
 ** optimizer
 ** statistics
 ** costs
 ** code generator for expressions



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] Standard / Convention for common connector metrics

[Chesnay Schepler]

@Robert: 
https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html#connectors


On 31.01.2019 11:03, Robert Metzger wrote:

Hey Becket,
thanks a lot for your proposal!

Do you have an overview over the current situation of the metrics in the
connectors?
Which connectors expose metrics at all?
Are they different?

On Thu, Jan 31, 2019 at 8:44 AM Becket Qin  wrote:


Hi folks,

I was trying to add some metrics to Kafka connectors and realized that
right now Flink does not have a common metric definition for the
connectors. This complicates the monitoring and operation because the
monitoring / alerts need to be set case by case.

To address this issue, I would like to see if is possible to have a set of
standardized common metrics for all sources and sinks. The following doc
describes the proposal. Feedback is very welcome.


https://docs.google.com/document/d/1q86bgj_3T6WFbSUoxLDJJXmUcBOUcvWfh2RZvHG-nPU/edit#

Thanks,

Jiangjie (Becket) Qin

[jira] [Created] (FLINK-11487) Support for writing data to Apache Flume

[ambition (JIRA)]

ambition created FLINK-11487:


 Summary: Support for writing data to Apache Flume
 Key: FLINK-11487
 URL: https://issues.apache.org/jira/browse/FLINK-11487
 Project: Flink
  Issue Type: New Feature
  Components: Streaming Connectors
Affects Versions: 1.7.1
 Environment: JDK 1.8

Scala 2.11

Flink 1.7.1

Apache Flume 1.6.0
Reporter: ambition
 Fix For: 1.7.2, 1.8.0


Flume is a distributed, reliable, and available service for efficiently 
collecting, aggregating, and moving large amounts of data, has many users. 
Unfortunately, Flink does not currently support  with data to Flume. 

The following is the official website of flume and github source address：

[Apache Flume website|http://flume.apache.org/index.html] 

[Apache Flume github|https://github.com/apache/flume]

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] Standard / Convention for common connector metrics

[Robert Metzger]

Hey Becket,
thanks a lot for your proposal!

Do you have an overview over the current situation of the metrics in the
connectors?
Which connectors expose metrics at all?
Are they different?

On Thu, Jan 31, 2019 at 8:44 AM Becket Qin  wrote:

> Hi folks,
>
> I was trying to add some metrics to Kafka connectors and realized that
> right now Flink does not have a common metric definition for the
> connectors. This complicates the monitoring and operation because the
> monitoring / alerts need to be set case by case.
>
> To address this issue, I would like to see if is possible to have a set of
> standardized common metrics for all sources and sinks. The following doc
> describes the proposal. Feedback is very welcome.
>
>
> https://docs.google.com/document/d/1q86bgj_3T6WFbSUoxLDJJXmUcBOUcvWfh2RZvHG-nPU/edit#
>
> Thanks,
>
> Jiangjie (Becket) Qin
>

Re: Request contributor permission

[Robert Metzger]

Hey Becket,

I gave you contributor permissions in our JIRA. (Assuming your JIRA id is
"becket_qin").

On Thu, Jan 31, 2019 at 10:37 AM Becket Qin  wrote:

> Hi,
>
> Could someone help me with getting the contributor's permission so I can
> create FLIPs and assign JIRA tickets?
>
> Thanks,
>
> Jiangjie (Becket) Qin
>

[jira] [Created] (FLINK-11486) Remove RecoveryITCase for jobmanager

[Shimin Yang (JIRA)]

Shimin Yang created FLINK-11486:
---

 Summary: Remove RecoveryITCase for jobmanager
 Key: FLINK-11486
 URL: https://issues.apache.org/jira/browse/FLINK-11486
 Project: Flink
  Issue Type: Sub-task
  Components: Tests
Reporter: Shimin Yang
Assignee: Shimin Yang


RecoveryITCase under org.apache.flink.runtime.jobmanager package is based on 
legacy mode and should be removed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Request contributor permission

[Becket Qin]

Hi,

Could someone help me with getting the contributor's permission so I can
create FLIPs and assign JIRA tickets?

Thanks,

Jiangjie (Becket) Qin