Re: Side-effects of DataSet::count

2016-05-30 Thread Eron Wright
JobGraph > execution, thus they also trigger writing to any previously defined sinks. > The idea behind this behavior is to enable interactive querying (the one > that you are used to get from a shell environment) and it is also a great > debugging tool. > > Best, > > Marto

Side-effects of DataSet::count

2016-05-29 Thread Eron Wright
I was curious as to how the `count` method on DataSet worked, and was surprised to see that it executes the entire program graph. Wouldn’t this cause undesirable side-effects like writing to sinks?Also strange that the graph is mutated with the addition of a sink (that isn’t subsequently

RE: Kerberos for Streaming & Kafka

2016-03-25 Thread Eron Wright
streaming job to survive, the original keytab is needed to re-authenticate. Spark Streaming solved this problem with `--keytab` on spark-submit (see AMDelegationTokenRenewer.scala). 2. Kafka doesn't support delegation tokens yet (see KIP-48 and KAFKA-1696). Thoughts? Thanks! - Eron Wright

Kerberos for Streaming & Kafka

2016-03-24 Thread Eron Wright
, the original keytab is needed to re-authenticate. Spark Streaming solved this problem with `--keytab` on spark-submit (see AMDelegationTokenRenewer.scala).Kafka doesn't support delegation tokens yet (see KIP-48 and KAFKA-1696). Thoughts? Thanks! - Eron Wright

RE: Data locality and scheduler

2016-04-30 Thread Eron Wright
My understanding of Flink on YARN is that the task managers are eagerly acquired when the session is started, before any job information (which could be mined for locality hints) is available. This makes sense because the session could be reused for other jobs. But this approach leads to

RE: Surprising order of events in union of two streams

2016-04-18 Thread Eron Wright
-scala/src/main/scala/org/numenta/nupic/flink/streaming/connectors/river/RiverSource.scala#L157 -Eron Wright > From: aljos...@apache.org > Date: Mon, 18 Apr 2016 08:20:11 + > Subject: Re: Surprising order of events in union of two streams > To: dev@flink.apache.org > CC: namluc.t

[DISCUSS] Mesos version dependency

2016-10-03 Thread Eron Wright
Hello, For Flink's initial support for Mesos, coming in Flink 1.2, does anyone have an objection to making the minimum requirement be Mesos 1.0+? There are some functional benefits to 1.0 and it would simplify the test effort. ThanksEron Wright

Re: [jira] [Created] (FLINK-4898) Refactor HTTP handlers and Netty server/client

2016-11-07 Thread Eron Wright
pipeline. Eron > On Oct 24, 2016, at 4:16 PM, Eron Wright (JIRA) <j...@apache.org> wrote: > > Eron Wright created FLINK-4898: > --- > > Summary: Refactor HTTP handlers and Netty server/client > Key: FLINK

Re: CheckpointedFunction applicability?

2017-01-05 Thread Eron Wright
lbacks on CheckpointedFunction. This is not the intended design and should be added. Best, Stefan > Am 05.01.2017 um 04:40 schrieb Eron Wright <ewri...@live.com>: > > Hello! > > The `CheckpointedFunction` interface was introduced to provide user functions > with a context

CheckpointedFunction applicability?

2017-01-04 Thread Eron Wright
Hello! The `CheckpointedFunction` interface was introduced to provide user functions with a context for partitioned state storage. This works for user functions that are provided directly to the operator (assuming it extends from `AbstractUdfStreamOperator`). However, I see that some

Mesos component status for 1.3

2017-04-21 Thread Eron Wright
Hello! Here's a list of related PRs that I'm tracking for 1.3 release: - FLINK-5974 - Mesos-DNS hostname support - FLINK-6336 - Mesos placement constraints - FLINK-5975 - Mesos volume support We need a committer to push these over the finish line. Also, I'd like to nominate myself as a

Re: Class Cache

2017-07-31 Thread Eron Wright
A Flink program is typically packaged as an 'uber-jar' containing its dependencies. The Flink quickstart project illustrates this (see the use of the shading plugin in pom.xml). Based on your description, the classes of mylib2.jar were copied into mylib1.jar when the latter was built. Try

Re: [DISCUSS] Service Authorization (redux)

2017-08-03 Thread Eron Wright
would break security for Scala 2.10. On the other hand > > people are already using Flink without this feature. > > > > Cheers, > > Till > > > > On Wed, Aug 2, 2017 at 7:21 PM, Eron Wright <eronwri...@gmail.com> > wrote: > > > >> Thanks Ti

Re: Using native library in Flink

2017-07-18 Thread Eron Wright
The solution mentioned by Timo works well with a standalone Flink cluster but might not work with a YARN or Mesos cluster. An alternative is to have your Java library contain the native library within itself, and to extract it to a temporary directory before calling `System.loadLibrary(...)`.

Re: Dropping Java 7 support

2017-07-21 Thread Eron Wright
Opened FLINK-7242 as an umbrella issue for this. On Wed, Jul 19, 2017 at 3:09 AM, Chesnay Schepler wrote: > Are the specific things we want to change right away? (build profiles > would be one thing) > > Would be neat to collect them in an umbrella issue. > > > On 18.07.2017

Re: [DISCUSS] A more thorough Pull Request check list and template

2017-07-24 Thread Eron Wright
uests do bring in a good piece of perspective, at > least, even if the code is not immediately suited for contribution... > > > On Mon, Jul 24, 2017 at 8:18 PM, Eron Wright <eronwri...@gmail.com> wrote: > > > This seems like a good step in establishing a better PR process.

[DISCUSS] Service Authorization (redux)

2017-07-23 Thread Eron Wright
Hello, now might be a good time to revisit an important enhancement to Flink security, so-called service authorization. This means the hardening of a Flink cluster against unauthorized use with some sort of authentication and authorization scheme. Today, Flink relies entirely on network

Re: [DISCUSS] A more thorough Pull Request check list and template

2017-07-24 Thread Eron Wright
This seems like a good step in establishing a better PR process. I believe the process could be improved to ensure timely and targeted review by component experts and committers. On Mon, Jul 17, 2017 at 9:36 AM, Stephan Ewen wrote: > Hi all! > > I have reflected a bit on the

Re: [DISCUSS] Service Authorization (redux)

2017-08-02 Thread Eron Wright
rote: > > > Hi Eron, > > > > I think after Dropping support for Java 7 we will move to Akka 2.4+, so > we > > should be good there. I think quite some users should find a (more) > secure > > Flink interesting. > > > > Best, > > Aljoscha >

Re: [DISCUSS] Changing Flink's shading model

2017-05-11 Thread Eron Wright
In my opinion, the ideal approach to mitigating conflicts between application code and Flink itself is to relocate all of Flink's dependencies. Rationale is to avoid placing the burden of relocation on the app developer, and ultimately to eliminate the need for an app uber-jar. For example,

Re: [DISCUSS] Release 1.3.0 RC1 (Non voting, testing release candidate)

2017-05-17 Thread Eron Wright
Robert, please add FLINK-6606 to the list of JIRAs that you're tracking, thanks. On Tue, May 16, 2017 at 8:30 AM, Robert Metzger wrote: > I totally forgot to post a document with testing tasks in the RC0 thread, > so I'll do it in the RC1 thread. > > Please use this

Re: [DISCUSS] Feature Freeze

2017-05-02 Thread Eron Wright
Robert, I'd like to see FLINK-5974 (Mesos DNS support) added to the list of important issues. A PR is ready. On Tue, May 2, 2017 at 4:30 AM, Kostas Kloudas wrote: > The only thing that I want to add in the features

Re: Run apache flink from intellij or eclipse

2017-09-15 Thread Eron Wright
When you run in the IDE, an embedded ("local") Flink environment is created. It doesn't make use of your Flink cluster as "flink run" would. I believe you can activate the Web UI by using the following API: StreamExecutionEnvironment::createLocalEnvironmentWithWebUI

Re: [DISCUSS] Releasing Flink 1.4

2017-10-16 Thread Eron Wright
+1 from our side on this plan. On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske wrote: > OK, sounds good to me. > > We have a couple of bugs to fix for the Table API / SQL but have PRs for > most of them. > > There's only one major issue that I'd like to include in 1.4.0 which

[DISCUSS] Watermark boundary condition

2017-08-30 Thread Eron Wright
Hello, I think I see a bug in a few places related to determining whether an input is to be considered late. Some components use the logic that (timestamp <= watermark) is considered late. Others use (timestamp < watermark). I think the former is correct according to the definition in

Re: Using Maven Archetype Plugin 3.0+ to create flink-quickstart-java project

2017-09-11 Thread Eron Wright
Michael, an alternative is to start your project with the released version of Flink (1.3) rather than 1.4-SNAPSHOT: https://ci.apache.org/projects/flink/flink-docs-release-1.3/quickstart/java_api_quickstart.html#create-project Afterwards you could update your dependencies (and add the Apache

Re: [DISCUSS] Service Authorization (redux)

2017-09-26 Thread Eron Wright
hether this feature would be useful to you. Thanks! Eron On Thu, Aug 3, 2017 at 11:11 AM, Eron Wright <eronwri...@gmail.com> wrote: > Till, with (c) are you suggesting that we'd use Akka 2.3 for Scala 2.10 > and Akka 2.4+ for Scala 2.11+? Sounds reasonable but I don't know how > feasibl

[DISCUSS] Service Authorization (SSL client authentication)

2017-11-27 Thread Eron Wright
objections to the technical plan? Thanks! Eron Wright Dell EMC

Re: [VOTE] Release 1.4.0, release candidate #2

2017-11-29 Thread Eron Wright
Unfortunately we've identified a blocker bug for Flink on Mesos - FLINK-8174.We'll have a patch ready on Thursday. Thanks, Eron On Wed, Nov 29, 2017 at 3:40 PM, Eron Wright <eronwri...@gmail.com> wrote: > On Dell EMC side, we're testing the RC2 on DCOS 1.10.0. Seeing a > pot

Re: [VOTE] Release 1.4.0, release candidate #2

2017-12-01 Thread Eron Wright
IRA issue for the the Hadoop 2.9.0 build problem [1]. > > Best, Fabian > > [1] https://issues.apache.org/jira/browse/FLINK-8177 > > 2017-11-30 4:35 GMT+01:00 Eron Wright <eronwri...@gmail.com>: > > > Unfortunately we've identified a blocker bug for Flink on Me

Re: [DISCUSS] Service Authorization (SSL client authentication)

2017-12-19 Thread Eron Wright
Folks, what is the next step to formally submit a FLIP?i.e. assign a number, drive to 'accepted' state? Given the introduction of new RPC and REST endpoints with FLIP-6, now is a good time to agree on the approach to securing Flink. Thanks! Eron On Mon, Nov 27, 2017 at 11:52 AM, Eron Wright

Re: [VOTE] Release 1.4.0, release candidate #3

2017-12-07 Thread Eron Wright
Just discovered: the removal of Flink's Future (FLINK-7252) causes a breaking change in connectors that use `org.apache.flink.runtime.checkpoint.MasterTriggerRestoreHook`, because `Future` is a type on one of the methods. To my knowledge, this affects only the Pravega connector. Curious to know

Re: TaskManager job lifecycle hooks

2017-12-07 Thread Eron Wright
Could you speak to whether the lifecycle provided by RichFunction (open/close) would fit the requirement? https://ci.apache.org/projects/flink/flink-docs-release-1.3/api/java/org/apache/flink/api/common/functions/RichFunction.html#open-org.apache.flink.configuration.Configuration- On Thu, Dec 7,

Re: [VOTE] Release 1.4.0, release candidate #2

2017-12-01 Thread Eron Wright
t; > > On 1. Dec 2017, at 18:39, Eron Wright <eronwri...@gmail.com> wrote: > > > > Update on reported Mesos issue (FLINK-8174): > > > > TLDR; a PR will be ready within 24 hours that will undo reservation > support. > > > > A couple of months ag

Re: [VOTE] Release 1.4.0, release candidate #2

2017-12-03 Thread Eron Wright
Update: PR for "FLINK-8174 - Mesos RM unable to accept offers for unreserved resources" is ready for review. We were able to solve (3), supporting any combination of reserved and unreserved resources. On Fri, Dec 1, 2017 at 2:24 PM, Eron Wright <eronwri...@gmail.com> wrote: &

[Discuss] FLIP-26 - SSL Mutual Authentication

2018-05-09 Thread Eron Wright
Hello, Given that some SSL enhancement bugs have been posted lately, I took some time to revise FLIP-26 which explores how to harden both external and internal communication. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=80453255 Some recent related issues: - FLINK-9312 -

Re: Time-stamp of output from a window operator

2017-12-22 Thread Eron Wright
That is correct, the window operator happily accepts late elements into the applicable window(s). Though the element is late (with respect to the current watermark) it might still be included in the main firing, or provoke a late firing. Elements occurring downstream of an operator that uses

Re: Timestamp/watermark support in Kinesis consumer

2018-02-12 Thread Eron Wright
m > working on though. We face issues with restart from older checkpoints where > parent and child shards are consumed in parallel. > > > -- > sent from mobile > > > On Feb 12, 2018 4:36 PM, "Eron Wright" <eronwri...@gmail.com> wrote: > > I'd like to kno

Re: Timestamp/watermark support in Kinesis consumer

2018-02-12 Thread Eron Wright
I'd like to know how you envision dealing with resharding in relation to the watermark state. Imagine that a given shard S1 has a watermark of T1, and is then split into two shards S2 and S3. The new shards are assigned to subtasks according to a hash function. The current watermarks of those

Re: Time-stamp of output from a window operator

2017-12-22 Thread Eron Wright
Each event in the stream has an associated timestamp as metadata. A timestamp assigner simply extracts the timestamp from the user object for that purpose. There is no per-operator watermark, but with `assignTimestampsAndWatermarks` you may insert an operator that overrides upstream watermarks.

Re: Flink-Yarn-Kerberos integration

2017-12-22 Thread Eron Wright
I agree that it is reasonable to use Hadoop DTs as you describe. That approach is even recommended in YARN's documentation (see Securing Long-lived YARN Services on the YARN Application Security page). But one of the goals of Kerberos integration is to support Kerberized data access for

Re: [Discuss] FLIP-26 - SSL Mutual Authentication

2018-07-27 Thread Eron Wright
the IP filter via such an > > authorizer. > > > > > > *Hostname Verification* > > > > For internal communication, and especially on dynamic environments like > > Kubernetes, it is very hard to work with certificates and have hostname > > verification on

Re: SQL Client Limitations

2018-08-25 Thread Eron Wright
I'd like to better understand how catalogs will work in SQL Client. I assume we'll be able to reference catalog classes from the environment file (e.g. FLINK-9172). Thanks On Tue, Aug 21, 2018 at 4:56 AM Fabian Hueske wrote: > Hi Dominik, > > The SQL Client supports the same subset of SQL

Re: [jira] [Created] (FLINK-10225) Cannot access state from a empty taskmanager

2018-08-28 Thread Eron Wright
Thanks for posting this issue, I also recently saw this. The cause appears to be that the TaskExecutor's queryable state proxy knows only about jobs that are using slot(s) from that executor. Note FLINK-10117 calls for queryable state to be exposed via the REST endpoint, which may indirectly

Re: FLIP-6 / Kubernetes

2018-03-20 Thread Eron Wright
Till, is it possible to package a Flink application as a self-contained deployment on Kubernetes? I mean, run a Flink application using 'flink run' such that it launches its own RM/JM and waits for a sufficient # of TMs to join? Thanks! On Mon, Mar 19, 2018 at 2:57 AM, Till Rohrmann

Re: [DISCUSS] Flink security improvements

2018-03-21 Thread Eron Wright
Please accept my apologies also. On Mon, Mar 19, 2018 at 2:46 AM, Till Rohrmann wrote: > Hi Shuyi, > > sorry for the unresponsiveness on your proposal. Currently, the community > is strongly focused on fixing and testing Flink 1.5 so that we can release > it soon. My gut

Re: FLIP-6 / Kubernetes

2018-03-21 Thread Eron Wright
use > k8s > > > in the future. This feature did not make it for 1.5, but should come > very > > > soon after. > > > > > > @Thomas An implementation of a ResourceManager for k8s should come in > the > > > near future. Would be happy to jump

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2019-01-01 Thread Eron Wright
Hi folks, there's clearly some incremental steps to be taken to introduce catalog support to SQL Client, complementary to what is proposed in the Flink-Hive Metastore design doc. I was quietly working on this over the holidays. I posted some new sub-tasks, PRs, and sample code to FLINK-10744.

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2019-01-06 Thread Eron Wright
! On Thu, Jan 3, 2019 at 9:37 AM Eron Wright wrote: > Would a couple folks raise their hand to make a review pass thru the 6 PRs > listed above? It is a lovely stack of PRs that is 'all green' at the > moment. I would be happy to open follow-on PRs to rapidly align with > other effor

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2019-01-03 Thread Eron Wright
of the ExternalCatalog interface; the code would not be obsolete if/when the catalog interface is enhanced as per the design doc. On Wed, Jan 2, 2019 at 1:35 PM Eron Wright wrote: > I propose that the community review and merge the PRs that I posted, and > then evolve the design thru 1.8 and beyond. I

Re: [DISCUSS] Watermark propagation with Sink API

2021-06-08 Thread Eron Wright
uld be part of this next/future FLIP that would > define > > idleness in the first place. > > 2. Block FLIP-167, until the idleness is fixed. > > > > I would vote for option number 1. > > > > Piotrek > > > > pon., 7 cze 2021 o 18:08 Eron Wright > &g

Re: [VOTE] Watermark propagation with Sink API

2021-06-08 Thread Eron Wright
Voting is re-open for FLIP-167 as-is (without idleness support as was the point of contention). On Fri, Jun 4, 2021 at 10:45 AM Eron Wright wrote: > Little update on this, more good discussion over the last few days, and > the FLIP will probably be amended to incorporate idleness.

Re: [DISCUSS] Definition of idle partitions

2021-06-08 Thread Eron Wright
It seems to me that idleness was introduced to deal with a very specific issue. In the pipeline, watermarks are aggregated not on a per-split basis but on a per-subtask basis. This works well when each subtask has exactly one split. When a sub-task has multiple splits, various complications

Re: [DISCUSS] Watermark propagation with Sink API

2021-06-07 Thread Eron Wright
ics of idleness. We could also cut the scope of the FLIP to exclude > idleness and go ahead without it (there should be enough binding votes > already). > > On Sat, Jun 5, 2021 at 12:09 AM Eron Wright .invalid> > wrote: > > > I understand your scenario but I disagree with

Re: [VOTE] Watermark propagation with Sink API

2021-06-10 Thread Eron Wright
; > > > > > > Piotr Nowojski 于2021年6月9日周三 下午10:04写道: > > > > > > > > > Thanks for driving this Eron, and sorry for causing the delay. > > > > > > > > > > +1 (binding) from my side > > > > > > > > > > Piotrek > >

Re: [DISCUSS] Definition of idle partitions

2021-06-10 Thread Eron Wright
Piotr Nowojski < > piotr.nowoj...@gmail.com > > > > > >> wrote: > > >> > > >> > Hi Eron, > > >> > > > >> > Can you elaborate a bit more what do you mean? I don’t understand > what > > >> do > > >&g

Re: [DISCUSS] Definition of idle partitions

2021-06-10 Thread Eron Wright
I quickly updated the draft PR that would propagate idleness information to the Sink function, based on the recent improvement provided by FLINK-18934. For illustration purposes. https://github.com/streamnative/flink/pull/2 On Thu, Jun 10, 2021 at 11:34 AM Eron Wright wrote: > Regard

PR: "Propagate watermarks to sink API"

2021-06-21 Thread Eron Wright
Would someone be willing and able to review the PR which adds watermarks to the sink API? https://github.com/apache/flink/pull/15950 Thanks! Eron

Re: [DISCUSS] Watermark propagation with Sink API

2021-05-18 Thread Eron Wright
o me. > > > > We have some experimental feature for Kafka [1], which pretty much > reflects > > your idea. Here we have an ugly workaround to be able to process the > > watermark by using a custom StreamSink task. We could also try to create > a > > FLIP t

Re: [DISCUSS] Watermark propagation with Sink API

2021-05-18 Thread Eron Wright
Update: opened an issue and a PR. https://issues.apache.org/jira/browse/FLINK-22700 https://github.com/apache/flink/pull/15950 On Tue, May 18, 2021 at 10:03 AM Eron Wright wrote: > Thanks Arvid and David for sharing your ideas on this subject. I'm glad > to hear that you're seeing use

[DISCUSS] Watermark propagation with Sink API

2021-05-17 Thread Eron Wright
! Eron Wright StreamNative

Re: [DISCUSS] Watermark propagation with Sink API

2021-05-20 Thread Eron Wright
I. But > that's something I can do. > > [1] > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/Flink*Improvement*Proposals__;Kys!!LpKI!2IQYKfnjRuBgkNRxnPbJeFvTdhWjpwN0urN3m0yz_6W11H74kY5dMnp6nc7o$ > [cwiki[.]apache[.]org] > > On Wed, May 19

Re: [DISCUSS] Watermark propagation with Sink API

2021-05-26 Thread Eron Wright
so > work >with ingestion time sufficiently? >6. How do exactly once sinks deal with written watermarks in case of >failure? I guess it's the same as normal records. (Either rollback of >transaction or deduplication on resumption) > > Best, > > Arvid > >

Re: [DISCUSS] Watermark propagation with Sink API

2021-05-25 Thread Eron Wright
Does anyone have further comment on FLIP-167? https://cwiki.apache.org/confluence/display/FLINK/FLIP-167%3A+Watermarks+for+Sink+API Thanks, Eron On Thu, May 20, 2021 at 5:02 PM Eron Wright wrote: > Filed FLIP-167: Watermarks for Sink API: > > https://cwiki.apache.org/confluence/disp

Re: [DISCUSS] Watermark propagation with Sink API

2021-05-28 Thread Eron Wright
s already on the exception list) > 5.+6. Awesome, I was also sure but wanted to double check. > > Best, > > Arvid > > > On Wed, May 26, 2021 at 7:29 PM Eron Wright .invalid> > wrote: > > > Arvid, > > > > 1. I assume that the method name `invoke`

Re: [DISCUSS] Watermark propagation with Sink API

2021-06-04 Thread Eron Wright
ng (no partition) > and as long as we do not have dynamic partition assignment, there will > never be a switch to active without restart (for the foreseeable future). > > On Fri, Jun 4, 2021 at 10:34 PM Eron Wright .invalid> > wrote: > > > Yes I'm talking about an impleme

Re: [DISCUSS] Watermark propagation with Sink API

2021-06-04 Thread Eron Wright
wall clock, you always run into the issues that I describe since you > are effectively mixing event time and processing time... > > > On Fri, Jun 4, 2021 at 6:28 PM Eron Wright .invalid> > wrote: > > > Dawid, I think you're mischaracterizing the idleness signal as &

Re: [DISCUSS] Watermark propagation with Sink API

2021-06-04 Thread Eron Wright
> would add unnecessary false late events. > > In my understanding a StreamStatus makes sense only when talking about > immediately consumed transient channels such as between operators within > a single job. > > Best, > > Dawid > > On 03/06/2021 23:31, Eron Wright wrote: > >

Re: [DISCUSS] Watermark propagation with Sink API

2021-06-03 Thread Eron Wright
ce/pages/viewpage.action?pageId=120731026#FlinkBylaws-Voting > > [2] > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Deprecating-Mesos-support-td50142.html > > > > On Fri, May 28, 2021 at 8:17 PM Eron Wright > .invalid> > >

Re: [DISCUSS] Watermark propagation with Sink API

2021-06-04 Thread Eron Wright
r an > immediate > >> > consumption. > >> > > >> > You could say the same thing about watermarks (note they are usually > >> > generated in Flink based on the incoming events) and I would not agree > >> with > >> > it in the same wa

Re: [VOTE] Watermark propagation with Sink API

2021-06-04 Thread Eron Wright
day, June 2, 2021 15:44 > > > > To: dev > > > > Subject: Re: [VOTE] Watermark propagation with Sink API > > > > > > > > > > > > [EXTERNAL EMAIL] > > > > > > > > +1 (binding) > > > > > > > >

[VOTE] Watermark propagation with Sink API

2021-06-01 Thread Eron Wright
Discussion thread: https://lists.apache.org/thread.html/r5194e1cf157d1fd5ba7ca9b567cb01723bd582aa12dda57d25bca37e%40%3Cdev.flink.apache.org%3E Implementation Issue: https://issues.apache.org/jira/browse/FLINK-22700 Thanks, Eron Wright StreamNative

Re: [DISCUSS] Watermark propagation with Sink API

2021-06-03 Thread Eron Wright
t be what the user > intended. > On the other hand, I don't immediately see a technical reason why the > downstream source wouldn't be able to infer that. > > > On Thu, Jun 3, 2021 at 9:14 PM Eron Wright .invalid> > wrote: > > > Thanks Piotr for bringing this up. I

Re: [DISCUSS] FLIP-182: Watermark alignment

2021-07-12 Thread Eron Wright
The notion of per-split watermarks seems quite interesting. I think the idleness feature could benefit from a per-split approach too, because idleness is typically related to whether any splits are assigned to a given operator instance. On Mon, Jul 12, 2021 at 3:06 AM 刘建刚 wrote: > +1 for the

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

2021-11-10 Thread Eron Wright
Jingsong, regarding the LogStore abstraction, I understand that you want to retain some flexibility as the implementation evolves. It makes sense that the abstract interfaces would be @Internal for now. Would you kindly ensure the minimal extensibility is in place, so that the Pulsar dev

Re: [DISCUSS] FLIP-180: Adjust StreamStatus and Idleness definition

2021-07-20 Thread Eron Wright
This proposal to narrow the definition of idleness to focus on the event-time clock is great. Please mention that the "temporary status toggle" code will be removed. I agree with adding the markActive() functionality, for symmetry. Speaking of symmetry, could we now include the minor

[jira] [Created] (FLINK-3929) Support for Kerberos Authentication with Keytab Credential

2016-05-18 Thread Eron Wright (JIRA)
Eron Wright created FLINK-3929: --- Summary: Support for Kerberos Authentication with Keytab Credential Key: FLINK-3929 URL: https://issues.apache.org/jira/browse/FLINK-3929 Project: Flink Issue

[jira] [Created] (FLINK-3931) Implement Transport Encryption (SSL/TLS)

2016-05-18 Thread Eron Wright (JIRA)
Eron Wright created FLINK-3931: --- Summary: Implement Transport Encryption (SSL/TLS) Key: FLINK-3931 URL: https://issues.apache.org/jira/browse/FLINK-3931 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-3883) Rename flink clients for inclusion on system path

2016-05-08 Thread Eron Wright (JIRA)
Eron Wright created FLINK-3883: --- Summary: Rename flink clients for inclusion on system path Key: FLINK-3883 URL: https://issues.apache.org/jira/browse/FLINK-3883 Project: Flink Issue Type

[jira] [Created] (FLINK-5533) DCOS Integration

2017-01-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5533: --- Summary: DCOS Integration Key: FLINK-5533 URL: https://issues.apache.org/jira/browse/FLINK-5533 Project: Flink Issue Type: New Feature Components

[jira] [Created] (FLINK-5534) Config option: 'flink-options'

2017-01-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5534: --- Summary: Config option: 'flink-options' Key: FLINK-5534 URL: https://issues.apache.org/jira/browse/FLINK-5534 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-5538) Config option: Kerberos

2017-01-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5538: --- Summary: Config option: Kerberos Key: FLINK-5538 URL: https://issues.apache.org/jira/browse/FLINK-5538 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-5537) Config option: SSL

2017-01-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5537: --- Summary: Config option: SSL Key: FLINK-5537 URL: https://issues.apache.org/jira/browse/FLINK-5537 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-5540) CLI: savepoint

2017-01-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5540: --- Summary: CLI: savepoint Key: FLINK-5540 URL: https://issues.apache.org/jira/browse/FLINK-5540 Project: Flink Issue Type: Sub-task Reporter: Eron

[jira] [Created] (FLINK-5535) Config option: HDFS

2017-01-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5535: --- Summary: Config option: HDFS Key: FLINK-5535 URL: https://issues.apache.org/jira/browse/FLINK-5535 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-5539) CLI: info/list/stop/cancel

2017-01-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5539: --- Summary: CLI: info/list/stop/cancel Key: FLINK-5539 URL: https://issues.apache.org/jira/browse/FLINK-5539 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-5536) Config option: HA

2017-01-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5536: --- Summary: Config option: HA Key: FLINK-5536 URL: https://issues.apache.org/jira/browse/FLINK-5536 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-4921) Upgrade to Mesos 1.0.1

2016-10-25 Thread Eron Wright (JIRA)
Eron Wright created FLINK-4921: --- Summary: Upgrade to Mesos 1.0.1 Key: FLINK-4921 URL: https://issues.apache.org/jira/browse/FLINK-4921 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-4922) Write documentation

2016-10-25 Thread Eron Wright (JIRA)
Eron Wright created FLINK-4922: --- Summary: Write documentation Key: FLINK-4922 URL: https://issues.apache.org/jira/browse/FLINK-4922 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-5030) Support hostname verification

2016-11-07 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5030: --- Summary: Support hostname verification Key: FLINK-5030 URL: https://issues.apache.org/jira/browse/FLINK-5030 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-5029) Implement KvState SSL

2016-11-07 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5029: --- Summary: Implement KvState SSL Key: FLINK-5029 URL: https://issues.apache.org/jira/browse/FLINK-5029 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-4900) Implement Docker image support

2016-10-24 Thread Eron Wright (JIRA)
Eron Wright created FLINK-4900: --- Summary: Implement Docker image support Key: FLINK-4900 URL: https://issues.apache.org/jira/browse/FLINK-4900 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-4899) Implement DCOS package

2016-10-24 Thread Eron Wright (JIRA)
Eron Wright created FLINK-4899: --- Summary: Implement DCOS package Key: FLINK-4899 URL: https://issues.apache.org/jira/browse/FLINK-4899 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-4897) Implement Dispatcher

2016-10-24 Thread Eron Wright (JIRA)
Eron Wright created FLINK-4897: --- Summary: Implement Dispatcher Key: FLINK-4897 URL: https://issues.apache.org/jira/browse/FLINK-4897 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-4898) Refactor HTTP handlers and Netty server/client

2016-10-24 Thread Eron Wright (JIRA)
Eron Wright created FLINK-4898: --- Summary: Refactor HTTP handlers and Netty server/client Key: FLINK-4898 URL: https://issues.apache.org/jira/browse/FLINK-4898 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-4901) Build DCOS Docker image

2016-10-24 Thread Eron Wright (JIRA)
Eron Wright created FLINK-4901: --- Summary: Build DCOS Docker image Key: FLINK-4901 URL: https://issues.apache.org/jira/browse/FLINK-4901 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-5091) Formalize the AppMaster environment for docker compability

2016-11-17 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5091: --- Summary: Formalize the AppMaster environment for docker compability Key: FLINK-5091 URL: https://issues.apache.org/jira/browse/FLINK-5091 Project: Flink Issue

[jira] [Created] (FLINK-5416) Function wrappers should support CheckpointedFunction

2017-01-05 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5416: --- Summary: Function wrappers should support CheckpointedFunction Key: FLINK-5416 URL: https://issues.apache.org/jira/browse/FLINK-5416 Project: Flink Issue

[jira] [Created] (FLINK-5364) Rework JAAS configuration to support user-supplied entries

2016-12-18 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5364: --- Summary: Rework JAAS configuration to support user-supplied entries Key: FLINK-5364 URL: https://issues.apache.org/jira/browse/FLINK-5364 Project: Flink Issue

[jira] [Created] (FLINK-5365) Mesos AppMaster/TaskManager should obey sigterm

2016-12-18 Thread Eron Wright (JIRA)
Eron Wright created FLINK-5365: --- Summary: Mesos AppMaster/TaskManager should obey sigterm Key: FLINK-5365 URL: https://issues.apache.org/jira/browse/FLINK-5365 Project: Flink Issue Type: Bug

  1   2   >