Re: [VOTE] Release 1.15.2, release candidate #1
+1 (binding) - signatures OK - all required artifacts on dist.apache.org - maven artifacts appear complete - tag exists - PR OK On 17/08/2022 10:32, Danny Cranmer wrote: Hi everyone, Please review and vote on the release candidate #1 for the version 1.15.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], * the official Apache source release and binary convenience releases to be deployed to dist.apache.org [2], which are signed with the key with fingerprint 125FD8DB [3], * all artifacts to be deployed to the Maven Central Repository [4], * source code tag "release-1.15.2-rc1" [5], * website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Danny Cranmer [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351829 [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.15.2-rc1/ [3] https://dist.apache.org/repos/dist/release/flink/KEYS [4] https://repository.apache.org/content/repositories/orgapacheflink-1521/ [5] https://github.com/apache/flink/tree/release-1.15.2-rc1 [6] https://github.com/apache/flink-web/pull/566
Re: [VOTE] Release 1.15.2, release candidate #1
It also seems like the distribution wasn't built in a clean environment; there's a .version.properties file in the flink-dist jar that shouldn't be there. (it's not there in 1.15.1) On 17/08/2022 13:02, Chesnay Schepler wrote: Oh hold on, what are those strange PaxHeader directories in the distribution? On 17/08/2022 12:48, Chesnay Schepler wrote: +1 (binding) - signatures OK - all required artifacts on dist.apache.org - maven artifacts appear complete - tag exists - PR OK On 17/08/2022 10:32, Danny Cranmer wrote: Hi everyone, Please review and vote on the release candidate #1 for the version 1.15.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], * the official Apache source release and binary convenience releases to be deployed to dist.apache.org [2], which are signed with the key with fingerprint 125FD8DB [3], * all artifacts to be deployed to the Maven Central Repository [4], * source code tag "release-1.15.2-rc1" [5], * website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Danny Cranmer [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351829 [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.15.2-rc1/ [3] https://dist.apache.org/repos/dist/release/flink/KEYS [4] https://repository.apache.org/content/repositories/orgapacheflink-1521/ [5] https://github.com/apache/flink/tree/release-1.15.2-rc1 [6] https://github.com/apache/flink-web/pull/566
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations and welcome to the committer team :-) Regards, Timo On 17.08.22 12:50, Yuxin Tan wrote: Congratulations, Lijie! Best, Yuxin Guowei Ma 于2022年8月17日周三 18:42写道： Congratulations, Lijie. Welcome on board~! Best, Guowei On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu wrote: Hi everyone, On behalf of the PMC, I'm very happy to announce Lijie Wang as a new Flink committer. Lijie has been contributing to Flink project for more than 2 years. He mainly works on the runtime/coordination part, doing feature development, problem debugging and code reviews. He has also driven the work of FLIP-187(Adaptive Batch Scheduler) and FLIP-224(Blocklist for Speculative Execution), which are important to run batch jobs. Please join me in congratulating Lijie for becoming a Flink committer! Cheers, Zhu
Re: [VOTE] Release 1.15.2, release candidate #1
Good spot Chesnay. The environment was clean as far as "git status" is concerned, however there must have been files hidden by .gitignore. I will rebuild from a fresh workspace. This release candidate is officially cancelled. On Wed, Aug 17, 2022 at 12:06 PM Chesnay Schepler wrote: > It also seems like the distribution wasn't built in a clean environment; > there's a .version.properties file in the flink-dist jar that shouldn't > be there. (it's not there in 1.15.1) > > On 17/08/2022 13:02, Chesnay Schepler wrote: > > Oh hold on, what are those strange PaxHeader directories in the > > distribution? > > > > On 17/08/2022 12:48, Chesnay Schepler wrote: > >> +1 (binding) > >> > >> - signatures OK > >> - all required artifacts on dist.apache.org > >> - maven artifacts appear complete > >> - tag exists > >> - PR OK > >> > >> On 17/08/2022 10:32, Danny Cranmer wrote: > >>> Hi everyone, > >>> > >>> Please review and vote on the release candidate #1 for the version > >>> 1.15.2, > >>> as follows: > >>> [ ] +1, Approve the release > >>> [ ] -1, Do not approve the release (please provide specific comments) > >>> > >>> > >>> The complete staging area is available for your review, which includes: > >>> * JIRA release notes [1], > >>> * the official Apache source release and binary convenience releases > >>> to be > >>> deployed to dist.apache.org [2], which are signed with the key with > >>> fingerprint 125FD8DB [3], > >>> * all artifacts to be deployed to the Maven Central Repository [4], > >>> * source code tag "release-1.15.2-rc1" [5], > >>> * website pull request listing the new release and adding > >>> announcement blog > >>> post [6]. > >>> > >>> The vote will be open for at least 72 hours. It is adopted by majority > >>> approval, with at least 3 PMC affirmative votes. > >>> > >>> Thanks, > >>> Danny Cranmer > >>> > >>> [1] > >>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351829 > >>> > >>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.15.2-rc1/ > >>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS > >>> [4] > >>> > https://repository.apache.org/content/repositories/orgapacheflink-1521/ > >>> [5] https://github.com/apache/flink/tree/release-1.15.2-rc1 > >>> [6] https://github.com/apache/flink-web/pull/566 > >>> > >> > > > >
[jira] [Created] (FLINK-29010) flink planner test is not correct
jackylau created FLINK-29010:
Summary: flink planner test is not correct
Key: FLINK-29010
URL: https://issues.apache.org/jira/browse/FLINK-29010
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 1.16.0
Reporter: jackylau
Fix For: 1.16.0
{code:java}
// code placeholder
private def prepareResult(seq: Seq[Row], isSorted: Boolean): Seq[String] = {
if (!isSorted) seq.map(_.toString).sortBy(s => s) else seq.map(_.toString)
} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations, Lijie! Best, Yuxin Guowei Ma 于2022年8月17日周三 18:42写道： > Congratulations, Lijie. Welcome on board~! > Best, > Guowei > > > On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu wrote: > > > Hi everyone, > > > > On behalf of the PMC, I'm very happy to announce Lijie Wang as > > a new Flink committer. > > > > Lijie has been contributing to Flink project for more than 2 years. > > He mainly works on the runtime/coordination part, doing feature > > development, problem debugging and code reviews. He has also > > driven the work of FLIP-187(Adaptive Batch Scheduler) and > > FLIP-224(Blocklist for Speculative Execution), which are important > > to run batch jobs. > > > > Please join me in congratulating Lijie for becoming a Flink committer! > > > > Cheers, > > Zhu > > >
[jira] [Created] (FLINK-29015) Flink stop with/without savepoint does not work with kinesis consumer
Vararu Vadim created FLINK-29015:
Summary: Flink stop with/without savepoint does not work with
kinesis consumer
Key: FLINK-29015
URL: https://issues.apache.org/jira/browse/FLINK-29015
Project: Flink
Issue Type: Bug
Components: Connectors / Kinesis
Affects Versions: 1.15.1
Environment: Flink 1.15.1. Same version for flink-connector-kinesis
1.15.1
Reporter: Vararu Vadim
Trying to migrate the job from Flink 1.14 to 1.15.1, observed that stop
with/without savepoint does not work anymore.
The checkpointing/savepointing by themselves are working OK.
The exception when stopping the job seems to come from the kinesis connector.
{code:java}
2022-08-17 12:57:16,266 INFO
org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher [] -
Starting shutdown of shard consumer threads and AWS SDK resources of subtask 2
...
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method) ~[?:?]
at
org.apache.flink.streaming.connectors.kinesis.internals.publisher.polling.PollingRecordPublisher.adjustRunLoopFrequency(PollingRecordPublisher.java:216)
~[blob_p-956a630701b5709969a8029f6fefe9d3cf05a778-c2e9ace4f09a95dc67692581fea51a5b:?]
at
org.apache.flink.streaming.connectors.kinesis.internals.publisher.polling.PollingRecordPublisher.run(PollingRecordPublisher.java:124)
~[blob_p-956a630701b5709969a8029f6fefe9d3cf05a778-c2e9ace4f09a95dc67692581fea51a5b:?]
at
org.apache.flink.streaming.connectors.kinesis.internals.publisher.polling.PollingRecordPublisher.run(PollingRecordPublisher.java:102)
~[blob_p-956a630701b5709969a8029f6fefe9d3cf05a778-c2e9ace4f09a95dc67692581fea51a5b:?]
at
org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer.run(ShardConsumer.java:114)
[blob_p-956a630701b5709969a8029f6fefe9d3cf05a778-c2e9ace4f09a95dc67692581fea51a5b:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:?]
at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
at java.lang.Thread.run(Unknown Source) [?:?]{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [VOTE] Release 1.15.2, release candidate #1
Oh hold on, what are those strange PaxHeader directories in the distribution? On 17/08/2022 12:48, Chesnay Schepler wrote: +1 (binding) - signatures OK - all required artifacts on dist.apache.org - maven artifacts appear complete - tag exists - PR OK On 17/08/2022 10:32, Danny Cranmer wrote: Hi everyone, Please review and vote on the release candidate #1 for the version 1.15.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], * the official Apache source release and binary convenience releases to be deployed to dist.apache.org [2], which are signed with the key with fingerprint 125FD8DB [3], * all artifacts to be deployed to the Maven Central Repository [4], * source code tag "release-1.15.2-rc1" [5], * website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Danny Cranmer [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351829 [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.15.2-rc1/ [3] https://dist.apache.org/repos/dist/release/flink/KEYS [4] https://repository.apache.org/content/repositories/orgapacheflink-1521/ [5] https://github.com/apache/flink/tree/release-1.15.2-rc1 [6] https://github.com/apache/flink-web/pull/566
[jira] [Created] (FLINK-29014) Use setting instead of merging for pipeline.jars in StreamExecutionEnvironment.configure
Timo Walther created FLINK-29014: Summary: Use setting instead of merging for pipeline.jars in StreamExecutionEnvironment.configure Key: FLINK-29014 URL: https://issues.apache.org/jira/browse/FLINK-29014 Project: Flink Issue Type: Sub-task Components: API / DataStream, Table SQL / API Reporter: Timo Walther Assignee: Timo Walther -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [Discuss] Let's Session Cluster JobManager take a breather (FLIP-257: Flink JobManager Process Split)
To be honest I'm terrified at the idea of splitting the Dispatcher into several processes, even more so if this is supposed to be opt-in and specific to session mode. It would fragment the coordination layer even more than it already is, and make ops more complicated (yet another set of processes to monitor, configure etc.). I'm not convinced that this proposal really gets us a lot of benefits; and would rather propose that you split your single session cluster into multiple session clusters (with the scheduling component in front of it to distribute jobs) to even the load. > The currently idling JobManagers could be utilized to take over some of the workload from the leader. This would also be the path I would go down if we'd try to tackle this. On 17/08/2022 16:22, Matthias Pohl wrote: Hi Conrad, thanks for reaching out to the community with your proposal. I looked through FLIP-257 [1]. Your motivation sounds interesting. Can you elaborate a bit more on the concrete use-cases you have in mind? How do these match the user-cases of the two favored execution modes, i.e. Flink's Session and Application mode? As mentioned in [2], Application Mode should be preferred for single long-running jobs to isolate the resources of each of those jobs from each other. In contrast, Session Mode is the natural choice when running rather small/short-lived jobs (e.g. FlinkSQL queries) or when deploying some kind of dev environment for testing out job implementations. It feels like your use-case is somewhere in between a bit? It would be interesting to get a better understanding of where you're coming from. Maybe, you could provide some workload statistics? That considered, I guess it's a topic worth looking into. Here are a few thoughts after looking into FLIP-257: - As far as I can see, the BlobServer is used for sharing configuration information (e.g. Classpath information) as part of the ExecutionGraph instantiation [3]. The JobGraph is not persisted through the BlobServer but rather stored in the JobGraphStore backed by the HighAvailabilityServices implementation. The HA side is not really covered in FLIP-257, yet. - The approach of having the current Dispatcher living next to the new JobMasterDispatcher (that encapsulates the logic around distributing the workload onto multiple runners) leaves me with some doubt whether there wouldn't be a better separation of concerns here. What about leaving the Dispatcher as is but adding some abstraction between JobManagerRunner/JobMaster and the Dispatcher that hides the logic around whether these instances are "deployed" on the same machine or somewhere else. - About distributing JobManager workload: The JobManager already utilizes leader election for faster recovery. Hence, one can set up multiple JobManagers in idle mode which wait to gain leadership and pick up the work (i.e. recovering the jobs) of the previously failed JobManager leader. What about utilizing this setup: The currently idling JobManagers could be utilized to take over some of the workload from the leader. I haven't thought this through, yet. But I'm wondering whether that would be a path we could go down. This would enable us to still stick to the JobManager/TaskManager setup which users are already used to rather than introducing another type of cluster node. - The JobManager initialization logic is kind of tricky to get your head around. There is some overhead, I hope, we could clean up as part of the efforts of removing the per-Job Mode from Flink [4]. It was decided to deprecate per-Job Mode in Flink 1.15. But we have to stick with it for some time (i.e. it's not going to be removed in 1.16) since it's a quite basic feature users might rely on. This shouldn't be a blocker. I just wanted to mention it to have it in the back of our minds when looking into ways to come up with a solid proposal for FLIP-257. - My concern is that this FLIP might turn out to be larger than expected and that it might be worth cutting it down into smaller chunks with each being covered in a separate FLIP down the road if we have some agreement and a clearer picture on how this should be tackled. I'm gonna add Chesnay and David to this discussion. Best, Matthias [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-257+Flink+JobManager+Process+Split [2] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/overview/#deployment-modes [3] https://github.com/apache/flink/blob/9ed70a1e8b5d59abdf9d7673bc5b44d421140ef0/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/DefaultExecutionGraph.java#L333 [4] https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n On Tue, Aug 16, 2022 at 11:43 AM Zheng Yu Chen wrote: Hi community ~ I think this title should be quite interesting. The idea is to reduce the workload of the JobManager and make the SessionCluster [2] more stable in the process of
Re: [Discuss] Let's Session Cluster JobManager take a breather (FLIP-257: Flink JobManager Process Split)
Hi Conrad, thanks for reaching out to the community with your proposal. I looked through FLIP-257 [1]. Your motivation sounds interesting. Can you elaborate a bit more on the concrete use-cases you have in mind? How do these match the user-cases of the two favored execution modes, i.e. Flink's Session and Application mode? As mentioned in [2], Application Mode should be preferred for single long-running jobs to isolate the resources of each of those jobs from each other. In contrast, Session Mode is the natural choice when running rather small/short-lived jobs (e.g. FlinkSQL queries) or when deploying some kind of dev environment for testing out job implementations. It feels like your use-case is somewhere in between a bit? It would be interesting to get a better understanding of where you're coming from. Maybe, you could provide some workload statistics? That considered, I guess it's a topic worth looking into. Here are a few thoughts after looking into FLIP-257: - As far as I can see, the BlobServer is used for sharing configuration information (e.g. Classpath information) as part of the ExecutionGraph instantiation [3]. The JobGraph is not persisted through the BlobServer but rather stored in the JobGraphStore backed by the HighAvailabilityServices implementation. The HA side is not really covered in FLIP-257, yet. - The approach of having the current Dispatcher living next to the new JobMasterDispatcher (that encapsulates the logic around distributing the workload onto multiple runners) leaves me with some doubt whether there wouldn't be a better separation of concerns here. What about leaving the Dispatcher as is but adding some abstraction between JobManagerRunner/JobMaster and the Dispatcher that hides the logic around whether these instances are "deployed" on the same machine or somewhere else. - About distributing JobManager workload: The JobManager already utilizes leader election for faster recovery. Hence, one can set up multiple JobManagers in idle mode which wait to gain leadership and pick up the work (i.e. recovering the jobs) of the previously failed JobManager leader. What about utilizing this setup: The currently idling JobManagers could be utilized to take over some of the workload from the leader. I haven't thought this through, yet. But I'm wondering whether that would be a path we could go down. This would enable us to still stick to the JobManager/TaskManager setup which users are already used to rather than introducing another type of cluster node. - The JobManager initialization logic is kind of tricky to get your head around. There is some overhead, I hope, we could clean up as part of the efforts of removing the per-Job Mode from Flink [4]. It was decided to deprecate per-Job Mode in Flink 1.15. But we have to stick with it for some time (i.e. it's not going to be removed in 1.16) since it's a quite basic feature users might rely on. This shouldn't be a blocker. I just wanted to mention it to have it in the back of our minds when looking into ways to come up with a solid proposal for FLIP-257. - My concern is that this FLIP might turn out to be larger than expected and that it might be worth cutting it down into smaller chunks with each being covered in a separate FLIP down the road if we have some agreement and a clearer picture on how this should be tackled. I'm gonna add Chesnay and David to this discussion. Best, Matthias [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-257+Flink+JobManager+Process+Split [2] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/overview/#deployment-modes [3] https://github.com/apache/flink/blob/9ed70a1e8b5d59abdf9d7673bc5b44d421140ef0/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/DefaultExecutionGraph.java#L333 [4] https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n On Tue, Aug 16, 2022 at 11:43 AM Zheng Yu Chen wrote: > Hi community ~ > > I think this title should be quite interesting. The idea is to reduce the > workload of the JobManager and make the SessionCluster [2] more stable in > the process of running jobs. I designed a plan for splitting the JobManager > on FLIP-257 [1]: > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-257+Flink+JobManager+Process+Split > < > https://cwiki.apache.org/confluence/display/FLINK/FLIP-257+Flink+JobManager+JobMaster+Thread+Split+to+Process > > > > This proposal proposes a splitting scheme for the current process and a new > process implementation idea that is compatible with the original process > model: splitting the internal JobMaster component of the JobManager, and > controlling whether to enable this new process through a parameter In the > split scheme, when the user configures, the JobMaster will make it run as > an independent service, reducing the workload of the JobManager. By > implementing a new Dispatcher to communicate and interact with a single > split JobMaster or multiple JobMasters, to achieve job
Re: [DISCUSS] FLIP-246: Multi Cluster Kafka Source
Hey Mason, I just had a look at the FLIP. If I understand correctly, you are proposing a very sophisticated way to read from multiple Kafka clusters / topics. I'm wondering whether we can share some of the code of the existing KafkaSource. I suppose you don't want to modify KafkaSource itself to avoid any breakage. But it would be good not to duplicate too much code, such that functionality that can be shared between the two implementations (e.g. the reader implementation). Some questions that I had when browsing the current version: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217389320 1. Why does KafkaMetadataService have the `describeStream` method in addition to `getAllStreams`? This may be redundant. Is the idea to get the updated metadata for a particular StreamId? 2. KafkaMetaDataService#isClusterActive serves the purpose to check for activeness, couldn't this be included in the metadata of KafkaStream? 3. Shouldn't MetadataUpdateEvent contain a full list of KafkaStream instead of `Map>`? 4. "In addition, restarting enumerators involve clearing outdated metrics" What metrics are we talking about here? 5. `StoppableKafkaEnumContextProxy` doesn't ring with me. How about `MultiKafkaSplitEnumeratorContext`? 6. What about the ConfigMap implementation? Are there any limitations on the type of configuration changes that we want to allow? For example, is it allowed to remove a cluster before it has been drained / deactivated? Is "not active" semantically identical to having the cluster / stream removed? This is an exciting new addition! Cheers, Max On 11.08.22 10:10, Mason Chen wrote: 5. At startup, GetMetadataUpdateEvent is also used to allow the MultiClusterKafkaSourceReader to get the latest metadata from the enumerator to filter out invalid splits This is how the reader can solve "removing" splits/topics in the startup case. Sorry for the late response, really appreciate you taking a look at the FLIP! Best, Mason On Thu, Aug 11, 2022 at 1:03 AM Mason Chen wrote: Hi Qingsheng, Thanks for the feedback--these are great points to raise. 1. This is something I missed that is now added. More generally, it can locate multiple topics in multiple clusters (1 topic on 1 cluster is the simplest case). 2. The KafkaMetadataService doesn't interact with the KafkaAdminClients. This source merely composes the functionality of the KafkaSource so KafkaAdminClient interaction is handled by the KafkaSubscriber. 3. There are no requirements for the two clusters--KafkaStream should clarify this question. For example, you could move from topicName1 in cluster 1 with 11 partitions to topicName2 in cluster 2 with 22 partitions--only the KafkaStream id needs to remain the same. If there are no offsets in checkpoint, the offsets are handled by the offsets initializer from KafkaSource and currently the design only exposes 1 option for all Kafka clusters, although this could be a valuable extension. 4. Regarding topic and cluster removal, metadata is checkpoint in state via the splits. Exactly once can be maintained with the assumption that required data from the dead cluster lives in the live cluster. This can be solved by not destroying the old Kafka cluster until consumers are already drained. In switchover, the consumer would consume from both old and new clusters. And finally, the metadata can be changed to point only to the new cluster when consumers are drained. With the regular KafkaSource, if Kafka deletes topic or a cluster is destroyed, the exactly once semantics are not preserved and the semantic is tightly coupled with storage. The design composes and delegates the responsibilities to KafkaSource components so it is limited to whatever KafkaSource can do for exactly once semantics. 5. Yes, I added more in the FLIP. GetMetadataUpdateEvent was added to make the order of steps in reader restart during split assignment deterministic. StoppableKafkaEnumContextProxy are used by the underlying KafkaSourceEnumerator to assign splits and do topic periodic partition discovery. So, these scheduled thread pools need to be cleaned up properly and splits need to be wrapped with cluster information. These details are added to the FLIP. Best, Mason On Fri, Jul 29, 2022 at 1:38 AM Qingsheng Ren wrote: Hi Mason, Thank you for starting this FLIP! From my first glance this FLIP looks like a collection of many new interfaces, but I can’t stitch them together. It’ll be great to have some brief descriptions about how the source works internally. Here are some questions in my mind and please correct me if I misunderstand your design. 1. I can’t find the definition (in code) of KafkaStream. As a part of the public interface KafkaMetadataService it has to be public too. If I understand correctly it locates a topic on a specific cluster. 2. I think there should be a default implementation / example for KafkaMetadataService for out-of-box usage, for example a
[jira] [Created] (FLINK-29016) Clarify Kryo limitations w.r.t. schema evolution
Chesnay Schepler created FLINK-29016: Summary: Clarify Kryo limitations w.r.t. schema evolution Key: FLINK-29016 URL: https://issues.apache.org/jira/browse/FLINK-29016 Project: Flink Issue Type: Improvement Components: API / Type Serialization System Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.16.0 While the schema evolution docs do state that Kryo isn't supported for schema evolution they don't really convey the gravity of the situation. They should clarify that if Kryo is used for anything, even some data-structure containing POJOs (like an ArrayList, scala option/collection), that schema evolution will not be possible. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29017) Some github links in released doc point to master
Ryan Skraba created FLINK-29017:
---
Summary: Some github links in released doc point to master
Key: FLINK-29017
URL: https://issues.apache.org/jira/browse/FLINK-29017
Project: Flink
Issue Type: Bug
Reporter: Ryan Skraba
For example:
{code:java}
The following example uses the example schema
[testdata.avsc](https://github.com/apache/flink/blob/master/flink-formats/flink-parquet/src/test/resources/avro/testdata.avsc):
{code}
should be using the {{gh_link}} shortcode to automatically create the link to
the the correct github branch:
{code:java}
The following example uses the example schema {{< gh_link
file="flink-formats/flink-parquet/src/test/resources/avro/testdata.avsc"
name="testdata.avsc" >}}:
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[VOTE] Apache Flink Table Store 0.2.0, release candidate #2
Hi everyone, Please review and vote on the release candidate #2 for the version 0.2.0 of Apache Flink Table Store, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of the following: a) Table Store canonical source distribution to be deployed to the release repository at dist.apache.org b) Table Store binary convenience releases to be deployed to the release repository at dist.apache.org c) Maven artifacts to be deployed to the Maven Central Repository **Staging Areas to Review** The staging areas containing the above mentioned artifacts are as follows, for your review: * All artifacts for a) and b) can be found in the corresponding dev repository at dist.apache.org [2] * All artifacts for c) can be found at the Apache Nexus Repository [3] All artifacts are signed with the key 2C2B6A653B07086B65E4369F7C76245E0A318150 [4] Other links for your review: * JIRA release notes [5] * source code tag "release-0.2.0-rc2" [6] * PR to update the website Downloads page to include Table Store links [7] **Vote Duration** The voting time will run for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Best, Jingsong Lee [1] https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+Table+Store+Release [2] https://dist.apache.org/repos/dist/dev/flink/flink-table-store-0.2.0-rc2/ [3] https://repository.apache.org/content/repositories/orgapacheflink-1523/ [4] https://dist.apache.org/repos/dist/release/flink/KEYS [5] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351570 [6] https://github.com/apache/flink-table-store/tree/release-0.2.0-rc2 [7] https://github.com/apache/flink-web/pull/562
[ANNOUNCE] New Apache Flink Committer - Lijie Wang
Hi everyone, On behalf of the PMC, I'm very happy to announce Lijie Wang as a new Flink committer. Lijie has been contributing to Flink project for more than 2 years. He mainly works on the runtime/coordination part, doing feature development, problem debugging and code reviews. He has also driven the work of FLIP-187(Adaptive Batch Scheduler) and FLIP-224(Blocklist for Speculative Execution), which are important to run batch jobs. Please join me in congratulating Lijie for becoming a Flink committer! Cheers, Zhu
[jira] [Created] (FLINK-29009) flink-end-to-end-tests-sql compile failed in hadoop3
Huang Xingbo created FLINK-29009:
Summary: flink-end-to-end-tests-sql compile failed in hadoop3
Key: FLINK-29009
URL: https://issues.apache.org/jira/browse/FLINK-29009
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Affects Versions: 1.16.0
Reporter: Huang Xingbo
Fix For: 1.16.0
{code:java}
2022-08-17T00:39:13.9082097Z Dependency convergence error for
com.nimbusds:nimbus-jose-jwt:4.41.1 paths to dependency are:
2022-08-17T00:39:13.9082987Z
+-org.apache.flink:flink-end-to-end-tests-sql:1.16-SNAPSHOT
2022-08-17T00:39:13.9083712Z +-org.apache.hadoop:hadoop-common:3.1.3
2022-08-17T00:39:13.9084340Z +-org.apache.hadoop:hadoop-auth:3.1.3
2022-08-17T00:39:13.9084963Z +-com.nimbusds:nimbus-jose-jwt:4.41.1
2022-08-17T00:39:13.9085616Z and
2022-08-17T00:39:13.9086212Z
+-org.apache.flink:flink-end-to-end-tests-sql:1.16-SNAPSHOT
2022-08-17T00:39:13.9086864Z +-org.apache.hadoop:hadoop-common:3.1.3
2022-08-17T00:39:13.9087499Z +-org.apache.kerby:kerb-simplekdc:1.0.1
2022-08-17T00:39:13.9088125Z +-org.apache.kerby:kerb-client:1.0.1
2022-08-17T00:39:13.9088753Z +-org.apache.kerby:token-provider:1.0.1
2022-08-17T00:39:13.9089381Z +-com.nimbusds:nimbus-jose-jwt:3.10
2022-08-17T00:39:13.9089596Z
2022-08-17T00:39:13.9090061Z [WARNING] Rule 0:
org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:
2022-08-17T00:39:13.9090651Z Failed while enforcing releasability. See above
detailed error message.
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=40084=logs=b1fcf054-9138-5463-c73c-a49979b9ac2a=9291ac46-dd95-5135-b799-3839e65a8691
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29008) SortMergeResultPartitionTest.testRelease failed with TestTimedOutException
Huang Xingbo created FLINK-29008:
Summary: SortMergeResultPartitionTest.testRelease failed with
TestTimedOutException
Key: FLINK-29008
URL: https://issues.apache.org/jira/browse/FLINK-29008
Project: Flink
Issue Type: Bug
Components: Runtime / Network
Affects Versions: 1.16.0
Reporter: Huang Xingbo
Fix For: 1.16.0
{code:java}
2022-08-17T02:58:01.8887337Z Aug 17 02:58:01 [ERROR]
SortMergeResultPartitionTest.testRelease Time elapsed: 60.843 s <<< ERROR!
2022-08-17T02:58:01.8887851Z Aug 17 02:58:01
org.junit.runners.model.TestTimedOutException: test timed out after 60 seconds
2022-08-17T02:58:01.8891703Z Aug 17 02:58:01at
org.apache.flink.runtime.io.network.partition.SortMergeResultPartitionTest.testRelease(SortMergeResultPartitionTest.java:374)
2022-08-17T02:58:01.8892705Z Aug 17 02:58:01at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-08-17T02:58:01.8893282Z Aug 17 02:58:01at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-08-17T02:58:01.8893966Z Aug 17 02:58:01at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-08-17T02:58:01.8898552Z Aug 17 02:58:01at
java.lang.reflect.Method.invoke(Method.java:498)
2022-08-17T02:58:01.8899210Z Aug 17 02:58:01at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
2022-08-17T02:58:01.8899844Z Aug 17 02:58:01at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
2022-08-17T02:58:01.8900581Z Aug 17 02:58:01at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
2022-08-17T02:58:01.8901226Z Aug 17 02:58:01at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
2022-08-17T02:58:01.8907420Z Aug 17 02:58:01at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
2022-08-17T02:58:01.8908068Z Aug 17 02:58:01at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
2022-08-17T02:58:01.8908735Z Aug 17 02:58:01at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
2022-08-17T02:58:01.8909447Z Aug 17 02:58:01at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
2022-08-17T02:58:01.8910255Z Aug 17 02:58:01at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
2022-08-17T02:58:01.8915691Z Aug 17 02:58:01at
java.lang.Thread.run(Thread.java:748)
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=40084=logs=77a9d8e1-d610-59b3-fc2a-4766541e0e33=125e07e7-8de0-5c6c-a541-a567415af3ef
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29011) Add Transformer for Binarizer
weibo zhao created FLINK-29011: -- Summary: Add Transformer for Binarizer Key: FLINK-29011 URL: https://issues.apache.org/jira/browse/FLINK-29011 Project: Flink Issue Type: New Feature Components: Library / Machine Learning Reporter: weibo zhao -- This message was sent by Atlassian Jira (v8.20.10#820010)
[VOTE] Release 1.15.2, release candidate #1
Hi everyone, Please review and vote on the release candidate #1 for the version 1.15.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], * the official Apache source release and binary convenience releases to be deployed to dist.apache.org [2], which are signed with the key with fingerprint 125FD8DB [3], * all artifacts to be deployed to the Maven Central Repository [4], * source code tag "release-1.15.2-rc1" [5], * website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Danny Cranmer [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351829 [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.15.2-rc1/ [3] https://dist.apache.org/repos/dist/release/flink/KEYS [4] https://repository.apache.org/content/repositories/orgapacheflink-1521/ [5] https://github.com/apache/flink/tree/release-1.15.2-rc1 [6] https://github.com/apache/flink-web/pull/566
[jira] [Created] (FLINK-29012) flink function doc is not correct
jackylau created FLINK-29012: Summary: flink function doc is not correct Key: FLINK-29012 URL: https://issues.apache.org/jira/browse/FLINK-29012 Project: Flink Issue Type: Bug Components: Documentation Affects Versions: 1.16.0 Reporter: jackylau Fix For: 1.16.0 Attachments: image-2022-08-17-17-15-39-702.png !image-2022-08-17-17-15-39-702.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29013) Fail to use "transform using" when record reader is binary record reader in Hive dialect
luoyuxia created FLINK-29013: Summary: Fail to use "transform using" when record reader is binary record reader in Hive dialect Key: FLINK-29013 URL: https://issues.apache.org/jira/browse/FLINK-29013 Project: Flink Issue Type: Bug Components: Connectors / Hive Affects Versions: 1.16.0 Reporter: luoyuxia Fix For: 1.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations, Lijie. Welcome on board~! Best, Guowei On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu wrote: > Hi everyone, > > On behalf of the PMC, I'm very happy to announce Lijie Wang as > a new Flink committer. > > Lijie has been contributing to Flink project for more than 2 years. > He mainly works on the runtime/coordination part, doing feature > development, problem debugging and code reviews. He has also > driven the work of FLIP-187(Adaptive Batch Scheduler) and > FLIP-224(Blocklist for Speculative Execution), which are important > to run batch jobs. > > Please join me in congratulating Lijie for becoming a Flink committer! > > Cheers, > Zhu >
[VOTE] Release 1.15.2, release candidate #2
Hi everyone, Please review and vote on the release candidate #2 for the version 1.15.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: - JIRA release notes [1], - the official Apache source release and binary convenience releases to be deployed to dist.apache.org [2], which are signed with the key with fingerprint 125FD8DB [3], - all artifacts to be deployed to the Maven Central Repository [4], - source code tag "release-1.15.2-rc2" [5], - website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. 1.15.2-RC1 was rejected for 2x issues: 1. Dist/src archives contained PaxHeader files when decompressing on Windows. Root cause was tar default archive format on Mac, fixed by using gnu-tar. I will follow up to update the release process to avoid this issue in the future. 2. Dist/src archives contained additional files. I had some locally gitignored files that ended up in the archive. New build used a fresh clone of Flink and I compared the archive contents of 1.15.1 with 1.15.2. Thanks, Danny Cranmer [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351829 [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.15.2-rc2/ [3] https://dist.apache.org/repos/dist/release/flink/KEYS [4] https://repository.apache.org/content/repositories/orgapacheflink-1524 [5] https://github.com/apache/flink/tree/release-1.15.2-rc2 [6] https://github.com/apache/flink-web/pull/566
Re: [ANNOUNCE] New Apache Flink Committer - Junhan Yang
Congratulations, Junhan! Best, Lijie Leonard Xu 于2022年8月18日周四 11:31写道： > Congratulations, Junhan! > > Best, > > > 2022年8月18日 上午11:27，Zhipeng Zhang 写道： > > > > Congratulations, Junhan! > > > > Xintong Song 于2022年8月18日周四 11:21写道： > >> > >> Hi everyone, > >> > >> On behalf of the PMC, I'm very happy to announce Junhan Yang as a new > Flink > >> committer. > >> > >> Junhan has been contributing to the Flink project for more than 1 year. > His > >> contributions are mostly identified in the web frontend, including > >> FLIP-241, FLIP-249 and various maintenance efforts of Flink's frontend > >> frameworks. > >> > >> Please join me in congratulating Junhan for becoming a Flink committer! > >> > >> Best, > >> Xintong > > > > > > > > -- > > best, > > Zhipeng > >
[jira] [Created] (FLINK-29023) Updating Jar statement document
dalongliu created FLINK-29023: - Summary: Updating Jar statement document Key: FLINK-29023 URL: https://issues.apache.org/jira/browse/FLINK-29023 Project: Flink Issue Type: Sub-task Components: Table SQL / API Affects Versions: 1.16.0 Reporter: dalongliu Fix For: 1.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29024) Add basic doc structure for Hive Compatibility
luoyuxia created FLINK-29024: Summary: Add basic doc structure for Hive Compatibility Key: FLINK-29024 URL: https://issues.apache.org/jira/browse/FLINK-29024 Project: Flink Issue Type: Sub-task Components: Connectors / Hive, Documentation Affects Versions: 1.16.0 Reporter: luoyuxia Fix For: 1.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29025) Add docs for Hive Dialect
luoyuxia created FLINK-29025: Summary: Add docs for Hive Dialect Key: FLINK-29025 URL: https://issues.apache.org/jira/browse/FLINK-29025 Project: Flink Issue Type: Sub-task Components: Connectors / Hive, Documentation Affects Versions: 1.16.0 Reporter: luoyuxia Fix For: 1.16.0 Moving some stuff from connectors/table/hive/hive_dialect. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29026) Add docs for HiveServer2 integration
luoyuxia created FLINK-29026: Summary: Add docs for HiveServer2 integration Key: FLINK-29026 URL: https://issues.apache.org/jira/browse/FLINK-29026 Project: Flink Issue Type: Sub-task Reporter: luoyuxia -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations, Lijie! Best, JunRui Timo Walther 于2022年8月17日周三 19:30写道： > Congratulations and welcome to the committer team :-) > > Regards, > Timo > > > On 17.08.22 12:50, Yuxin Tan wrote: > > Congratulations, Lijie! > > > > Best, > > Yuxin > > > > > > Guowei Ma 于2022年8月17日周三 18:42写道： > > > >> Congratulations, Lijie. Welcome on board~! > >> Best, > >> Guowei > >> > >> > >> On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu wrote: > >> > >>> Hi everyone, > >>> > >>> On behalf of the PMC, I'm very happy to announce Lijie Wang as > >>> a new Flink committer. > >>> > >>> Lijie has been contributing to Flink project for more than 2 years. > >>> He mainly works on the runtime/coordination part, doing feature > >>> development, problem debugging and code reviews. He has also > >>> driven the work of FLIP-187(Adaptive Batch Scheduler) and > >>> FLIP-224(Blocklist for Speculative Execution), which are important > >>> to run batch jobs. > >>> > >>> Please join me in congratulating Lijie for becoming a Flink committer! > >>> > >>> Cheers, > >>> Zhu > >>> > >> > > > >
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations, Lijie! Best, Yanfei JunRui Lee 于2022年8月18日周四 10:35写道： > Congratulations, Lijie! > > Best, > JunRui > > Timo Walther 于2022年8月17日周三 19:30写道： > > > Congratulations and welcome to the committer team :-) > > > > Regards, > > Timo > > > > > > On 17.08.22 12:50, Yuxin Tan wrote: > > > Congratulations, Lijie! > > > > > > Best, > > > Yuxin > > > > > > > > > Guowei Ma 于2022年8月17日周三 18:42写道： > > > > > >> Congratulations, Lijie. Welcome on board~! > > >> Best, > > >> Guowei > > >> > > >> > > >> On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu wrote: > > >> > > >>> Hi everyone, > > >>> > > >>> On behalf of the PMC, I'm very happy to announce Lijie Wang as > > >>> a new Flink committer. > > >>> > > >>> Lijie has been contributing to Flink project for more than 2 years. > > >>> He mainly works on the runtime/coordination part, doing feature > > >>> development, problem debugging and code reviews. He has also > > >>> driven the work of FLIP-187(Adaptive Batch Scheduler) and > > >>> FLIP-224(Blocklist for Speculative Execution), which are important > > >>> to run batch jobs. > > >>> > > >>> Please join me in congratulating Lijie for becoming a Flink > committer! > > >>> > > >>> Cheers, > > >>> Zhu > > >>> > > >> > > > > > > > >
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations Lijie! yanfei lei 于2022年8月18日周四 10:44写道： > Congratulations, Lijie! > > Best, > Yanfei > > JunRui Lee 于2022年8月18日周四 10:35写道： > > > Congratulations, Lijie! > > > > Best, > > JunRui > > > > Timo Walther 于2022年8月17日周三 19:30写道： > > > > > Congratulations and welcome to the committer team :-) > > > > > > Regards, > > > Timo > > > > > > > > > On 17.08.22 12:50, Yuxin Tan wrote: > > > > Congratulations, Lijie! > > > > > > > > Best, > > > > Yuxin > > > > > > > > > > > > Guowei Ma 于2022年8月17日周三 18:42写道： > > > > > > > >> Congratulations, Lijie. Welcome on board~! > > > >> Best, > > > >> Guowei > > > >> > > > >> > > > >> On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu wrote: > > > >> > > > >>> Hi everyone, > > > >>> > > > >>> On behalf of the PMC, I'm very happy to announce Lijie Wang as > > > >>> a new Flink committer. > > > >>> > > > >>> Lijie has been contributing to Flink project for more than 2 years. > > > >>> He mainly works on the runtime/coordination part, doing feature > > > >>> development, problem debugging and code reviews. He has also > > > >>> driven the work of FLIP-187(Adaptive Batch Scheduler) and > > > >>> FLIP-224(Blocklist for Speculative Execution), which are important > > > >>> to run batch jobs. > > > >>> > > > >>> Please join me in congratulating Lijie for becoming a Flink > > committer! > > > >>> > > > >>> Cheers, > > > >>> Zhu > > > >>> > > > >> > > > > > > > > > > > > > -- Best, Benchao Li
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations, Lijie! Best, Lincoln Lee Benchao Li 于2022年8月18日周四 10:51写道： > Congratulations Lijie! > > yanfei lei 于2022年8月18日周四 10:44写道： > > > Congratulations, Lijie! > > > > Best, > > Yanfei > > > > JunRui Lee 于2022年8月18日周四 10:35写道： > > > > > Congratulations, Lijie! > > > > > > Best, > > > JunRui > > > > > > Timo Walther 于2022年8月17日周三 19:30写道： > > > > > > > Congratulations and welcome to the committer team :-) > > > > > > > > Regards, > > > > Timo > > > > > > > > > > > > On 17.08.22 12:50, Yuxin Tan wrote: > > > > > Congratulations, Lijie! > > > > > > > > > > Best, > > > > > Yuxin > > > > > > > > > > > > > > > Guowei Ma 于2022年8月17日周三 18:42写道： > > > > > > > > > >> Congratulations, Lijie. Welcome on board~! > > > > >> Best, > > > > >> Guowei > > > > >> > > > > >> > > > > >> On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu wrote: > > > > >> > > > > >>> Hi everyone, > > > > >>> > > > > >>> On behalf of the PMC, I'm very happy to announce Lijie Wang as > > > > >>> a new Flink committer. > > > > >>> > > > > >>> Lijie has been contributing to Flink project for more than 2 > years. > > > > >>> He mainly works on the runtime/coordination part, doing feature > > > > >>> development, problem debugging and code reviews. He has also > > > > >>> driven the work of FLIP-187(Adaptive Batch Scheduler) and > > > > >>> FLIP-224(Blocklist for Speculative Execution), which are > important > > > > >>> to run batch jobs. > > > > >>> > > > > >>> Please join me in congratulating Lijie for becoming a Flink > > > committer! > > > > >>> > > > > >>> Cheers, > > > > >>> Zhu > > > > >>> > > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > Best, > Benchao Li >
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congrats, Lijie Best, Xingbo Lincoln Lee 于2022年8月18日周四 11:01写道： > Congratulations, Lijie! > > Best, > Lincoln Lee > > > Benchao Li 于2022年8月18日周四 10:51写道： > > > Congratulations Lijie! > > > > yanfei lei 于2022年8月18日周四 10:44写道： > > > > > Congratulations, Lijie! > > > > > > Best, > > > Yanfei > > > > > > JunRui Lee 于2022年8月18日周四 10:35写道： > > > > > > > Congratulations, Lijie! > > > > > > > > Best, > > > > JunRui > > > > > > > > Timo Walther 于2022年8月17日周三 19:30写道： > > > > > > > > > Congratulations and welcome to the committer team :-) > > > > > > > > > > Regards, > > > > > Timo > > > > > > > > > > > > > > > On 17.08.22 12:50, Yuxin Tan wrote: > > > > > > Congratulations, Lijie! > > > > > > > > > > > > Best, > > > > > > Yuxin > > > > > > > > > > > > > > > > > > Guowei Ma 于2022年8月17日周三 18:42写道： > > > > > > > > > > > >> Congratulations, Lijie. Welcome on board~! > > > > > >> Best, > > > > > >> Guowei > > > > > >> > > > > > >> > > > > > >> On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu > wrote: > > > > > >> > > > > > >>> Hi everyone, > > > > > >>> > > > > > >>> On behalf of the PMC, I'm very happy to announce Lijie Wang as > > > > > >>> a new Flink committer. > > > > > >>> > > > > > >>> Lijie has been contributing to Flink project for more than 2 > > years. > > > > > >>> He mainly works on the runtime/coordination part, doing feature > > > > > >>> development, problem debugging and code reviews. He has also > > > > > >>> driven the work of FLIP-187(Adaptive Batch Scheduler) and > > > > > >>> FLIP-224(Blocklist for Speculative Execution), which are > > important > > > > > >>> to run batch jobs. > > > > > >>> > > > > > >>> Please join me in congratulating Lijie for becoming a Flink > > > > committer! > > > > > >>> > > > > > >>> Cheers, > > > > > >>> Zhu > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best, > > Benchao Li > > >
[ANNOUNCE] New Apache Flink Committer - Junhan Yang
Hi everyone, On behalf of the PMC, I'm very happy to announce Junhan Yang as a new Flink committer. Junhan has been contributing to the Flink project for more than 1 year. His contributions are mostly identified in the web frontend, including FLIP-241, FLIP-249 and various maintenance efforts of Flink's frontend frameworks. Please join me in congratulating Junhan for becoming a Flink committer! Best, Xintong
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations Lijie, and welcome~! Best, Xintong On Thu, Aug 18, 2022 at 11:12 AM Xingbo Huang wrote: > Congrats, Lijie > > Best, > Xingbo > > Lincoln Lee 于2022年8月18日周四 11:01写道： > > > Congratulations, Lijie! > > > > Best, > > Lincoln Lee > > > > > > Benchao Li 于2022年8月18日周四 10:51写道： > > > > > Congratulations Lijie! > > > > > > yanfei lei 于2022年8月18日周四 10:44写道： > > > > > > > Congratulations, Lijie! > > > > > > > > Best, > > > > Yanfei > > > > > > > > JunRui Lee 于2022年8月18日周四 10:35写道： > > > > > > > > > Congratulations, Lijie! > > > > > > > > > > Best, > > > > > JunRui > > > > > > > > > > Timo Walther 于2022年8月17日周三 19:30写道： > > > > > > > > > > > Congratulations and welcome to the committer team :-) > > > > > > > > > > > > Regards, > > > > > > Timo > > > > > > > > > > > > > > > > > > On 17.08.22 12:50, Yuxin Tan wrote: > > > > > > > Congratulations, Lijie! > > > > > > > > > > > > > > Best, > > > > > > > Yuxin > > > > > > > > > > > > > > > > > > > > > Guowei Ma 于2022年8月17日周三 18:42写道： > > > > > > > > > > > > > >> Congratulations, Lijie. Welcome on board~! > > > > > > >> Best, > > > > > > >> Guowei > > > > > > >> > > > > > > >> > > > > > > >> On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu > > wrote: > > > > > > >> > > > > > > >>> Hi everyone, > > > > > > >>> > > > > > > >>> On behalf of the PMC, I'm very happy to announce Lijie Wang > as > > > > > > >>> a new Flink committer. > > > > > > >>> > > > > > > >>> Lijie has been contributing to Flink project for more than 2 > > > years. > > > > > > >>> He mainly works on the runtime/coordination part, doing > feature > > > > > > >>> development, problem debugging and code reviews. He has also > > > > > > >>> driven the work of FLIP-187(Adaptive Batch Scheduler) and > > > > > > >>> FLIP-224(Blocklist for Speculative Execution), which are > > > important > > > > > > >>> to run batch jobs. > > > > > > >>> > > > > > > >>> Please join me in congratulating Lijie for becoming a Flink > > > > > committer! > > > > > > >>> > > > > > > >>> Cheers, > > > > > > >>> Zhu > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best, > > > Benchao Li > > > > > >
Re: [ANNOUNCE] New Apache Flink Committer - Junhan Yang
Congratulations, Junhan! Best, > 2022年8月18日 上午11:27，Zhipeng Zhang 写道： > > Congratulations, Junhan! > > Xintong Song 于2022年8月18日周四 11:21写道： >> >> Hi everyone, >> >> On behalf of the PMC, I'm very happy to announce Junhan Yang as a new Flink >> committer. >> >> Junhan has been contributing to the Flink project for more than 1 year. His >> contributions are mostly identified in the web frontend, including >> FLIP-241, FLIP-249 and various maintenance efforts of Flink's frontend >> frameworks. >> >> Please join me in congratulating Junhan for becoming a Flink committer! >> >> Best, >> Xintong > > > > -- > best, > Zhipeng
[jira] [Created] (FLINK-29019) Updating parquet format document that support complex type
dalongliu created FLINK-29019: - Summary: Updating parquet format document that support complex type Key: FLINK-29019 URL: https://issues.apache.org/jira/browse/FLINK-29019 Project: Flink Issue Type: New Feature Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) Affects Versions: 1.16.0 Reporter: dalongliu Fix For: 1.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29020) Add document for CTAS feature
dalongliu created FLINK-29020: - Summary: Add document for CTAS feature Key: FLINK-29020 URL: https://issues.apache.org/jira/browse/FLINK-29020 Project: Flink Issue Type: Sub-task Components: Table SQL / API Affects Versions: 1.16.0 Reporter: dalongliu Fix For: 1.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29021) [Umbrella] Add docs for Hive Compatibility
luoyuxia created FLINK-29021: Summary: [Umbrella] Add docs for Hive Compatibility Key: FLINK-29021 URL: https://issues.apache.org/jira/browse/FLINK-29021 Project: Flink Issue Type: Improvement Components: Documentation Affects Versions: 1.16.0 Reporter: luoyuxia Fix For: 1.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29018) KafkaITCase.testMultipleSourcesOnePartition failed with TopicExistsException
Huang Xingbo created FLINK-29018:
Summary: KafkaITCase.testMultipleSourcesOnePartition failed with
TopicExistsException
Key: FLINK-29018
URL: https://issues.apache.org/jira/browse/FLINK-29018
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Affects Versions: 1.16.0
Reporter: Huang Xingbo
Fix For: 1.16.0
{code:java}
2022-08-17T22:52:03.5376972Z Aug 17 22:52:03 [ERROR] Tests run: 23, Failures:
1, Errors: 0, Skipped: 0, Time elapsed: 133.5 s <<< FAILURE! - in
org.apache.flink.streaming.connectors.kafka.KafkaITCase
2022-08-17T22:52:03.5378664Z Aug 17 22:52:03 [ERROR]
org.apache.flink.streaming.connectors.kafka.KafkaITCase.testMultipleSourcesOnePartition
Time elapsed: 3.967 s <<< FAILURE!
2022-08-17T22:52:03.5380231Z Aug 17 22:52:03 java.lang.AssertionError: Create
test topic : manyToOneTopic failed,
org.apache.kafka.common.errors.TopicExistsException: Topic 'manyToOneTopic'
already exists.
2022-08-17T22:52:03.5381136Z Aug 17 22:52:03at
org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:207)
2022-08-17T22:52:03.5381983Z Aug 17 22:52:03at
org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:97)
2022-08-17T22:52:03.5382935Z Aug 17 22:52:03at
org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:217)
2022-08-17T22:52:03.5384026Z Aug 17 22:52:03at
org.apache.flink.streaming.connectors.kafka.KafkaConsumerTestBase.runMultipleSourcesOnePartitionExactlyOnceTest(KafkaConsumerTestBase.java:1053)
2022-08-17T22:52:03.5385057Z Aug 17 22:52:03at
org.apache.flink.streaming.connectors.kafka.KafkaITCase.testMultipleSourcesOnePartition(KafkaITCase.java:105)
2022-08-17T22:52:03.5385741Z Aug 17 22:52:03at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-08-17T22:52:03.5386344Z Aug 17 22:52:03at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-08-17T22:52:03.5387038Z Aug 17 22:52:03at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-08-17T22:52:03.5387658Z Aug 17 22:52:03at
java.lang.reflect.Method.invoke(Method.java:498)
2022-08-17T22:52:03.5388350Z Aug 17 22:52:03at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
2022-08-17T22:52:03.5389118Z Aug 17 22:52:03at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
2022-08-17T22:52:03.5389875Z Aug 17 22:52:03at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
2022-08-17T22:52:03.5390556Z Aug 17 22:52:03at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
2022-08-17T22:52:03.5391564Z Aug 17 22:52:03at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
2022-08-17T22:52:03.5392325Z Aug 17 22:52:03at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
2022-08-17T22:52:03.5392985Z Aug 17 22:52:03at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
2022-08-17T22:52:03.5393687Z Aug 17 22:52:03at
java.lang.Thread.run(Thread.java:748)
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=40124=logs=aa18c3f6-13b8-5f58-86bb-c1cffb239496=502fb6c0-30a2-5e49-c5c2-a00fa3acb203=37261
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink Committer - Junhan Yang
Congratulations, Junhan! Xintong Song 于2022年8月18日周四 11:21写道： > > Hi everyone, > > On behalf of the PMC, I'm very happy to announce Junhan Yang as a new Flink > committer. > > Junhan has been contributing to the Flink project for more than 1 year. His > contributions are mostly identified in the web frontend, including > FLIP-241, FLIP-249 and various maintenance efforts of Flink's frontend > frameworks. > > Please join me in congratulating Junhan for becoming a Flink committer! > > Best, > Xintong -- best, Zhipeng
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations, Lijie! Xintong Song 于2022年8月18日周四 11:23写道： > > Congratulations Lijie, and welcome~! > > Best, > > Xintong > > > > On Thu, Aug 18, 2022 at 11:12 AM Xingbo Huang wrote: > > > Congrats, Lijie > > > > Best, > > Xingbo > > > > Lincoln Lee 于2022年8月18日周四 11:01写道： > > > > > Congratulations, Lijie! > > > > > > Best, > > > Lincoln Lee > > > > > > > > > Benchao Li 于2022年8月18日周四 10:51写道： > > > > > > > Congratulations Lijie! > > > > > > > > yanfei lei 于2022年8月18日周四 10:44写道： > > > > > > > > > Congratulations, Lijie! > > > > > > > > > > Best, > > > > > Yanfei > > > > > > > > > > JunRui Lee 于2022年8月18日周四 10:35写道： > > > > > > > > > > > Congratulations, Lijie! > > > > > > > > > > > > Best, > > > > > > JunRui > > > > > > > > > > > > Timo Walther 于2022年8月17日周三 19:30写道： > > > > > > > > > > > > > Congratulations and welcome to the committer team :-) > > > > > > > > > > > > > > Regards, > > > > > > > Timo > > > > > > > > > > > > > > > > > > > > > On 17.08.22 12:50, Yuxin Tan wrote: > > > > > > > > Congratulations, Lijie! > > > > > > > > > > > > > > > > Best, > > > > > > > > Yuxin > > > > > > > > > > > > > > > > > > > > > > > > Guowei Ma 于2022年8月17日周三 18:42写道： > > > > > > > > > > > > > > > >> Congratulations, Lijie. Welcome on board~! > > > > > > > >> Best, > > > > > > > >> Guowei > > > > > > > >> > > > > > > > >> > > > > > > > >> On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu > > > wrote: > > > > > > > >> > > > > > > > >>> Hi everyone, > > > > > > > >>> > > > > > > > >>> On behalf of the PMC, I'm very happy to announce Lijie Wang > > as > > > > > > > >>> a new Flink committer. > > > > > > > >>> > > > > > > > >>> Lijie has been contributing to Flink project for more than 2 > > > > years. > > > > > > > >>> He mainly works on the runtime/coordination part, doing > > feature > > > > > > > >>> development, problem debugging and code reviews. He has also > > > > > > > >>> driven the work of FLIP-187(Adaptive Batch Scheduler) and > > > > > > > >>> FLIP-224(Blocklist for Speculative Execution), which are > > > > important > > > > > > > >>> to run batch jobs. > > > > > > > >>> > > > > > > > >>> Please join me in congratulating Lijie for becoming a Flink > > > > > > committer! > > > > > > > >>> > > > > > > > >>> Cheers, > > > > > > > >>> Zhu > > > > > > > >>> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best, > > > > Benchao Li > > > > > > > > > -- best, Zhipeng
Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang
Congratulations, Lijie! Best, Leonard > 2022年8月18日 上午11:26，Zhipeng Zhang 写道： > > Congratulations, Lijie! > > Xintong Song 于2022年8月18日周四 11:23写道： >> >> Congratulations Lijie, and welcome~! >> >> Best, >> >> Xintong >> >> >> >> On Thu, Aug 18, 2022 at 11:12 AM Xingbo Huang wrote: >> >>> Congrats, Lijie >>> >>> Best, >>> Xingbo >>> >>> Lincoln Lee 于2022年8月18日周四 11:01写道： >>> Congratulations, Lijie! Best, Lincoln Lee Benchao Li 于2022年8月18日周四 10:51写道： > Congratulations Lijie! > > yanfei lei 于2022年8月18日周四 10:44写道： > >> Congratulations, Lijie! >> >> Best, >> Yanfei >> >> JunRui Lee 于2022年8月18日周四 10:35写道： >> >>> Congratulations, Lijie! >>> >>> Best, >>> JunRui >>> >>> Timo Walther 于2022年8月17日周三 19:30写道： >>> Congratulations and welcome to the committer team :-) Regards, Timo On 17.08.22 12:50, Yuxin Tan wrote: > Congratulations, Lijie! > > Best, > Yuxin > > > Guowei Ma 于2022年8月17日周三 18:42写道： > >> Congratulations, Lijie. Welcome on board~! >> Best, >> Guowei >> >> >> On Wed, Aug 17, 2022 at 6:25 PM Zhu Zhu wrote: >> >>> Hi everyone, >>> >>> On behalf of the PMC, I'm very happy to announce Lijie Wang >>> as >>> a new Flink committer. >>> >>> Lijie has been contributing to Flink project for more than 2 > years. >>> He mainly works on the runtime/coordination part, doing >>> feature >>> development, problem debugging and code reviews. He has also >>> driven the work of FLIP-187(Adaptive Batch Scheduler) and >>> FLIP-224(Blocklist for Speculative Execution), which are > important >>> to run batch jobs. >>> >>> Please join me in congratulating Lijie for becoming a Flink >>> committer! >>> >>> Cheers, >>> Zhu >>> >> > >>> >> > > > -- > > Best, > Benchao Li > >>> > > > > -- > best, > Zhipeng
[jira] [Created] (FLINK-29022) Add document for CREATE FUNCTIOn USING JAR feature
dalongliu created FLINK-29022: - Summary: Add document for CREATE FUNCTIOn USING JAR feature Key: FLINK-29022 URL: https://issues.apache.org/jira/browse/FLINK-29022 Project: Flink Issue Type: Sub-task Components: Table SQL / API Affects Versions: 1.16.0 Reporter: dalongliu Fix For: 1.16.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
