[jira] [Created] (FLINK-18149) Taskmanager logs could not show up in native K8s deployment

2020-06-05 Thread Yang Wang (Jira)
Yang Wang created FLINK-18149: - Summary: Taskmanager logs could not show up in native K8s deployment Key: FLINK-18149 URL: https://issues.apache.org/jira/browse/FLINK-18149 Project: Flink Issue

Re: [ANNOUNCE] New Apache Flink Committer - Xintong Song

2020-06-05 Thread Yang Wang
Congratulations, Xintong. Well deserved. Best, Yang Weihua Hu 于2020年6月5日周五 下午1:57写道: > Congratulations! > > Best > Weihua Hu > > > 2020年6月5日 13:50,Xingbo Huang 写道: > > > > Congratulations, Xintong. > > > > Best, > > Xingbo > > > > Jiayi Liao 于2020年6月5日周五 下午1:49写道: > > > >> Congratulations! >

[jira] [Created] (FLINK-18087) Uploading user artifact for Yarn job cluster could not work

2020-06-03 Thread Yang Wang (Jira)
Yang Wang created FLINK-18087: - Summary: Uploading user artifact for Yarn job cluster could not work Key: FLINK-18087 URL: https://issues.apache.org/jira/browse/FLINK-18087 Project: Flink Issue

[jira] [Created] (FLINK-18085) Manually test application mode for standalone, Yarn, K8s

2020-06-03 Thread Yang Wang (Jira)
Yang Wang created FLINK-18085: - Summary: Manually test application mode for standalone, Yarn, K8s Key: FLINK-18085 URL: https://issues.apache.org/jira/browse/FLINK-18085 Project: Flink Issue

[jira] [Created] (FLINK-18048) "--host" option could not take effect for standalone application cluster

2020-06-01 Thread Yang Wang (Jira)
Yang Wang created FLINK-18048: - Summary: "--host" option could not take effect for standalone application cluster Key: FLINK-18048 URL: https://issues.apache.org/jira/browse/FLINK-18048 Proj

[jira] [Created] (FLINK-18035) Executors#newCachedThreadPool could not work as expected

2020-05-29 Thread Yang Wang (Jira)
Yang Wang created FLINK-18035: - Summary: Executors#newCachedThreadPool could not work as expected Key: FLINK-18035 URL: https://issues.apache.org/jira/browse/FLINK-18035 Project: Flink Issue

[jira] [Created] (FLINK-17958) Kubernetes session constantly allocates taskmanagers after cancel a job

2020-05-27 Thread Yang Wang (Jira)
Yang Wang created FLINK-17958: - Summary: Kubernetes session constantly allocates taskmanagers after cancel a job Key: FLINK-17958 URL: https://issues.apache.org/jira/browse/FLINK-17958 Project: Flink

[jira] [Created] (FLINK-17935) Logs could not show up when deploying Flink on Yarn via "--executor"

2020-05-25 Thread Yang Wang (Jira)
Yang Wang created FLINK-17935: - Summary: Logs could not show up when deploying Flink on Yarn via "--executor" Key: FLINK-17935 URL: https://issues.apache.org/jira/browse/FLINK-17935 Proj

Re: [ANNOUNCE] Apache Flink 1.11.0, release candidate #1

2020-05-25 Thread Yang Wang
Since the official image for 1.11 has not been release, I just want to post how to build the image with provided Flink binary. I think many users are running Flink in container environment(e.g. docker, Kubernetes, etc.) and hope this could help them with the testing. git clone

Re: [DISCUSS] Semantics of our JIRA fields

2020-05-25 Thread Yang Wang
+1 from this useful proposal. This makes me clearer about "Resolve" and "Close" since I used to be confused by this two button. Best, Yang Jingsong Li 于2020年5月25日周一 下午3:10写道: > +1 for the proposal. > It makes me clearer. > > Best, > Jingsong Lee > > On Mon, May 25, 2020 at 2:51 PM Zhijiang

[jira] [Created] (FLINK-17857) All Kubernetes e2e tests could not run on Mac after migration

2020-05-20 Thread Yang Wang (Jira)
Yang Wang created FLINK-17857: - Summary: All Kubernetes e2e tests could not run on Mac after migration Key: FLINK-17857 URL: https://issues.apache.org/jira/browse/FLINK-17857 Project: Flink

[jira] [Created] (FLINK-17814) Translate native kubernetes document to Chinese

2020-05-19 Thread Yang Wang (Jira)
Yang Wang created FLINK-17814: - Summary: Translate native kubernetes document to Chinese Key: FLINK-17814 URL: https://issues.apache.org/jira/browse/FLINK-17814 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-17810) Add document for K8s application mode

2020-05-19 Thread Yang Wang (Jira)
Yang Wang created FLINK-17810: - Summary: Add document for K8s application mode Key: FLINK-17810 URL: https://issues.apache.org/jira/browse/FLINK-17810 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17796) Application mode should respect user specified classpath configuration

2020-05-18 Thread Yang Wang (Jira)
Yang Wang created FLINK-17796: - Summary: Application mode should respect user specified classpath configuration Key: FLINK-17796 URL: https://issues.apache.org/jira/browse/FLINK-17796 Project: Flink

[jira] [Created] (FLINK-17648) YarnApplicationClusterEntryPoint should override getRPCPortRange

2020-05-12 Thread Yang Wang (Jira)
Yang Wang created FLINK-17648: - Summary: YarnApplicationClusterEntryPoint should override getRPCPortRange Key: FLINK-17648 URL: https://issues.apache.org/jira/browse/FLINK-17648 Project: Flink

[jira] [Created] (FLINK-17632) Support to specify a remote path for job jar

2020-05-12 Thread Yang Wang (Jira)
Yang Wang created FLINK-17632: - Summary: Support to specify a remote path for job jar Key: FLINK-17632 URL: https://issues.apache.org/jira/browse/FLINK-17632 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17624) "TPC-DS end-to-end test (Blink planner)' is unstable

2020-05-11 Thread Yang Wang (Jira)
Yang Wang created FLINK-17624: - Summary: "TPC-DS end-to-end test (Blink planner)' is unstable Key: FLINK-17624 URL: https://issues.apache.org/jira/browse/FLINK-17624 Project: Flink Issue

[jira] [Created] (FLINK-17618) Update the outdated comments in the log4j properties files

2020-05-11 Thread Yang Wang (Jira)
Yang Wang created FLINK-17618: - Summary: Update the outdated comments in the log4j properties files Key: FLINK-17618 URL: https://issues.apache.org/jira/browse/FLINK-17618 Project: Flink Issue

[jira] [Created] (FLINK-17613) Run K8s related e2e tests with multiple K8s versions

2020-05-11 Thread Yang Wang (Jira)
Yang Wang created FLINK-17613: - Summary: Run K8s related e2e tests with multiple K8s versions Key: FLINK-17613 URL: https://issues.apache.org/jira/browse/FLINK-17613 Project: Flink Issue Type

Re: [DISCUSS] Have foreground processes also create log files

2020-05-05 Thread Yang Wang
;> > > > >> > Hi everyone, > > > >> > > > > >> > thanks for starting this discussion Chesnay. > > > >> > > > > >> > I think it would be nice if we also displayed the logs when > starting > &g

Re: [DISCUSS] Introduce The Batch/Stream ExecutionEnvironment in Yarn mode

2020-05-05 Thread Yang Wang
Hi Roc Marshal, I have a question about making the Yarn deployment ASAP. In my opinion, using the "ExecutionEnvironment" instead of "flink run -m yarn-cluster" to deploy a Flink cluster on Yarn do not help to reduce the time cost. Since we still need to ship the user jars, flink libs to the HDFS

Re: [DISCUSS][FLIP-108] Problems regarding the class loader and dependency

2020-04-29 Thread Yang Wang
I am also in favor of the option3. Since the Flink FileSystem has the very similar implementation via plugin mechanism. It has a map "FS_FACTORIES" to store the plugin-loaded specific FileSystem(e.g. S3, AzureFS, OSS, etc.). And provide some common interfaces. Best, Yang Yangze Guo

Re: [DISCUSS] Have foreground processes also create log files

2020-04-28 Thread Yang Wang
Thanks for Chesnay starting this discussion. In FLINK-17166 implementation[1], we are trying to use "tee" instead of introducing the stream redirection(redirect the out/err to files). However, a side effect is that the logging will be duplicated both in .log and .out files. Then it may consume

Re: [DISCUSS] Intermediary releases of the flink-docker images

2020-04-28 Thread Yang Wang
Thanks for starting this discussion. In general, i am also in favor of making docker image release could be partly decoupled from Flink release. However, i think it should only happen when we want to make some changes that are independent from Flink majar release(e.g. JDK11, change base image,

Re: [DISCUSS] Add custom labels on AbstractPrometheusReporter like PrometheusPushGatewayReporter's groupingKey

2020-04-26 Thread Yang Wang
Hi jinhai, I think it is a good improvement to also add groupingKey to PrometheusReporter.And making the PrometheusPushGatewayReporter and PrometheusReporter always share the same logics is important. AFAIK, they should not have differences except for the metrics exposing mechanism

Re: [DISCUSS] Adding support for Hadoop 3 and removing flink-shaded-hadoop

2020-04-22 Thread Yang Wang
Thanks Robert for starting this significant discussion. Since hadoop3 has been released for long time and many companies have already put it in production. No matter you are using flink-shaded-hadoop2 or not, currently Flink could already run in yarn3(not sure about HDFS). Since the yarn api is

[jira] [Created] (FLINK-17264) taskmanager.sh could not work on Mac

2020-04-20 Thread Yang Wang (Jira)
Yang Wang created FLINK-17264: - Summary: taskmanager.sh could not work on Mac Key: FLINK-17264 URL: https://issues.apache.org/jira/browse/FLINK-17264 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-17225) Support native k8s for scala shell

2020-04-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-17225: - Summary: Support native k8s for scala shell Key: FLINK-17225 URL: https://issues.apache.org/jira/browse/FLINK-17225 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-17154) Make ResourceManager#internalDeregisterApplication and implementation asynchronous

2020-04-15 Thread Yang Wang (Jira)
Yang Wang created FLINK-17154: - Summary: Make ResourceManager#internalDeregisterApplication and implementation asynchronous Key: FLINK-17154 URL: https://issues.apache.org/jira/browse/FLINK-17154 Project

[jira] [Created] (FLINK-17127) Make pod creating retry interval configurable

2020-04-13 Thread Yang Wang (Jira)
Yang Wang created FLINK-17127: - Summary: Make pod creating retry interval configurable Key: FLINK-17127 URL: https://issues.apache.org/jira/browse/FLINK-17127 Project: Flink Issue Type: New

Re: [DISCUSS] Generating java code?

2020-04-13 Thread Yang Wang
Hi Niels, Thanks a lot for starting this discussion. Although the latter option is more stable, i think it is not acceptable for all the developers to execute `mvn generate-sources` first. Otherwise, the Flink project is just broken and could not run tests, Flink jobs in IDE. So i think the

[jira] [Created] (FLINK-17085) Remove FlinkKubeClient#handleException since it does not do anything

2020-04-10 Thread Yang Wang (Jira)
Yang Wang created FLINK-17085: - Summary: Remove FlinkKubeClient#handleException since it does not do anything Key: FLINK-17085 URL: https://issues.apache.org/jira/browse/FLINK-17085 Project: Flink

Re: [VOTE] FLIP-111: Docker image unification

2020-04-07 Thread Yang Wang
Thanks Andrey for the efforts of docker image unification. +1 (non-binding) Best, Yang Till Rohrmann 于2020年4月7日周二 下午11:04写道: > Thanks for driving this effort Andrey. > > +1 (binding) > > Cheers, > Till > > On Tue, Apr 7, 2020 at 4:48 PM Ufuk Celebi wrote: > > > Thanks for starting this FLIP.

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-07 Thread Yang Wang
Thanks Yangze for the efforts to support GPU extended resources. +1 for this FLIP Best, Yang Till Rohrmann 于2020年4月2日周四 下午11:10写道: > Thanks for driving this effort Yangze. > > +1 > > Cheers, > Till > > On Wed, Apr 1, 2020 at 12:41 PM Canbin Zheng > wrote: > > > Thanks Yangze for driving the

[jira] [Created] (FLINK-17024) Make RpcEndpoint#getRpcService#getExecutor return a custom thread pool

2020-04-07 Thread Yang Wang (Jira)
Yang Wang created FLINK-17024: - Summary: Make RpcEndpoint#getRpcService#getExecutor return a custom thread pool Key: FLINK-17024 URL: https://issues.apache.org/jira/browse/FLINK-17024 Project: Flink

Re: [DISCUSS] FLIP-111: Docker image unification

2020-04-02 Thread Yang Wang
Hi Ufuk, Thanks for make the conclusion and directly point out what need to be done in FLIP-111. I agree with you that we should narrow down the scope and focus the most important and basic part about docker image unification. (1) Extend the entrypoint script in apache/flink-docker to start the

Re: [ANNOUNCE] New Committers and PMC member

2020-04-01 Thread Yang Wang
Congratulations all. Best, Yang Leonard Xu 于2020年4月1日周三 下午10:15写道: > Congratulations Konstantin, Dawid and Zhijiang! Well deserved! > > Best, > Leonard Xu > > 在 2020年4月1日,21:22,Jark Wu 写道: > > > > Congratulations to you all! > > > > Best, > > Jark > > > > On Wed, 1 Apr 2020 at 20:33, Kurt

[jira] [Created] (FLINK-16856) taskmanager.sh is broken because of invalid arguments of head

2020-03-29 Thread Yang Wang (Jira)
Yang Wang created FLINK-16856: - Summary: taskmanager.sh is broken because of invalid arguments of head Key: FLINK-16856 URL: https://issues.apache.org/jira/browse/FLINK-16856 Project: Flink

Re: [DISCUSS] FLIP-118: Improve Flink’s ID system

2020-03-29 Thread Yang Wang
Hi Konstantin, I think it is a good idea. Currently, our users also report a similar issue with resourceId of standalone cluster. When we start a standalone cluster now, the `TaskManagerRunner` always generates a uuid for the resourceId. It will be used to register to the jobmanager and not

[jira] [Created] (FLINK-16760) Support the yaml file submission for native Kubernetes integration

2020-03-24 Thread Yang Wang (Jira)
Yang Wang created FLINK-16760: - Summary: Support the yaml file submission for native Kubernetes integration Key: FLINK-16760 URL: https://issues.apache.org/jira/browse/FLINK-16760 Project: Flink

[jira] [Created] (FLINK-16749) Support to set node selector for JM/TM pod

2020-03-24 Thread Yang Wang (Jira)
Yang Wang created FLINK-16749: - Summary: Support to set node selector for JM/TM pod Key: FLINK-16749 URL: https://issues.apache.org/jira/browse/FLINK-16749 Project: Flink Issue Type: Sub-task

Re: [VOTE] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-19 Thread Yang Wang
+1 (non-binding). This FLIP will make jobmanager memory configuration keep up with taskmanager. Best, Yang Xintong Song 于2020年3月19日周四 上午11:23写道: > +1 (non-binding) from my side. > > Thank you~ > > Xintong Song > > > > On Wed, Mar 18, 2020 at 10:16 PM Andrey Zagrebin > wrote: > > > Hi All, >

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-17 Thread Yang Wang
gt; >> To customise this in the current mainstream image is difficult, this >> would >> >> require to ship it w/o Java at all. >> >> Or this is a separate discussion whether we want to distribute docker >> hub >> >> images with different Jav

Re: How to change the flink web-ui jobServer?

2020-03-15 Thread Yang Wang
For K8s deployment(including standalone session/perjob, native integration), Flink could not support port range for rest options. You need to set the `rest.bind-port` exactly same with `rest.port`. Hi LakeShen I am trying to understand your problem and do not think it is about the port

Re: [VOTE] [FLIP-85] Flink Application Mode

2020-03-12 Thread Yang Wang
Thanks a lot for moving this FLIP forward. Looking forward to the JIRAs. Best, Yang Flavio Pompermaier 于2020年3月12日周四 下午9:05写道: > +1 (non-binding). > There's also a related issue that I opened a long time ago > https://issues.apache.org/jira/browse/FLINK-10879 that could be closed > once >

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2020-03-10 Thread Yang Wang
Thanks for your response. @Kostas Kloudas Could we update the cli changes and how to fetch the user jars to FLIP document? I think other dev or users may have the similar questions. Best, Yang Aljoscha Krettek 于2020年3月10日周二 下午9:03写道: > On 10.03.20 03:31, Yang Wang wrote: > > For

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2020-03-09 Thread Yang Wang
Hi Aljoscha, Kostas, I would be in favour of something like "bin/flink run-application", > maybe we should even have "run-job" in the future to differentiate. I have no preference for the "-R/--remote-deploy" option of "flink run" or new introduced "flink run-application". If we always bind

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2020-03-09 Thread Yang Wang
CLI / configuration / > metrics, etc. Can we update the FLIP to include the conclusion we have here > in the ML? > > Thanks, > > Jiangjie (Becket) Qin > > On Mon, Mar 9, 2020 at 11:59 AM Yang Wang wrote: > >> Hi Becket, >> >> Thanks for jumping out and sha

[jira] [Created] (FLINK-16499) Flink shaded hadoop could not work when Yarn timeline service is enabled

2020-03-09 Thread Yang Wang (Jira)
Yang Wang created FLINK-16499: - Summary: Flink shaded hadoop could not work when Yarn timeline service is enabled Key: FLINK-16499 URL: https://issues.apache.org/jira/browse/FLINK-16499 Project: Flink

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2020-03-08 Thread Yang Wang
they have been discussed in the ML earlier, but I think they should >> be part of the FLIP also. >> >> Thanks, >> >> Jiangjie (Becket) Qin >> >> On Thu, Mar 5, 2020 at 10:09 PM Kostas Kloudas >> wrote: >> >>> Also from my side +1 to start

Re: [VOTE] [FLIP-85] Flink Application Mode

2020-03-06 Thread Yang Wang
Hi Kostas, Thanks a lot for kicking off the vote for FLIP-85. It is really a useful feature for platform developers who manage hundreds of Flink jobs on Yarn in production. Also the native Kubernetes job cluster will benefit a lot from this. +1 (non-binding) from my side Best, Yang Kostas

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-05 Thread Yang Wang
Hi Andrey, Thanks for driving this significant FLIP. From the user ML, we could also know there are many users running Flink in container environment. Then the docker image will be the very basic requirement. Just as you say, we should provide a unified place for all various usage(e.g. session,

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2020-03-04 Thread Yang Wang
unch > a per job cluster with the main function executed in cluster > entry-point. +1 for the execution plan. > > > > Best Regards > Peter Huang > > > > > On Tue, Mar 3, 2020 at 7:11 AM Yang Wang wrote: > >> Hi Peter, >> >> Having the appl

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2020-03-03 Thread Yang Wang
> >> while in the application mode, the main is executed on the cluster. > >> I do not think we have to offer "application mode" with running the > >> main on the client side as this is exactly what the per-job mode does > >> currently and, as you descr

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2020-03-02 Thread Yang Wang
t# > > Please let me know what you think and please keep the discussion in the ML > :) > > Thanks for starting the discussion and I hope that soon we will be > able to vote on the FLIP. > > Cheers, > Kostas > > On Thu, Jan 16, 2020 at 3:40 AM Yang Wang wrote: > &g

Re: [DISCUSS] FLINK-16194: Refactor the Kubernetes architecture design

2020-02-27 Thread Yang Wang
;> >>> Thanks for your clarification Yang! We're on the same page. >>> >>> Best, >>> tison. >>> >>> >>> Yang Wang 于2020年2月25日周二 下午10:07写道: >>> >>>> Hi tison, >>>> >>>> I do not mean to keep two decor

Re: [DISCUSS] FLINK-16194: Refactor the Kubernetes architecture design

2020-02-25 Thread Yang Wang
unit tested. >> >> However, it makes no sense we have two codepaths from FlinkKubeClient to >> decorators >> since these two version of decorators are not api compatible and there is >> no reason we keep both >> of them. >> >> Best, >> tison.

Re: [DISCUSS] FLINK-16194: Refactor the Kubernetes architecture design

2020-02-25 Thread Yang Wang
I think if we could, splitting into as many PRs as possible is good. Maybe we could introduce the new designed decorators and parameter parser first, and leave the existing decorators as legacy. Once all the new decorators is ready and well tested, we could remove the legacy codes and use the new

Re: [DISCUSS] FLINK-16194: Refactor the Kubernetes architecture design

2020-02-24 Thread Yang Wang
It seems that we could benefit a lot from the "Parameters Parser". Let's start to add the dedicated jobmanager/taskmanager config parser classes. Best, Yang Canbin Zheng 于2020年2月24日周一 上午10:39写道: > Hi, Yang Wang, > > Thanks for the feedback. > > > Parameters Pa

Re: [DISCUSS] FLINK-16194: Refactor the Kubernetes architecture design

2020-02-23 Thread Yang Wang
Just to make sure, it is not a fundamentally different approach compared > to > > what we have right now, is it? > > > > If this is the case, then I think it makes sense to reuse code as much as > > possible and to create small code units which are easier to test. > > &

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Yang Wang
> > > > > > +1 (non-binding) > > > > > > > > I like the ideas of having a list of all log files, and make them > > > > downloadable. > > > > > > > > Thank you~ > > > > > > > > Xintong Song > >

Re: [VOTE] FLIP-98: Better Back Pressure Detection

2020-02-20 Thread Yang Wang
+1 non-binding It is really straightforward. Best, Yang Jark Wu 于2020年2月21日周五 下午1:52写道: > +1 (binding) > > On Fri, 21 Feb 2020 at 11:51, Kurt Young wrote: > > > +1 (binding) > > > > > > > > On Fri, Feb 21, 2020 at 1:09 AM Zhijiang > .invalid> > > wrote: > > > > > +1 (binding). > > > It

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Yang Wang
Congratulations!Jingsong. Well deserved. Best, Yang Zhijiang 于2020年2月21日周五 下午1:18写道: > Congrats Jingsong! Welcome on board! > > Best, > Zhijiang > > -- > From:Zhenghua Gao > Send Time:2020 Feb. 21 (Fri.) 12:49 > To:godfrey he >

Re: [Discuss] Update the pull request description template.

2020-02-20 Thread Yang Wang
I second xintong's suggestion. When i open a PR, i also check the item list in the template. It help to know whether i should test the PR in a real cluster(Yarn/K8s/Mesos). Or i should be more careful when touching the per-record code paths.If we have some dependencies changes, i will need to

Re: [DISCUSS] FLINK-16194: Refactor the Kubernetes architecture design

2020-02-20 Thread Yang Wang
Hi Canbing, Thanks a lot for sharing your thoughts to improve the Flink on K8s native integration. Frankly speaking, your discussion title confuses me and i am wondering whether you want to refactor the whole design. However, after i dive into the details and the provided document, i realize

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Yang Wang
+1 non-binding The log files list are shown in the new design. I like the idea. Since the user defined logs(e.g. my-log.file) could be found here now. Best, Yang Yadong Xie 于2020年2月20日周四 下午5:56写道: > Hi all > > I want to start the vote for FLIP-103, which proposes to provide better > user log

Re: [DISCUSS] Kicking off the 1.11 release cycle

2020-02-20 Thread Yang Wang
Thanks Stephan for kicking off the 1.11 release and Zhijiang and Piotr for the volunteering. A short release cycle is really good. Since we will not push many features simultaneously in every release. +1 for feature freeze date for end of April. Best, Yang Hequn Cheng 于2020年2月20日周四 下午8:27写道:

Re: [DISCUSS] Extend (or maintain) "shell" script support for Windows

2020-02-19 Thread Yang Wang
I tend to drop the windows bat script to start/stop the cluster. Since there are various windows versions(e.g. xp, win7, win10, etc.) and do not have sensational compatibility. Also we do not have the e2e tests to guarantee the scripts work well. On the other hand, i completely agree that local

[jira] [Created] (FLINK-16116) Remove shading from oss filesystems build

2020-02-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-16116: - Summary: Remove shading from oss filesystems build Key: FLINK-16116 URL: https://issues.apache.org/jira/browse/FLINK-16116 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-16115) Aliyun oss filesystem could not work with plugin mechanism

2020-02-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-16115: - Summary: Aliyun oss filesystem could not work with plugin mechanism Key: FLINK-16115 URL: https://issues.apache.org/jira/browse/FLINK-16115 Project: Flink Issue

Re: [DISCUSS] Improve history server with log support

2020-02-16 Thread Yang Wang
Hi Rong Rong, Thanks for starting this discussion. I think the log is an important part of improving user experience of Flink. The logs is very important for debugging problems or checking the expected output. Some users, especially for machine learning, print global steps or residual to the

[jira] [Created] (FLINK-16013) List config option could not be parsed correctly

2020-02-11 Thread Yang Wang (Jira)
Yang Wang created FLINK-16013: - Summary: List config option could not be parsed correctly Key: FLINK-16013 URL: https://issues.apache.org/jira/browse/FLINK-16013 Project: Flink Issue Type: Bug

Re: [VOTE] Release 1.10.0, release candidate #3

2020-02-10 Thread Yang Wang
+1 non-binding - Building from source with all tests skipped - Build a custom image with 1.10-rc3 - K8s tests * Deploy a standalone session cluster on K8s and submit multiple jobs * Deploy a standalone per-job cluster * Deploy a native session cluster on K8s with/without HA

Re: [DISCUSS] FLINK-15831: Add Docker image publication to release documentation

2020-02-09 Thread Yang Wang
+1 to make flink-docker repository self-contained, including the document. And others refer to it. Best, Yang Till Rohrmann 于2020年2月9日周日 下午5:35写道: > Sounds good to me Patrick. +1 for these changes. > > Cheers, > Till > > On Fri, Feb 7, 2020 at 3:25 PM Patrick Lucas > wrote: > > > Hi all, > >

[jira] [Created] (FLINK-15948) Resource will be wasted when the task manager memory is not a multiple of Yarn minimum allocation

2020-02-07 Thread Yang Wang (Jira)
Yang Wang created FLINK-15948: - Summary: Resource will be wasted when the task manager memory is not a multiple of Yarn minimum allocation Key: FLINK-15948 URL: https://issues.apache.org/jira/browse/FLINK-15948

[jira] [Created] (FLINK-15871) Support to start sidecar container

2020-02-03 Thread Yang Wang (Jira)
Yang Wang created FLINK-15871: - Summary: Support to start sidecar container Key: FLINK-15871 URL: https://issues.apache.org/jira/browse/FLINK-15871 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-15836) Throw fatal exception when the watcher in Fabric8FlinkKubeClient is closed

2020-01-31 Thread Yang Wang (Jira)
Yang Wang created FLINK-15836: - Summary: Throw fatal exception when the watcher in Fabric8FlinkKubeClient is closed Key: FLINK-15836 URL: https://issues.apache.org/jira/browse/FLINK-15836 Project: Flink

Re: [VOTE] Integrate Flink Docker image publication into Flink release process

2020-01-29 Thread Yang Wang
+1 (non-binding) Best, Yang Rong Rong 于2020年1月30日周四 上午12:53写道: > +1 > > -- > Rong > > On Wed, Jan 29, 2020 at 8:51 AM Ismaël Mejía wrote: > > > +1 (non-binding) > > > > No more maintenance work for us Patrick! Just kidding :), it was mostly > > done by Patrick, all kudos to him. > > Just one

Re: [ANNOUNCE] Yu Li became a Flink committer

2020-01-23 Thread Yang Wang
Congratulations, Yu. Best, Yang tison 于2020年1月23日周四 下午7:07写道: > Congratulations! > > Best, > tison. > > > Dian Fu 于2020年1月23日周四 下午7:06写道: > > > Congrats Yu! > > > > > 在 2020年1月23日,下午6:47,Hequn Cheng 写道: > > > > > > Congratulations Yu! > > > Thanks a lot for being the release manager of the

Re: [DISCUSS] Active Kubernetes integration phase2

2020-01-19 Thread Yang Wang
some ideas about the new features. Join in us, let's make Flink on Kubernetes moving forward together. [1]. https://slack.com/share/ISUKQ2WG5/cXwlw5HZgp3AE5ElE8m580Pk/enQtOTEyNjcwMDk4NTQ5LWZiNTU0ZmJiZTU4MWU1OTk2YjllNmE0OTg2YjIxYjA2YmE1MWFlOTE4NWFhMDBkMzE4NDQzYjk1YmQwMDI2MzU Yang Wang 于2020年1月19

[DISCUSS] Active Kubernetes integration phase2

2020-01-19 Thread Yang Wang
Hi everyone, Currently Flink supports the resource management system YARN and Mesos. However, they were not designed for fast moving cloud native architectures, and they could not support mixed workloads (e.g. batch, streaming, deep learning, web services, etc.) relatively well. At the same

[jira] [Created] (FLINK-15643) Support to start multiple Flink masters to achieve faster recovery

2020-01-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-15643: - Summary: Support to start multiple Flink masters to achieve faster recovery Key: FLINK-15643 URL: https://issues.apache.org/jira/browse/FLINK-15643 Project: Flink

[jira] [Created] (FLINK-15642) Support to set JobManager liveness check

2020-01-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-15642: - Summary: Support to set JobManager liveness check Key: FLINK-15642 URL: https://issues.apache.org/jira/browse/FLINK-15642 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-15641) Support to start sidecar container and init container

2020-01-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-15641: - Summary: Support to start sidecar container and init container Key: FLINK-15641 URL: https://issues.apache.org/jira/browse/FLINK-15641 Project: Flink Issue Type

[jira] [Created] (FLINK-15640) Support to set label and node selector

2020-01-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-15640: - Summary: Support to set label and node selector Key: FLINK-15640 URL: https://issues.apache.org/jira/browse/FLINK-15640 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-15639) Support to set toleration for jobmanager and taskmanger

2020-01-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-15639: - Summary: Support to set toleration for jobmanager and taskmanger Key: FLINK-15639 URL: https://issues.apache.org/jira/browse/FLINK-15639 Project: Flink Issue Type

[jira] [Created] (FLINK-15632) Zookeeper HA service could not work for active kubernetes integration

2020-01-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-15632: - Summary: Zookeeper HA service could not work for active kubernetes integration Key: FLINK-15632 URL: https://issues.apache.org/jira/browse/FLINK-15632 Project: Flink

Re: [ANNOUNCE] Dian Fu becomes a Flink committer

2020-01-17 Thread Yang Wang
Congratulations! Best, Yang Terry Wang 于2020年1月17日周五 下午3:28写道: > Congratulations! > > Best, > Terry Wang > > > > 2020年1月17日 14:09,Biao Liu 写道: > > Congrats! > > Thanks, > Biao /'bɪ.aʊ/ > > > > On Fri, 17 Jan 2020 at 13:43, Rui Li wrote: > >> Congratulations Dian, well deserved! >> >> On

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2020-01-15 Thread Yang Wang
the jar downloading will be a painful thing. > We can consider about PoC and performance evaluation first. If the end to > end experience is good enough, then we can consider > proceeding with the solution. > > Looking forward to more opinions from @Yang Wang @Zili > Chen @Dian

[jira] [Created] (FLINK-15483) Support to set environment variables for jobmanager pod

2020-01-06 Thread Yang Wang (Jira)
Yang Wang created FLINK-15483: - Summary: Support to set environment variables for jobmanager pod Key: FLINK-15483 URL: https://issues.apache.org/jira/browse/FLINK-15483 Project: Flink Issue Type

[jira] [Created] (FLINK-15470) Remove YARN magic properties file

2020-01-03 Thread Yang Wang (Jira)
Yang Wang created FLINK-15470: - Summary: Remove YARN magic properties file Key: FLINK-15470 URL: https://issues.apache.org/jira/browse/FLINK-15470 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-15457) Remove the finished TODO in the YarnEntrypointUtils

2020-01-02 Thread Yang Wang (Jira)
Yang Wang created FLINK-15457: - Summary: Remove the finished TODO in the YarnEntrypointUtils Key: FLINK-15457 URL: https://issues.apache.org/jira/browse/FLINK-15457 Project: Flink Issue Type

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2019-12-30 Thread Yang Wang
t the > > dispatch first, then we can use > > a ClusterEnvironment similar to ContextEnvironment to submit the job with > > jobName the local > > dispatcher. For the details, we need more investigation. Let's wait > > for @Aljoscha > > Krettek

[jira] [Created] (FLINK-15405) Make Yarn staging directory configurable

2019-12-26 Thread Yang Wang (Jira)
Yang Wang created FLINK-15405: - Summary: Make Yarn staging directory configurable Key: FLINK-15405 URL: https://issues.apache.org/jira/browse/FLINK-15405 Project: Flink Issue Type: New Feature

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2019-12-25 Thread Yang Wang
in my case , there is no > > jar to upload but only a sql string . > > do u had more suggestion to improve for sql mode or it is only a > > switch for api programme? > > > > > > best > > bai jj > > > > > > Yang Wang 于2019年12月18日周三 下午

[jira] [Created] (FLINK-15360) Yarn e2e test is broken with building docker image

2019-12-22 Thread Yang Wang (Jira)
Yang Wang created FLINK-15360: - Summary: Yarn e2e test is broken with building docker image Key: FLINK-15360 URL: https://issues.apache.org/jira/browse/FLINK-15360 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-15354) Start and stop minikube only in kubernetes related e2e tests

2019-12-20 Thread Yang Wang (Jira)
Yang Wang created FLINK-15354: - Summary: Start and stop minikube only in kubernetes related e2e tests Key: FLINK-15354 URL: https://issues.apache.org/jira/browse/FLINK-15354 Project: Flink

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

2019-12-20 Thread Yang Wang
runs user program on the cluster side. >> >> Best, >> tison. >> >> >> Peter Huang 于2019年12月19日周四 上午2:54写道: >> >>> Hi Yang, >>> >>> Thanks for your input, I can see the master side job graph generation is >>> a common re

Re: [DISCUSS] Integrate Flink Docker image publication into Flink release process

2019-12-20 Thread Yang Wang
Big +1 for this effort. It is really exciting we have started this great work. More and more companies start to use Flink in container environment(docker, Kubernetes, Mesos, even Yarn-3.x). So it is very important that we could have unified official image building and releasing process. The

Re: [Question] How to use different filesystem between checkpoint data and user data sink

2019-12-18 Thread Yang Wang
You could have a try the new plugin mechanism. Create a new directory named "myhdfs" under $FLINK_HOME/plugins, and then put your filesystem related jars in it. Different plugins will be loaded by separate classloader to avoid conflict. Best, Yang vino yang 于2019年12月18日周三 下午6:46写道: > Hi

<    1   2   3   4   5   >