[GitHub] [flink-kubernetes-operator] kingsathurthi commented on pull request #552: Fixing the container vulnerability by updating maven dependency

2023-04-09 Thread via GitHub


kingsathurthi commented on PR #552:
URL: 
https://github.com/apache/flink-kubernetes-operator/pull/552#issuecomment-1501393713

   Hi Yang, I'm still active on this is issue, please allow me some time to 
close this. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Updated] (FLINK-31374) ProxyStreamPartitioner should implement ConfigurableStreamPartitioner

2023-04-09 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-31374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-31374:
---
Labels: pull-request-available  (was: )

> ProxyStreamPartitioner should implement ConfigurableStreamPartitioner
> -
>
> Key: FLINK-31374
> URL: https://issues.apache.org/jira/browse/FLINK-31374
> Project: Flink
>  Issue Type: Bug
>  Components: Library / Machine Learning
>Affects Versions: ml-2.2.0
>Reporter: Zhipeng Zhang
>Priority: Major
>  Labels: pull-request-available
>
> In flink-ml-iterations module, we use ProxyStreamPartitioner to wrap 
> StreamPartitioner to deal with records in iterations.
>  
> However, it did not implement ConfigurableStreamPartitioner interface. Thus 
> that maxParallelism information is lost.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[GitHub] [flink-ml] jiangxin369 opened a new pull request, #231: [FLINK-31374] ProxyStreamPartitioner should implement ConfigurableStreamPartitioner

2023-04-09 Thread via GitHub


jiangxin369 opened a new pull request, #231:
URL: https://github.com/apache/flink-ml/pull/231

   
   
   ## What is the purpose of the change
   
   Make the ProxyStreamPartitioner implement ConfigurableStreamPartitioner. 
Otherwise, the KeyGroupRange that is computed in 
StreamTaskStateInitializerImpl.java doesn't match the subTaskId that is 
computed in KeyGroupStreamPartitioner.java.
   
   ## Brief change log
   
 - Make the ProxyStreamPartitioner implement ConfigurableStreamPartitioner.
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): (no)
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
   
   ## Documentation
   
 - Does this pull request introduce a new feature? (no)
 - If yes, how is the feature documented? (not applicable)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[GitHub] [flink-kubernetes-operator] yangjf2019 commented on pull request #552: Fixing the container vulnerability by updating maven dependency

2023-04-09 Thread via GitHub


yangjf2019 commented on PR #552:
URL: 
https://github.com/apache/flink-kubernetes-operator/pull/552#issuecomment-1501357652

   Hi @kingsathurthi , sorry to bother you! If you are no longer following up, 
please close this issue and I will take over the management of it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Assigned] (FLINK-31344) Support to update nested columns in update statement

2023-04-09 Thread luoyuxia (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-31344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

luoyuxia reassigned FLINK-31344:


Assignee: Aitozi  (was: luoyuxia)

> Support to update nested columns in update statement
> 
>
> Key: FLINK-31344
> URL: https://issues.apache.org/jira/browse/FLINK-31344
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API
>Reporter: luoyuxia
>Assignee: Aitozi
>Priority: Major
>
> Currently, it'll throw exception while using update statement to update 
> nested column;
> For the following sql:
>  
> {code:java}
> create table (t ROW<`a` INT>) with (xxx);
> update t set s.a = 1;{code}
> It'll throw the exception:
> {code:java}
> Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered "." 
> at line 1, column 15.
> Was expecting:
>     "=" ...
>     
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:46382)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:46190)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlUpdate(FlinkSqlParserImpl.java:14389)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmt(FlinkSqlParserImpl.java:4121)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtList(FlinkSqlParserImpl.java:2998)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtList(FlinkSqlParserImpl.java:306)
>     at 
> org.apache.calcite.sql.parser.SqlParser.parseStmtList(SqlParser.java:198)
>     ... 33 more {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (FLINK-31344) Support to update nested columns in update statement

2023-04-09 Thread luoyuxia (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-31344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

luoyuxia reassigned FLINK-31344:


Assignee: luoyuxia

> Support to update nested columns in update statement
> 
>
> Key: FLINK-31344
> URL: https://issues.apache.org/jira/browse/FLINK-31344
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API
>Reporter: luoyuxia
>Assignee: luoyuxia
>Priority: Major
>
> Currently, it'll throw exception while using update statement to update 
> nested column;
> For the following sql:
>  
> {code:java}
> create table (t ROW<`a` INT>) with (xxx);
> update t set s.a = 1;{code}
> It'll throw the exception:
> {code:java}
> Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered "." 
> at line 1, column 15.
> Was expecting:
>     "=" ...
>     
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:46382)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:46190)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlUpdate(FlinkSqlParserImpl.java:14389)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmt(FlinkSqlParserImpl.java:4121)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtList(FlinkSqlParserImpl.java:2998)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtList(FlinkSqlParserImpl.java:306)
>     at 
> org.apache.calcite.sql.parser.SqlParser.parseStmtList(SqlParser.java:198)
>     ... 33 more {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-31344) Support to update nested columns in update statement

2023-04-09 Thread Aitozi (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-31344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710023#comment-17710023
 ] 

Aitozi commented on FLINK-31344:


Thanks for your reply. Totally agree with you. I have basically verified the 
first work, can you help assign this ticket to me ?

> Support to update nested columns in update statement
> 
>
> Key: FLINK-31344
> URL: https://issues.apache.org/jira/browse/FLINK-31344
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API
>Reporter: luoyuxia
>Priority: Major
>
> Currently, it'll throw exception while using update statement to update 
> nested column;
> For the following sql:
>  
> {code:java}
> create table (t ROW<`a` INT>) with (xxx);
> update t set s.a = 1;{code}
> It'll throw the exception:
> {code:java}
> Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered "." 
> at line 1, column 15.
> Was expecting:
>     "=" ...
>     
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:46382)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:46190)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlUpdate(FlinkSqlParserImpl.java:14389)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmt(FlinkSqlParserImpl.java:4121)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtList(FlinkSqlParserImpl.java:2998)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtList(FlinkSqlParserImpl.java:306)
>     at 
> org.apache.calcite.sql.parser.SqlParser.parseStmtList(SqlParser.java:198)
>     ... 33 more {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-31757) Optimize Flink un-balanced tasks scheduling

2023-04-09 Thread RocMarshal (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710022#comment-17710022
 ] 

RocMarshal commented on FLINK-31757:


[~Weijie Guo] Glad to get your attention and reminding~:)

I'll add the background and cases description later.
Looking forward your discussion after that~
Thanks a lot.

> Optimize Flink un-balanced tasks scheduling
> ---
>
> Key: FLINK-31757
> URL: https://issues.apache.org/jira/browse/FLINK-31757
> Project: Flink
>  Issue Type: Improvement
>  Components: Runtime / Task
>Reporter: RocMarshal
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-31757) Optimize Flink un-balanced tasks scheduling

2023-04-09 Thread Weijie Guo (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710021#comment-17710021
 ] 

Weijie Guo commented on FLINK-31757:


Hi [~RocMarshal], Can you add more details to this ticket? I don't really know 
what unbalanced scheduling is.

> Optimize Flink un-balanced tasks scheduling
> ---
>
> Key: FLINK-31757
> URL: https://issues.apache.org/jira/browse/FLINK-31757
> Project: Flink
>  Issue Type: Improvement
>  Components: Runtime / Task
>Reporter: RocMarshal
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (FLINK-31757) Optimize Flink un-balanced tasks scheduling

2023-04-09 Thread RocMarshal (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

RocMarshal updated FLINK-31757:
---
Summary: Optimize Flink un-balanced tasks scheduling  (was: Optimize Flink 
tasks un-balanced scheduling)

> Optimize Flink un-balanced tasks scheduling
> ---
>
> Key: FLINK-31757
> URL: https://issues.apache.org/jira/browse/FLINK-31757
> Project: Flink
>  Issue Type: Improvement
>  Components: Runtime / Task
>Reporter: RocMarshal
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (FLINK-31757) Optimize Flink tasks un-balanced scheduling

2023-04-09 Thread RocMarshal (Jira)
RocMarshal created FLINK-31757:
--

 Summary: Optimize Flink tasks un-balanced scheduling
 Key: FLINK-31757
 URL: https://issues.apache.org/jira/browse/FLINK-31757
 Project: Flink
  Issue Type: Improvement
  Components: Runtime / Task
Reporter: RocMarshal






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-31686) Filesystem connector should replace the shallow copy with deep copy

2023-04-09 Thread luoyuxia (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-31686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710020#comment-17710020
 ] 

luoyuxia commented on FLINK-31686:
--

+1 for adding a new api. +1 for public discussion.

> Filesystem connector should replace the shallow copy with deep copy
> ---
>
> Key: FLINK-31686
> URL: https://issues.apache.org/jira/browse/FLINK-31686
> Project: Flink
>  Issue Type: Bug
>  Components: Connectors / FileSystem
>Affects Versions: 1.16.1
>Reporter: tanjialiang
>Priority: Major
> Attachments: image-2023-04-01-16-18-48-762.png, 
> image-2023-04-01-16-18-56-075.png
>
>
> Hi team, when i using the following sql
> {code:java}
> CREATE TABLE student (
>     `id` STRING,
>     `name` STRING,
>     `age` INT
> ) WITH (
>   'connector' = 'filesystem',
>   'path' = '...',
>   'format' = 'orc'
> );
> select
>     t1.total,
>     t2.total
> from
>     (
>         select
>             count(*) as total,
>             1 as join_key
>         from student
>         where name = 'tanjialiang'
>     ) t1
>     LEFT JOIN (
>         select
>             count(*) as total,
>             1 as join_key
>         from student;
>     ) t2 
>     ON t1.join_key = t2.join_key; {code}
>  
> it will throw an error
> !image-2023-04-01-16-18-48-762.png!
>  
> I tried to solve it, and i found filesystem connector's copy function using a 
> shallow copy instread of deep copy.   It lead to all of query from a same 
> table source reuse the same bulkWriterFormat, and my query have filter 
> condition, which will push down into the bulkWriterFormat, so the filter 
> condition maybe reuse.   
> I found the DynamicTableSource and DynamicTableSink's copy function comment 
> to ask we should impletement it with deep copy, but i found every connector 
> are using shallow copy to impletement it.   So i think not only the 
> filesystem connector have this problem.
> !image-2023-04-01-16-18-56-075.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-31344) Support to update nested columns in update statement

2023-04-09 Thread luoyuxia (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-31344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710011#comment-17710011
 ] 

luoyuxia commented on FLINK-31344:
--

[~aitozi] I think it's fine to make SqlParser to accept the compound 
identifier. But about the modifcation about {{SupportsRowLevelUpdate}} , I 
perfer to keep for a while to see whether do we really need such modifcation to 
avoid over design.

> Support to update nested columns in update statement
> 
>
> Key: FLINK-31344
> URL: https://issues.apache.org/jira/browse/FLINK-31344
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API
>Reporter: luoyuxia
>Priority: Major
>
> Currently, it'll throw exception while using update statement to update 
> nested column;
> For the following sql:
>  
> {code:java}
> create table (t ROW<`a` INT>) with (xxx);
> update t set s.a = 1;{code}
> It'll throw the exception:
> {code:java}
> Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered "." 
> at line 1, column 15.
> Was expecting:
>     "=" ...
>     
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:46382)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:46190)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlUpdate(FlinkSqlParserImpl.java:14389)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmt(FlinkSqlParserImpl.java:4121)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtList(FlinkSqlParserImpl.java:2998)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtList(FlinkSqlParserImpl.java:306)
>     at 
> org.apache.calcite.sql.parser.SqlParser.parseStmtList(SqlParser.java:198)
>     ... 33 more {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] (FLINK-31344) Support to update nested columns in update statement

2023-04-09 Thread luoyuxia (Jira)


[ https://issues.apache.org/jira/browse/FLINK-31344 ]


luoyuxia deleted comment on FLINK-31344:
--

was (Author: luoyuxia):
[~aitozi] Why need to modify the iinterface SupportsRowLevelUpdate?

> Support to update nested columns in update statement
> 
>
> Key: FLINK-31344
> URL: https://issues.apache.org/jira/browse/FLINK-31344
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API
>Reporter: luoyuxia
>Priority: Major
>
> Currently, it'll throw exception while using update statement to update 
> nested column;
> For the following sql:
>  
> {code:java}
> create table (t ROW<`a` INT>) with (xxx);
> update t set s.a = 1;{code}
> It'll throw the exception:
> {code:java}
> Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered "." 
> at line 1, column 15.
> Was expecting:
>     "=" ...
>     
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:46382)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:46190)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlUpdate(FlinkSqlParserImpl.java:14389)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmt(FlinkSqlParserImpl.java:4121)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtList(FlinkSqlParserImpl.java:2998)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtList(FlinkSqlParserImpl.java:306)
>     at 
> org.apache.calcite.sql.parser.SqlParser.parseStmtList(SqlParser.java:198)
>     ... 33 more {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (FLINK-31344) Support to update nested columns in update statement

2023-04-09 Thread luoyuxia (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-31344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710009#comment-17710009
 ] 

luoyuxia commented on FLINK-31344:
--

[~aitozi] Why need to modify the iinterface SupportsRowLevelUpdate?

> Support to update nested columns in update statement
> 
>
> Key: FLINK-31344
> URL: https://issues.apache.org/jira/browse/FLINK-31344
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API
>Reporter: luoyuxia
>Priority: Major
>
> Currently, it'll throw exception while using update statement to update 
> nested column;
> For the following sql:
>  
> {code:java}
> create table (t ROW<`a` INT>) with (xxx);
> update t set s.a = 1;{code}
> It'll throw the exception:
> {code:java}
> Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered "." 
> at line 1, column 15.
> Was expecting:
>     "=" ...
>     
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:46382)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:46190)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlUpdate(FlinkSqlParserImpl.java:14389)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmt(FlinkSqlParserImpl.java:4121)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtList(FlinkSqlParserImpl.java:2998)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtList(FlinkSqlParserImpl.java:306)
>     at 
> org.apache.calcite.sql.parser.SqlParser.parseStmtList(SqlParser.java:198)
>     ... 33 more {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[GitHub] [flink] flinkbot commented on pull request #22371: [FLINK-28171] [flink-kubernates] enable add appProtocol via the configuration and verify it is not overridden by Default port defintion

2023-04-09 Thread via GitHub


flinkbot commented on PR #22371:
URL: https://github.com/apache/flink/pull/22371#issuecomment-1501130037

   
   ## CI report:
   
   * 60aac8039125ce954c91b63f578ea35467e7a054 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (FLINK-28171) Adjust Job and Task manager port definitions to work with Istio+mTLS

2023-04-09 Thread Sigalit Aliazov (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-28171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17709974#comment-17709974
 ] 

Sigalit Aliazov commented on FLINK-28171:
-

hello, [~wangyang0918]  and [~martijnvisser] we created PR for this based on 
flink 1.14.6, [https://github.com/apache/flink/pull/22371] we will appreciate 
your assistance in code review, merging and verifying it also goes to newer 
versions.

> Adjust Job and Task manager port definitions to work with Istio+mTLS
> 
>
> Key: FLINK-28171
> URL: https://issues.apache.org/jira/browse/FLINK-28171
> Project: Flink
>  Issue Type: Improvement
>  Components: Deployment / Kubernetes
>Affects Versions: 1.14.4
> Environment: flink-kubernetes-operator 1.0.0
> Flink 1.14-java11
> Kubernetes v1.19.5
> Istio 1.7.6
>Reporter: Moshe Elisha
>Assignee: Moshe Elisha
>Priority: Major
>  Labels: pull-request-available
>
> Hello,
>  
> We are launching Flink deployments using the [Flink Kubernetes 
> Operator|https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/]
>  on a Kubernetes cluster with Istio and mTLS enabled.
>  
> We found that the TaskManager is unable to communicate with the JobManager on 
> the jobmanager-rpc port:
>  
> {{2022-06-15 15:25:40,508 WARN  akka.remote.ReliableDeliverySupervisor        
>                [] - Association with remote system 
> [akka.tcp://[flink@amf-events-to-inference-and-central.nwdaf-edge|mailto:flink@amf-events-to-inference-and-central.nwdaf-edge]:6123]
>  has failed, address is now gated for [50] ms. Reason: [Association failed 
> with 
> [akka.tcp://[flink@amf-events-to-inference-and-central.nwdaf-edge|mailto:flink@amf-events-to-inference-and-central.nwdaf-edge]:6123]]
>  Caused by: [The remote system explicitly disassociated (reason unknown).]}}
>  
> The reason for the issue is that the JobManager service port definitions are 
> not following the Istio guidelines 
> [https://istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/]
>  (see example below).
>  
> There was also an email discussion around this topic in the users mailing 
> group under the subject "Flink Kubernetes Operator with K8S + Istio + mTLS - 
> port definitions".
> With the help of the community, we were able to work around the issue but it 
> was very hard and forced us to skip Istio proxy which is not ideal.
>  
> We would like you to consider changing the default port definitions, either
>  # Rename the ports – I understand it is Istio specific guideline but maybe 
> it is better to at least be aligned with one (popular) vendor guideline 
> instead of none at all.
>  # Add the “appProtocol” property[1] that is not specific to any vendor but 
> requires Kubernetes >= 1.19 where it was introduced as beta and moved to 
> stable in >= 1.20. The option to add appProtocol property was added only in 
> [https://github.com/fabric8io/kubernetes-client/releases/tag/v5.10.0] with 
> [#3570|https://github.com/fabric8io/kubernetes-client/issues/3570].
>  # Or allow a way to override the defaults.
>  
> [https://kubernetes.io/docs/concepts/services-networking/_print/#application-protocol]
>  
>  
> {{# k get service inference-results-to-analytics-engine -o yaml}}
> {{apiVersion: v1}}
> {{kind: Service}}
> {{...}}
> {{spec:}}
> {{  clusterIP: None}}
> {{  ports:}}
> {{  - name: jobmanager-rpc *# should start with “tcp-“ or add "appProtocol" 
> property*}}
> {{    port: 6123}}
> {{    protocol: TCP}}
> {{    targetPort: 6123}}
> {{  - name: blobserver *# should start with "tcp-" or add "appProtocol" 
> property*}}
> {{    port: 6124}}
> {{    protocol: TCP}}
> {{    targetPort: 6124}}
> {{...}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[GitHub] [flink] sigalite opened a new pull request, #22371: [FLINK-28171] [flink-kubernates] enable add appProtocol via the configuration and verify it is not overridden by Default port defintion

2023-04-09 Thread via GitHub


sigalite opened a new pull request, #22371:
URL: https://github.com/apache/flink/pull/22371

   
   ## What is the purpose of the change
   enable the job manager and task manager communicate in a Kubernetes cluster 
with Istio and mTLS enabled.
   
   ## Brief change log
   Add the “appProtocol” in the task manager and job manager configuration, 
verify they are not overridden by the default definition in the decorators 
   
   ## Verifying this change
   This change is already covered by existing tests
   
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): no
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: no
 - The serializers: no
 - The runtime per-record code paths (performance sensitive): don't know
 - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: don't know
 - The S3 file system connector: no
   
   ## Documentation
   
 - Does this pull request introduce a new feature? (no)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (FLINK-31344) Support to update nested columns in update statement

2023-04-09 Thread Aitozi (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-31344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17709961#comment-17709961
 ] 

Aitozi commented on FLINK-31344:


Hi [~luoyuxia], when solving the FLINK-31301 I notice this ticket. After 
spending some time investigate on this ticket, I think we can support update 
nested columns in the following step:
 - Modify the SqlParser to accept the compound identifier for the UPDATE SET 
clause
 - Modify the interface SupportsRowLevelUpdate, which will be better to work 
with {{int[][]}} instead of {{Column}} (It will make the control of required 
column finer).

WDYT ?

> Support to update nested columns in update statement
> 
>
> Key: FLINK-31344
> URL: https://issues.apache.org/jira/browse/FLINK-31344
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API
>Reporter: luoyuxia
>Priority: Major
>
> Currently, it'll throw exception while using update statement to update 
> nested column;
> For the following sql:
>  
> {code:java}
> create table (t ROW<`a` INT>) with (xxx);
> update t set s.a = 1;{code}
> It'll throw the exception:
> {code:java}
> Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered "." 
> at line 1, column 15.
> Was expecting:
>     "=" ...
>     
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:46382)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:46190)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlUpdate(FlinkSqlParserImpl.java:14389)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmt(FlinkSqlParserImpl.java:4121)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtList(FlinkSqlParserImpl.java:2998)
>     at 
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtList(FlinkSqlParserImpl.java:306)
>     at 
> org.apache.calcite.sql.parser.SqlParser.parseStmtList(SqlParser.java:198)
>     ... 33 more {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)