[jira] [Commented] (FLINK-20036) Join Has NoUniqueKey when using mini-batch

2023-01-04 Thread Jianhui Dong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654367#comment-17654367
 ] 

Jianhui Dong commented on FLINK-20036:
--

This issue is duplicated with Flink-27851.

> Join Has NoUniqueKey when using mini-batch
> --
>
> Key: FLINK-20036
> URL: https://issues.apache.org/jira/browse/FLINK-20036
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / Planner
>Affects Versions: 1.11.2
>Reporter: Rex Remind
>Priority: Minor
>  Labels: auto-deprioritized-major
> Fix For: 1.17.0
>
>
> Hello,
>  
> We tried out mini-batch mode and our Join suddenly had NoUniqueKey.
> Join:
> {code:java}
> Table membershipsTable = tableEnv.from(SOURCE_MEMBERSHIPS)
>   .renameColumns($("id").as("membership_id"))
>   .select($("*")).join(usersTable, $("user_id").isEqual($("id")));
> {code}
> Mini-batch config:
> {code:java}
> configuration.setString("table.exec.mini-batch.enabled", "true"); // enable 
> mini-batch optimization
> configuration.setString("table.exec.mini-batch.allow-latency", "5 s"); // use 
> 5 seconds to buffer input records
> configuration.setString("table.exec.mini-batch.size", "5000"); // the maximum 
> number of records can be buffered by each aggregate operator task
> {code}
>  
> Join with mini-batch:
> {code:java}
>  Join(joinType=[InnerJoin], where=[(user_id = id0)], select=[id, 
> group_id, user_id, uuid, owner, id0, deleted_at], 
> leftInputSpec=[NoUniqueKey], rightInputSpec=[NoUniqueKey]) 
> {code}
> Join without mini-batch:
> {code:java}
> Join(joinType=[InnerJoin], where=[(user_id = id0)], select=[id, group_id, 
> user_id, uuid, owner, id0, deleted_at], leftInputSpec=[HasUniqueKey], 
> rightInputSpec=[JoinKeyContainsUniqueKey])
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (FLINK-20036) Join Has NoUniqueKey when using mini-batch

2020-11-16 Thread Jark Wu (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17233237#comment-17233237
 ] 

Jark Wu commented on FLINK-20036:
-

This is a performance improvement, because when the input sepc is  
{{HasUniqueKey}} or {{JoinKeyContainsUniqueKey}}, the join operator will choose 
a better state structure. Therefore, update the fix version to 1.13. 

> Join Has NoUniqueKey when using mini-batch
> --
>
> Key: FLINK-20036
> URL: https://issues.apache.org/jira/browse/FLINK-20036
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / Planner
>Affects Versions: 1.11.2
>Reporter: Rex Remind
>Priority: Major
> Fix For: 1.13.0
>
>
> Hello,
>  
> We tried out mini-batch mode and our Join suddenly had NoUniqueKey.
> Join:
> {code:java}
> Table membershipsTable = tableEnv.from(SOURCE_MEMBERSHIPS)
>   .renameColumns($("id").as("membership_id"))
>   .select($("*")).join(usersTable, $("user_id").isEqual($("id")));
> {code}
> Mini-batch config:
> {code:java}
> configuration.setString("table.exec.mini-batch.enabled", "true"); // enable 
> mini-batch optimization
> configuration.setString("table.exec.mini-batch.allow-latency", "5 s"); // use 
> 5 seconds to buffer input records
> configuration.setString("table.exec.mini-batch.size", "5000"); // the maximum 
> number of records can be buffered by each aggregate operator task
> {code}
>  
> Join with mini-batch:
> {code:java}
>  Join(joinType=[InnerJoin], where=[(user_id = id0)], select=[id, 
> group_id, user_id, uuid, owner, id0, deleted_at], 
> leftInputSpec=[NoUniqueKey], rightInputSpec=[NoUniqueKey]) 
> {code}
> Join without mini-batch:
> {code:java}
> Join(joinType=[InnerJoin], where=[(user_id = id0)], select=[id, group_id, 
> user_id, uuid, owner, id0, deleted_at], leftInputSpec=[HasUniqueKey], 
> rightInputSpec=[JoinKeyContainsUniqueKey])
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-20036) Join Has NoUniqueKey when using mini-batch

2020-11-06 Thread Jark Wu (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227717#comment-17227717
 ] 

Jark Wu commented on FLINK-20036:
-

Thanks for reporting this. 

> Join Has NoUniqueKey when using mini-batch
> --
>
> Key: FLINK-20036
> URL: https://issues.apache.org/jira/browse/FLINK-20036
> Project: Flink
>  Issue Type: Bug
>  Components: Table SQL / Planner
>Affects Versions: 1.11.2
>Reporter: Rex Remind
>Priority: Major
> Fix For: 1.12.0
>
>
> Hello,
>  
> We tried out mini-batch mode and our Join suddenly had NoUniqueKey.
> Join:
> {code:java}
> Table membershipsTable = tableEnv.from(SOURCE_MEMBERSHIPS)
>   .renameColumns($("id").as("membership_id"))
>   .select($("*")).join(usersTable, $("user_id").isEqual($("id")));
> {code}
> Mini-batch config:
> {code:java}
> configuration.setString("table.exec.mini-batch.enabled", "true"); // enable 
> mini-batch optimization
> configuration.setString("table.exec.mini-batch.allow-latency", "5 s"); // use 
> 5 seconds to buffer input records
> configuration.setString("table.exec.mini-batch.size", "5000"); // the maximum 
> number of records can be buffered by each aggregate operator task
> {code}
>  
> Join with mini-batch:
> {code:java}
>  Join(joinType=[InnerJoin], where=[(user_id = id0)], select=[id, 
> group_id, user_id, uuid, owner, id0, deleted_at], 
> leftInputSpec=[NoUniqueKey], rightInputSpec=[NoUniqueKey]) 
> {code}
> Join without mini-batch:
> {code:java}
> Join(joinType=[InnerJoin], where=[(user_id = id0)], select=[id, group_id, 
> user_id, uuid, owner, id0, deleted_at], leftInputSpec=[HasUniqueKey], 
> rightInputSpec=[JoinKeyContainsUniqueKey])
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)