[
https://issues.apache.org/jira/browse/HDFS-15757?focusedWorklogId=544844&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-544844
]
ASF GitHub Bot logged work on HDFS-15757:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Jan/21 11:10
Start Date: 31/Jan/21 11:10
Worklog Time Spent: 10m
Work Description: Hexiaoqiao commented on a change in pull request #2651:
URL: https://github.com/apache/hadoop/pull/2651#discussion_r567408154
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/ConnectionContext.java
##########
@@ -42,7 +44,10 @@
private int numThreads = 0;
/** If the connection is closed. */
private boolean closed = false;
-
+ /** Last timestamp the connection was active. */
+ private long lastActiveTs = 0;
+ /** The connection's active status would expire after this window. */
+ private long activeWindow = TimeUnit.SECONDS.toMillis(30);
Review comment:
`activeWindow` should be final static constant here?
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/ConnectionPool.java
##########
@@ -252,19 +252,23 @@ public synchronized void addConnection(ConnectionContext
conn) {
*/
public synchronized List<ConnectionContext> removeConnections(int num) {
List<ConnectionContext> removed = new LinkedList<>();
-
- // Remove and close the last connection
- List<ConnectionContext> tmpConnections = new ArrayList<>();
- for (int i=0; i<this.connections.size(); i++) {
- ConnectionContext conn = this.connections.get(i);
- if (i < this.minSize || i < this.connections.size() - num) {
- tmpConnections.add(conn);
- } else {
- removed.add(conn);
+ if (this.connections.size() > this.minSize) {
+ int targetCount = Math.min(num, this.connections.size() - this.minSize);
Review comment:
IIRC, `connections` is not thread safe, so is it possible `targetCount`
is negative here？
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/ConnectionPool.java
##########
@@ -252,19 +252,23 @@ public synchronized void addConnection(ConnectionContext
conn) {
*/
public synchronized List<ConnectionContext> removeConnections(int num) {
List<ConnectionContext> removed = new LinkedList<>();
-
- // Remove and close the last connection
- List<ConnectionContext> tmpConnections = new ArrayList<>();
- for (int i=0; i<this.connections.size(); i++) {
- ConnectionContext conn = this.connections.get(i);
- if (i < this.minSize || i < this.connections.size() - num) {
- tmpConnections.add(conn);
- } else {
- removed.add(conn);
+ if (this.connections.size() > this.minSize) {
+ int targetCount = Math.min(num, this.connections.size() - this.minSize);
+ // Remove and close targetCount of connections
+ List<ConnectionContext> tmpConnections = new ArrayList<>();
+ for (int i=0; i<this.connections.size(); i++) {
Review comment:
nit: code style.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 544844)
Time Spent: 1h 20m (was: 1h 10m)
> RBF: Improving Router Connection Management
> -------------------------------------------
>
> Key: HDFS-15757
> URL: https://issues.apache.org/jira/browse/HDFS-15757
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
> Reporter: Fengnan Li
> Assignee: Fengnan Li
> Priority: Major
> Labels: pull-request-available
> Attachments: RBF_ Improving Router Connection Management_v2.pdf, RBF_
> Router Connection Management.pdf
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> We have seen high number of connections from Router to namenodes, leaving
> namenodes unstable.
> This ticket is trying to reduce connections through some changes. Please take
> a look at the design and leave comments.
> Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
