[kudu-CR] master: reduce timeout for master to tserver rpcs

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change.

Change subject: master: reduce timeout for master to tserver rpcs
..


Patch Set 3:

> I do recall seeing a once-in-a-blue-moon AlterTable() timeout that
 > I traced back to this 30s timeout and a coinciding leader election,
 > but I can't remember the details now. I'll retest without the
 > timeout to see if I can retrigger it.

I ran the test 1000 times and it didn't fail. So I'll abandon this.

-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] master: reduce timeout for master to tserver rpcs

[Adar Dembo (Code Review)]

Adar Dembo has abandoned this change.

Change subject: master: reduce timeout for master to tserver rpcs
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] master: reduce timeout for master to tserver rpcs

[Kudu Jenkins (Code Review)]

Kudu Jenkins has posted comments on this change.

Change subject: master: reduce timeout for master to tserver rpcs
..


Patch Set 3:

Build Started http://104.196.14.100/job/kudu-gerrit/2319/

-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] master: reduce timeout for master to tserver rpcs

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change.

Change subject: master: reduce timeout for master to tserver rpcs
..


Patch Set 2:

Filed KUDU-1526 for the test failure (which appears to be an unrelated flake).

 > I think we bumped this to be pretty high because we found that
 > creating a new tablet can actually take reasonably long (it
 > involves a bunch of fsyncs, preallocating log, etc) especially when
 > there are lots of them going on at the same time (as when you
 > create a table with 10s of tablets per server).

Note that this only changes the "generic" timeout. AsyncCreateReplica() uses 
its own timeout gflag which is still set to 30s. That one is more dangerous to 
dial down, as a timeout would cause the master to replace the tablet in 
question. But, a discrepancy between the two gflags is weird, so I am tempted 
to revert this.

I do recall seeing a once-in-a-blue-moon AlterTable() timeout that I traced 
back to this 30s timeout and a coinciding leader election, but I can't remember 
the details now. I'll retest without the timeout to see if I can retrigger it.

-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] master: reduce timeout for master to tserver rpcs

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change.

Change subject: master: reduce timeout for master to tserver rpcs
..


Patch Set 2:

> I think we bumped this to be pretty high because we found that
 > creating a new tablet can actually take reasonably long (it
 > involves a bunch of fsyncs, preallocating log, etc) especially when
 > there are lots of them going on at the same time (as when you
 > create a table with 10s of tablets per server).

-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] master: reduce timeout for master to tserver rpcs

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change.

Change subject: master: reduce timeout for master to tserver rpcs
..


Patch Set 2:

I think we bumped this to be pretty high because we found that creating a new 
tablet can actually take reasonably long (it involves a bunch of fsyncs, 
preallocating log, etc) especially when there are lots of them going on at the 
same time (as when you create a table with 10s of tablets per server).

-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] master: reduce timeout for master to tserver rpcs

[Kudu Jenkins (Code Review)]

Kudu Jenkins has posted comments on this change.

Change subject: master: reduce timeout for master to tserver rpcs
..


Patch Set 2:

Build Started http://104.196.14.100/job/kudu-gerrit/2299/

-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] master: reduce timeout for master to tserver rpcs

[Kudu Jenkins (Code Review)]

Kudu Jenkins has posted comments on this change.

Change subject: master: reduce timeout for master to tserver rpcs
..


Patch Set 1:

Build Started http://104.196.14.100/job/kudu-gerrit/2267/

-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] master: reduce timeout for master to tserver rpcs

[Adar Dembo (Code Review)]

Hello Dan Burkert, David Ribeiro Alves, Todd Lipcon,

I'd like you to do a code review.  Please visit

http://gerrit.cloudera.org:8080/3606

to review the following change.

Change subject: master: reduce timeout for master to tserver rpcs
..

master: reduce timeout for master to tserver rpcs

Thirty seconds seems awfully high, especially since all RPCs retry on
failure anyway. Furthermore, some RPCs are sent to leader replicas, so if an
RPC coincides with a leader election, it may have to wait up to thirty
seconds to retry on the new leader.

Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
---
M src/kudu/master/catalog_manager.cc
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/06/3606/1
-- 
To view, visit http://gerrit.cloudera.org:8080/3606
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I06b5f32c0295f0a9d24ff42695905858ae6101f6
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Todd Lipcon