[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Reviewed-on: http://gerrit.cloudera.org:8080/12119 Reviewed-by: Grant Henke Tested-by: Grant Henke --- A _posts/2019-04-30-location-awareness.md 1 file changed, 380 insertions(+), 0 deletions(-) Approvals: Grant Henke: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: merged Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 13 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 12: Verified+1 Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 12 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Tue, 30 Apr 2019 15:40:33 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, Grant Henke, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#12). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2019-04-30-location-awareness.md 1 file changed, 380 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/12 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 12 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 11: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 11 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 22 Apr 2019 19:48:19 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 11: (9 comments) http://gerrit.cloudera.org:8080/#/c/12119/11//COMMIT_MSG Commit Message: PS11: Rendered version is at: https://gist.github.com/alexeyserbin/a6049124e63f32a8cf344a3fe7c55489 http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md File _posts/2019-04-16-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@48 PS10, Line 48: : > Is there a Jira tracking this? It will be useful in context of KUDU-2604. http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@62 PS10, Line 62: : > How should users detect this? Does the tablet server fail? Is there a log t As one criterion, non-registered tablet servers does not appear in the list of tablet servers at masters' web UI. Also, masters will not return addresses of non-registered tablet servers in the ListTabletServers -- that makes them unusable to Kudu clients. Of course masters log about inability to determine location for a table server. I added a few details about that. http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@73 PS10, Line 73: > client's assigned Done http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@81 PS10, Line 81: > follow the following just sounds funny because it's two forms for "follow". That sounds funny, indeed :) Done http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@85 PS10, Line 85: > as the replica placement policy Done http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@87 PS10, Line 87: > remove "the" Done http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@103 PS10, Line 103: > Is thee a jira to track this? I opened a new one upstream: KUDU-2780 http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@368 PS10, Line 368: > Now that these are linked inline, are they needed here too? I prefer references to be listed in a separate section as well. That helps to build a better context of a blog/paper even without reading it thoroughly. -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 11 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 22 Apr 2019 17:29:50 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, Grant Henke, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#11). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2019-04-23-location-awareness.md 1 file changed, 380 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/11 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 11 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 10: (8 comments) Just a few small comments. I think this is almost good to go. http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md File _posts/2019-04-16-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@48 PS10, Line 48: leverage the hierarchical structure : in future releases. Is there a Jira tracking this? http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@62 PS10, Line 62: (the latter renders the tablet : server unusable to Kudu clients). How should users detect this? Does the tablet server fail? Is there a log to look for? http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@73 PS10, Line 73: assigned client's client's assigned http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@81 PS10, Line 81: follow the following follow the following just sounds funny because it's two forms for "follow". How about "adhere to the following"? http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@85 PS10, Line 85: as _replica placement policy_ as the replica placement policy http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@87 PS10, Line 87: the remove "the" http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@103 PS10, Line 103: in future releases that might be done automatically in background Is thee a jira to track this? http://gerrit.cloudera.org:8080/#/c/12119/10/_posts/2019-04-16-location-awareness.md@368 PS10, Line 368: References Now that these are linked inline, are they needed here too? -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 10 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Wed, 17 Apr 2019 18:36:55 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 10: > Patch Set 9: > > Now that the design doc is merged the link can be updated. Yes, indeed. Thank you for kind reminder! -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 10 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 15 Apr 2019 22:22:21 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, Grant Henke, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#10). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2019-04-16-location-awareness.md 1 file changed, 372 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/10 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 10 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 9: Now that the design doc is merged the link can be updated. -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 9 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 15 Apr 2019 14:17:08 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 9: (1 comment) http://gerrit.cloudera.org:8080/#/c/12119/9//COMMIT_MSG Commit Message: PS9: The rendered version of this blogpost is available at https://github.com/alexeyserbin/kudu/blob/gh-pages/_posts/2019-03-25-location-awareness.md -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 9 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Tue, 26 Mar 2019 18:07:27 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 8: (28 comments) Thank you for the thorough review. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md File _posts/2019-03-25-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@1 PS8, Line 1: --- > Can you push this to the gh_pages branch in your github fork so a rendered Done: https://github.com/alexeyserbin/kudu/blob/gh-pages/_posts/2019-03-25-location-awareness.md http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@15 PS8, Line 15: > Should this be removed? I'll remove it once I'm getting +1 or +2 on the latest patchset. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@19 PS8, Line 19: first cut > initial implementation? Done http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@19 PS8, Line 19: starting 1.9.0 > ...starting *with the* 1.9.0... Done http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@20 PS8, Line 20: is built for the following use case: > I am not sure this is a "use case" per se, but instead what the term "locat I rephrased it using term 'requirement'. Does it sound better now? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@26 PS8, Line 26: A rack failure might happen because of a failure of a hardware component shared : among servers in the rack: network switch, power supply, etc. > A rack failure can occur when a hardware component shared Done http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@31 PS8, Line 31: network latency between datacenters is low. > This is a good opportunity to explicitly mention that this is why we call t Done http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@37 PS8, Line 37: are : supposed to > should Done http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@38 PS8, Line 38: physical or cloud-defined hierarchy of the : deployed cluster > I am not sure I understand what this means in relation to location awarenes I recall trying to introduce 'fault domain' term in this blog , I got feedback that we better not introduce it since 1) readers might be not aware of that concept, 2) we don't use that term in how design doc. That's why you don't see 'failure domain' or 'fault domain' here at all. If you feel strong about the necessity to speak in 'proper' terms here, let me know and I'll introduce those terms with corresponding references. As of now, I just added the examples you suggested. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@41 PS8, Line 41: However, we want to keep the hierarchy : there to make it possible to exploit it later > However, we plan to leverage the hierarchical structure in future releases. Done http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@43 PS8, Line 43: compatibility with HDFS > Perhaps this should be moved up and describe a bit more in detail as a desi Done http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@52 PS8, Line 52: etc > What is the "etc"? What else does it use it for? The text below contains information about what those locations are used for, so 'etc' here is for: * client finding the closest tablet server to its own location * other usages of the location hierarchy which we haven't leveraged yet I removed the 'etc'. Does it look better to you now? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@55 PS8, Line 55: location string for the specified IP address/hostname. > The script below specifically shows ip-address. How do I use hostname? The script below is an example. You are free to use hostname to perform any flavor of mapping 'hostname' --> 'location string' you want. I added corresponding note into the example's comment. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@59 PS8, Line 59: tablet server restarts > Is this dependent on `--follower_unavailable_considered_failed_sec`? Or wil It's not dependent of '--follower_unavailable_considered_failed_sec'. That's about registering tablet servers, not tracking tablet replicas. Any restart of a tablet server will cause assignment of location to the tablet server. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@59 PS8, Line 59: Kudu tablet servers are location : agnostic, at least for now, so the assigned location is not reported back : to the tablet
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, Grant Henke, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#9). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2019-03-25-location-awareness.md 1 file changed, 374 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/9 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 9 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 8: (28 comments) http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md File _posts/2019-03-25-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@1 PS8, Line 1: --- Can you push this to the gh_pages branch in your github fork so a rendered version can be proofed? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@15 PS8, Line 15: Should this be removed? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@19 PS8, Line 19: first cut initial implementation? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@19 PS8, Line 19: starting 1.9.0 ...starting *with the* 1.9.0... http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@20 PS8, Line 20: is built for the following use case: I am not sure this is a "use case" per se, but instead what the term "location awareness" currently means in Kudu. Maybe say something like: "In the initial implementation of location awareness in Kudu, when a Kudu cluster consists of multiple servers spread across several racks, Kudu will place the replicas of a tablet in such a way that the tablet stays available even if all the servers in a single rack become unavailable." http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@26 PS8, Line 26: A rack failure might happen because of a failure of a hardware component shared : among servers in the rack: network switch, power supply, etc. A rack failure can occur when a hardware component shared among servers in the rack, such as a network switch or power supply, fails. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@31 PS8, Line 31: network latency between datacenters is low. This is a good opportunity to explicitly mention that this is why we call the feature location awareness and not rack awareness. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@37 PS8, Line 37: are : supposed to should http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@38 PS8, Line 38: physical or cloud-defined hierarchy of the : deployed cluster I am not sure I understand what this means in relation to location awareness utility. I suspect it's saying that the components should map to the hierarchical levels of "failure domains". You could then give a private data center example: `/data-center-0/rack-09` And a cloud example: `/region-0/availability-zone-01` http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@41 PS8, Line 41: However, we want to keep the hierarchy : there to make it possible to exploit it later However, we plan to leverage the hierarchical structure in future releases. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@43 PS8, Line 43: compatibility with HDFS Perhaps this should be moved up and describe a bit more in detail as a design choice? It's useful to know that you can use the same locations as your HDFS nodes, because it's common to deploy Kudu along size HDFS. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@52 PS8, Line 52: etc What is the "etc"? What else does it use it for? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@55 PS8, Line 55: location string for the specified IP address/hostname. The script below specifically shows ip-address. How do I use hostname? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@59 PS8, Line 59: tablet server restarts Is this dependent on `--follower_unavailable_considered_failed_sec`? Or will a "quick" restart cause the location to be reset? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@59 PS8, Line 59: Kudu tablet servers are location : agnostic, at least for now, so the assigned location is not reported back : to the tablet server. Maybe this paragraph would flow better if you moved this part to the bottom. That would make it so you describe how the master uses the location configurations, and then tack on at the end that the tablet servers do not need/use it. http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@64 PS8, Line 64: masters provide connected clients How do they do this? http://gerrit.cloudera.org:8080/#/c/12119/8/_posts/2019-03-25-location-awareness.md@62 PS8, Line 62: to try to place replicas evenly across : locations and to keep tablets
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#8). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2019-03-25-location-awareness.md 1 file changed, 348 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/8 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 8 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 7: Code-Review+2 Don't forget to update the date like Todd said. -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 7 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 31 Jan 2019 18:30:34 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Greg Solovyev has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 7: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 7 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Wed, 30 Jan 2019 00:27:08 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 5: (2 comments) > (2 comments) > > cool post, enjoyed catching up on what you've accomplished with > this feature Yep, Will and I did some progress on that since the LA project started. Fengling contributed few patches as well. That's a team effort :) Thank you for the review! http://gerrit.cloudera.org:8080/#/c/12119/5/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/5/_posts/2018-12-20-location-awareness.md@1 PS5, Line 1: --- > just a reminder: should rename the file to the current date before committi Thank you for the reminder -- I added a TODO for that. http://gerrit.cloudera.org:8080/#/c/12119/5/_posts/2018-12-20-location-awareness.md@28 PS5, Line 28: replace 'rack' with any other aggregation of nodes (e.g., chassis, site, etc.) > might be good to add "cloud availability zone" to this list Done -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Wed, 30 Jan 2019 00:12:04 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#7). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2018-12-20-location-awareness.md 1 file changed, 349 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/7 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 7 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/12119/6/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: PS6: The rendered version is at: https://gist.github.com/alexeyserbin/c6f03f1d20151d906bf230e9b014dfb7 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 6 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Tue, 29 Jan 2019 19:30:34 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#6). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2018-12-20-location-awareness.md 1 file changed, 348 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/6 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 6 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 5: > What about a link to a sample script or an inline sample script in > the post? Whoops, I need to add that, sure. I'll do that tonight. Thanks for the reminder! -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Tue, 29 Jan 2019 02:41:46 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Greg Solovyev has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 5: What about a link to a sample script or an inline sample script in the post? -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Tue, 29 Jan 2019 00:44:50 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 5: (2 comments) cool post, enjoyed catching up on what you've accomplished with this feature http://gerrit.cloudera.org:8080/#/c/12119/5/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/5/_posts/2018-12-20-location-awareness.md@1 PS5, Line 1: --- just a reminder: should rename the file to the current date before committing so it shows up as new when it's posted on the blog http://gerrit.cloudera.org:8080/#/c/12119/5/_posts/2018-12-20-location-awareness.md@28 PS5, Line 28: replace 'rack' with any other aggregation of nodes (e.g., chassis, site, etc.) might be good to add "cloud availability zone" to this list -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 28 Jan 2019 23:15:50 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 28 Jan 2019 22:25:47 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 28 Jan 2019 22:25:40 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 5: (18 comments) http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@50 PS4, Line 50: > the Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@55 PS4, Line 55: location string for the specified IP address/hostname. : > Is this meant to be a paragraph break? Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@61 PS4, Line 61: as > to try Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@61 PS4, Line 61: ation infor > replicas Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@62 PS4, Line 62: o > Remove. Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@62 PS4, Line 62: a pl > to keep Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@64 PS4, Line 64: de conn > Remove, or remove "Also". Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@65 PS4, Line 65: nt's location, so the clients can make inform > I'd say "attempt to read from the closest tablet server" since that's all w Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@67 PS4, Line 67: : How to make a Kudu cluster location-aware : == > This is redundant with the previous paragraph. Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@84 PS4, Line 84: > a Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@92 PS4, Line 92: ke tablets unavail > I prefer to say "placement policy" singular since we only have one. Also I Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@94 PS4, Line 94: That's referred to as _replica placement policy_ or just _placement policy_. : In Kudu, both the initial placement of tablet replicas and the automatic : re-replication are governed by the that policy. > This is basically the placement policy, so I'd just make it clear that "pla Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@101 PS4, Line 101: s, > Here and below, like I said, I prefer "policy" since we have just one. Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@139 PS4, Line 139: > each location Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@142 PS4, Line 142: own. Use > This section is really nice. Thanks for adding it. I added it while waiting for my PHX --> SJC flight in the Phoenix airport :) http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@147 PS4, Line 147: > nit: rectangles or boxes Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@181 PS4, Line 181: > resulting Done http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@188 PS4, Line 188: o lo > receive Done -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Fri, 25 Jan 2019 22:59:00 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#5). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2018-12-20-location-awareness.md 1 file changed, 284 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/5 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 4: (18 comments) http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@50 PS4, Line 50: the http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@55 PS4, Line 55: location string for the specified IP address/hostname. : The Is this meant to be a paragraph break? http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@61 PS4, Line 61: try to try http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@61 PS4, Line 61: the replica replicas http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@62 PS4, Line 62: if Remove. http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@62 PS4, Line 62: keep to keep http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@64 PS4, Line 64: as well Remove, or remove "Also". http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@65 PS4, Line 65: about to connect to the closest tablet server I'd say "attempt to read from the closest tablet server" since that's all we actually use it for. http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@67 PS4, Line 67: The Kudu leader master assigns location for a client when it connects to a : cluster. The assigned location string is sent back from the leader master : to the client along with other cluster-specific information. This is redundant with the previous paragraph. http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@84 PS4, Line 84: a http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@92 PS4, Line 92: placement policies I prefer to say "placement policy" singular since we only have one. Also I think you ought to define the placement policy here (i.e. restate the "use case" from the beginning as the placement policy). http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@94 PS4, Line 94: The automatic re-replication and the initial placement of new replicas both : attempt to spread replicas across locations so that the failure of tablet : servers in one location does not make tablets unavailable. This is basically the placement policy, so I'd just make it clear that "placement policy" is referring to this condition. http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@101 PS4, Line 101: policies Here and below, like I said, I prefer "policy" since we have just one. http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@139 PS4, Line 139: location each location http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@142 PS4, Line 142: Examples This section is really nice. Thanks for adding it. http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@147 PS4, Line 147: squares nit: rectangles or boxes http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@181 PS4, Line 181: result resulting http://gerrit.cloudera.org:8080/#/c/12119/4/_posts/2018-12-20-location-awareness.md@188 PS4, Line 188: land receive -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 24 Jan 2019 18:57:47 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, Greg Solovyev, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#4). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2018-12-20-location-awareness.md 1 file changed, 282 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/4 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 3: (1 comment) > (1 comment) > > > We have this script which we use for testing: > > https://github.com/apache/kudu/blob/master/src/kudu/scripts/assign-location.py > > While that script is awesome for testing it's a really bad example > for users because it doesn't actually assign locations based on the > network topology. We should make a small example that assigns > locations based on common subnet or something like that. SGTM. I agree it's better to have something that closer reflects the reality. After some consideration, I think out test scripts might be really confusing for people trying to understand what it's all about. > > > Maybe, I should put some TODO in the text as a reminder? > > Good idea. Just make it a comment. Done. http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@37 PS2, Line 37: `/dc-0/rack-09/host-01` > This isn't a good example location because it would place the host in its o Done -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Sat, 19 Jan 2019 04:48:21 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 3: (1 comment) > We have this script which we use for testing: > https://github.com/apache/kudu/blob/master/src/kudu/scripts/assign-location.py While that script is awesome for testing it's a really bad example for users because it doesn't actually assign locations based on the network topology. We should make a small example that assigns locations based on common subnet or something like that. > Maybe, I should put some TODO in the text as a reminder? Good idea. Just make it a comment. http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@37 PS2, Line 37: `/dc-0/rack-09/host-01` This isn't a good example location because it would place the host in its own location, which probably isn't what a user wants. I think `/dc-0/rack-09` is better. -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Sat, 19 Jan 2019 00:58:32 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 3: > It would be good to provide a working code sample for location > mapping executable in order to demonstrate input/output > expectations. Even though this seems trivial having a code sample > will eliminate potential ambiguity. We have this script which we use for testing: https://github.com/apache/kudu/blob/master/src/kudu/scripts/assign-location.py Do you think it's good enough or the idea is to have something as a snippet in the blog post itself? > > It would also be good to mention in which version of Kudu this is > available. Yep, we discussed that in the prior comments with Will. The idea is to put the version in the very top as soon as 1.9 get released. Maybe, I should put some TODO in the text as a reminder? -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Sat, 19 Jan 2019 00:35:34 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Greg Solovyev has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 3: It would be good to provide a working code sample for location mapping executable in order to demonstrate input/output expectations. Even though this seems trivial having a code sample will eliminate potential ambiguity. It would also be good to mention in which version of Kudu this is available. -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Greg Solovyev Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Fri, 18 Jan 2019 16:22:27 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/12119/3/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: PS3: The rendered version is available at: https://gist.github.com/alexeyserbin/c6f03f1d20151d906bf230e9b014dfb7 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 17 Jan 2019 04:35:02 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 2: (21 comments) http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@70 PS2, Line 70: > a Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@98 PS2, Line 98: > a Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@112 PS2, Line 112: when > where Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@114 PS2, Line 114: is > are Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@115 PS2, Line 115: a > the Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@117 PS2, Line 117: > the Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@117 PS2, Line 117: kudu > Either Kudu, or put in backticks `kudu`. Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@121 PS2, Line 121: very > Remove. Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@121 PS2, Line 121: At > In Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@122 PS2, Line 122: policies > policy Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@122 PS2, Line 122: re-establishes > reestablish Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@124 PS2, Line 124: continue to the cross-location rebalancing. > Add "phase" at the end. Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@126 PS2, Line 126: Second phase is cross-location rebalancing, i.e. moving tablet replicas between > "The second phase" Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@127 PS2, Line 127: location > locations Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@128 PS2, Line 128: loads > What is the load of a location? I added a sentence about that. http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@128 PS2, Line 128: equalizing loads of locations throughout the cluster. Use the > the Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@132 PS2, Line 132: Third phase is intra-location rebalancing, i.e. balance tablet replica > The Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@132 PS2, Line 132: replica > replicas Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@133 PS2, Line 133: distribution > Remove. Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@133 PS2, Line 133: such > the Done http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@135 PS2, Line 135: > A very quick example showing the moves for each of the three phases would b Done. -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 17 Jan 2019 04:33:11 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#3). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2018-12-20-location-awareness.md 1 file changed, 280 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/3 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 2: (21 comments) http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@70 PS2, Line 70: a http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@98 PS2, Line 98: a http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@112 PS2, Line 112: when where http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@114 PS2, Line 114: is are http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@115 PS2, Line 115: a the http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@117 PS2, Line 117: the http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@117 PS2, Line 117: kudu Either Kudu, or put in backticks `kudu`. http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@121 PS2, Line 121: very Remove. http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@121 PS2, Line 121: At In http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@122 PS2, Line 122: policies policy http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@122 PS2, Line 122: re-establishes reestablish http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@124 PS2, Line 124: continue to the cross-location rebalancing. Add "phase" at the end. http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@126 PS2, Line 126: Second phase is cross-location rebalancing, i.e. moving tablet replicas between "The second phase" http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@127 PS2, Line 127: location locations http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@128 PS2, Line 128: equalizing loads of locations throughout the cluster. Use the the http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@128 PS2, Line 128: loads What is the load of a location? http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@132 PS2, Line 132: replica replicas http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@132 PS2, Line 132: Third phase is intra-location rebalancing, i.e. balance tablet replica The http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@133 PS2, Line 133: such the http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@133 PS2, Line 133: distribution Remove. http://gerrit.cloudera.org:8080/#/c/12119/2/_posts/2018-12-20-location-awareness.md@135 PS2, Line 135: A very quick example showing the moves for each of the three phases would be good. It would illustrate the three phases and the role of "load" in cross-location rebalancing. -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Tue, 08 Jan 2019 21:40:44 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 2: (46 comments) Thank you for the thorough review! http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@7 PS1, Line 7: an overview > An overview is always high-level, so remove "high-level". Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@9 PS1, Line 9: principles of the design > principles of the design Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@11 PS1, Line 11: potential future enhancements and extensions > potential future enhancements and extensions Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@17 PS1, Line 17: the fo > for Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@17 PS1, Line 17: loc > Remove. Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@20 PS1, Line 20: In a Kudu cluster consisting of multiple servers s > In a Kudu cluster consisting of multiple servers spread over several racks Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@20 PS1, Line 20: ever > replicas Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@21 PS1, Line 21: that t > Remove. Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@21 PS1, Line 21: s > in Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@21 PS1, Line 21: place the > Remove. Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@21 PS1, Line 21: vailable : even if all the ser > Remove. Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@22 PS1, Line 22: > become Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@22 PS1, Line 22: ers in a single rack bec > even if all the servers Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@25 PS1, Line 25: among servers in the rack: network switch, power supply, etc. More generally, : replace 'rack' wit > Remove (redundant). Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@28 PS1, Line 28: lies to a datacenter if the network latency between datacenters : is low. > I don't think we need to use this concept very much so I wouldn't include i OK. I thought this might look easier to digest for people who know what fault domain is, but I agree we don't use the concept of fault domain in this post, so it's not necessary to include. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@31 PS1, Line 31: > Remove. Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@31 PS1, Line 31: Locations in > nit: I prefer "More generally", but "To generalize" is ok. Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@31 PS1, Line 31: Locations in Kudu > Join this paragraph with the previous paragraph. Done http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@32 PS1, Line 32: === > Is a chassis a common term for something multiple servers would run on? To In big server farms, they sometimes have aggregates of physical hosts/nodes (sometimes hot-pluggable) which share common power, e.g.: https://ark.intel.com/products/81878/Intel-Server-Chassis-H2216XXKR2 (check the 'Product images' page to get a better idea how it looks like). Those are called 'chassis'. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@32 PS1, Line 32: > Does a site differ from a datacenter? Site might mean a sub-unit in a datacenter (e.g., a server room in a big datacenter). http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@33 PS1, Line 33: a string that begins with a slash (`/`) and : consists of slash-separated tokens each of which contains only characters from : the set `[a-zA-Z0-9_-.]`. The components > Rephrase this more positively: "This even applies to a datacenter if the ne I like that positive attitude :) http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@37 PS1, Line 37: deployed cluster, e.g. `/dc-0/rack-09/host-01`. As of now, Kudu does not : exploit the hierarchical > I'm not sure what our publishing schedule is, but I think we ought not publ OK, I removed this
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Hello Will Berkeley, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12119 to look at the new patch set (#2). Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2018-12-20-location-awareness.md 1 file changed, 154 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/2 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Will Berkeley
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/12119 ) Change subject: [blog] a blogpost about location awareness in Kudu .. Patch Set 1: (46 comments) Gerrit gets funny when there's a lot of comments on one file, it seems, so I'm posting this first batch and I will continue reviewing later. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md File _posts/2018-12-20-location-awareness.md: http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@7 PS1, Line 7: a high-level overview An overview is always high-level, so remove "high-level". http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@9 PS1, Line 9: principles behind the design decisions principles of the design http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@11 PS1, Line 11: ways how can be extended and enhanced in the future potential future enhancements and extensions http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@17 PS1, Line 17: the Remove. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@17 PS1, Line 17: to fit for http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@20 PS1, Line 20: data replicas http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@20 PS1, Line 20: Having servers in multiple racks in a Kudu cluster In a Kudu cluster consisting of multiple servers spread over several racks http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@21 PS1, Line 21: 's data Remove. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@21 PS1, Line 21: replicated Remove. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@21 PS1, Line 21: in http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@22 PS1, Line 22: becomes become http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@21 PS1, Line 21: for : reading and writing Remove. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@22 PS1, Line 22: if any subset of servers even if all the servers http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@25 PS1, Line 25: In other words, a single rack failure should not lead to data unavailability : in a Kudu cluster. Remove (redundant). http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@28 PS1, Line 28: In that sense, a rack is a fault domain (usually, a fault domain : defined is a set of hardware components that share a single point of failure). I don't think we need to use this concept very much so I wouldn't include it. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@31 PS1, Line 31: To generalize, replace 'rack' with any other physical aggregation of nodes Join this paragraph with the previous paragraph. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@31 PS1, Line 31: To generalize nit: I prefer "More generally", but "To generalize" is ok. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@31 PS1, Line 31: physical Remove. http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@32 PS1, Line 32: chassis Is a chassis a common term for something multiple servers would run on? To me it sounds synonymous with a single physical host, unless the cluster is running in VMs and some VMs share a physical host. If that's the case, I think we ought to say "shared VM host" or some such instead of "chassis". http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@32 PS1, Line 32: site Does a site differ from a datacenter? http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@33 PS1, Line 33: It might be applicable even to a datacenter : given that the network latency between datacenters is negligible : (which, frankly, isn't quite realistic). Rephrase this more positively: "This even applies to a datacenter if the network latency between datacenters is low." http://gerrit.cloudera.org:8080/#/c/12119/1/_posts/2018-12-20-location-awareness.md@37 PS1, Line 37: At this point, the location awareness feature is a work in progress and may be : released with Kudu 1.9.0. I'm not sure what our publishing schedule is, but I think we ought not publish this until the feature is sure to be released with the next release and the next release is coming in the next month or two. In that case, we shouldn't include this sentence, and we should mention the release the
[kudu-CR](gh-pages) [blog] a blogpost about location awareness in Kudu
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12119 Change subject: [blog] a blogpost about location awareness in Kudu .. [blog] a blogpost about location awareness in Kudu Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 --- A _posts/2018-12-20-location-awareness.md 1 file changed, 155 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/12119/1 -- To view, visit http://gerrit.cloudera.org:8080/12119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newchange Gerrit-Change-Id: I10b30a80d5661fb889a11285b8118cdea1a87cd2 Gerrit-Change-Number: 12119 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
