On Mon, 7 Dec 2015, Martin Millnert wrote: > > Note that on a largish cluster the public/client traffic is all > > north-south, while the backend traffic is also mostly north-south to the > > top-of-rack and then east-west. I.e., within the rack, almost everything > > is north-south, and client and replication traffic don't look that > > different. > > This problem domain is one of the larger challenges. I worry about > network timeouts for critical cluster traffic in one of the clusters due > to hosts having 2x1GbE. I.e. in our case I want to > prioritize/guarantee/reserve a minimum amount of bandwidth for cluster > health traffic primarily, and secondarily cluster replication. Client > write replication should then be least prioritized.
One word of caution here: the health traffic should really be the same path and class of service as the inter-osd traffic, or else it will not identify failures. e.g., if the health traffic is prioritized, and lower-priority traffic is starved/dropped, we won't notice. > To support this I need our network equipment to perform the CoS job, and > in order to do that at some level in the stack I need to be able to > classify traffic. And furthermore, I'd like to do this with as little > added state as possible. I seem to recall a conversation a year or so ago about tagging stream/sockets so that the network layer could do this. I don't think we got anywhere, though... sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
