drcrallen commented on a change in pull request #6629: Add support parallel
combine in brokers
URL: https://github.com/apache/incubator-druid/pull/6629#discussion_r240789022
##########
File path:
server/src/main/java/org/apache/druid/client/selector/ConnectionCountServerSelectorStrategy.java
##########
@@ -27,16 +27,20 @@
import java.util.Comparator;
import java.util.List;
import java.util.Set;
+import java.util.stream.Collectors;
public class ConnectionCountServerSelectorStrategy implements
ServerSelectorStrategy
{
- private static final Comparator<QueryableDruidServer> COMPARATOR =
- Comparator.comparingInt(s -> s.getClient().getNumOpenConnections());
+ private static final Comparator<RemoteDruidServer> COMPARATOR =
+ Comparator.comparingInt(s -> s.getQueryRunner().getNumOpenConnections());
@Override
public QueryableDruidServer pick(Set<QueryableDruidServer> servers,
DataSegment segment)
{
- return Collections.min(servers, COMPARATOR);
+ return Collections.min(
+ servers.stream().map(server -> (RemoteDruidServer)
server).collect(Collectors.toSet()),
Review comment:
Would it be possible to add a `getWeight()` method to `QueryableDruidServer`
instead? For example:
```java
private static final Comparator<QueryableDruidServer> COMPARATOR =
Comparator.comparingInt(QueryableDruidServer::getWeight);
```
and then for `RemoteDruidServer` it is just
```java
return getQueryRunner().getNumOpenConnections();
```
That might require some reconfiguring of how the strategies are done
though... so maybe not the best thing at this point.
Basically it seems odd here to take `QueryableDruidServer` but require
`RemoteDruidServer`. That is a missmatch in expectations. Either
`QueryableDruidServer` needs extended somehow to account for arbitrary
`ServerSelectorStrategy`, or `ConnectionCountServerSelectorStrategy` needs
changed a bit to allow for `QueryableDruidServer` that don't have a connection
count.
A very simple counter-example is if we have an HTTP/2 connection between the
brokers and historicals at some point in the future (like with gRPC, or maybe
just straight HTTP/2). In such a scenario you are actually just wanting to send
the query to "the historical with the least outstanding queries that I know
about" but don't have "connection count" as a metric. The code as is will break
in such future extensions.
I'm having a hard time thinking of the right abstraction. If you have any
ideas here that don't require hopeful casting please do call them out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
