JNSimba commented on code in PR #173:
URL:
https://github.com/apache/doris-spark-connector/pull/173#discussion_r1436214087
##########
spark-doris-connector/src/main/java/org/apache/doris/spark/rest/RestService.java:
##########
@@ -561,48 +583,77 @@ static List<BackendRow> parseBackend(String response,
Logger logger) throws Dori
}
/**
- * get Doris BE node list.
+ * get Doris BE nodes.
* @param logger slf4j logger
- * @return the Doris BE node list
+ * @return the Doris BE node
* @throws IllegalArgumentException BE nodes is illegal
*/
- @VisibleForTesting
- public static List<BackendV2.BackendRowV2> getBackendRows(SparkSettings
sparkSettings, Logger logger) throws DorisException {
- List<String> feNodeList =
allEndpoints(sparkSettings.getProperty(DORIS_FENODES), logger);
- for (String feNode : feNodeList){
+ public static List<BackendV2.BackendRowV2> getBeNodes(SparkSettings
sparkSettings, Logger logger) throws DorisException {
+ List<String> backends =
allBeEndpoints(sparkSettings.getProperty(DORIS_BENODES),logger);
+ List<BackendV2.BackendRowV2> backendRowV2s = new
ArrayList<BackendV2.BackendRowV2>();
+ if (backends == null || backends.isEmpty()) {
+ logger.error(ILLEGAL_ARGUMENT_MESSAGE, "beNodes", backends);
+ throw new IllegalArgumentException("beNodes",
String.valueOf(backends));
+ }
+ BackendV2.BackendRowV2 backendRowV2 = new BackendV2.BackendRowV2();
+ for (int i = 0; i < backends.size(); i++) {
+ String ip =
backends.get(i).substring(0,backends.get(i).indexOf(":"));
try {
- String beUrl = String.format("http://%s"; + BACKENDS_V2,
feNode);
- HttpGet httpGet = new HttpGet(beUrl);
- String response = send(sparkSettings, httpGet, logger);
- logger.info("Backend Info:{}", response);
- List<BackendV2.BackendRowV2> backends =
parseBackendV2(response, logger);
- logger.trace("Parse beNodes '{}'.", backends);
- if (backends == null || backends.isEmpty()) {
- logger.error(ILLEGAL_ARGUMENT_MESSAGE, "beNodes",
backends);
- throw new IllegalArgumentException("beNodes",
String.valueOf(backends));
- }
- return backends;
- } catch (ConnectedFailedException e) {
- logger.info("Doris FE node {} is unavailable: {}, Request the
next Doris FE node", feNode, e.getMessage());
+ Integer port =
Integer.valueOf(backends.get(i).substring(backends.get(i).indexOf(":")+1,backends.get(i).length()));
+ BackendV2.BackendRowV2 backend = backendRowV2.clone();
Review Comment:
Are there any benefits to cloning here? Is it possible to create one
directly by referring to this?
https://github.com/apache/doris-flink-connector/blob/master/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/models/BackendV2.java#L78-L84
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
