github-actions[bot] commented on code in PR #70232:
URL: https://github.com/apache/doris/pull/70232#discussion_r4474784302
##########
regression-test/suites/external_table_p0/tvf/test_http_tvf.groovy:
##########
@@ -349,71 +354,71 @@ suite("test_http_tvf", "p0") {
);
"""
- // hf
- def hfCountOneFile = sql """
+ // Stable mirrors of the HuggingFace fixtures. Keep the regression
independent
+ // from public HuggingFace availability and mutable dataset branches.
+ qt_sql15 """
select count(*) from
http(
- "uri" =
"hf://datasets/fka/awesome-chatgpt-prompts/blob/main/prompts.csv",
- "format" = "csv"
+ "uri" =
"${stableRemoteDataUrl}/awesome-chatgpt-prompts/61229c8c07ea12c6e219b2b2b728f23f2842901b/prompts.csv",
+ "format" = "csv",
+ "skip_lines" = "1"
);
"""
- assertTrue(Long.parseLong(hfCountOneFile[0][0].toString()) > 0)
- def hfCountWildcard = sql """
+ qt_sql16 """
select count(*) from
http(
- "uri" =
"hf://datasets/fka/awesome-chatgpt-prompts/blob/main/*.csv",
- "format" = "csv"
+ "uri" =
"${stableRemoteDataUrl}/awesome-chatgpt-prompts/61229c8c07ea12c6e219b2b2b728f23f2842901b/prompts.csv",
+ "format" = "csv",
+ "skip_lines" = "1"
);
"""
- assertTrue(Long.parseLong(hfCountWildcard[0][0].toString()) > 0)
qt_sql17 """
desc function
http(
- "uri" =
"hf://datasets/fka/awesome-chatgpt-prompts/blob/main/*.csv",
- "format" = "csv"
+ "uri" =
"${stableRemoteDataUrl}/awesome-chatgpt-prompts/61229c8c07ea12c6e219b2b2b728f23f2842901b/prompts.csv",
+ "format" = "csv",
+ "skip_lines" = "1"
);
"""
- // branch
qt_sql18 """
select * from
http(
- "uri" = "hf://datasets/stanfordnlp/imdb@script/dataset_infos.json",
+ "uri" =
"${stableRemoteDataUrl}/stanfordnlp-imdb/script/dataset_infos.json",
"format" = "json"
);
"""
qt_sql19 """
select * from
http(
- "uri" =
"hf://datasets/stanfordnlp/imdb@main/plain_text/test-00000-of-00001.parquet",
+ "uri" =
"${stableRemoteDataUrl}/stanfordnlp-imdb/main/plain_text/test-00000-of-00001.parquet",
Review Comment:
[P1] Avoid making the mandatory P0 suite depend on unbounded 20 MB
downloads. The head commit reports `sql19` hitting the 900 s query timeout at
roughly 10 KB/s, and `sql19`-`sql21` were not run locally; each query causes FE
metadata checks and BE range reads against the new OSS object. A throttled CI
region can therefore stall or fail the whole P0 run. Use a small/local fixture
or otherwise enforce a bounded, validated artifact for these cases.
##########
regression-test/suites/external_table_p0/tvf/test_http_tvf.groovy:
##########
@@ -349,71 +354,71 @@ suite("test_http_tvf", "p0") {
);
"""
- // hf
- def hfCountOneFile = sql """
+ // Stable mirrors of the HuggingFace fixtures. Keep the regression
independent
+ // from public HuggingFace availability and mutable dataset branches.
+ qt_sql15 """
select count(*) from
http(
- "uri" =
"hf://datasets/fka/awesome-chatgpt-prompts/blob/main/prompts.csv",
- "format" = "csv"
+ "uri" =
"${stableRemoteDataUrl}/awesome-chatgpt-prompts/61229c8c07ea12c6e219b2b2b728f23f2842901b/prompts.csv",
Review Comment:
[P2] Preserve HF/glob integration coverage when moving fixtures to the
mirror. This replacement turns the old `hf://` cases into repeated exact HTTPS
reads, so the P0 suite no longer exercises `HttpTableValuedFunction`'s `hf://`
dispatch, `HFUtils.expandGlob()` (`*`, `**`, `[1]`), or `@revision` parsing;
`sql15`/`sql16` and `sql19`/`sql20` now assert the same file. Keep
deterministic HF/glob cases, or add a hermetic equivalent, instead of dropping
all end-to-end coverage.
##########
regression-test/suites/external_table_p0/tvf/test_http_tvf.groovy:
##########
@@ -151,12 +151,17 @@ suite("test_http_tvf", "p0") {
String httpServerHost = context.config.otherConfigs.get("httpTvfHost")
if (httpServerHost == null || httpServerHost.trim().isEmpty()) {
httpServerHost = context.config.otherConfigs.get("externalEnvIp")
- }
- if (httpServerHost == null || httpServerHost.trim().isEmpty()) {
- httpServerHost = InetAddress.getLocalHost().getHostAddress()
+ // A loopback or wildcard externalEnvIp would point a BE on another
host at itself.
+ if (httpServerHost == null || httpServerHost.trim().isEmpty()
+ || InetAddress.getByName(httpServerHost).isLoopbackAddress()
+ || InetAddress.getByName(httpServerHost).isAnyLocalAddress()) {
+ httpServerHost = InetAddress.getLocalHost().getHostAddress()
Review Comment:
[P2] Keep the fallback address compatible with the listener and URL syntax.
With the default loopback config, this branch can select an IPv6 result from
`getLocalHost()`, but the server is bound only to IPv4 `0.0.0.0` and the URL is
built without IPv6 brackets. On an IPv6-resolving runner every local HTTP TVF
query becomes unreachable or malformed; choose a reachable IPv4 address or bind
and format IPv6 explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
