173lyb opened a new issue, #6108: URL: https://github.com/apache/seatunnel/issues/6108
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened **1.page循环那块代码有问题，如果遇到http请求抛出异常或者接口404,500等，那么陷入死循环了** 发现代码如下： ``` public void internalPollNext(Collector<SeaTunnelRow> output) throws Exception { try { if (pageInfoOptional.isPresent()) { noMoreElementFlag = false; Long pageIndex = 1L; while (!noMoreElementFlag) { PageInfo info = pageInfoOptional.get(); // increment page info.setPageIndex(pageIndex); // set request param updateRequestParam(info); pollAndCollectData(output); pageIndex += 1; } } else { pollAndCollectData(output); } ``` 这个pollAndCollectData去支持调用http请求，http请求失败后noMoreElementFlag始终为false ### SeaTunnel Version dev ### SeaTunnel Config ```conf env { execution.parallelism = 1 job.mode = "BATCH" } source { Http { result_table_name = "http" url = "https://ssssssssss"; method = "GET" params = { startDate = "2023-01-01" endDate = "2023-11-28" pageSize = 100 pageNum = 1 } format = "json" json_field = { ID = "$.rows[*].id" } pageing = { total_page_size = 7 page_field = pageNum } schema = { fields { ID = string } } } } transform { } sink { Jdbc { url = "xxxxxx" driver = "com.xugu.cloudjdbc.Driver" user = "SYSDBA" password = "xxxx" password_decrypt = false generate_sink_sql = true database = xxxx table = SYSDBA.xxxxx enable_upsert = true #primary_keys = [EVENTID] batch_size = 1000 } } ``` ### Running Command ```shell ./seatunnel.sh -e local --config job/test.conf ``` ### Error Exception ```log Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) ~[?:1.8.0_221] at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_221] at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_221] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_221] at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) ~[?:1.8.0_221] at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) ~[?:1.8.0_221] at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) ~[?:1.8.0_221] at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.client.HttpClientProvider.lambda$retryWithException$1(HttpClientProvider.java:407) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at com.github.rholder.retry.Retryer.call(Retryer.java:160) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] ... 17 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[?:1.8.0_221] at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[?:1.8.0_221] at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) ~[?:1.8.0_221] at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392) ~[?:1.8.0_221] at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_221] at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_221] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_221] at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) ~[?:1.8.0_221] at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) ~[?:1.8.0_221] at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) ~[?:1.8.0_221] at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.client.HttpClientProvider.lambda$retryWithException$1(HttpClientProvider.java:407) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at com.github.rholder.retry.Retryer.call(Retryer.java:160) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] ... 17 more 2023-12-28 13:07:38,007 WARN org.apache.seatunnel.connectors.seatunnel.http.client.HttpClientProvider - [1] request http failed javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946) ~[?:1.8.0_221] at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316) ~[?:1.8.0_221] at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) ~[?:1.8.0_221] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639) ~[?:1.8.0_221] at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) ~[?:1.8.0_221] at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) ~[?:1.8.0_221] at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) ~[?:1.8.0_221] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) ~[?:1.8.0_221] at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.client.HttpClientProvider.lambda$retryWithException$1(HttpClientProvider.java:407) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) ~[connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at com.github.rholder.retry.Retryer.call(Retryer.java:160) [connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.client.HttpClientProvider.retryWithException(HttpClientProvider.java:407) [connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.client.HttpClientProvider.getResponse(HttpClientProvider.java:393) [connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.client.HttpClientProvider.doGet(HttpClientProvider.java:217) [connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.client.HttpClientProvider.execute(HttpClientProvider.java:160) [connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader.executeRequest(HttpSourceReader.java:165) [connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader.pollAndCollectData(HttpSourceReader.java:127) [connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader.pollNext(HttpSourceReader.java:287) [connector-http-base-2.3.3-sign-list-https.jar:2.3.3] at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:150) [seatunnel-starter.jar:2.3.3] at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:95) [seatunnel-starter.jar:2.3.3] at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) [seatunnel-starter.jar:2.3.3] at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:100) [seatunnel-starter.jar:2.3.3] at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613) [seatunnel-starter.jar:2.3.3] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_221] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_221] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_221] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_221] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_221] Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) ~[?:1.8.0_221] at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_221] at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_221] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_221] ... 39 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[?:1.8.0_221] at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[?:1.8.0_221] at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) ~[?:1.8.0_221] at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392) ~[?:1.8.0_221] at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) ~[?:1.8.0_221] at sun.security.validator.Validator.validate(Validator.java:262) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:1.8.0_221] at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:1.8.0_221] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[?:1.8.0_221] ... 39 more ``` ### Zeta or Flink or Spark Version Zeta ### Java or Scala Version 1.8 ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
