This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/dev/pr-12974-2eccdc412a6dce28fe0f80b5d26973737fa876d7 in repository https://gitbox.apache.org/repos/asf/seatunnel.git
commit 84e47e957539de49ef2cd36d38d46a580e041227 Author: SiriusTL666 <[email protected]> AuthorDate: Wed Nov 4 15:00:21 2026 +0000 [Fix][Connector-V2][Socket] bound the socket connect timeout (#12974) Co-authored-by: SiriusTL666 <[email protected]> --- docs/en/connectors/sink/Socket.md | 1 + docs/en/connectors/source/Socket.md | 2 + .../introduction/concepts/incompatible-changes.md | 7 ++ docs/zh/connectors/sink/Socket.md | 1 + docs/zh/connectors/source/Socket.md | 2 + .../introduction/concepts/incompatible-changes.md | 7 ++ .../socket/config/SocketCommonOptions.java | 17 ++++ .../seatunnel/socket/config/SocketConfig.java | 4 + .../socket/config/SocketSourceOptions.java | 23 ++++- .../seatunnel/socket/sink/SocketClient.java | 28 +++++- .../seatunnel/socket/sink/SocketSinkFactory.java | 3 + .../socket/source/SocketSourceFactory.java | 7 ++ .../socket/source/SocketSourceReader.java | 22 +++- .../seatunnel/socket/SocketFactoryTest.java | 98 ++++++++++++++++++ .../socket/source/SocketSourceReaderTest.java | 111 +++++++++++++++++++++ .../e2e/connector/socket/SocketSinkIT.java | 2 + .../src/test/resources/fake_to_socket.conf | 3 + 17 files changed, 329 insertions(+), 9 deletions(-) diff --git a/docs/en/connectors/sink/Socket.md b/docs/en/connectors/sink/Socket.md index fa5ed24548..bd05aef0c4 100644 --- a/docs/en/connectors/sink/Socket.md +++ b/docs/en/connectors/sink/Socket.md @@ -36,6 +36,7 @@ own. | host | String | Yes | | socket server host | | port | Integer | Yes | | socket server port | | max_retries | Integer | No | 3 | The number of retries to send record failed. Must be `>= 0`; set to `0` to fail immediately on the first write error. | +| connect_timeout_ms | Integer | No | 10000 | Connect timeout in milliseconds. `0` disables the application-level timeout and leaves the OS default, which is what this connector used to do unconditionally. | | common-options | | No | - | Sink plugin common parameters, please refer to [Sink Common Options](../common-options/sink-common-options.md) for details | :::tip diff --git a/docs/en/connectors/source/Socket.md b/docs/en/connectors/source/Socket.md index ce07d45a83..446d330df5 100644 --- a/docs/en/connectors/source/Socket.md +++ b/docs/en/connectors/source/Socket.md @@ -46,6 +46,8 @@ Socket source reads each incoming line as a string record. |----------------|---------|----------|---------|--------------------------------------------------------------------------------------------------------------------| | host | String | Yes | - | socket server host | | port | Integer | Yes | - | socket server port | +| connect_timeout_ms | Integer | No | 10000 | Connect timeout in milliseconds. `0` disables the application-level timeout and leaves the OS default, which is what this connector used to do unconditionally. | +| read_timeout_ms | Integer | No | 0 | `SO_TIMEOUT` for reads in milliseconds. `0` (default) keeps the blocking-read behaviour long-lived socket sources rely on. | | common-options | | no | - | Source plugin common parameters, please refer to [Source Common Options](../common-options/source-common-options.md) for details. | :::tip diff --git a/docs/en/introduction/concepts/incompatible-changes.md b/docs/en/introduction/concepts/incompatible-changes.md index 62a1967254..2540e6c74c 100644 --- a/docs/en/introduction/concepts/incompatible-changes.md +++ b/docs/en/introduction/concepts/incompatible-changes.md @@ -49,6 +49,13 @@ Remove any manual offset used to compensate for this defect. Existing stored val - **Impact**: Before this change, the Doris sink followed the v3.0.0 downgrade contract and dropped the timezone offset while keeping the wall-clock digits unchanged. Stored `DATETIME` values for the same `TIMESTAMP_TZ` input can differ when the JVM default zone differs from the Doris session zone, or when `sink.datetime-timezone` is set explicitly. When the option is unset and the table has `TIMESTAMP_TZ` columns, the writer logs a warning. - **Migration Guide**: Set `sink.datetime-timezone` to the Doris session zone. Verify existing stored data if the JVM default zone and Doris session zone differed. (#11524) +### Socket connector: connections now have a 10-second connect timeout by default + +- **Behavior change: `connect_timeout_ms` defaults to 10000 instead of leaving the wait to the operating system** + - **Description**: The Socket source and sink used to establish a TCP connection with no application-level timeout (`connect(address, 0)` for the source, `new Socket(host, port)` for the sink). A peer that was unreachable, stopped responding, or silently dropped the SYN blocked the connector thread until the OS TCP timeout expired. Both now default to a 10-second connect bound. + - **Impact**: Jobs whose target took longer than 10 seconds to accept a connection now fail at startup or on reconnect where they previously succeeded after a longer OS-managed wait. This applies to existing configurations that do not set the option. The source's blocking read behaviour is unchanged: `read_timeout_ms` defaults to `0`, so an idle stream is not treated as a failure. + - **Migration Guide**: Set `connect_timeout_ms = 0` on the source or sink to restore the previous OS-governed behaviour, or raise the value above 10000 for a legitimately slow peer. `read_timeout_ms` is source-only; it has no effect on the sink, which never reads from its socket. + ### Zeta: runtime metric counters restart from zero after a pipeline failover restore - **Behavior change: runtime counters no longer accumulate values from a failed pipeline attempt** diff --git a/docs/zh/connectors/sink/Socket.md b/docs/zh/connectors/sink/Socket.md index ed5aee98de..57f65e22c0 100644 --- a/docs/zh/connectors/sink/Socket.md +++ b/docs/zh/connectors/sink/Socket.md @@ -33,6 +33,7 @@ import ChangeLog from '../changelog/connector-socket.md'; | host | String | 是 | - | socket 服务器主机 | | port | Integer | 是 | - | socket 服务器端口 | | max_retries | Integer | 否 | 3 | 发送失败后的最大重试次数。必须 `>= 0`,设置为 `0` 表示第一次写失败就立即抛出异常。 | +| connect_timeout_ms | Integer | 否 | 10000 | 连接超时(毫秒)。`0` 表示不设应用层超时、由操作系统默认值决定,这也是本连接器此前的固定行为。 | | common-options | | 否 | - | Sink 插件通用参数,详见 [Sink 通用选项](../common-options/sink-common-options.md) | :::tip diff --git a/docs/zh/connectors/source/Socket.md b/docs/zh/connectors/source/Socket.md index bf5d8b7c36..d8886e9038 100644 --- a/docs/zh/connectors/source/Socket.md +++ b/docs/zh/connectors/source/Socket.md @@ -43,6 +43,8 @@ Socket Source 会把每一行输入读取为字符串。 |--------|------|------|--------|------| | host | String | 是 | - | socket 服务器主机 | | port | Integer | 是 | - | socket 服务器端口 | +| connect_timeout_ms | Integer | 否 | 10000 | 连接超时(毫秒)。`0` 表示不设应用层超时、由操作系统默认值决定,这也是本连接器此前的固定行为。 | +| read_timeout_ms | Integer | 否 | 0 | 读取的 `SO_TIMEOUT`(毫秒)。默认 `0` 表示不设读超时,保持长连接 source 依赖的阻塞读取行为。 | | common-options | | 否 | - | 源插件通用参数,请参考 [源通用选项](../common-options/source-common-options.md) 详见。 | :::tip diff --git a/docs/zh/introduction/concepts/incompatible-changes.md b/docs/zh/introduction/concepts/incompatible-changes.md index 0c5342f45c..6f51a64e40 100644 --- a/docs/zh/introduction/concepts/incompatible-changes.md +++ b/docs/zh/introduction/concepts/incompatible-changes.md @@ -44,6 +44,13 @@ MySQL JDBC Sink 现在直接绑定 SeaTunnel `TIMESTAMP`(`LocalDateTime`), - **影响**:在此变更之前,Doris Sink 遵循 v3.0.0 的写入约定,仅丢弃时区偏移并保留原 wall-clock 数字。升级后,当 JVM 默认时区与 Doris 会话时区不同,或显式配置了 `sink.datetime-timezone` 时,同一 `TIMESTAMP_TZ` 输入写入的 `DATETIME` 可能不同。未配置该选项且表中有 `TIMESTAMP_TZ` 列时,writer 会输出警告日志。 - **迁移指南**:将 `sink.datetime-timezone` 设置为 Doris 会话时区。若 JVM 默认时区与 Doris 会话时区曾不一致,请核对已存储数据。(#11524) +### Socket 连接器:连接默认使用 10 秒连接超时 + +- **行为变更:`connect_timeout_ms` 默认为 10000,不再把等待交给操作系统** + - **描述**:Socket source 与 sink 此前建立 TCP 连接时没有应用层超时(source 使用 `connect(address, 0)`,sink 使用 `new Socket(host, port)`)。当对端不可达、停止响应或静默丢弃 SYN 时,连接器线程会阻塞到操作系统 TCP 超时为止。现在两端的连接等待默认限制为 10 秒。 + - **影响**:若目标主机建立连接所需时间超过 10 秒,此前能够成功建立连接、现在会在启动或重连时失败。这会影响未设置该选项的既有配置。source 的阻塞读取行为不变:`read_timeout_ms` 默认为 `0`,空闲的数据流不会被判定为失败。 + - **迁移指南**:在 source 或 sink 上设置 `connect_timeout_ms = 0` 可恢复此前由操作系统决定的行为;若对端确实较慢,可将该值调大至 10000 以上。`read_timeout_ms` 仅适用于 source,对 sink 无效(sink 不从 socket 读取数据)。 + ### Zeta:管道故障转移恢复后运行时计数器从零重新计数 - **行为变更:运行时计数器不再累计失败管道执行的数值** diff --git a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketCommonOptions.java b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketCommonOptions.java index 12e180ddc3..563a8cc85e 100644 --- a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketCommonOptions.java +++ b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketCommonOptions.java @@ -29,4 +29,21 @@ public class SocketCommonOptions { public static final Option<Integer> PORT = Options.key("port").intType().noDefaultValue().withDescription("socket port"); + + public static final int DEFAULT_CONNECT_TIMEOUT_MS = 10000; + + /** + * Bound on the TCP connect for both source and sink. Without it an unreachable or filtered peer + * blocks the connector thread until the OS-level TCP timeout, which is minutes. Note that 0 + * means "no application-level timeout", so the OS default applies. + */ + public static final Option<Integer> CONNECT_TIMEOUT_MS = + Options.key("connect_timeout_ms") + .intType() + .defaultValue(DEFAULT_CONNECT_TIMEOUT_MS) + .withDescription( + "Socket connect timeout in milliseconds. A value of 0 disables the " + + "application-level timeout, leaving the OS default. Default is " + + DEFAULT_CONNECT_TIMEOUT_MS + + " ms"); } diff --git a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketConfig.java b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketConfig.java index 7a1ff52eb2..18947a502b 100644 --- a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketConfig.java +++ b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketConfig.java @@ -28,10 +28,14 @@ public class SocketConfig implements Serializable { private String host; private int port; private int maxNumRetries; + private int connectTimeoutMs; + private int readTimeoutMs; public SocketConfig(ReadonlyConfig config) { this.host = config.get(SocketCommonOptions.HOST); this.port = config.get(SocketCommonOptions.PORT); this.maxNumRetries = config.get(SocketSinkOptions.MAX_RETRIES); + this.connectTimeoutMs = config.get(SocketCommonOptions.CONNECT_TIMEOUT_MS); + this.readTimeoutMs = config.get(SocketSourceOptions.READ_TIMEOUT_MS); } } diff --git a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketSourceOptions.java b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketSourceOptions.java index 71823fea29..061ea3cdb4 100644 --- a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketSourceOptions.java +++ b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/config/SocketSourceOptions.java @@ -17,4 +17,25 @@ package org.apache.seatunnel.connectors.seatunnel.socket.config; -public class SocketSourceOptions extends SocketCommonOptions {} +import org.apache.seatunnel.api.configuration.Option; +import org.apache.seatunnel.api.configuration.Options; + +public class SocketSourceOptions extends SocketCommonOptions { + + public static final int DEFAULT_READ_TIMEOUT_MS = 0; + + /** + * SO_TIMEOUT for the source's blocking read. Defaults to 0, which keeps the historical blocking + * behaviour that long-lived socket sources rely on. This option is source-only: the sink never + * reads from its socket, so a read timeout would be a no-op there. + */ + public static final Option<Integer> READ_TIMEOUT_MS = + Options.key("read_timeout_ms") + .intType() + .defaultValue(DEFAULT_READ_TIMEOUT_MS) + .withDescription( + "Socket read (SO_TIMEOUT) in milliseconds for the source. A value of 0 " + + "(default) means no application-level timeout is applied, so " + + "the read blocks until data arrives or the socket closes; " + + "the OS default then governs the connection."); +} diff --git a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/sink/SocketClient.java b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/sink/SocketClient.java index 4d5b4029f3..703d8d8726 100644 --- a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/sink/SocketClient.java +++ b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/sink/SocketClient.java @@ -27,6 +27,7 @@ import lombok.extern.slf4j.Slf4j; import java.io.IOException; import java.io.OutputStream; +import java.net.InetSocketAddress; import java.net.Socket; @Slf4j @@ -36,6 +37,7 @@ public class SocketClient { private final int port; private int retries; private final int maxNumRetries; + private final int connectTimeoutMs; private transient Socket client; private transient OutputStream outputStream; private final SerializationSchema serializationSchema; @@ -48,14 +50,30 @@ public class SocketClient { this.serializationSchema = serializationSchema; retries = config.getMaxNumRetries(); maxNumRetries = config.getMaxNumRetries(); + connectTimeoutMs = config.getConnectTimeoutMs(); } private void createConnection() throws IOException { - client = new Socket(hostName, port); - client.setKeepAlive(true); - client.setTcpNoDelay(true); - - outputStream = client.getOutputStream(); + Socket socket = new Socket(); + try { + // Connect with a bounded timeout so an unreachable sink fails the task instead of + // blocking the writer thread forever. + socket.connect(new InetSocketAddress(hostName, port), connectTimeoutMs); + socket.setKeepAlive(true); + socket.setTcpNoDelay(true); + // The sink only writes; SO_TIMEOUT governs reads and has no effect here, so no read + // timeout is exposed for this direction. + outputStream = socket.getOutputStream(); + } catch (IOException e) { + // Do not leave an unconnected socket behind when the connect attempt fails. + try { + socket.close(); + } catch (IOException closeFailure) { + e.addSuppressed(closeFailure); + } + throw e; + } + client = socket; } public void open() throws IOException { diff --git a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/sink/SocketSinkFactory.java b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/sink/SocketSinkFactory.java index f948f0473c..7d78226385 100644 --- a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/sink/SocketSinkFactory.java +++ b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/sink/SocketSinkFactory.java @@ -42,6 +42,9 @@ public class SocketSinkFactory implements TableSinkFactory { .optional( SocketSinkOptions.MAX_RETRIES, Conditions.greaterOrEqual(SocketSinkOptions.MAX_RETRIES, 0)) + .optional( + SocketSinkOptions.CONNECT_TIMEOUT_MS, + Conditions.greaterOrEqual(SocketSinkOptions.CONNECT_TIMEOUT_MS, 0)) .build(); } diff --git a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceFactory.java b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceFactory.java index 4bc6d38210..c03f88f651 100644 --- a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceFactory.java +++ b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceFactory.java @@ -17,6 +17,7 @@ package org.apache.seatunnel.connectors.seatunnel.socket.source; +import org.apache.seatunnel.api.configuration.util.Conditions; import org.apache.seatunnel.api.configuration.util.OptionRule; import org.apache.seatunnel.api.source.SeaTunnelSource; import org.apache.seatunnel.api.source.SourceSplit; @@ -41,6 +42,12 @@ public class SocketSourceFactory implements TableSourceFactory { public OptionRule optionRule() { return OptionRule.builder() .required(SocketSourceOptions.HOST, SocketSourceOptions.PORT) + .optional( + SocketSourceOptions.CONNECT_TIMEOUT_MS, + Conditions.greaterOrEqual(SocketSourceOptions.CONNECT_TIMEOUT_MS, 0)) + .optional( + SocketSourceOptions.READ_TIMEOUT_MS, + Conditions.greaterOrEqual(SocketSourceOptions.READ_TIMEOUT_MS, 0)) .build(); } diff --git a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReader.java b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReader.java index 975e3fca70..677e14b23c 100644 --- a/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReader.java +++ b/seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReader.java @@ -53,9 +53,25 @@ public class SocketSourceReader extends AbstractSingleSplitReader<SeaTunnelRow> "connect socket server, host:[{}], port:[{}] ", this.parameter.getHost(), this.parameter.getPort()); - socket.connect( - new InetSocketAddress(this.parameter.getHost(), this.parameter.getPort()), 0); - reader = new BufferedReader(new InputStreamReader(socket.getInputStream())); + try { + // Bounded connect: an unreachable or filtered host would otherwise block the reader + // thread indefinitely with no way to observe a failure. + socket.connect( + new InetSocketAddress(this.parameter.getHost(), this.parameter.getPort()), + this.parameter.getConnectTimeoutMs()); + // 0 keeps the historical blocking-read behaviour, which long-lived socket sources rely + // on. + socket.setSoTimeout(this.parameter.getReadTimeoutMs()); + reader = new BufferedReader(new InputStreamReader(socket.getInputStream())); + } catch (Exception e) { + // Do not leave an unconnected socket behind when open() fails. + try { + socket.close(); + } catch (Exception closeFailure) { + e.addSuppressed(closeFailure); + } + throw e; + } } @Override diff --git a/seatunnel-connectors-v2/connector-socket/src/test/java/org/apache/seatunnel/connectors/seatunnel/socket/SocketFactoryTest.java b/seatunnel-connectors-v2/connector-socket/src/test/java/org/apache/seatunnel/connectors/seatunnel/socket/SocketFactoryTest.java index 89737ab1b5..3ef7153dbc 100644 --- a/seatunnel-connectors-v2/connector-socket/src/test/java/org/apache/seatunnel/connectors/seatunnel/socket/SocketFactoryTest.java +++ b/seatunnel-connectors-v2/connector-socket/src/test/java/org/apache/seatunnel/connectors/seatunnel/socket/SocketFactoryTest.java @@ -21,14 +21,26 @@ import org.apache.seatunnel.api.configuration.ReadonlyConfig; import org.apache.seatunnel.api.configuration.util.ConfigValidator; import org.apache.seatunnel.api.configuration.util.OptionRule; import org.apache.seatunnel.api.configuration.util.OptionValidationException; +import org.apache.seatunnel.api.table.type.BasicType; +import org.apache.seatunnel.api.table.type.SeaTunnelDataType; +import org.apache.seatunnel.api.table.type.SeaTunnelRowType; +import org.apache.seatunnel.connectors.seatunnel.socket.config.SocketConfig; import org.apache.seatunnel.connectors.seatunnel.socket.config.SocketSinkOptions; +import org.apache.seatunnel.connectors.seatunnel.socket.exception.SocketConnectorException; +import org.apache.seatunnel.connectors.seatunnel.socket.sink.SocketClient; import org.apache.seatunnel.connectors.seatunnel.socket.sink.SocketSinkFactory; import org.apache.seatunnel.connectors.seatunnel.socket.source.SocketSourceFactory; +import org.apache.seatunnel.format.json.JsonSerializationSchema; import org.junit.jupiter.api.Assertions; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; +import org.mockito.MockedConstruction; +import org.mockito.Mockito; +import java.io.ByteArrayOutputStream; +import java.net.InetSocketAddress; +import java.net.Socket; import java.util.HashMap; import java.util.Map; @@ -85,4 +97,90 @@ class SocketFactoryTest { cfg.put(SocketSinkOptions.MAX_RETRIES.key(), -1); Assertions.assertThrows(OptionValidationException.class, () -> validateSink(cfg)); } + + @Test + void testSinkOptionRuleWithNegativeConnectTimeoutFails() { + Map<String, Object> cfg = baseSinkConfig(); + cfg.put(SocketSinkOptions.CONNECT_TIMEOUT_MS.key(), -1); + Assertions.assertThrows(OptionValidationException.class, () -> validateSink(cfg)); + } + + @Test + void testSinkOptionRuleAcceptsZeroConnectTimeout() { + Map<String, Object> cfg = baseSinkConfig(); + cfg.put(SocketSinkOptions.CONNECT_TIMEOUT_MS.key(), 0); + Assertions.assertDoesNotThrow(() -> validateSink(cfg)); + } + + @Test + void testSinkDoesNotExposeReadTimeout() { + // The sink only writes, so a read timeout would be a no-op there and must not be offered. + Assertions.assertFalse( + sinkRule.getOptionalOptions().stream() + .anyMatch(o -> "read_timeout_ms".equals(o.key())), + "read_timeout_ms must not be part of the sink option rule"); + } + + @Test + void testSourceOptionRuleWithNegativeConnectTimeoutFails() { + OptionRule sourceRule = new SocketSourceFactory().optionRule(); + Map<String, Object> cfg = new HashMap<>(); + cfg.put(SocketSinkOptions.HOST.key(), "localhost"); + cfg.put(SocketSinkOptions.PORT.key(), 8080); + cfg.put(SocketSinkOptions.CONNECT_TIMEOUT_MS.key(), -1); + Assertions.assertThrows( + OptionValidationException.class, + () -> ConfigValidator.of(ReadonlyConfig.fromMap(cfg)).validate(sourceRule)); + } + + @Test + void testSinkClientPassesConnectTimeoutToSocket() throws Exception { + Map<String, Object> cfg = baseSinkConfig(); + cfg.put(SocketSinkOptions.CONNECT_TIMEOUT_MS.key(), 6543); + SocketConfig socketConfig = new SocketConfig(ReadonlyConfig.fromMap(cfg)); + + try (MockedConstruction<Socket> mocked = + Mockito.mockConstruction( + Socket.class, + (socket, ctx) -> + Mockito.when(socket.getOutputStream()) + .thenReturn(new ByteArrayOutputStream()))) { + SocketClient client = + new SocketClient( + socketConfig, + new JsonSerializationSchema( + new SeaTunnelRowType( + new String[] {"id"}, + new SeaTunnelDataType<?>[] {BasicType.INT_TYPE}))); + client.open(); + + Socket created = mocked.constructed().get(0); + Mockito.verify(created).connect(Mockito.any(InetSocketAddress.class), Mockito.eq(6543)); + } + } + + @Test + void testSinkClientClosesSocketWhenConnectFails() throws Exception { + SocketConfig socketConfig = new SocketConfig(ReadonlyConfig.fromMap(baseSinkConfig())); + + try (MockedConstruction<Socket> mocked = + Mockito.mockConstruction( + Socket.class, + (socket, ctx) -> + Mockito.doThrow(new java.net.ConnectException("refused")) + .when(socket) + .connect( + Mockito.any(InetSocketAddress.class), + Mockito.anyInt()))) { + SocketClient client = + new SocketClient( + socketConfig, + new JsonSerializationSchema( + new SeaTunnelRowType( + new String[] {"id"}, + new SeaTunnelDataType<?>[] {BasicType.INT_TYPE}))); + Assertions.assertThrows(SocketConnectorException.class, client::open); + Mockito.verify(mocked.constructed().get(0)).close(); + } + } } diff --git a/seatunnel-connectors-v2/connector-socket/src/test/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReaderTest.java b/seatunnel-connectors-v2/connector-socket/src/test/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReaderTest.java index 257dfc0b33..6b59adb2e3 100644 --- a/seatunnel-connectors-v2/connector-socket/src/test/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReaderTest.java +++ b/seatunnel-connectors-v2/connector-socket/src/test/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReaderTest.java @@ -18,6 +18,9 @@ package org.apache.seatunnel.connectors.seatunnel.socket.source; import org.apache.seatunnel.api.configuration.ReadonlyConfig; +import org.apache.seatunnel.api.configuration.util.ConfigValidator; +import org.apache.seatunnel.api.configuration.util.OptionRule; +import org.apache.seatunnel.api.configuration.util.OptionValidationException; import org.apache.seatunnel.api.source.Boundedness; import org.apache.seatunnel.api.source.Collector; import org.apache.seatunnel.api.source.SourceReader; @@ -25,13 +28,19 @@ import org.apache.seatunnel.api.table.type.SeaTunnelRow; import org.apache.seatunnel.connectors.seatunnel.common.source.SingleSplitReaderContext; import org.apache.seatunnel.connectors.seatunnel.socket.config.SocketCommonOptions; import org.apache.seatunnel.connectors.seatunnel.socket.config.SocketConfig; +import org.apache.seatunnel.connectors.seatunnel.socket.config.SocketSourceOptions; import org.junit.jupiter.api.Assertions; import org.junit.jupiter.api.Test; +import org.mockito.MockedConstruction; +import org.mockito.Mockito; import java.io.BufferedWriter; +import java.io.ByteArrayInputStream; import java.io.IOException; import java.io.OutputStreamWriter; +import java.net.ConnectException; +import java.net.InetSocketAddress; import java.net.ServerSocket; import java.net.Socket; import java.nio.charset.StandardCharsets; @@ -71,9 +80,14 @@ class SocketSourceReaderTest { } private static SocketSourceReader createReader(int port) { + return createReader(port, new HashMap<>()); + } + + private static SocketSourceReader createReader(int port, Map<String, Object> extraOptions) { Map<String, Object> options = new HashMap<>(); options.put(SocketCommonOptions.HOST.key(), "127.0.0.1"); options.put(SocketCommonOptions.PORT.key(), port); + options.putAll(extraOptions); SourceReader.Context context = mock(SourceReader.Context.class); when(context.getBoundedness()).thenReturn(Boundedness.BOUNDED); return new SocketSourceReader( @@ -81,6 +95,103 @@ class SocketSourceReaderTest { new SingleSplitReaderContext(context)); } + @Test + void shouldApplyConfiguredConnectAndReadTimeouts() { + Map<String, Object> defaults = new HashMap<>(); + defaults.put(SocketCommonOptions.HOST.key(), "127.0.0.1"); + defaults.put(SocketCommonOptions.PORT.key(), 1); + SocketConfig defaultConfig = new SocketConfig(ReadonlyConfig.fromMap(defaults)); + // connect must stay bounded by default; reads stay unbounded to preserve streaming + Assertions.assertEquals( + SocketCommonOptions.DEFAULT_CONNECT_TIMEOUT_MS, + defaultConfig.getConnectTimeoutMs()); + Assertions.assertEquals( + SocketSourceOptions.DEFAULT_READ_TIMEOUT_MS, defaultConfig.getReadTimeoutMs()); + + Map<String, Object> overrides = new HashMap<>(defaults); + overrides.put(SocketCommonOptions.CONNECT_TIMEOUT_MS.key(), 1234); + overrides.put(SocketSourceOptions.READ_TIMEOUT_MS.key(), 4321); + SocketConfig overridden = new SocketConfig(ReadonlyConfig.fromMap(overrides)); + Assertions.assertEquals(1234, overridden.getConnectTimeoutMs()); + Assertions.assertEquals(4321, overridden.getReadTimeoutMs()); + } + + /** + * Deterministic replacement for a "wait for an unreachable host" test. + * + * <p>Wall-clock assertions on real network addresses are not reliable here: on some platforms a + * saturated accept backlog or a filtered route fails immediately with a reset, so the test + * would pass even without the fix. Asserting the arguments handed to Socket.connect is exact + * and fails as soon as the timeout stops being propagated. + */ + @Test + void shouldPassConfiguredTimeoutToConnect() throws Exception { + Map<String, Object> options = new HashMap<>(); + options.put(SocketCommonOptions.HOST.key(), "127.0.0.1"); + options.put(SocketCommonOptions.PORT.key(), 12345); + options.put(SocketCommonOptions.CONNECT_TIMEOUT_MS.key(), 4321); + SourceReader.Context context = mock(SourceReader.Context.class); + when(context.getBoundedness()).thenReturn(Boundedness.BOUNDED); + + try (MockedConstruction<Socket> mocked = + Mockito.mockConstruction( + Socket.class, + (socket, ctx) -> + Mockito.when(socket.getInputStream()) + .thenReturn(new ByteArrayInputStream(new byte[0])))) { + SocketSourceReader reader = + new SocketSourceReader( + new SocketConfig(ReadonlyConfig.fromMap(options)), + new SingleSplitReaderContext(context)); + reader.open(); + + Socket created = mocked.constructed().get(0); + Mockito.verify(created).connect(Mockito.any(InetSocketAddress.class), Mockito.eq(4321)); + Mockito.verify(created).setSoTimeout(0); + } + } + + @Test + void shouldCloseSocketWhenConnectFails() throws Exception { + Map<String, Object> options = new HashMap<>(); + options.put(SocketCommonOptions.HOST.key(), "127.0.0.1"); + options.put(SocketCommonOptions.PORT.key(), 12345); + SourceReader.Context context = mock(SourceReader.Context.class); + when(context.getBoundedness()).thenReturn(Boundedness.BOUNDED); + + try (MockedConstruction<Socket> mocked = + Mockito.mockConstruction( + Socket.class, + (socket, ctx) -> + Mockito.doThrow(new ConnectException("refused")) + .when(socket) + .connect( + Mockito.any(InetSocketAddress.class), + Mockito.anyInt()))) { + SocketSourceReader reader = + new SocketSourceReader( + new SocketConfig(ReadonlyConfig.fromMap(options)), + new SingleSplitReaderContext(context)); + Assertions.assertThrows(ConnectException.class, reader::open); + Mockito.verify(mocked.constructed().get(0)).close(); + } + } + + @Test + void shouldRejectNegativeConnectTimeout() { + Map<String, Object> options = new HashMap<>(); + options.put(SocketCommonOptions.HOST.key(), "localhost"); + options.put(SocketCommonOptions.PORT.key(), 1); + options.put(SocketCommonOptions.CONNECT_TIMEOUT_MS.key(), -1); + SocketConfig config = new SocketConfig(ReadonlyConfig.fromMap(options)); + Assertions.assertEquals(-1, config.getConnectTimeoutMs()); + + OptionRule rule = new SocketSourceFactory().optionRule(); + Assertions.assertThrows( + OptionValidationException.class, + () -> ConfigValidator.of(ReadonlyConfig.fromMap(options)).validate(rule)); + } + private static void writeLine(BufferedWriter writer, String value) throws IOException { writer.write(value); writer.newLine(); diff --git a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-socket-e2e/src/test/java/org/apache/seatunnel/e2e/connector/socket/SocketSinkIT.java b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-socket-e2e/src/test/java/org/apache/seatunnel/e2e/connector/socket/SocketSinkIT.java index 63877c28c2..a186ed9638 100644 --- a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-socket-e2e/src/test/java/org/apache/seatunnel/e2e/connector/socket/SocketSinkIT.java +++ b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-socket-e2e/src/test/java/org/apache/seatunnel/e2e/connector/socket/SocketSinkIT.java @@ -110,6 +110,8 @@ public class SocketSinkIT extends TestSuiteBase implements TestResource { public void testFakeSourceWritesJsonLinesToSocket(TestContainer container) throws Exception { truncateSocketOutput(); + // The config sets a non-default connect_timeout_ms, so this also covers the option + // surviving job parsing, factory validation and sink instantiation. Container.ExecResult execResult = container.executeJob("/fake_to_socket.conf"); Assertions.assertEquals(0, execResult.getExitCode(), execResult.getStderr()); diff --git a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-socket-e2e/src/test/resources/fake_to_socket.conf b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-socket-e2e/src/test/resources/fake_to_socket.conf index d21eff3688..a32e68ee0d 100644 --- a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-socket-e2e/src/test/resources/fake_to_socket.conf +++ b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-socket-e2e/src/test/resources/fake_to_socket.conf @@ -38,5 +38,8 @@ sink { Socket { host = "socket-server" port = 9999 + # Deliberately not the 10000 ms default: the packaged job config has to carry the option + # through parsing and factory validation, which a unit test on the factory cannot show. + connect_timeout_ms = 5000 } }
