liunaijie commented on code in PR #6975:
URL: https://github.com/apache/seatunnel/pull/6975#discussion_r1637790885
##########
seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/source/ClickhouseSourceFactory.java:
##########
@@ -37,11 +61,60 @@ public String factoryIdentifier() {
return "Clickhouse";
}
+ @Override
+ public <T, SplitT extends SourceSplit, StateT extends Serializable>
+ TableSource<T, SplitT, StateT>
createSource(TableSourceFactoryContext context) {
+ ReadonlyConfig readonlyConfig = context.getOptions();
+ List<ClickHouseNode> nodes =
ClickhouseUtil.createNodes(readonlyConfig);
+
+ String sql = readonlyConfig.get(SQL);
+ ClickHouseNode currentServer =
nodes.get(ThreadLocalRandom.current().nextInt(nodes.size()));
+ try (ClickHouseClient client =
ClickHouseClient.newInstance(currentServer.getProtocol());
+ ClickHouseResponse response =
+ client.connect(currentServer)
+
.format(ClickHouseFormat.RowBinaryWithNamesAndTypes)
+ .query(modifySQLToLimit1(sql))
+ .executeAndWait()) {
+ TableSchema.Builder builder = TableSchema.builder();
+ int columnSize = response.getColumns().size();
+ for (int i = 0; i < columnSize; i++) {
+ String columnName =
response.getColumns().get(i).getColumnName();
+ SeaTunnelDataType<?> seaTunnelDataType =
+ TypeConvertUtil.convert(response.getColumns().get(i));
+ builder.column(
+ PhysicalColumn.of(
+ columnName, seaTunnelDataType, Long.MAX_VALUE,
true, null, null));
+ }
+ CatalogTable catalogTable =
+ CatalogTable.of(
+ TableIdentifier.of(
+ factoryIdentifier(),
readonlyConfig.get(DATABASE), "default"),
+ builder.build(),
+ Collections.emptyMap(),
+ Collections.emptyList(),
+ "",
+ factoryIdentifier());
+ return () ->
+ (SeaTunnelSource<T, SplitT, StateT>)
+ new ClickhouseSource(nodes, catalogTable, sql);
+ } catch (ClickHouseException e) {
+ throw new ClickhouseConnectorException(
+ SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED,
+ String.format(
+ "PluginName: %s, PluginType: %s, Message: %s",
+ factoryIdentifier(), PluginType.SOURCE,
e.getMessage()));
+ }
+ }
+
Review Comment:
thanks, will update the code.
and one more thing i want say. now clickhouse source doesn't has
`table_name` parameter, only has `query`. So we can't know the table name we
queryed (or need parse the sql).
Maybe we can add `table_name` is next version, then when we generate the
CatalogTable, we can pass the real name instead of `default`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
