EricJoy2048 commented on code in PR #4989:
URL: https://github.com/apache/seatunnel/pull/4989#discussion_r1254176365
##########
docs/en/connector-v2/source/OceanBase.md:
##########
@@ -0,0 +1,168 @@
+# OceanBase
+
+> JDBC OceanBase Source Connector
+
+## Support Those Engines
+
+> Spark<br/>
+> Flink<br/>
+> Seatunnel Zeta<br/>
+
+## Key Features
+
+- [x] [batch](../../concept/connector-v2-features.md)
+- [ ] [stream](../../concept/connector-v2-features.md)
+- [x] [exactly-once](../../concept/connector-v2-features.md)
+- [x] [column projection](../../concept/connector-v2-features.md)
+- [x] [parallelism](../../concept/connector-v2-features.md)
+- [x] [support user-defined split](../../concept/connector-v2-features.md)
+
+## Description
+
+Read external data source data through JDBC.
+
+## Supported DataSource Info
+
+| Datasource | Supported versions | Driver |
Url | Maven
|
+|------------|--------------------------------|---------------------------|--------------------------------------|-------------------------------------------------------------------------------|
+| OceanBase | All OceanBase server versions. | com.oceanbase.jdbc.Driver |
jdbc:oceanbase://localhost:2883/test |
[Download](https://mvnrepository.com/artifact/com.oceanbase/oceanbase-client) |
+
+## Database Dependency
+
+> Please download the support list corresponding to 'Maven' and copy it to the
'$SEATNUNNEL_HOME/plugins/jdbc/lib/' working directory<br/>
+> For example: cp oceanbase-client-xxx.jar $SEATNUNNEL_HOME/plugins/jdbc/lib/
+
+## Data Type Mapping
+
+### Mysql Mode
+
+| Mysql Data type
|
Seatunnel Data type
|
+|-----------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
+| BIT(1)<br/>INT UNSIGNED
| BOOLEAN
|
+| TINYINT<br/>TINYINT UNSIGNED<br/>SMALLINT<br/>SMALLINT
UNSIGNED<br/>MEDIUMINT<br/>MEDIUMINT UNSIGNED<br/>INT<br/>INTEGER<br/>YEAR |
INT
|
+| INT UNSIGNED<br/>INTEGER UNSIGNED<br/>BIGINT
| BIGINT
|
+| BIGINT UNSIGNED
| DECIMAL(20,0)
|
+| DECIMAL(x,y)(Get the designated column's specified column size.<38)
| DECIMAL(x,y)
|
+| DECIMAL(x,y)(Get the designated column's specified column size.>38)
| DECIMAL(38,18)
|
+| DECIMAL UNSIGNED
| DECIMAL((Get the
designated column's specified column size)+1,<br/>(Gets the designated column's
number of digits to right of the decimal point.))) |
+| FLOAT<br/>FLOAT UNSIGNED
| FLOAT
|
+| DOUBLE<br/>DOUBLE UNSIGNED
| DOUBLE
|
+| CHAR<br/>VARCHAR<br/>TINYTEXT<br/>MEDIUMTEXT<br/>TEXT<br/>LONGTEXT<br/>JSON
| STRING
|
+| DATE
| DATE
|
+| TIME
| TIME
|
+| DATETIME<br/>TIMESTAMP
| TIMESTAMP
|
+|
TINYBLOB<br/>MEDIUMBLOB<br/>BLOB<br/>LONGBLOB<br/>BINARY<br/>VARBINAR<br/>BIT(n)
| BYTES
|
+| GEOMETRY<br/>UNKNOWN
| Not supported yet
|
+
+### Oracle Mode
+
+| Oracle Data type | Seatunnel Data
type |
+|-----------------------------------------------------------|---------------------|
+| Number(p), p <= 9 | INT
|
+| Number(p), p <= 18 | BIGINT
|
+| Number(p), p > 18 | DECIMAL(38,18)
|
+| REAL<br/> BINARY_FLOAT | FLOAT
|
+| BINARY_DOUBLE | DOUBLE
|
+| CHAR<br/>NCHAR<br/>NVARCHAR2<br/>NCLOB<br/>CLOB<br/>ROWID | STRING
|
+| DATE | DATE
|
+| TIMESTAMP<br/>TIMESTAMP WITH LOCAL TIME ZONE | TIMESTAMP
|
+| BLOB<br/>RAW<br/>LONG RAW<br/>BFILE | BYTES
|
+| UNKNOWN | Not supported
yet |
+
+## Source Options
+
+| Name | Type | Required | Default |
Description
|
+|------------------------------|--------|----------|-----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| url | String | Yes | - | The URL
of the JDBC connection. Refer to a case: jdbc:oceanbase://localhost:2883/test
|
+| driver | String | Yes | - | The
jdbc class name used to connect to the remote data source, should be
`com.oceanbase.jdbc.Driver`.
|
+| user | String | No | - |
Connection instance user name
|
+| password | String | No | - |
Connection instance password
|
+| compatible_mode | String | Yes | - | The
compatible mode of OceanBase, can be 'mysql' or 'oracle'.
|
+| query | String | Yes | - | Query
statement
|
+| connection_check_timeout_sec | Int | No | 30 | The
time in seconds to wait for the database operation used to validate the
connection to complete
|
+| partition_column | String | No | - | The
column name for parallelism's partition, only support numeric type primary key
and only support single column.
|
Review Comment:
```suggestion
| partition_column | String | No | - | The
column name for parallelism's partition, only support numeric type column and
string type column.
|
```
##########
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-2/src/test/resources/jdbc_oceanbase_oracle_source_and_sink.conf:
##########
@@ -0,0 +1,60 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+######
+###### This config file is a demonstration of streaming processing in
seatunnel config
+######
+
+env {
+ # You can set flink configuration here
+ execution.parallelism = 1
+ job.mode = "BATCH"
+ #execution.checkpoint.interval = 10000
+ #execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
+}
+
+source {
+ jdbc{
+ # This is a example source plugin **only for test and demonstrate the
feature source plugin**
+ url = "jdbc:oceanbase://e2e_oceanbase_oracle:2883/seatunnel"
+ driver = com.oceanbase.jdbc.Driver
+ user = "root"
+ password = ""
+ query = "SELECT * FROM source"
+ compatible_mode = "oracle"
+ }
+}
+
+transform {
+}
+
+sink {
+ Assert {
Review Comment:
Hello, contribution du, thank you for your contribution. A good test case
should not just check whether the number of rows is correct. You should check
the field type of each column in each row and whether the value of the field is
completely correct.
You can reference
https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-clickhouse-e2e/src/test/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/ClickhouseIT.java
to to it.
Thanks.
##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSourceFactory.java:
##########
@@ -83,7 +83,10 @@ TableSource<T, SplitT, StateT>
createSource(TableFactoryContext context) {
JdbcConnectionProvider connectionProvider =
new
SimpleJdbcConnectionProvider(config.getJdbcConnectionConfig());
final String querySql = config.getQuery();
- JdbcDialect dialect =
JdbcDialectLoader.load(config.getJdbcConnectionConfig().getUrl());
+ JdbcDialect dialect =
Review Comment:
Add new option to `public OptionRule optionRule() `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
