TyrantLucifer commented on issue #2371:
URL:
https://github.com/apache/incubator-seatunnel/issues/2371#issuecomment-1207612744
> > This is a common feature, for example, kafka, HTTP, File, Pulsar, etc;
My opinion is to conform to the habit of SQL;
> > ```
> > # conf file
> > Source {
> > schema {
> > field = "STRING"
> > field2 = "INT"
> > field3 = "DECIMAL(30, 3)"
> > }
> > }
> > ```
>
> I think it's a good idea that add a common feature in connector to support
user-defined schema. In my option, I think we can add a new config option
`schema` in all source connector source configs and for each connector they can
parse their own schema in `getProducedType` method. The implement function of
parsing `schema` we can add it in module `seatunnel-common`. And I agree with
your advice @ashulin to conform schema type to the habit of SQL the same as
defined in code.
>
>
By the way, the row data format information also should defined in source
connector. The final source config as the following:
```hcon
Source {
Kafka {
shema {
fields {
field = "STRING"
field2 = "INT"
field3 = "DECIMAL(30, 3)"
}
format = "json" // format = "text"
}
}
}
```
In most use cases, I think we can support `json` or `text`, if we assign
`text`, we also assign `delimiter` in config file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
