[
https://issues.apache.org/jira/browse/FLINK-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16341346#comment-16341346
]
ASF GitHub Bot commented on FLINK-8240:
---------------------------------------
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/5240#discussion_r164131675
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/CSV.scala
---
@@ -139,26 +139,26 @@ class CSV extends EncodingDescriptor("csv") {
}
/**
- * Internal method for encoding properties conversion.
+ * Internal method for format properties conversion.
*/
- override protected def addEncodingProperties(properties:
NormalizedProperties): Unit = {
- fieldDelim.foreach(properties.putString("field-delimiter", _))
- lineDelim.foreach(properties.putString("line-delimiter", _))
- properties.putTableSchema("fields", encodingSchema.toIndexedSeq)
- quoteCharacter.foreach(properties.putCharacter("quote-character", _))
- commentPrefix.foreach(properties.putString("comment-prefix", _))
- isIgnoreFirstLine.foreach(properties.putBoolean("ignore-first-line",
_))
- lenient.foreach(properties.putBoolean("ignore-parse-errors", _))
+ override protected def addFormatProperties(properties:
NormalizedProperties): Unit = {
+
fieldDelim.foreach(properties.putString(DescriptorUtils.FIELD_DELIMITER, _))
--- End diff --
I would not define the constants globally. Some constants should be global,
but constants for specific connectors or formats, should go to the respective
descriptor.
IMO, it would be better to have these keys in `CSV` or the class that
validates the properties of a certain type.
> Create unified interfaces to configure and instatiate TableSources
> ------------------------------------------------------------------
>
> Key: FLINK-8240
> URL: https://issues.apache.org/jira/browse/FLINK-8240
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
> Reporter: Timo Walther
> Assignee: Timo Walther
> Priority: Major
>
> At the moment every table source has different ways for configuration and
> instantiation. Some table source are tailored to a specific encoding (e.g.,
> {{KafkaAvroTableSource}}, {{KafkaJsonTableSource}}) or only support one
> encoding for reading (e.g., {{CsvTableSource}}). Each of them might implement
> a builder or support table source converters for external catalogs.
> The table sources should have a unified interface for discovery, defining
> common properties, and instantiation. The {{TableSourceConverters}} provide a
> similar functionality but use an external catalog. We might generialize this
> interface.
> In general a table source declaration depends on the following parts:
> {code}
> - Source
> - Type (e.g. Kafka, Custom)
> - Properties (e.g. topic, connection info)
> - Encoding
> - Type (e.g. Avro, JSON, CSV)
> - Schema (e.g. Avro class, JSON field names/types)
> - Rowtime descriptor/Proctime
> - Watermark strategy and Watermark properties
> - Time attribute info
> - Bucketization
> {code}
> This issue needs a design document before implementation. Any discussion is
> very welcome.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
