ferenc-csaky commented on code in PR #10:
URL:
https://github.com/apache/flink-connector-http/pull/10#discussion_r2607242594
##########
docs/content.zh/docs/connectors/table/http.md:
##########
@@ -241,6 +241,27 @@ POST, PUT and GET operations. This query creator allows
you to issue json reques
your own custom http connector. The mappings from columns to the json request
are supplied in the query creator configuration
parameters `http.request.query-param-fields`, `http.request.body-fields` and
`http.request.url-map`.
+### Format considerations
+
+#### For http requests
Review Comment:
http -> HTTP
##########
docs/content.zh/docs/connectors/table/http.md:
##########
@@ -241,6 +241,27 @@ POST, PUT and GET operations. This query creator allows
you to issue json reques
your own custom http connector. The mappings from columns to the json request
are supplied in the query creator configuration
parameters `http.request.query-param-fields`, `http.request.body-fields` and
`http.request.url-map`.
+### Format considerations
+
+#### For http requests
+In order to use a custom format, users have to specify the option
`'lookup-request.format' = '{customFormatName}'`, where `{customFormatName}` is
the identifier of the custom format factory.
+Additionally, it is possible to pass custom query format options from table's
DDL.
+This can be done by:
`'lookup-request.format.{customFormatName}.{customFormatProperty}' =
'{propertyValue}'`, where {customFormatProperty} is the name of a custom
+property and {propertyValue} is the property value.
+For example:
+`'lookup-request.format.myCustomFormatName.foo' = 'baa'`.
+
+With the default configuration, flink-Json format is used for
`GenericGetQueryCreator`; all options defined in
[json-format](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/formats/json/)
can be passed through the table DDL.
+For example:
+`'lookup-request.format.json.fail-on-missing-field' = 'true'`.
+
+#### For http responses
Review Comment:
http -> HTTP
##########
docs/content/docs/connectors/table/http.md:
##########
@@ -240,6 +241,27 @@ POST, PUT and GET operations. This query creator allows
you to issue json reques
your own custom http connector. The mappings from columns to the json request
are supplied in the query creator configuration
parameters `http.request.query-param-fields`, `http.request.body-fields` and
`http.request.url-map`.
+### Format considerations
+
+#### For http requests
Review Comment:
http -> HTTP
##########
flink-connector-http/src/main/java/org/apache/flink/connector/http/table/lookup/HttpLookupTableSourceFactory.java:
##########
@@ -97,18 +97,25 @@ public DynamicTableSource createDynamicTableSource(Context
dynamicTableContext)
FactoryUtil.createTableFactoryHelper(this,
dynamicTableContext);
ReadableConfig readable = helper.getOptions();
+
+ // Discover and validate the decoding format first - this validates
format-specific options
Review Comment:
IMO this comment is redundant, we have 6 more 5 lines later that are a lot
more specific. Comments has to be maintained just like code, so it should have
a reason to be. :)
##########
docs/content/docs/connectors/table/http.md:
##########
@@ -240,6 +241,27 @@ POST, PUT and GET operations. This query creator allows
you to issue json reques
your own custom http connector. The mappings from columns to the json request
are supplied in the query creator configuration
parameters `http.request.query-param-fields`, `http.request.body-fields` and
`http.request.url-map`.
+### Format considerations
+
+#### For http requests
+In order to use a custom format, users have to specify the option
`'lookup-request.format' = '{customFormatName}'`, where `{customFormatName}` is
the identifier of the custom format factory.
+Additionally, it is possible to pass custom query format options from table's
DDL.
+This can be done by:
`'lookup-request.format.{customFormatName}.{customFormatProperty}' =
'{propertyValue}'`, where {customFormatProperty} is the name of a custom
+property and {propertyValue} is the property value.
+For example:
+`'lookup-request.format.myCustomFormatName.foo' = 'baa'`.
+
+With the default configuration, flink-Json format is used for
`GenericGetQueryCreator`; all options defined in
[json-format](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/formats/json/)
can be passed through the table DDL.
+For example:
+`'lookup-request.format.json.fail-on-missing-field' = 'true'`.
+
+#### For http responses
Review Comment:
http -> HTTP
##########
flink-connector-http/src/main/java/org/apache/flink/connector/http/table/lookup/HttpTableLookupFunction.java:
##########
@@ -120,11 +120,16 @@ public Collection<RowData> lookup(RowData keyRow) {
}
}
// if we did not get the physical arity from the http response
physical row then get it from
- // the producedDataType, which is set when we have metadata
- if (physicalArity == -1 && producedDataType != null) {
- List<LogicalType> childrenLogicalTypes =
- producedDataType.getLogicalType().getChildren();
- physicalArity = childrenLogicalTypes.size() - metadataArity;
+ // the producedDataType. which is set when we have metadata or when
there's no data
+ if (physicalArity == -1) {
+ if (producedDataType != null) {
Review Comment:
Lets invert this check, so we can return empty first, and then spare the
extra indentation level for the code that updates `physicalArity`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
