luoyuxia commented on code in PR #20696: URL: https://github.com/apache/flink/pull/20696#discussion_r970252678
########## docs/content/docs/dev/table/hiveCompatibility/hiveDialect/Queries/transform.md: ########## @@ -0,0 +1,130 @@ +--- +title: "Transform Clause" +weight: 10 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Transform Clause + +## Description + +The `TRANSFORM` clause allows user to transform inputs using user-specified command or script. + +## Syntax + +```sql +query: + SELECT TRANSFORM ( expression [ , ... ] ) + [ inRowFormat ] + [ inRecordWriter ] + USING command_or_script + [ AS colName [ colType ] [ , ... ] ] + [ outRowFormat ] + [ outRecordReader ] + +rowFormat + : ROW FORMAT + (DELIMITED [FIELDS TERMINATED BY char] + [COLLECTION ITEMS TERMINATED BY char] + [MAP KEYS TERMINATED BY char] + [ESCAPED BY char] + [LINES SEPARATED BY char] + | + SERDE serde_name [WITH SERDEPROPERTIES + property_name=property_value, + property_name=property_value, ...]) + +outRowFormat : rowFormat +inRowFormat : rowFormat +outRecordReader : RECORDREADER className +inRecordWriter: RECORDWRITER record_write_class +``` + +{{< hint warning >}} +**Note:** + +- `MAP ..` and `REDUCE ..` are syntactic transformations of `SELECT TRANSFORM ( ... )` in Hive dialect for such query. + So you can use `MAP` / `REDUCE` to replace `SELECT TRANSFORM`. + {{< /hint >}} + +## Parameters + +- inRowFormat + + Specific use what row format to feed to input data into the running script. + By default, columns will be transformed to `STRING` and delimited by `TAB` before feeding to the user script; + Similarly, all `NULL` values will be converted to the literal string `\N` in order to differentiate `NULL` values from empty strings. + +- outRowFormat + + Specific use what row format to read the output from the running script. + By default, the standard output of the user script will be treated as TAB-separated `STRING` columns, + any cell containing only `\N` will be re-interpreted as a `NULL`, + and then the resulting `STRING` column will be cast to the data type specified in the table declaration in the usual way. + +- inRecordWriter + Specific use what writer(fully-qualified class name) to write the input data. The default is `org.apache.hadoop.hive.ql.exec.TextRecordWriter` + +- outRecordReader + Specific use what reader(fully-qualified class name) to read the output data. The default is `org.apache.hadoop.hive.ql.exec.TextRecordReader` + +- command_or_script + Specifies a command or a path to script to process data. Review Comment: good suggesion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
