talatuyarer commented on code in PR #35572: URL: https://github.com/apache/beam/pull/35572#discussion_r2207009271
########## website/www/site/content/en/documentation/dsls/sql/extensions/create-external-table.md: ########## @@ -644,6 +644,151 @@ TYPE text LOCATION '/home/admin/orders' ``` +## DataGen + +The **DataGen** connector allows for creating tables based on in-memory data generation. This is useful for developing and testing queries locally without requiring access to external systems. The DataGen connector is built-in; no additional dependencies are required. + +Tables can be either **bounded** (generating a fixed number of rows) or **unbounded** (generating a stream of rows at a specific rate). The connector provides fine-grained controls to customize the generated values for each field, including support for event-time windowing. + +### Syntax + +```sql +CREATE EXTERNAL TABLE [ IF NOT EXISTS ] tableName (tableElement [, tableElement ]*) +TYPE datagen +[TBLPROPERTIES tblProperties] +``` + +### Table Properties (`TBLPROPERTIES`) + +The `TBLPROPERTIES` JSON object is used to configure the generator's behavior. + + +#### General Options + +| Key | Required | Description | +| :--- | :--- | :--- | +| `number-of-rows` | **Yes** (or `rows-per-second`) | Creates a **bounded** table with a specified total number of rows. | +| `rows-per-second`| **Yes** (or `number-of-rows`) | Creates an **unbounded** table that generates rows at the specified rate. | + +#### Event-Time and Watermark Configuration + +| Key | Required | Description | +| :--- | :--- | :--- | +| `timestamp.behavior` | No | Specifies the time handling. Can be `'processing_time'` (default) or `'event_time'`. | +| `event_time.timestamp_column` | **Yes**, if `timestamp.behavior` is `event_time` | The name of the column that will be used to drive the event-time watermark for the stream. | +| `event_time.max_out_of_orderness` | No | When using `event_time`, this sets the maximum out-of-orderness in **milliseconds** for generated timestamps (e.g., `'5000'` for 5 seconds). Defaults to `0`. | Review Comment: Done I switched all configuration and conf values kabab case 😋 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org