HeartSaVioR commented on a change in pull request #34333:
URL: https://github.com/apache/spark/pull/34333#discussion_r732395419
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -517,6 +517,8 @@ There are a few built-in sources.
- **Rate source (for testing)** - Generates data at the specified number of
rows per second, each output row contains a `timestamp` and `value`. Where
`timestamp` is a `Timestamp` type containing the time of message dispatch, and
`value` is of `Long` type containing the message count, starting from 0 as the
first row. This source is intended for testing and benchmarking.
Review comment:
The set of available options between rate source and rate-epoch source
are different, like, `rampUpTime` is only valid for rate source, whereas
`startTimestamp` and `advanceMillisPerEpoch` are only valid for rate-epoch
source.
So if we have a unified data source (mix up `rate-epoch` into current
`rate`), then there will be two sets of options, which defines the behavior,
which IMHO is confusing.
Could we please enumerate the upside of adding this as different mode of
rate source?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
