Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/16258#discussion_r92338712
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ---
@@ -61,13 +61,20 @@ class StateOperatorProgress private[sql](
* @param id An unique query id that persists across restarts. See
`StreamingQuery.id()`.
* @param runId A query id that is unique for every start/restart. See
`StreamingQuery.runId()`.
* @param name User-specified name of the query, null if not specified.
- * @param timestamp Timestamp (ms) of the beginning of the trigger.
+ * @param timestamp Beginning time of the trigger in ISO8601 format, i.e.
UTC timestamps.
* @param batchId A unique id for the current batch of data being
processed. Note that in the
* case of retries after a failure a given batchId my be
executed more than once.
* Similarly, when there is no data to be processed, the
batchId will not be
* incremented.
* @param durationMs The amount of time taken to perform various
operations in milliseconds.
- * @param currentWatermark The current event time watermark in milliseconds
+ * @param eventTime Statistics of event time seen in this batch. It may
contain the following keys:
+ * {
+ * "max" -> "2016-12-05T20:54:20.827Z" // maximum event
time seen in this trigger
--- End diff --
Hi all, I am just leaving a comment as a gentle reminder to note that we
probably should replace `<` or `>` to other ones such as `{@literal <}` or
`{@literal >}` in the future. Please refer
https://github.com/apache/spark/pull/16013#discussion_r89666914. This causes
javadoc8 break.
```
[error]
.../java/org/apache/spark/sql/streaming/StreamingQueryProgress.java:19: error:
bad use of '>'
[error] * "max" -> "2016-12-05T20:54:20.827Z" //
maximum event time seen in this trigger
[error] ^
[error]
.../java/org/apache/spark/sql/streaming/StreamingQueryProgress.java:20: error:
bad use of '>'
[error] * "min" -> "2016-12-05T20:54:20.827Z" //
minimum event time seen in this trigger
[error] ^
[error]
.../java/org/apache/spark/sql/streaming/StreamingQueryProgress.java:21: error:
bad use of '>'
[error] * "avg" -> "2016-12-05T20:54:20.827Z" //
average event time seen in this trigger
[error] ^
[error]
.../java/org/apache/spark/sql/streaming/StreamingQueryProgress.java:22: error:
bad use of '>'
[error] * "watermark" -> "2016-12-05T20:54:20.827Z" //
watermark used in this trigger
[error] ^
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
