Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13335#discussion_r65411829
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/Offset.scala
---
@@ -17,11 +17,14 @@
package org.apache.spark.sql.execution.streaming
+import com.fasterxml.jackson.annotation.JsonTypeInfo
+
/**
* An offset is a monotonically increasing metric used to track progress
in the computation of a
* stream. An [[Offset]] must be comparable, and the result of `compareTo`
must be consistent
* with `equals` and `hashcode`.
*/
+@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include =
JsonTypeInfo.As.PROPERTY, property = "@class")
--- End diff --
In my experience (with jackson specifically and people who have read the
docs) you end up with surprises down the road due to magic.
@zsxwing In this particular case, maybe we should remove the offsets from
the status messages, or turn the to strings or something. These are going to
be implemented by specific sinks, so I'm not sure we want to expose the details
of arbitrary classes to the listener bus.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
