wankunde commented on a change in pull request #575:
URL: https://github.com/apache/griffin/pull/575#discussion_r444802163
##########
File path: measure/src/main/scala/org/apache/griffin/measure/sink/Sink.scala
##########
@@ -18,30 +18,57 @@
package org.apache.griffin.measure.sink
import org.apache.spark.rdd.RDD
+import org.apache.spark.sql.DataFrame
import org.apache.griffin.measure.Loggable
/**
- * sink metric and record
+ * Base trait for batch and Streaming Sinks.
+ * To implement custom sinks, extend your classes with this trait.
*/
trait Sink extends Loggable with Serializable {
- val metricName: String
+
+ val jobName: String
val timeStamp: Long
val config: Map[String, Any]
val block: Boolean
- def available(): Boolean
+ /**
+ * Ensures that the pre-requisites (if any) of the Sink are met before
opening it.
+ */
+ def validate(): Boolean
- def start(msg: String): Unit
- def finish(): Unit
+ /**
+ * Allows initialization of the connection to the sink (if required).
+ *
+ * @param applicationId Spark Application ID
+ */
+ def open(applicationId: String): Unit
Review comment:
What's the use of `applicationId `? Can we use `jobName` instead?
##########
File path: measure/src/main/scala/org/apache/griffin/measure/sink/Sink.scala
##########
@@ -18,30 +18,57 @@
package org.apache.griffin.measure.sink
import org.apache.spark.rdd.RDD
+import org.apache.spark.sql.DataFrame
import org.apache.griffin.measure.Loggable
/**
- * sink metric and record
+ * Base trait for batch and Streaming Sinks.
+ * To implement custom sinks, extend your classes with this trait.
*/
trait Sink extends Loggable with Serializable {
- val metricName: String
+
+ val jobName: String
Review comment:
It's better to unify the names of variable, and easier to understand.
In `DQConfig` is `name`, in `BatchDQApp` is `metricName`, in `DQContext` is
`name`, in `SinkFactory` is
`jobName`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
