guoyuepeng commented on a change in pull request #504: build and run transform
steps with multiple threads
URL: https://github.com/apache/griffin/pull/504#discussion_r299350383
##########
File path:
measure/src/main/scala/org/apache/griffin/measure/step/transform/TransformStep.scala
##########
@@ -28,4 +36,72 @@ trait TransformStep extends DQStep {
val cache: Boolean
+ var status = PENDING
+
+ val parentSteps = new HashSet[TransformStep]
+
+ def doExecute(context: DQContext): Boolean
+
+ def execute(context: DQContext): Boolean = {
+ println("Running step : \n" + debugString())
+ // Submit parents Steps
+ val parentStepFutures = parentSteps.filter(checkAndUpdateStatus).map {
parentStep =>
+ Future {
Review comment:
steps in parallel , but can we have some test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
