[
https://issues.apache.org/jira/browse/GOBBLIN-1187?focusedWorklogId=446863&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-446863
]
ASF GitHub Bot logged work on GOBBLIN-1187:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 16/Jun/20 20:03
Start Date: 16/Jun/20 20:03
Worklog Time Spent: 10m
Work Description: autumnust commented on a change in pull request #3027:
URL: https://github.com/apache/incubator-gobblin/pull/3027#discussion_r441108741
##########
File path:
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/job_monitor/KafkaJobMonitor.java
##########
@@ -100,17 +101,29 @@ protected void shutdownMetrics()
@Override
protected void processMessage(DecodeableKafkaRecord<byte[],byte[]> message) {
try {
- Collection<Either<JobSpec, URI>> parsedCollection =
parseJobSpec(message.getValue());
- for (Either<JobSpec, URI> parsedMessage : parsedCollection) {
- if (parsedMessage instanceof Either.Left) {
- this.newSpecs.inc();
- this.jobCatalog.put(((Either.Left<JobSpec, URI>)
parsedMessage).getLeft());
- } else if (parsedMessage instanceof Either.Right) {
- this.removedSpecs.inc();
- URI jobSpecUri = ((Either.Right<JobSpec, URI>)
parsedMessage).getRight();
- this.jobCatalog.remove(jobSpecUri);
- // Delete the job state if it is a delete spec request
- deleteStateStore(jobSpecUri);
+ Collection<JobSpec> parsedCollection = parseJobSpec(message.getValue());
+ for (JobSpec parsedMessage : parsedCollection) {
+ SpecExecutor.Verb verb =
SpecExecutor.Verb.valueOf(parsedMessage.getMetadata().get(VERB_KEY));
+
+ switch (verb) {
+ case ADD:
+ case UPDATE:
+ case UNKNOWN: // unknown are considered as add request to maintain
backward compatibility
Review comment:
Shouldn't the default block be the place to maintain compatibility? Is
the VERB guaranteed to be set previously ?
##########
File path:
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/job_monitor/KafkaJobMonitor.java
##########
@@ -100,17 +101,29 @@ protected void shutdownMetrics()
@Override
protected void processMessage(DecodeableKafkaRecord<byte[],byte[]> message) {
try {
- Collection<Either<JobSpec, URI>> parsedCollection =
parseJobSpec(message.getValue());
- for (Either<JobSpec, URI> parsedMessage : parsedCollection) {
- if (parsedMessage instanceof Either.Left) {
- this.newSpecs.inc();
- this.jobCatalog.put(((Either.Left<JobSpec, URI>)
parsedMessage).getLeft());
- } else if (parsedMessage instanceof Either.Right) {
- this.removedSpecs.inc();
- URI jobSpecUri = ((Either.Right<JobSpec, URI>)
parsedMessage).getRight();
- this.jobCatalog.remove(jobSpecUri);
- // Delete the job state if it is a delete spec request
- deleteStateStore(jobSpecUri);
+ Collection<JobSpec> parsedCollection = parseJobSpec(message.getValue());
+ for (JobSpec parsedMessage : parsedCollection) {
+ SpecExecutor.Verb verb =
SpecExecutor.Verb.valueOf(parsedMessage.getMetadata().get(VERB_KEY));
+
+ switch (verb) {
+ case ADD:
+ case UPDATE:
+ case UNKNOWN: // unknown are considered as add request to maintain
backward compatibility
+ this.newSpecs.inc();
+ this.jobCatalog.put(parsedMessage);
+ break;
+ case DELETE:
+ this.removedSpecs.inc();
+ URI jobSpecUri = parsedMessage.getUri();
+ this.jobCatalog.remove(jobSpecUri);
+ // Delete the job state if it is a delete spec request
+ deleteStateStore(jobSpecUri);
+ break;
+ case CANCEL:
+ this.jobCatalog.cancel(parsedMessage);
+ break;
+ default:
Review comment:
Shall we throw exceptions here instead of just printing log ?
##########
File path: gobblin-modules/gobblin-kafka-09/build.gradle
##########
@@ -87,6 +87,9 @@ artifacts {
}
test {
+ // this module's test cases hang often when running in parallel.
+ // making every test class execute in a forked test process till tests are
fixed
+ forkEvery = 1
Review comment:
Q: Is this the only change targeting for the travis issue ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 446863)
Time Spent: 1h 20m (was: 1h 10m)
> cancel a gaas flow when the dag manager is disabled
> ---------------------------------------------------
>
> Key: GOBBLIN-1187
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1187
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
