Aleksandr Borgatin created GRIFFIN-232:
------------------------------------------
Summary: Add support for running batch jobs when an external event
occurs
Key: GRIFFIN-232
URL: https://issues.apache.org/jira/browse/GRIFFIN-232
Project: Griffin (Incubating)
Issue Type: Improvement
Reporter: Aleksandr Borgatin
Presently we can only add batch jobs that run on a cron expression.
Proposal is to add support starting batch jobs by external events, for example
create/update hive table, create/update file in HDFS.
Moreover, so that everyone can create their own events without changing the
code of the Griffin Service, we can make this feature pluggable.
In connection with, I propose to add new object of model - ListenerJob. The
example of request for adding ListenerJob:
{code:java}
{
"job.name": "listener_job",
"cron.expression": "0 * * * * ?",
"cron.time.zone": "GMT+8:00",
"job.class": "org.apache.griffin.listener.FileScanJob",
"job.config": {
"FILE_NAME": "/Users/test/file.txt”
}
}
{code}
As a result, this object will be saved in the database and a listener in quartz
is scheduled with the following parameters from the query: the type from the
“job.class”, the schedule from the “cron.expression”, and the JobDataMap from
the “job.config”.
After that we can add a batch job and instead of a cron expression write in the
request the listener identifier to subscribe to the previously created listener.
When the desired event occurs, quartz job will find all the batch jobs that are
subscribed to this listener, and call them to run once.
The class from the “job.class” must be child of AbstractListenerJob, which
provides service for running jobs by listener identifier and has one abstract
method:
{code:java}
@DisallowConcurrentExecution
@PersistJobDataAfterExecution
public abstract class AbstractListenerJob implements Job {
public static final String JOB_ID_CONFIG_MAP_PROPERTY =
"JOB_ID_CONFIG_MAP_PROPERTY";
@Autowired
private ListenerJobService listenerJobService;
@Override
public void execute(JobExecutionContext context) throws
JobExecutionException {
if (needNotify(context)) {
long jobId =
context.getMergedJobDataMap().getLong(JOB_ID_CONFIG_MAP_PROPERTY);
listenerJobService.triggerJobsByListenerJobId(jobId);
}
}
protected abstract boolean needNotify(JobExecutionContext context) throws
JobExecutionException;
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
