[
https://issues.apache.org/jira/browse/GRIFFIN-232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772188#comment-16772188
]
Nikolay Sokolov commented on GRIFFIN-232:
-----------------------------------------
I think it should be fit this into AbstractJob model from API end. Can you
please cross-check your API request example against that format ("job.type",
"predicate.config")?
Another thing is that looks like there is something relevant in existing code
base already, specifically "predicate.config". It looks like the place where
all "predicates", like checking whether file exists, should go. In fact, there
is a code already doing similar checks, and supported options can be found in
org.apache.griffin.core.job.factory.PredicatorFactory.
So looks like basic infrastructure for doing such checks is in place, but it's
not pluggable. I suggest the following:
1) do not create new job types
2) research what predicate.config is doing, and document it
service/predicates.md
3) implement pluggable Predicators in PredicatorFactory, allowing to specify
custom perdicators by classname
> Add support for running batch jobs when an external event occurs
> ----------------------------------------------------------------
>
> Key: GRIFFIN-232
> URL: https://issues.apache.org/jira/browse/GRIFFIN-232
> Project: Griffin (Incubating)
> Issue Type: Improvement
> Reporter: Aleksandr Borgatin
> Priority: Minor
>
> Presently we can only add batch jobs that run on a cron expression.
> Proposal is to add support starting batch jobs by external events, for
> example create/update hive table, create/update file in HDFS.
> Moreover, so that everyone can create their own events without changing the
> code of the Griffin Service, we can make this feature pluggable.
> In connection with, I propose to add new object of model - ListenerJob. The
> example of request for adding ListenerJob:
>
> {code:java}
> {
> "job.name": "listener_job",
> "cron.expression": "0 * * * * ?",
> "cron.time.zone": "GMT+8:00",
> "job.class": "org.apache.griffin.listener.FileScanJob",
> "job.config": {
> "FILE_NAME": "/Users/test/file.txt”
> }
> }
> {code}
> As a result, this object will be saved in the database and a listener in
> quartz is scheduled with the following parameters from the query: the type
> from the “job.class”, the schedule from the “cron.expression”, and the
> JobDataMap from the “job.config”.
> After that we can add a batch job and instead of a cron expression write in
> the request the listener identifier to subscribe to the previously created
> listener.
> When the desired event occurs, quartz job will find all the batch jobs that
> are subscribed to this listener, and call them to run once.
> The class from the “job.class” must be child of AbstractListenerJob, which
> provides service for running jobs by listener identifier and has one abstract
> method:
> {code:java}
> @DisallowConcurrentExecution
> @PersistJobDataAfterExecution
> public abstract class AbstractListenerJob implements Job {
> public static final String JOB_ID_CONFIG_MAP_PROPERTY =
> "JOB_ID_CONFIG_MAP_PROPERTY";
> @Autowired
> private ListenerJobService listenerJobService;
> @Override
> public void execute(JobExecutionContext context) throws
> JobExecutionException {
> if (needNotify(context)) {
> long jobId =
> context.getMergedJobDataMap().getLong(JOB_ID_CONFIG_MAP_PROPERTY);
> listenerJobService.triggerJobsByListenerJobId(jobId);
> }
> }
> protected abstract boolean needNotify(JobExecutionContext context) throws
> JobExecutionException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
