Hi Soheil,
Yes, It's possible to force your application to sleep after Job
do {
// Your spark job goes here
Thread.sleep(3600000);
} while(true);
But maybe AirFlow is better option if you need scheduler on your Spark Job.
On Wed, Jan 23, 2019 at 9:26 AM Soheil Pourbafrani <soheil.i...@gmail.com>
wrote:
> Hi,
>
> I want to submit a job in YARN cluster to read data from Cassandra and
> write them in HDFS, every hour, for example.
>
> Is it possible to make Spark Application sleep in a while true loop and
> awake every hour to process data?
>
--
Moein Hosseini
Data Engineer
mobile: +98 912 468 1859 <+98+912+468+1859>
site: www.moein.xyz
email: moein...@gmail.com
[image: linkedin] <https://www.linkedin.com/in/moeinhm>
[image: twitter] <https://twitter.com/moein7tl>
