Hello,
We are looking for a developer to help us with a small ETL project using Spark
and Kubernetes. Here are some of the requirements:
1. We need a REST API to run and schedule jobs. We would prefer this done in
Node.js but can be done using Java. The REST API will not be available to the
public.
2. We need an easy way to create new jobs in Java without deploying the whole
server again.
3. We want jobs deployed/ran using Kubernetes.
4. Must be able to scale to 1000s of ETL jobs.
5. Source for data will be one REST API.
6. Destination for data will be one Couchbase Database cluster. (Couchbase also
uses a REST API)
7. I am not sure how many records will be processed per job.
8. The data is mostly sales related data.
I know there are commercial ETL solutions that do everything I want. We are
looking for something simple and do not need a fancy UI to describe our ETL. We
want to use Spark and Java to programmatically describe out ETL jobs.
Please let me know if you are interested.
Thanks,
Warren Bell
--
**
This
email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are
addressed. They may not be disseminated or distributed to persons or
entities other than the ones intended without the authority of the
sender.
If you have received this email in error or are not the
intended
recipient, you may not use, copy, disseminate or distribute
it. Delete it
immediately from your system and notify the sender
promptly
by email that
you have done so. This footnote also confirms that this
email message has
been scanned for the presence of computer viruses.
**
Please consider the environment before printing
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org