Or even better
./spot-ingest/python
./spot-ingest/pyspark-streaming
Also, I could help to track down the files that were created for ingestion.
- kostas
On 15/01/2020 06:28, Nate Smith wrote:
Perhaps separating by framework would be good,
./spot-ingest/python
./spot-ingest/spark-streaming
Just my 2 cents,
- nathanael
On Jan 14, 2020, at 4:45 PM, Skip Cruse <[email protected]> wrote:
We should keep the name /spot-ingest/ for the original ingester, but move the
new ingester to /spot-ingest-sparkstreaming/ or similar. Hopefully we can use
the ticket to track down the files that were created, so we can move them to a
new home easily.
Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Tadd Wood <[email protected]>
Sent: Tuesday, January 14, 2020 5:51 PM
To: [email protected]
Subject: [SPOT-INGEST] Ingest file organization
I noticed that after SPOT-141 was introduced (a new kind of Spot Ingest,
using PySpark Streaming) that it overlaid the new code on top of the old
code on /spot-ingest/. When debugging the code, it makes it hard to
determine which files are relevant to the new or the old ingest process. We
should split them apart. Thoughts?
Thank you,
Tadd Wood
