Or even better
Also, I could help to track down the files that were created for ingestion.
On 15/01/2020 06:28, Nate Smith wrote:
Perhaps separating by framework would be good,
Just my 2 cents,
On Jan 14, 2020, at 4:45 PM, Skip Cruse <f8al.er...@gmail.com> wrote:
We should keep the name /spot-ingest/ for the original ingester, but move the
new ingester to /spot-ingest-sparkstreaming/ or similar. Hopefully we can use
the ticket to track down the files that were created, so we can move them to a
new home easily.
Get Outlook for iOS<https://aka.ms/o0ukef>
From: Tadd Wood <taddw...@apache.org>
Sent: Tuesday, January 14, 2020 5:51 PM
Subject: [SPOT-INGEST] Ingest file organization
I noticed that after SPOT-141 was introduced (a new kind of Spot Ingest,
using PySpark Streaming) that it overlaid the new code on top of the old
code on /spot-ingest/. When debugging the code, it makes it hard to
determine which files are relevant to the new or the old ingest process. We
should split them apart. Thoughts?