Hi Dylan, "Forever" is still not ideal due to the fact that task logs are not rotated until tasks exit, and they can get unwieldy. But you can safely run with longer durations. I haven't seen anyone doing days, but I have seen a few people doing 4-8 hours and that works fine.
On Wed, Apr 4, 2018 at 4:25 AM, Dylan Wylie <dylanwy...@gmail.com> wrote: > Hey all, > > With the Kafka Indexing Service now supporting incremental handoffs how > are folks configuring the taskDuration of the kafka indexing service? > > If the service was ingesting hourly data into hourly segments, would there > be any downside to setting the taskDuration to one or more days? Or even > just have the task run forever? > > The theorised advantage being when the task runs forever the service > doesn't abruptly write smaller segments. > > Best regards, > Dylan > >
