Re: Kafka Indexing Service - Decoupling segments from consumer tasks

2018-05-10 Thread Dylan Wylie
affic at Kafka but at least it's simple! It > follows a general rule of making Kafka do as much of the work as possible. > > What do you think? > > On Thu, May 3, 2018 at 1:53 PM, Dylan Wylie <dylanwy...@gmail.com> wrote: > > > Hey Gian, > > > > Thanks for your

Kafka Indexing Service - Decoupling segments from consumer tasks

2018-05-02 Thread Dylan Wylie
Hey there, With the recent improvements to the Kafka Indexing Service we've been migrating over from Tranquility and have had a very positive experience. However one of the downsides to using the KIS, is that the number of segments generated for each period can't be smaller than the number of

Re: Auto-closing old PRs

2019-02-28 Thread Dylan Wylie
02aab13a08a9c37356466e6750c007@%3Cdev.druid.apache.org%3E > > and > > > > > https://lists.apache.org/thread.html/b9cd3aaf2d01801751f16ee0b2beb2cebc39e2a42160ffb268dc6918@%3Cdev.druid.apache.org%3E > > for the discussion of the milestone policy). > > > > I think we should make bug PRs to be not auto-clo

Re: Auto-closing old PRs

2019-03-01 Thread Dylan Wylie
rote: > > > IMO, 60 days is nothing in Druid terms. I suggest making it 6 months. > > > > On Thu, 28 Feb 2019 at 07:36, Dylan Wylie wrote: > > > > > Infra got this switched on this morning for the repository, anyone who > > gets > > > email notification

Re: docker build

2019-02-08 Thread Dylan Wylie
I believe all we have to do is submit a ticket to Apache's Infrastructure team and then we'll have some automatic process that'll automatically update docker-hub with images relating to each release. I guess there's two open questions I think we should reach a consensus on (others feel free to

FOSDEM 2019

2019-01-30 Thread Dylan Wylie
Anyone planning to be at Fosdem this year? If enough of us are attending a quick impromptu Druid gathering might be fun.

Re: 0.16.0-incubating

2019-08-23 Thread Dylan Wylie
Hey all, This causes (what seems to be) a key JAR to be missing from the build - https://github.com/apache/incubator-druid/issues/8338 It could well be something weird in our setup & since we're not seeing anyone else reporting it, that seems likely. Anyone able to triage the issue & my

Segment Load Contention

2020-05-06 Thread Dylan Wylie
Hey folks, Discovered recently that when taking historicals down for maintenance that we get pretty significant query latency spikes across that node's tier. These spikes seem be related to contention from ZKCoordinator threads unzipping segments from deep storage to replace those from the