It sounds like the general feeling is +1 on Kafka and maybe wait another release for SQL. I will do a PR to mark Kafka ingest as non-experimental, then, and on SQL we'll see whether #6742 and #6862 look solid in 0.14.
On Tue, Jan 15, 2019 at 8:39 AM Gian Merlino <g...@apache.org> wrote: > Hi Mat, > > Ah, right. IMO https://github.com/apache/incubator-druid/pull/6742 is a > decent workaround towards making #6176 less of a problem. It would prevent > incorrect results from happening (the broker will not start up its http > server & announce itself, and so it won't get picked up by clients, if it > never got the initialization event). If paired with monitoring that > restarts unhealthy brokers, the issue should be fully worked-around in > practice. > > Even though there's an (imo) viable workaround, it would still be good to > fix the root cause of #6176. I just raised > https://github.com/apache/incubator-druid/pull/6862 to update Curator and > see if that helps -- there is a bug fixed in the latest release that looks > like it could cause the behavior we're seeing ( > https://issues.apache.org/jira/browse/CURATOR-476). > > My feeling is that it's still reasonable to remove the experimental label > from Druid SQL in 0.14, especially since #6742 will make SQL and native > queries behave at parity (initialization getting missed will delay broker > startup for _both_ cases). So in that sense they are at least on the same > footing. And hopefully #6862 will fix them both, together. > > On Tue, Jan 15, 2019 at 7:56 AM Pierre-Emile Ferron <pe.fer...@gmail.com> > wrote: > >> A remaining issue with SQL is >> https://github.com/apache/incubator-druid/issues/6176 >> >> We've seen it happen several times in production on 0.12, where thankfully >> SQL doesn't power anything critical. The current workarounds are: >> 1. Restart the broker. Obviously not a good solution. >> 2. Migrate to HTTP segment discovery. I'm fine with that, and we are >> actually planning to do it soon in our clusters, but I'm still concerned >> about other Druid users—the default setting is still ZK, which means that >> SQL would still have this issue by default. >> >> Before marking SQL as non-experimental, I'd suggest either fixing the root >> cause, or making HTTP segment discovery the default and then explicitly >> deprecating ZK segment discovery. >> >> >> On Mon, Jan 14, 2019 at 2:18 PM Gian Merlino <g...@apache.org> wrote: >> >> > I'd like to propose graduating a couple of features out of >> 'experimental' >> > status in 0.14. Both are popular features (judging by mailing list & >> github >> > issue/PR activity). Both have been around for a while and have attained >> a >> > good level of quality and stability of API & behavior. I believe >> removing >> > the 'experimental' banner from these features would more accurately >> reflect >> > reality, and be a good signal to the user community. >> > >> > 1) Kafka indexing service. First introduced in Druid 0.9.1, it went >> through >> > a major protocol change in Druid 0.12.0 that added incremental >> publishing, >> > & 'mixing' of data from different partitions. Subjectively, quality >> appears >> > to be getting more solid, based on frequency of bug reports and also >> based >> > on our own experiences running this in production. Finally- I believe >> it is >> > already much more robust than Tranquility, the only 'stable' >> alternative. >> > >> > 2) Druid SQL. First introduced in Druid 0.10.0. It isn't feature >> complete >> > yet (multi-value dimensions, datasketches, etc, remain unsupported) but >> the >> > API and behavior have been generally stable. No major issues around >> memory >> > / performance / etc regressions relative to native Druid queries are >> > outstanding. IMO, it is well on its way to becoming a first class way to >> > query Druid, and it is a good time to remove the 'experimental' banner. >> > >> >
