Hey all, Reviving this thread. Now that https://github.com/apache/incubator-druid/pull/6742 and https://github.com/apache/incubator-druid/pull/6862 have been released in 0.14, I'm re-proposing graduating Druid SQL from experimental status in the next release, 0.15. I don't think we need a formal vote on this, but if there seems to be general consensus, I'll do a PR before the next 3-monthly 0.15 code freeze (which is in about 2 weeks).
On Thu, Jan 31, 2019 at 9:20 AM Gian Merlino <g...@apache.org> wrote: > It sounds like the general feeling is +1 on Kafka and maybe wait another > release for SQL. I will do a PR to mark Kafka ingest as non-experimental, > then, and on SQL we'll see whether #6742 and #6862 look solid in 0.14. > > On Tue, Jan 15, 2019 at 8:39 AM Gian Merlino <g...@apache.org> wrote: > >> Hi Mat, >> >> Ah, right. IMO https://github.com/apache/incubator-druid/pull/6742 is a >> decent workaround towards making #6176 less of a problem. It would prevent >> incorrect results from happening (the broker will not start up its http >> server & announce itself, and so it won't get picked up by clients, if it >> never got the initialization event). If paired with monitoring that >> restarts unhealthy brokers, the issue should be fully worked-around in >> practice. >> >> Even though there's an (imo) viable workaround, it would still be good to >> fix the root cause of #6176. I just raised >> https://github.com/apache/incubator-druid/pull/6862 to update Curator >> and see if that helps -- there is a bug fixed in the latest release that >> looks like it could cause the behavior we're seeing ( >> https://issues.apache.org/jira/browse/CURATOR-476). >> >> My feeling is that it's still reasonable to remove the experimental label >> from Druid SQL in 0.14, especially since #6742 will make SQL and native >> queries behave at parity (initialization getting missed will delay broker >> startup for _both_ cases). So in that sense they are at least on the same >> footing. And hopefully #6862 will fix them both, together. >> >> On Tue, Jan 15, 2019 at 7:56 AM Pierre-Emile Ferron <pe.fer...@gmail.com> >> wrote: >> >>> A remaining issue with SQL is >>> https://github.com/apache/incubator-druid/issues/6176 >>> >>> We've seen it happen several times in production on 0.12, where >>> thankfully >>> SQL doesn't power anything critical. The current workarounds are: >>> 1. Restart the broker. Obviously not a good solution. >>> 2. Migrate to HTTP segment discovery. I'm fine with that, and we are >>> actually planning to do it soon in our clusters, but I'm still concerned >>> about other Druid users—the default setting is still ZK, which means that >>> SQL would still have this issue by default. >>> >>> Before marking SQL as non-experimental, I'd suggest either fixing the >>> root >>> cause, or making HTTP segment discovery the default and then explicitly >>> deprecating ZK segment discovery. >>> >>> >>> On Mon, Jan 14, 2019 at 2:18 PM Gian Merlino <g...@apache.org> wrote: >>> >>> > I'd like to propose graduating a couple of features out of >>> 'experimental' >>> > status in 0.14. Both are popular features (judging by mailing list & >>> github >>> > issue/PR activity). Both have been around for a while and have >>> attained a >>> > good level of quality and stability of API & behavior. I believe >>> removing >>> > the 'experimental' banner from these features would more accurately >>> reflect >>> > reality, and be a good signal to the user community. >>> > >>> > 1) Kafka indexing service. First introduced in Druid 0.9.1, it went >>> through >>> > a major protocol change in Druid 0.12.0 that added incremental >>> publishing, >>> > & 'mixing' of data from different partitions. Subjectively, quality >>> appears >>> > to be getting more solid, based on frequency of bug reports and also >>> based >>> > on our own experiences running this in production. Finally- I believe >>> it is >>> > already much more robust than Tranquility, the only 'stable' >>> alternative. >>> > >>> > 2) Druid SQL. First introduced in Druid 0.10.0. It isn't feature >>> complete >>> > yet (multi-value dimensions, datasketches, etc, remain unsupported) >>> but the >>> > API and behavior have been generally stable. No major issues around >>> memory >>> > / performance / etc regressions relative to native Druid queries are >>> > outstanding. IMO, it is well on its way to becoming a first class way >>> to >>> > query Druid, and it is a good time to remove the 'experimental' banner. >>> > >>> >>
