What does this mean for release of new query features without Druid SQL support?
On Mon, Apr 22, 2019 at 2:07 PM Jihoon Son <jihoon...@apache.org> wrote: > +1 > I'm mostly using only SQL. > > Jihoon > > On Mon, Apr 22, 2019 at 12:24 PM Jonathan Wei <jon...@apache.org> wrote: > > > +1, I think it has enough feature parity with the native JSON queries, > and > > it's stable enough to be moved out of experimental. > > > > On Thu, Apr 18, 2019 at 6:23 PM Fangjin Yang <fang...@imply.io> wrote: > > > > > Strong +1 > > > > > > I think there's been enough production usage of Druid SQL, it matches > > what > > > native JSON-over-http can do, and it should no longer be labeled as > > > experimental. > > > > > > On Thu, Apr 18, 2019 at 6:06 PM Gian Merlino <g...@apache.org> wrote: > > > > > > > Hey all, > > > > > > > > Reviving this thread. Now that > > > > https://github.com/apache/incubator-druid/pull/6742 and > > > > https://github.com/apache/incubator-druid/pull/6862 have been > released > > > in > > > > 0.14, I'm re-proposing graduating Druid SQL from experimental status > in > > > the > > > > next release, 0.15. I don't think we need a formal vote on this, but > if > > > > there seems to be general consensus, I'll do a PR before the next > > > 3-monthly > > > > 0.15 code freeze (which is in about 2 weeks). > > > > > > > > On Thu, Jan 31, 2019 at 9:20 AM Gian Merlino <g...@apache.org> > wrote: > > > > > > > > > It sounds like the general feeling is +1 on Kafka and maybe wait > > > another > > > > > release for SQL. I will do a PR to mark Kafka ingest as > > > non-experimental, > > > > > then, and on SQL we'll see whether #6742 and #6862 look solid in > > 0.14. > > > > > > > > > > On Tue, Jan 15, 2019 at 8:39 AM Gian Merlino <g...@apache.org> > > wrote: > > > > > > > > > >> Hi Mat, > > > > >> > > > > >> Ah, right. IMO > https://github.com/apache/incubator-druid/pull/6742 > > > is a > > > > >> decent workaround towards making #6176 less of a problem. It would > > > > prevent > > > > >> incorrect results from happening (the broker will not start up its > > > http > > > > >> server & announce itself, and so it won't get picked up by > clients, > > if > > > > it > > > > >> never got the initialization event). If paired with monitoring > that > > > > >> restarts unhealthy brokers, the issue should be fully > worked-around > > in > > > > >> practice. > > > > >> > > > > >> Even though there's an (imo) viable workaround, it would still be > > good > > > > to > > > > >> fix the root cause of #6176. I just raised > > > > >> https://github.com/apache/incubator-druid/pull/6862 to update > > Curator > > > > >> and see if that helps -- there is a bug fixed in the latest > release > > > that > > > > >> looks like it could cause the behavior we're seeing ( > > > > >> https://issues.apache.org/jira/browse/CURATOR-476). > > > > >> > > > > >> My feeling is that it's still reasonable to remove the > experimental > > > > label > > > > >> from Druid SQL in 0.14, especially since #6742 will make SQL and > > > native > > > > >> queries behave at parity (initialization getting missed will delay > > > > broker > > > > >> startup for _both_ cases). So in that sense they are at least on > the > > > > same > > > > >> footing. And hopefully #6862 will fix them both, together. > > > > >> > > > > >> On Tue, Jan 15, 2019 at 7:56 AM Pierre-Emile Ferron < > > > > pe.fer...@gmail.com> > > > > >> wrote: > > > > >> > > > > >>> A remaining issue with SQL is > > > > >>> https://github.com/apache/incubator-druid/issues/6176 > > > > >>> > > > > >>> We've seen it happen several times in production on 0.12, where > > > > >>> thankfully > > > > >>> SQL doesn't power anything critical. The current workarounds are: > > > > >>> 1. Restart the broker. Obviously not a good solution. > > > > >>> 2. Migrate to HTTP segment discovery. I'm fine with that, and we > > are > > > > >>> actually planning to do it soon in our clusters, but I'm still > > > > concerned > > > > >>> about other Druid users—the default setting is still ZK, which > > means > > > > that > > > > >>> SQL would still have this issue by default. > > > > >>> > > > > >>> Before marking SQL as non-experimental, I'd suggest either fixing > > the > > > > >>> root > > > > >>> cause, or making HTTP segment discovery the default and then > > > explicitly > > > > >>> deprecating ZK segment discovery. > > > > >>> > > > > >>> > > > > >>> On Mon, Jan 14, 2019 at 2:18 PM Gian Merlino <g...@apache.org> > > > wrote: > > > > >>> > > > > >>> > I'd like to propose graduating a couple of features out of > > > > >>> 'experimental' > > > > >>> > status in 0.14. Both are popular features (judging by mailing > > list > > > & > > > > >>> github > > > > >>> > issue/PR activity). Both have been around for a while and have > > > > >>> attained a > > > > >>> > good level of quality and stability of API & behavior. I > believe > > > > >>> removing > > > > >>> > the 'experimental' banner from these features would more > > accurately > > > > >>> reflect > > > > >>> > reality, and be a good signal to the user community. > > > > >>> > > > > > >>> > 1) Kafka indexing service. First introduced in Druid 0.9.1, it > > went > > > > >>> through > > > > >>> > a major protocol change in Druid 0.12.0 that added incremental > > > > >>> publishing, > > > > >>> > & 'mixing' of data from different partitions. Subjectively, > > quality > > > > >>> appears > > > > >>> > to be getting more solid, based on frequency of bug reports and > > > also > > > > >>> based > > > > >>> > on our own experiences running this in production. Finally- I > > > believe > > > > >>> it is > > > > >>> > already much more robust than Tranquility, the only 'stable' > > > > >>> alternative. > > > > >>> > > > > > >>> > 2) Druid SQL. First introduced in Druid 0.10.0. It isn't > feature > > > > >>> complete > > > > >>> > yet (multi-value dimensions, datasketches, etc, remain > > unsupported) > > > > >>> but the > > > > >>> > API and behavior have been generally stable. No major issues > > around > > > > >>> memory > > > > >>> > / performance / etc regressions relative to native Druid > queries > > > are > > > > >>> > outstanding. IMO, it is well on its way to becoming a first > class > > > way > > > > >>> to > > > > >>> > query Druid, and it is a good time to remove the 'experimental' > > > > banner. > > > > >>> > > > > > >>> > > > > >> > > > > > > > > > >
