I haven't seen objection, so, I will do a PR to make this change soon. On Mon, Apr 22, 2019 at 7:04 PM Gian Merlino <g...@apache.org> wrote:
> I think it's fine to release new query features that don't have Druid SQL > support, although I would certainly encourage people to add SQL support > even if it's not required. In the long run I wish that SQL could become the > primary query language for Druid, because in Druid's space (analytical > databases) SQL is what users generally expect to see. > > On Mon, Apr 22, 2019 at 6:43 PM Eyal Yurman > <eyurma...@verizonmedia.com.invalid> wrote: > >> What does this mean for release of new query features without Druid SQL >> support? >> >> On Mon, Apr 22, 2019 at 2:07 PM Jihoon Son <jihoon...@apache.org> wrote: >> >> > +1 >> > I'm mostly using only SQL. >> > >> > Jihoon >> > >> > On Mon, Apr 22, 2019 at 12:24 PM Jonathan Wei <jon...@apache.org> >> wrote: >> > >> > > +1, I think it has enough feature parity with the native JSON queries, >> > and >> > > it's stable enough to be moved out of experimental. >> > > >> > > On Thu, Apr 18, 2019 at 6:23 PM Fangjin Yang <fang...@imply.io> >> wrote: >> > > >> > > > Strong +1 >> > > > >> > > > I think there's been enough production usage of Druid SQL, it >> matches >> > > what >> > > > native JSON-over-http can do, and it should no longer be labeled as >> > > > experimental. >> > > > >> > > > On Thu, Apr 18, 2019 at 6:06 PM Gian Merlino <g...@apache.org> >> wrote: >> > > > >> > > > > Hey all, >> > > > > >> > > > > Reviving this thread. Now that >> > > > > https://github.com/apache/incubator-druid/pull/6742 and >> > > > > https://github.com/apache/incubator-druid/pull/6862 have been >> > released >> > > > in >> > > > > 0.14, I'm re-proposing graduating Druid SQL from experimental >> status >> > in >> > > > the >> > > > > next release, 0.15. I don't think we need a formal vote on this, >> but >> > if >> > > > > there seems to be general consensus, I'll do a PR before the next >> > > > 3-monthly >> > > > > 0.15 code freeze (which is in about 2 weeks). >> > > > > >> > > > > On Thu, Jan 31, 2019 at 9:20 AM Gian Merlino <g...@apache.org> >> > wrote: >> > > > > >> > > > > > It sounds like the general feeling is +1 on Kafka and maybe wait >> > > > another >> > > > > > release for SQL. I will do a PR to mark Kafka ingest as >> > > > non-experimental, >> > > > > > then, and on SQL we'll see whether #6742 and #6862 look solid in >> > > 0.14. >> > > > > > >> > > > > > On Tue, Jan 15, 2019 at 8:39 AM Gian Merlino <g...@apache.org> >> > > wrote: >> > > > > > >> > > > > >> Hi Mat, >> > > > > >> >> > > > > >> Ah, right. IMO >> > https://github.com/apache/incubator-druid/pull/6742 >> > > > is a >> > > > > >> decent workaround towards making #6176 less of a problem. It >> would >> > > > > prevent >> > > > > >> incorrect results from happening (the broker will not start up >> its >> > > > http >> > > > > >> server & announce itself, and so it won't get picked up by >> > clients, >> > > if >> > > > > it >> > > > > >> never got the initialization event). If paired with monitoring >> > that >> > > > > >> restarts unhealthy brokers, the issue should be fully >> > worked-around >> > > in >> > > > > >> practice. >> > > > > >> >> > > > > >> Even though there's an (imo) viable workaround, it would still >> be >> > > good >> > > > > to >> > > > > >> fix the root cause of #6176. I just raised >> > > > > >> https://github.com/apache/incubator-druid/pull/6862 to update >> > > Curator >> > > > > >> and see if that helps -- there is a bug fixed in the latest >> > release >> > > > that >> > > > > >> looks like it could cause the behavior we're seeing ( >> > > > > >> https://issues.apache.org/jira/browse/CURATOR-476). >> > > > > >> >> > > > > >> My feeling is that it's still reasonable to remove the >> > experimental >> > > > > label >> > > > > >> from Druid SQL in 0.14, especially since #6742 will make SQL >> and >> > > > native >> > > > > >> queries behave at parity (initialization getting missed will >> delay >> > > > > broker >> > > > > >> startup for _both_ cases). So in that sense they are at least >> on >> > the >> > > > > same >> > > > > >> footing. And hopefully #6862 will fix them both, together. >> > > > > >> >> > > > > >> On Tue, Jan 15, 2019 at 7:56 AM Pierre-Emile Ferron < >> > > > > pe.fer...@gmail.com> >> > > > > >> wrote: >> > > > > >> >> > > > > >>> A remaining issue with SQL is >> > > > > >>> https://github.com/apache/incubator-druid/issues/6176 >> > > > > >>> >> > > > > >>> We've seen it happen several times in production on 0.12, >> where >> > > > > >>> thankfully >> > > > > >>> SQL doesn't power anything critical. The current workarounds >> are: >> > > > > >>> 1. Restart the broker. Obviously not a good solution. >> > > > > >>> 2. Migrate to HTTP segment discovery. I'm fine with that, and >> we >> > > are >> > > > > >>> actually planning to do it soon in our clusters, but I'm still >> > > > > concerned >> > > > > >>> about other Druid users—the default setting is still ZK, which >> > > means >> > > > > that >> > > > > >>> SQL would still have this issue by default. >> > > > > >>> >> > > > > >>> Before marking SQL as non-experimental, I'd suggest either >> fixing >> > > the >> > > > > >>> root >> > > > > >>> cause, or making HTTP segment discovery the default and then >> > > > explicitly >> > > > > >>> deprecating ZK segment discovery. >> > > > > >>> >> > > > > >>> >> > > > > >>> On Mon, Jan 14, 2019 at 2:18 PM Gian Merlino <g...@apache.org >> > >> > > > wrote: >> > > > > >>> >> > > > > >>> > I'd like to propose graduating a couple of features out of >> > > > > >>> 'experimental' >> > > > > >>> > status in 0.14. Both are popular features (judging by >> mailing >> > > list >> > > > & >> > > > > >>> github >> > > > > >>> > issue/PR activity). Both have been around for a while and >> have >> > > > > >>> attained a >> > > > > >>> > good level of quality and stability of API & behavior. I >> > believe >> > > > > >>> removing >> > > > > >>> > the 'experimental' banner from these features would more >> > > accurately >> > > > > >>> reflect >> > > > > >>> > reality, and be a good signal to the user community. >> > > > > >>> > >> > > > > >>> > 1) Kafka indexing service. First introduced in Druid 0.9.1, >> it >> > > went >> > > > > >>> through >> > > > > >>> > a major protocol change in Druid 0.12.0 that added >> incremental >> > > > > >>> publishing, >> > > > > >>> > & 'mixing' of data from different partitions. Subjectively, >> > > quality >> > > > > >>> appears >> > > > > >>> > to be getting more solid, based on frequency of bug reports >> and >> > > > also >> > > > > >>> based >> > > > > >>> > on our own experiences running this in production. Finally- >> I >> > > > believe >> > > > > >>> it is >> > > > > >>> > already much more robust than Tranquility, the only 'stable' >> > > > > >>> alternative. >> > > > > >>> > >> > > > > >>> > 2) Druid SQL. First introduced in Druid 0.10.0. It isn't >> > feature >> > > > > >>> complete >> > > > > >>> > yet (multi-value dimensions, datasketches, etc, remain >> > > unsupported) >> > > > > >>> but the >> > > > > >>> > API and behavior have been generally stable. No major issues >> > > around >> > > > > >>> memory >> > > > > >>> > / performance / etc regressions relative to native Druid >> > queries >> > > > are >> > > > > >>> > outstanding. IMO, it is well on its way to becoming a first >> > class >> > > > way >> > > > > >>> to >> > > > > >>> > query Druid, and it is a good time to remove the >> 'experimental' >> > > > > banner. >> > > > > >>> > >> > > > > >>> >> > > > > >> >> > > > > >> > > > >> > > >> > >> >
