Re: Accidentally cloned Superset to druid-io

Thank you, Xavier! On Thu, May 18, 2023 at 8:47 AM Xavier Léauté wrote: > No worries, I can take care of it. > > On Thu, May 18, 2023 at 7:45 AM Charles Allen wrote: > > > https://github.com/druid-io/superset > > > > I clicked too fast and didn't realize cl

Accidentally cloned Superset to druid-io

https://github.com/druid-io/superset I clicked too fast and didn't realize cloning went to druid-io instead of my own stuff. Sorry about that! I'm going to work with the repo admins to rectify and minimize the chance of it happening again. Cheers, Charles Allen

Pinot tiered storage

https://docs.google.com/document/d/1Z4FLg3ezHpqvc6zhy0jR6Wi2OL8wLO_lRC6aLkskFgs/edit FYI, this proposal just came across the incubator mailing list.

Re: Druid and machine learning

neck > and > > therefore no significant reason to use Druid instead of just plugging > Spark > > engine to Druid segments. > > > > At indexing side, Druid indexer may be considered a general-purpose job > > scheduler so that somebody who already has Druid may levera

Re: Druid and machine learning

Druid segments. > > > > > > At indexing side, Druid indexer may be considered a general-purpose job > > > scheduler so that somebody who already has Druid may leverage it > instead > > of > > > setting up a separate Airflow scheduler. > > > > &

Re: Druid and machine learning

> it makes more sense to have tooling around Druid, to do slice and dice the data that you need, and do the ml stuff in sklearn, or even in spark I agree with this sentiment. Druid as an execution engine is very good at doing distributed aggregation (distributed reduce). What advantage does

Re: stop() method for extensions module

I had problems with this as well. Check out https://github.com/apache/incubator-druid/pull/6798 for some ways to handle this. On Wed, Dec 4, 2019 at 9:22 AM Krishna Likhitha Katakam < krishna.likhi...@phonepe.com> wrote: > Hi, > > I have a basic question: > When we write a custom Druid

Re: Discussion: Moving DataSketches to core

Any time we discuss moving things into core Druid I would love to see a list of dependencies that comes with it. On Wed, Oct 30, 2019, 6:08 PM Jihoon Son wrote: > +1 on moving too. > > On Mon, Oct 28, 2019 at 12:46 PM Fangjin Yang wrote: > > > +1 on moving datasketches to core > > > > On Mon,

Re: Graduation

+1 On Fri, Jun 7, 2019, 9:27 PM Gian Merlino wrote: > Hey Druids, > > Druid has been in the incubator for a while, and we have done 4 releases so > far (0.13.0, 0.14.0, 0.14.1, and 0.14.2) with a 5th on the way. There has > been some discussion off-list recently about pushing for graduation

Re: Druid community weekly meetings

meeting, can you please call them out? Cheers, Charles Allen On Tue, May 21, 2019 at 10:14 AM Anastasia Braginsky wrote: > Hi Everyone, > Are there no weekly meetings anymore? I recall there were on Tuesdays... > Have it been moved to some other time? > Thanks,Anastasia >

Approx N-tile and complex object return values

quantiles singularly in different aggregations. Which in this case I believe would be an array of `NaN`. Thoughts? Charles Allen - To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org For additional commands, e-mail: dev-h...@druid.apache.org

Re: decompression time metric, ie `query/decompress/time`

We use the gcp profiler to get this kind of info. I'm torn, because on one hand it is handy to be able to have such information in the metrics system. But on the other hand you are kind of adding in extra profiling stuff when there are other profiling tools out there. As another example would we

Re: JDK 11 support

https://github.com/apache/incubator-druid/issues/5589 desire is there, but needs some footwork. There are some enterprises that cannot migrate hadoop deployments to java11, so that would have to be sorted out on how long it would be supported. I think trying to keep up with JDK long term support

Re: Data Types

For my team we start from the other direction. What are people DOING with the data. For example, if they are doing counts and sums with basic predicates, then in what ways does the existing feature set not meet those needs? If they are doing other things, what is the end result they are trying to

Re: dns lookups cached for kafka?

I believe so. That's what I do On Thu, Mar 21, 2019 at 8:52 AM Don Bowman wrote: > On Thu, 21 Mar 2019 at 11:48, Charles Allen .invalid> > wrote: > > > Druid assumes the network layer handles whatever tuning is needed > regarding > > DNS resolution or IP routing. I

Re: dns lookups cached for kafka?

Druid assumes the network layer handles whatever tuning is needed regarding DNS resolution or IP routing. In general this means making sure you have your java settings correct (see https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-jvm-ttl.html for a related article). On Thu, Mar

Re: Proposed website migration plan

Are there other projects who have transitioned an independently successful domain name to an apache one? On Tue, Mar 5, 2019 at 2:13 PM David Lim wrote: > Who has control over the druid.io domain? Charles would that be you? > > We'd need support from them for the DNS redirect. > > On Tue, Mar

Re: docker build

ckerfile) > - Tell users who want to use MySQL that they must publish that extra > layered image themselves > > --dave > > > On Tue, Mar 5, 2019 at 9:59 AM Charles Allen .invalid> > wrote: > > > Honestly we're at a very strange impasse. On one hand I don't th

Re: docker build

Honestly we're at a very strange impasse. On one hand I don't think the ASF project can adopt an official docker image unless ASF legal says its ok. "Official" releases are source code anyways (as my understanding goes), and binary artifacts are convenience things. Unfortunately I do not see a

Re: docker build

haha, sorry, I mean I agree with Don On Tue, Mar 5, 2019 at 9:37 AM Charles Allen wrote: > I agree with Gian on this sentiment. > > On Tue, Feb 19, 2019 at 7:47 AM Don Bowman wrote: > >> On Mon, 18 Feb 2019 at 19:14, Gaurav Bhatnagar >> wrote: >> >> > I

Re: docker build

I agree with Gian on this sentiment. On Tue, Feb 19, 2019 at 7:47 AM Don Bowman wrote: > On Mon, 18 Feb 2019 at 19:14, Gaurav Bhatnagar wrote: > > > I have been thinking if automated scripts can be provided to end users in > > Druid for "Additional Dependencies" for user initiated installation

Re: Datasketches

: > What scope would you suggest for the label or github project? > > There seem to be discussions going on around making DataSketches HLL and/or > Quantiles more 'default' options for their respective areas -- are you > thinking that kind of thing? > > On Mon, Feb 25, 2019 at

Datasketches

things can be tracked easier. Anyone have any opinions or desires here? Thanks, Charles Allen

Re: Knowledge sharing between Druid developers via technical talks

such guidance on how to handle videos released on behalf of ASF projects? Cheers, Charles Allen On Fri, Feb 22, 2019 at 11:20 AM Furkan KAMACI wrote: > Hi, > > It would be really great to organize/attend such tech talks/meetups and > record them. > > On the other hand, we can create

Dev sync

I am unable to host the dev sync this week. Is anyone finding utility out of these? the dev list seems pretty active these days, so the legacy utility of the dev sync is very muted (this is a good thing). Unless people are finding specific utility out of a weekly video sync up, I propose it be

Re: json logging enable

ssorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > > org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:427) > at > > org.apache.druid.java.util.common.lifecycle.Lifecycle.start(Lifecycle.java:323) > at org.apache

Re: Forbiddenapis Plugin

Is this indicative of latent bugs the generated sources have? On Thu, Jan 31, 2019 at 8:55 AM Gian Merlino wrote: > I get those sometimes with generated sources -- typically doing a "mvn > clean" beforehand clears it up. We might be able to add exclusions for the > generated source directories

Re: Slow download of segments from deep storage

I *think* the HTTP coordination already enables this On Wed, Jan 30, 2019 at 4:20 PM Samarth Jain wrote: > We noticed that it takes a long time for the historicals to download > segments from deep storage (in our case S3). Looking closer at the code in > ZKCoordinator, I noticed that the

Re: Dev sync this week

sorry, yes, that is the information I meant. that Feature Freeze is in its final stages On Tue, Jan 29, 2019 at 12:21 PM David Glasser wrote: > On Tue, Jan 29, 2019 at 10:45 AM Charles Allen > wrote: > >* *0.14 *release is in final stages, any blockers for 0.14 should be

Dev sync today

My computer is having issues. Can anyone else start the dev sync today?

Re: Docker image

The idea has been toyed around with internally here. What would your expectations be of such an image? On Wed, Jan 16, 2019 at 2:35 PM Don Bowman wrote: > Is anyone working on a docker image? I mean, there are quite a few out > there but they have some various issues, usually security based as

Sync up this week

To join the video meeting, click this link: https://meet.google.com/ozi-rtfg-ags Otherwise, to join by phone, dial +1 442-666-1256 and enter this PIN: 6867# To view more phone numbers, click this link: https://tel.meet/ozi-rtfg-ags?hs=5 Cheers!

Re: dev sync this week

not much interest this week, ducking out On Tue, Jan 8, 2019 at 10:06 AM Charles Allen wrote: > https://meet.google.com/ozi-rtfg-ags > > sorry for the late notice >

Re: Watermarks!

me range has reached some "complete" criteria (number of events, or signal from external system could make sense). The nice thing here is also with automated checks, which can wait until the watermark has progressed before querying the druid cluster for some data. Hopefully that answers

Watermarks!

on forever if desired. An alternative I would like to propose is that the primitives for the watermark feature be added to core druid, and the extension points be added to their respective places (mysql extension and google extension to name two explicitly). Let me know what you think! Charles Allen

Re: Writing a Druid extension

https://github.com/apache/incubator-druid/pull/6798 Please check it out Nikita On Sat, Dec 29, 2018 at 11:59 PM Nikita Dolgov wrote: > I was experimenting with a Druid extension prototype and encountered some > difficulties. The experiment is to build something like >

Re: Writing a Druid extension

oint setup in an extension wouldn't be too hard, but have not tried such a thing. This one would probably be worth having its own discussion thread (maybe an issue?) on how to handle. You are on the right track! Charles Allen On Sat, Dec 29, 2018 at 11:59 PM Nikita Dolgov wrote: > I was

Re: Drop 0. from the version

it is sort of a marketing/mentality thing, so it's an > > opportunity for us to declare that we feel Druid has reached some > > milestone. My feeling at this time is probably ASF graduation or > > completeness of vision (see my earlier mail for thoughts there) are the &

Re: Drop 0. from the version

X-2).? is not supported. Some things that will not have tools might have warning logs printed that the functionality will change (should we change these to alerts?) If this sounds reasonable then jumping straight to Apache Druid 14 on the first official apache release would make a lot of sense. Cheers, Char

Unable to start dev sync this week

I'm unable to start the dev sync this week. Can someone else start it up? Thanks, Charles Allenn

Re: This week's dev sync

processing, please make sure it is known in the community. * It is proposed the dev sync for Dec 25th and Jan 1st be skipped. On Tue, Dec 11, 2018 at 9:51 AM Charles Allen wrote: > To join the video meeting, click this link: > https://meet.google.com/ozi-rtfg-ags > Otherwise, to join

This week's dev sync

To join the video meeting, click this link: https://meet.google.com/ozi-rtfg-ags Otherwise, to join by phone, dial +1 442-666-1256 and enter this PIN: 6867# To view more phone numbers, click this link: https://tel.meet/ozi-rtfg-ags?hs=5

Dev sync this week

I have a conflict for the meeting this week (again, unfortunately) Is anyone else able to start the meeting?

Sync up this week

Hi all! I have an off-site today so will not be able to host the sync up. Is anyone else able to host? Thank you, Charles Allen

Dev sync this week

I have a conflict. Can someone else start the dev sync this week? Thank you, Charles Allen

Bug in Jackson Smile 2.6.5 (-Pspark2)

If anyone is building druid with -Pspark2, there is a weird bug in older versions of Smile documented in https://github.com/apache/incubator-druid/issues/6553

This week's dev sync

https://meet.google.com/ozi-rtfg-ags Cheers! Charles Allen

Dev sync this week

I have a conflict this week. Is anyone else able to start the sync up?

Re: Druid + Theta Sketches performance

, 2018 at 9:17 PM alex.rnv...@gmail.com wrote: > > > On 2018/10/19 14:42:18, Charles Allen > wrote: > > This is a good callout. Those numbers still seem very slow. One item I'm > > curious of is if you are dropping the id when you index, or if the id is > > als

Re: Druid + Theta Sketches performance

ave a very limited scope use case. Now, to Theta Sketches itself, I am not as familiar with the Theta Sketches code paths. It is possible there are performance gains to be had. Hope this helps, Charles Allen On Fri, Oct 19, 2018 at 3:38 AM alex.rnv...@gmail.com wrote: > Hi Druid devs, > I

Dev Sync

https://meet.google.com/ozi-rtfg-ags This week's dev sync Cheers! Charles Allen

Re: TopN on two metrics

":"metric1", > "type":"fieldAccess" > }, > { >"fieldName":"metric2", >"name":"metric2", >"type":"fieldA

Re: TopN on two metrics

For the vast majority of use cases, Yes! For example, lets say you have two metrics "Cost" and "Taxes" as USD cents. When you add the two together you get the total that was charged (or something similar). To get the total that was charged across everything, you simply sum the Cost and sum the

Re: [EXTERNAL] This week's dev sync

great question! They typically occur at 10am pacific time every Tuesday On Tue, Oct 9, 2018 at 10:59 AM Mohammad.J.Khan wrote: > Hi Charles, > > What time do the week's dev sync take place? > > Thanks, > Mohammad > > On 10/9/18, 11:55 AM, "Charles A

This week's dev sync

https://meet.google.com/ozi-rtfg-ags Cheers!

Unique Sketch aggregations and bias correction

, Charles Allen

This week's dev sync

https://meet.google.com/ozi-rtfg-ags Cheers! Charles Allen

This week's dev sync

To join the video meeting, click this link: https://meet.google.com/ozi-rtfg-ags Otherwise, to join by phone, dial +1 442-666-1256 and enter this PIN: 6867# To view more phone numbers, click this link: https://tel.meet/ozi-rtfg-ags?hs=5 Cheers!

Re: Towards 0.13 (Apache release)

I just merged to one of the PRs I'm working on, and after upping the rename limit to pretty high it was relatively painless On Thu, Aug 30, 2018 at 10:24 AM Slim Bouguerra wrote: > Thanks Gian worked for me! > > > On Aug 30, 2018, at 10:06 AM, Gian Merlino wrote: > > > > That PR is merged now!

dev Sync this week

To join the video meeting, click this link: https://meet.google.com/ozi-rtfg-ags Otherwise, to join by phone, dial +1 442-666-1256 and enter this PIN: 6867# To view more phone numbers, click this link: https://tel.meet/ozi-rtfg-ags?hs=5

September LA Meetup!

please stop by for some good tech talks and networking opportunities! Best Regards, Charles Allen

This week's dev sync

https://meet.google.com/ozi-rtfg-ags sorry for the delay

Re: Dev sync this week

zgIOkZCdAAEE. > > Best, > Jihoon > > On Tue, Aug 14, 2018 at 9:51 AM Charles Allen > wrote: > > > I will be late to the dev sync this week and might not be able to make > it, > > can anyone else start it? > > > > Thank you, > > Charles Allen > > >

Re: Druid 0.12.2 release vote

attached, maybe the mailing list ate it? > > On Wed, Aug 8, 2018 at 4:15 PM Charles Allen .invalid> > wrote: > > > Blue is 0.12.2 with some minor backports not perf related. Red is from > the > > 0.11.x series. This is effectively a bucketed PDF of the query times for >

Re: Druid 0.12.2 release vote

internal 0.12.2 release. This is good! Cheers, Charles Allen [image: Screen Shot 2018-08-08 at 4.01.24 PM.png] On Wed, Aug 8, 2018 at 3:11 PM David Lim wrote: > +1, thank you! > > On Wed, Aug 8, 2018 at 3:16 PM Jonathan Wei wrote: > > > +1, thanks Jihoon! > > > >

Re: Druid 0.12.2 release vote

I don't think it should be a blocker to release, but I have to run perf tests for rollouts anyways so I figured I'd publish what I find :-P Cheers, Charles Allen On Wed, Aug 8, 2018 at 12:33 PM Gian Merlino wrote: > That being said, Charles I am definitely looking forward to your rep

Re: Druid 0.12.2 release vote

((don't let this ask block the release)) Is there a way to get a preview of what the release notice will look like? On Mon, Aug 6, 2018 at 3:38 PM Fangjin Yang wrote: > +1 > > On Mon, Aug 6, 2018 at 3:03 PM, Jihoon Son wrote: > > > Hi all, > > > > Druid 0.12.2-rc1

This week's dev sync

https://meet.google.com/ozi-rtfg-ags See you soon!

Dev Sync today

I will likely be late to the dev sync, is anyone else able to start it this week?

Re: Build failure on 0.13.SNAPSHOT

OOME seems to be showing up in some of the Travis testing as well for group by related stuff. Unsure what's going on there. On Tue, Jul 24, 2018 at 9:46 PM Dongjin Lee wrote: > After some experiments, I figured out the following: > > 1. Druid uses above 8gb of memory for testing.

Dev sync

Is someone else able to start the dev sync this week? I'm out of town at a conference.

Re: Multi-threaded Druid Tests/Benchmarks

18, 2018 at 12:35 AM Anastasia Braginsky wrote: > So this is probably where we can help with the Oak-based incremental > index.Can you please give me any reference to those tests? Any descriptions? > Thanks! > > On Tuesday, July 17, 2018, 8:59:57 PM GMT+3, Charles Allen

Re: Multi-threaded Druid Tests/Benchmarks

Unfortunately I think multi-threaded test coverage is kind of weak and historically very hart to test. There are some topN benchmarks but they are very limited as they don't scale well (heap gets blasted from incremental index) with a large concurrency level. On Sun, Jul 15, 2018 at 6:35 AM

TopN folding and result ordering (and maybe group by)

inalize flag of a query to determine the sort order, or adding a new query context to determine if the sort order should be specification order (default) or lexicographic order (internal override) ? Thanks, Charles Allen

Re: This week's dev sync

Some on the call mentioned there were some oddities with logins this week. Using incognito mode worked to fix the login issues On Tue, Jul 17, 2018 at 9:50 AM Charles Allen wrote: > To join the video meeting, click this link: > https://meet.google.com/ozi-rtfg-ags > Otherwise, to join

This week's dev sync

To join the video meeting, click this link: https://meet.google.com/ozi-rtfg-ags Otherwise, to join by phone, dial +1 442-666-1256 and enter this PIN: 6867# To view more phone numbers, click this link: https://tel.meet/ozi-rtfg-ags?hs=5

Java script engine to be removed

http://openjdk.java.net/jeps/335 https://bugs.openjdk.java.net/browse/JDK-8202786 The javascript Nashorn engine is deprecated and slated to be removed in the next long term support release of Java. https://github.com/apache/incubator-druid/issues/5589 is the ticket for maintaining future java

Re: Druid 0.12.2-rc1 vote

Brought this up in the dev sync: I saw a lot of PRs and fixes for Coordinator segment balancing related to some regressions that happened in 0.12.x . Is anyone able to give a rundown of the state of coordinator segment management for the 0.12.2 RC? On Tue, Jul 10, 2018 at 10:26 AM Nishant

This week's dev sync

https://meet.google.com/ozi-rtfg-ags

Sync this week

I am indisposed this week. Can anyone else start the sync up?

This week's dev sync

To join the video meeting, click this link: https://meet.google.com/ozi-rtfg-ags Otherwise, to join by phone, dial +1 442-666-1256 and enter this PIN: 6867# To view more phone numbers, click this link: https://tel.meet/ozi-rtfg-ags?hs=5

Re: Update log4j version to >= 2.8

This would actually be helpful as more recent log4j json formats have better and more controllable options. This helps when druid stdout/stderr logs get pumped directly into a rich logging system (like stackdriver or splunk or sumologic). But there are a few changes in tests that need to be fixed

New Druid Meetup group for LA / Venice / Santa Monica!

in the LA area please feel free to join! Regards, Charles Allen

Committers

or collaborator structure in GitHub? Thank you, Charles Allen

Dev sync this week:

https://meet.google.com/ozi-rtfg-ags Cheers! Charles Allen

Re: Apache project name for Druid

http://tsdr.uspto.gov/#caseNumber=85681881=SERIAL_NO=statusSearch just for note, this is not expected to be a concern. +1 On Mon, Jun 4, 2018 at 3:40 PM Jihoon Son wrote: > +1 > > On Mon, Jun 4, 2018 at 2:54 PM Xavier Léauté wrote: > > > +1 > > > > On Mon, Jun 4, 2018 at 2:49 PM Maxime

Re: Apache Incubation task items

Thank you Atul! The items are open to anyone who has the capacity to help with them. There are a few items like Intellectual Property assignments that are hard for the general community to assist with, but there are many others which would be great. Cheers, Charles Allen On Thu, May 31, 2018

Apache Incubation task items

not take the relative order or issue number as any sort of indicator. Thank you all for your assistance as we go along this exciting path! Cheers, Charles Allen

Re: Access to jira

May 31, 2018 at 9:15 AM, Charles Allen wrote: > > > Hi all > > > > http://incubator.apache.org/projects/druid.html says that > > https://issues.apache.org/jira/browse/DRUID is our issue tracker, but I > > don't seem to have access to it. Does anyone know how to

Access to jira

Hi all http://incubator.apache.org/projects/druid.html says that https://issues.apache.org/jira/browse/DRUID is our issue tracker, but I don't seem to have access to it. Does anyone know how to apply for access using an existing Apache JIRA login? Thanks, Charles Allen

Re: Druid 0.12.1 release vote

+1 On Tue, May 29, 2018 at 11:04 AM Himanshu wrote: > +1 > > On Tue, May 29, 2018 at 11:02 AM, Jonathan Wei wrote: > > > As discussed on the sync up call this morning, let's vote on the 0.12.1 > > release. > > > > Thanks, > > Jon > > > >

This week's dev sync

https://meet.google.com/ozi-rtfg-ags

Sync up this week

https://meet.google.com/ozi-rtfg-ags

This week's dev sync

https://meet.google.com/ozi-rtfg-ags

Sync-up today

Is some one else able to start the sync-up today? I will be unable to make it again. Thank you, Charles Allen

This week's sync up

To join the video meeting, click this link: https://meet.google.com/ozi-rtfg-ags Otherwise, to join by phone, dial +1 442-666-1256 and enter this PIN: 6867# To view more phone numbers, click this link: https://tel.meet/ozi-rtfg-ags?hs=5

This week's Dev Sync

To join the video meeting, click this link: https://meet.google.com/ozi-rtfg-ags Otherwise, to join by phone, dial +1 442-666-1256 and enter this PIN: 6867# To view more phone numbers, click this link: https://tel.meet/ozi-rtfg-ags?hs=5

RE: [druid-dev] any reason to still keep overlord as separate node?

I am one of those odd cases and agree with Gian on both accounts. -Original Message- From: Gian Merlino Sent: Saturday, March 31, 2018 11:15 AM To: druid-developm...@googlegroups.com; dev@druid.apache.org Subject: Re: [druid-dev] any reason to still keep overlord as

RE: Roll call?

Its been called out in the weekly sync up and sent to the mailing lists. I think it is safe to move things here in the sense that there’s not much else we can do for anyone who may have not moved over yet. Cheers, Charles Allen From: Julian Hyde <