[GitHub] [druid] skyrocknroll commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
skyrocknroll commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598556893 > is it because you have some data reprocessing and you want to re-process only subpart of the data and the only way know /

[GitHub] [druid] skyrocknroll removed a comment on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
skyrocknroll removed a comment on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598556604 > is it because you have some data reprocessing and you want to re-process only subpart of the data and the only

[GitHub] [druid] skyrocknroll commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
skyrocknroll commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598556604 > is it because you have some data reprocessing and you want to re-process only subpart of the data and the only way know /

[GitHub] [druid] jihoonson commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
jihoonson commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598498946 @JulianJaffePinterest thanks for more details. > The output of an intra-day run needs to overshadow any existing output

[GitHub] [druid] himanshug commented on issue #9511: Unable to debug tests from the IDE

2020-03-12 Thread GitBox
himanshug commented on issue #9511: Unable to debug tests from the IDE URL: https://github.com/apache/druid/issues/9511#issuecomment-598497335 Just tried from Intellij, worked for me. ``` ~/work/druid[(HEAD detached at apache/master)]$ java -version java version "1.8.0_171"

[GitHub] [druid] himanshug commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
himanshug commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598491420 Yeah, I agree with your concerns about 100+ dataSource scale in union. My guess is that it could still be handled by pushing

[GitHub] [druid] abhishekrb19 opened a new issue #9511: Unable to debug tests from the IDE

2020-03-12 Thread GitBox
abhishekrb19 opened a new issue #9511: Unable to debug tests from the IDE URL: https://github.com/apache/druid/issues/9511 Hi, I am trying to debug the `CompactionTest`, but unfortunately I am unable to step through the code with breakpoints setup from my IDE (Intellij). Note: I am

[GitHub] [druid] JulianJaffePinterest commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
JulianJaffePinterest commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598465621 Let me explain our use case, which will hopefully clarify some of the questions here: We have multiple

[druid] branch master updated (ddc6f87 -> 6b02991)

2020-03-12 Thread ccaominh
This is an automated email from the ASF dual-hosted git repository. ccaominh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from ddc6f87 Web console: standardize the spec format (#9477) add 6b02991 Match GREATEST/LEAST function behavior

[GitHub] [druid] ccaominh merged pull request #9488: Match GREATEST/LEAST function behavior to other DBs

2020-03-12 Thread GitBox
ccaominh merged pull request #9488: Match GREATEST/LEAST function behavior to other DBs URL: https://github.com/apache/druid/pull/9488 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [druid] clintropolis commented on a change in pull request #9493: threshold based automatic query prioritization

2020-03-12 Thread GitBox
clintropolis commented on a change in pull request #9493: threshold based automatic query prioritization URL: https://github.com/apache/druid/pull/9493#discussion_r391928090 ## File path: docs/configuration/index.md ## @@ -1473,22 +1473,41 @@ These Broker configurations

[GitHub] [druid] jihoonson commented on a change in pull request #9492: add manual laning strategy, integration test

2020-03-12 Thread GitBox
jihoonson commented on a change in pull request #9492: add manual laning strategy, integration test URL: https://github.com/apache/druid/pull/9492#discussion_r391885566 ## File path: docs/configuration/index.md ## @@ -1505,7 +1505,16 @@ This strategy can be enabled by

[GitHub] [druid] jihoonson commented on a change in pull request #9492: add manual laning strategy, integration test

2020-03-12 Thread GitBox
jihoonson commented on a change in pull request #9492: add manual laning strategy, integration test URL: https://github.com/apache/druid/pull/9492#discussion_r391917386 ## File path: integration-tests/src/test/java/org/apache/druid/tests/query/ITWikipediaQueryTest.java ##

[GitHub] [druid] jihoonson commented on a change in pull request #9492: add manual laning strategy, integration test

2020-03-12 Thread GitBox
jihoonson commented on a change in pull request #9492: add manual laning strategy, integration test URL: https://github.com/apache/druid/pull/9492#discussion_r391919871 ## File path: server/src/main/java/org/apache/druid/server/scheduling/ManualQueryLaningStrategy.java ##

[GitHub] [druid] vogievetsky commented on issue #9477: Web console: standardize the spec format

2020-03-12 Thread GitBox
vogievetsky commented on issue #9477: Web console: standardize the spec format URL: https://github.com/apache/druid/pull/9477#issuecomment-598423762 Sweet! Thank you! This is an automated message from the Apache Git Service.

[GitHub] [druid] vogievetsky merged pull request #9477: Web console: standardize the spec format

2020-03-12 Thread GitBox
vogievetsky merged pull request #9477: Web console: standardize the spec format URL: https://github.com/apache/druid/pull/9477 This is an automated message from the Apache Git Service. To respond to the message, please log

[druid] branch master updated (1ba1a3c -> ddc6f87)

2020-03-12 Thread vogievetsky
This is an automated email from the ASF dual-hosted git repository. vogievetsky pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 1ba1a3c fix worker category on Indexer node (#9510) add ddc6f87 Web console: standardize the spec format

[GitHub] [druid] vogievetsky closed issue #9144: Missing "spec" in the batch ingestion spec created by the web console

2020-03-12 Thread GitBox
vogievetsky closed issue #9144: Missing "spec" in the batch ingestion spec created by the web console URL: https://github.com/apache/druid/issues/9144 This is an automated message from the Apache Git Service. To respond to

[GitHub] [druid] vogievetsky commented on issue #9504: is there any way to get version 0.10.1 documents ?

2020-03-12 Thread GitBox
vogievetsky commented on issue #9504: is there any way to get version 0.10.1 documents ? URL: https://github.com/apache/druid/issues/9504#issuecomment-598422915 Yay a simple question! ❤️ This is an automated message from

[GitHub] [druid] vogievetsky closed issue #9504: is there any way to get version 0.10.1 documents ?

2020-03-12 Thread GitBox
vogievetsky closed issue #9504: is there any way to get version 0.10.1 documents ? URL: https://github.com/apache/druid/issues/9504 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [druid] glasser commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
glasser commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598420758 @himanshug Yeah, I don't know much about union queries and that might be a better way to go.

[druid] branch master updated (ff59d2e -> 1ba1a3c)

2020-03-12 Thread himanshug
This is an automated email from the ASF dual-hosted git repository. himanshug pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from ff59d2e Move RowSignature from druid-sql to druid-processing and make use of it. (#9508) add 1ba1a3c fix

[GitHub] [druid] himanshug merged pull request #9510: fix worker category on Indexer node

2020-03-12 Thread GitBox
himanshug merged pull request #9510: fix worker category on Indexer node URL: https://github.com/apache/druid/pull/9510 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [druid] himanshug closed issue #9451: druid indexer ignores druid.worker.category

2020-03-12 Thread GitBox
himanshug closed issue #9451: druid indexer ignores druid.worker.category URL: https://github.com/apache/druid/issues/9451 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [druid] himanshug commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
himanshug commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598417933 @JulianJaffePinterest thanks for explaining the difficulties. However considering the use case at hand and the alternatives

[GitHub] [druid] himanshug edited a comment on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process

2020-03-12 Thread GitBox
himanshug edited a comment on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process URL: https://github.com/apache/druid/issues/9496#issuecomment-598355639 you can't point to service, druid(actually zookeeper client) needs to have all of

[GitHub] [druid] ceastman-ibm commented on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process

2020-03-12 Thread GitBox
ceastman-ibm commented on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process URL: https://github.com/apache/druid/issues/9496#issuecomment-598414828 @himanshug thank you for the information. i will give these things a try.

[GitHub] [druid] b-slim commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
b-slim commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598385728 I have read the proposal and my understanding is that you are trying to index data form the same source with the same interval of

[GitHub] [druid] glasser commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
glasser commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598380163 It seems like an interesting application of namespaces is that it allows you to do reingestion and retention policies on a subset

[GitHub] [druid] b-slim commented on issue #9488: Match GREATEST/LEAST function behavior to other DBs

2020-03-12 Thread GitBox
b-slim commented on issue #9488: Match GREATEST/LEAST function behavior to other DBs URL: https://github.com/apache/druid/pull/9488#issuecomment-598376839 +1 This is an automated message from the Apache Git Service. To

[GitHub] [druid] jihoonson commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
jihoonson commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598371248 I'm also curious exactly what namespaces are for. AFAIK, Druid currently allows users to ingest data into the same datasource

[GitHub] [druid] ccaominh commented on issue #9488: Match GREATEST/LEAST function behavior to other DBs

2020-03-12 Thread GitBox
ccaominh commented on issue #9488: Match GREATEST/LEAST function behavior to other DBs URL: https://github.com/apache/druid/pull/9488#issuecomment-598370482 @b-slim Let me know if you have any comments. (I'd like to get this in before the 0.18.0 branch cut in a few days.)

[GitHub] [druid] jihoonson commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
jihoonson commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598368248 @JulianJaffePinterest thanks for the proposal and a nice blog post. The problems of the union query you mentioned above are

[GitHub] [druid] himanshug edited a comment on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process

2020-03-12 Thread GitBox
himanshug edited a comment on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process URL: https://github.com/apache/druid/issues/9496#issuecomment-598355639 you can't point to service, druid(actually zookeeper client) needs to have all of

[GitHub] [druid] himanshug edited a comment on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process

2020-03-12 Thread GitBox
himanshug edited a comment on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process URL: https://github.com/apache/druid/issues/9496#issuecomment-598355639 you can't point to service, druid(actually zookeeper client) needs to have all of

[GitHub] [druid] himanshug commented on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process

2020-03-12 Thread GitBox
himanshug commented on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process URL: https://github.com/apache/druid/issues/9496#issuecomment-598355639 you can't point to service, druid(actually zookeeper client) needs to have all of zookeeper

[GitHub] [druid] ccaominh commented on a change in pull request #9488: Match GREATEST/LEAST function behavior to other DBs

2020-03-12 Thread GitBox
ccaominh commented on a change in pull request #9488: Match GREATEST/LEAST function behavior to other DBs URL: https://github.com/apache/druid/pull/9488#discussion_r391809838 ## File path: core/src/main/java/org/apache/druid/math/expr/Function.java ## @@ -976,6 +981,163

[GitHub] [druid] JulianJaffePinterest commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
JulianJaffePinterest commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598346730 We have both differing dimension and differing metrics between the namespaces. I investigated modifying union

[GitHub] [druid] ceastman-ibm commented on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process

2020-03-12 Thread GitBox
ceastman-ibm commented on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process URL: https://github.com/apache/druid/issues/9496#issuecomment-598343132 @himanshug currently i have pointing to a kubernetes service front ending a number of

[GitHub] [druid] fjy merged pull request #9508: Move RowSignature from druid-sql to druid-processing and make use of it.

2020-03-12 Thread GitBox
fjy merged pull request #9508: Move RowSignature from druid-sql to druid-processing and make use of it. URL: https://github.com/apache/druid/pull/9508 This is an automated message from the Apache Git Service. To respond to

[druid] branch master updated (3082b92 -> ff59d2e)

2020-03-12 Thread fjy
This is an automated email from the ASF dual-hosted git repository. fjy pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 3082b92 Fix NPE when using IndexedTable and all left rows are filtered out (#9490) add ff59d2e Move RowSignature

[GitHub] [druid] himanshug commented on issue #9463: Add namespaces to Druid segments within a data source

2020-03-12 Thread GitBox
himanshug commented on issue #9463: Add namespaces to Druid segments within a data source URL: https://github.com/apache/druid/issues/9463#issuecomment-598335522 I read your email on dev list and consequently

[GitHub] [druid] himanshug commented on a change in pull request #9493: threshold based automatic query prioritization

2020-03-12 Thread GitBox
himanshug commented on a change in pull request #9493: threshold based automatic query prioritization URL: https://github.com/apache/druid/pull/9493#discussion_r391786815 ## File path: docs/configuration/index.md ## @@ -1473,22 +1473,41 @@ These Broker configurations can

[GitHub] [druid] himanshug commented on a change in pull request #9493: threshold based automatic query prioritization

2020-03-12 Thread GitBox
himanshug commented on a change in pull request #9493: threshold based automatic query prioritization URL: https://github.com/apache/druid/pull/9493#discussion_r391785447 ## File path: docs/configuration/index.md ## @@ -1473,22 +1473,41 @@ These Broker configurations can

[GitHub] [druid] himanshug commented on issue #9451: druid indexer ignores druid.worker.category

2020-03-12 Thread GitBox
himanshug commented on issue #9451: druid indexer ignores druid.worker.category URL: https://github.com/apache/druid/issues/9451#issuecomment-598312195 https://github.com/apache/druid/pull/9510 should fix this This is an

[GitHub] [druid] himanshug opened a new pull request #9510: fix worker category on Indexer node

2020-03-12 Thread GitBox
himanshug opened a new pull request #9510: fix worker category on Indexer node URL: https://github.com/apache/druid/pull/9510 Fixes #9451 ### Description Fixes worker announcement in CliIndexer to publish configured worker category. This PR has: - [x] been

[GitHub] [druid] himanshug commented on issue #9451: druid indexer ignores druid.worker.category

2020-03-12 Thread GitBox
himanshug commented on issue #9451: druid indexer ignores druid.worker.category URL: https://github.com/apache/druid/issues/9451#issuecomment-598311058 I think this is a trivial typo kind of problem. This is an automated

[GitHub] [druid] himanshug commented on issue #9457: Make delete from deep storage multi-threaded

2020-03-12 Thread GitBox
himanshug commented on issue #9457: Make delete from deep storage multi-threaded URL: https://github.com/apache/druid/issues/9457#issuecomment-598308010 another avenue to explore would be to batch the delete calls from FS if Hdfs supports batch deletion. That could potentially be much

[GitHub] [druid] himanshug commented on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process

2020-03-12 Thread GitBox
himanshug commented on issue #9496: druid management ui in kubernetes fails if more than one coordinator/overlord process URL: https://github.com/apache/druid/issues/9496#issuecomment-59829 a wild guess: that sounds like something up with zookeeper. you need make sure all zookeeper

[GitHub] [druid] himanshug commented on issue #9506: Official Docker Image Error Response

2020-03-12 Thread GitBox
himanshug commented on issue #9506: Official Docker Image Error Response URL: https://github.com/apache/druid/issues/9506#issuecomment-598298038 not exactly the fix but you can try to get a specific version, that works.

[GitHub] [druid] himanshug commented on issue #9504: is there any way to get version 0.10.1 documents ?

2020-03-12 Thread GitBox
himanshug commented on issue #9504: is there any way to get version 0.10.1 documents ? URL: https://github.com/apache/druid/issues/9504#issuecomment-598296727 you can still find the doc source at https://github.com/apache/druid/tree/0.10.1/docs

[GitHub] [druid] Elvis0607 edited a comment on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

2020-03-12 Thread GitBox
Elvis0607 edited a comment on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service URL: https://github.com/apache/druid/issues/8715#issuecomment-598228794 > Hi @Elvis0607, just wondering, why are you wanting to customize the

[GitHub] [druid] Elvis0607 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

2020-03-12 Thread GitBox
Elvis0607 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service URL: https://github.com/apache/druid/issues/8715#issuecomment-598228794 > Hi @Elvis0607, just wondering, why are you wanting to customize the

[GitHub] [druid] codecov-io commented on issue #9509: add kinesis lag metric

2020-03-12 Thread GitBox
codecov-io commented on issue #9509: add kinesis lag metric URL: https://github.com/apache/druid/pull/9509#issuecomment-598183215 # [Codecov](https://codecov.io/gh/apache/druid/pull/9509?src=pr=h1) Report > Merging [#9509](https://codecov.io/gh/apache/druid/pull/9509?src=pr=desc) into

[GitHub] [druid] clintropolis opened a new pull request #9509: add kinesis lag metric

2020-03-12 Thread GitBox
clintropolis opened a new pull request #9509: add kinesis lag metric URL: https://github.com/apache/druid/pull/9509 Fixes #6739. ### Description This PR adds an additional set of `SeekableStreamSupervisor` lag metrics, based on time in millis behind the latest offsets of the

[GitHub] [druid] gianm opened a new pull request #9508: Move RowSignature from druid-sql to druid-processing and make use of it.

2020-03-12 Thread GitBox
gianm opened a new pull request #9508: Move RowSignature from druid-sql to druid-processing and make use of it. URL: https://github.com/apache/druid/pull/9508 I split this out from another patch I'm working on that allows the broker to inline subqueries (see

[GitHub] [druid] yangliang9004 commented on issue #8663: Kafka indexing service duplicate entry exception in druid_pendingSegments

2020-03-12 Thread GitBox
yangliang9004 commented on issue #8663: Kafka indexing service duplicate entry exception in druid_pendingSegments URL: https://github.com/apache/druid/issues/8663#issuecomment-598030104 I also meet the problem, it may be a bug.

[GitHub] [druid] himanshug opened a new pull request #9507: optionally disable all of hardcoded zookeeper use

2020-03-12 Thread GitBox
himanshug opened a new pull request #9507: optionally disable all of hardcoded zookeeper use URL: https://github.com/apache/druid/pull/9507 Paving Path Towards #9053 ### Description This patch adds a new configuration property `druid.zk.service.enabled=true/false, default =