[GitHub] [druid] himanshug commented on issue #8992: druid extension for OpenID Connect auth using pac4j lib
himanshug commented on issue #8992: druid extension for OpenID Connect auth using pac4j lib URL: https://github.com/apache/druid/pull/8992#issuecomment-609592760 @soumyajose0784 that sounds like that the ssl cert returned from server configured in `druid.auth.pac4j.oidc.discoveryURI` has either expired or is "self signed" or something else is wrong with it. if you visited that url on browser, does that successfully validate the cert ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] himanshug commented on a change in pull request #9573: Fix some flaws of KafkaEmitter
himanshug commented on a change in pull request #9573: Fix some flaws of
KafkaEmitter
URL: https://github.com/apache/druid/pull/9573#discussion_r403854106
##
File path:
extensions-contrib/kafka-emitter/src/main/java/org/apache/druid/emitter/kafka/KafkaEmitter.java
##
@@ -130,21 +120,21 @@ public void start()
private void sendMetricToKafka()
{
-sendToKafka(config.getMetricTopic(), metricQueue);
+sendToKafka(config.getMetricTopic(), metricQueue,
setProducerCallback(metricLost));
}
private void sendAlertToKafka()
{
-sendToKafka(config.getAlertTopic(), alertQueue);
+sendToKafka(config.getAlertTopic(), alertQueue,
setProducerCallback(alertLost));
}
- private void sendToKafka(final String topic,
MemoryBoundLinkedBlockingQueue recordQueue)
+ private void sendToKafka(final String topic,
MemoryBoundLinkedBlockingQueue recordQueue, Callback callback)
{
ObjectContainer objectToSend;
try {
while (true) {
objectToSend = recordQueue.take();
-producer.send(new ProducerRecord<>(topic, objectToSend.getData()),
producerCallback);
+producer.send(new ProducerRecord<>(topic, objectToSend.getData()),
callback);
}
}
catch (InterruptedException e) {
Review comment:
```suggestion
catch (Throwable e) {
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] soumyajose0784 commented on issue #8992: druid extension for OpenID Connect auth using pac4j lib
soumyajose0784 commented on issue #8992: druid extension for OpenID Connect auth using pac4j lib URL: https://github.com/apache/druid/pull/8992#issuecomment-609586100 We need the ssl property for common.runtime.properties, to enable connection to ssl enabled discovery URI. We get following error javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at org.pac4j.oidc.config.OidcConfiguration.internalInit(OidcConfiguration.java:147) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch master updated: Eliminate common subfilters when converting it to a CNF (#9608)
This is an automated email from the ASF dual-hosted git repository.
himanshug pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 40e84a1 Eliminate common subfilters when converting it to a CNF
(#9608)
40e84a1 is described below
commit 40e84a171b8be8e87db20b0c5c189aef1b860f41
Author: Jihoon Son
AuthorDate: Sun Apr 5 22:29:41 2020 -0700
Eliminate common subfilters when converting it to a CNF (#9608)
---
.../druid/benchmark/FilterPartitionBenchmark.java | 4 +-
.../apache/druid/query/filter/BooleanFilter.java | 5 +-
.../apache/druid/query/filter/DimFilterUtils.java | 4 +-
.../apache/druid/query/filter/TrueDimFilter.java | 8 +-
.../segment/QueryableIndexStorageAdapter.java | 9 +-
.../org/apache/druid/segment/filter/AndFilter.java | 33 ++--
.../org/apache/druid/segment/filter/Filters.java | 56 +++---
.../org/apache/druid/segment/filter/NotFilter.java | 32 +++-
.../org/apache/druid/segment/filter/OrFilter.java | 28 ++-
.../apache/druid/segment/filter/TrueFilter.java| 9 +-
.../segment/join/filter/JoinFilterAnalyzer.java| 8 +-
.../druid/query/filter/AndDimFilterTest.java | 29 ++-
...dDimFilterTest.java => DimFilterTestUtils.java} | 34 ++--
.../apache/druid/query/filter/OrDimFilterTest.java | 53 +
.../druid/segment/filter/BaseFilterTest.java | 2 +-
.../druid/segment/filter/FilterPartitionTest.java | 4 +-
.../filter/FilterTestUtils.java} | 38 ++--
.../apache/druid/segment/filter/FiltersTest.java | 213 -
...tFilterTest.java => NotFilterEvaluateTest.java} | 6 +-
.../apache/druid/segment/filter/NotFilterTest.java | 84 ++--
20 files changed, 473 insertions(+), 186 deletions(-)
diff --git
a/benchmarks/src/test/java/org/apache/druid/benchmark/FilterPartitionBenchmark.java
b/benchmarks/src/test/java/org/apache/druid/benchmark/FilterPartitionBenchmark.java
index b52cde1..547784e 100644
---
a/benchmarks/src/test/java/org/apache/druid/benchmark/FilterPartitionBenchmark.java
+++
b/benchmarks/src/test/java/org/apache/druid/benchmark/FilterPartitionBenchmark.java
@@ -377,7 +377,7 @@ public class FilterPartitionBenchmark
Filter orFilter = new OrFilter(Arrays.asList(filter, filter2));
StorageAdapter sa = new QueryableIndexStorageAdapter(qIndex);
-Sequence cursors = makeCursors(sa, Filters.convertToCNF(orFilter));
+Sequence cursors = makeCursors(sa, Filters.toCNF(orFilter));
readCursors(cursors, blackhole);
}
@@ -451,7 +451,7 @@ public class FilterPartitionBenchmark
);
StorageAdapter sa = new QueryableIndexStorageAdapter(qIndex);
-Sequence cursors = makeCursors(sa,
Filters.convertToCNF(dimFilter3.toFilter()));
+Sequence cursors = makeCursors(sa,
Filters.toCNF(dimFilter3.toFilter()));
readCursors(cursors, blackhole);
}
diff --git
a/processing/src/main/java/org/apache/druid/query/filter/BooleanFilter.java
b/processing/src/main/java/org/apache/druid/query/filter/BooleanFilter.java
index 9bbbdb6..e11153e 100644
--- a/processing/src/main/java/org/apache/druid/query/filter/BooleanFilter.java
+++ b/processing/src/main/java/org/apache/druid/query/filter/BooleanFilter.java
@@ -23,12 +23,13 @@ import org.apache.druid.segment.ColumnSelector;
import org.apache.druid.segment.ColumnSelectorFactory;
import java.util.HashSet;
-import java.util.List;
import java.util.Set;
public interface BooleanFilter extends Filter
{
- List getFilters();
+ ValueMatcher[] EMPTY_VALUE_MATCHER_ARRAY = new ValueMatcher[0];
+
+ Set getFilters();
/**
* Get a ValueMatcher that applies this filter to row values.
diff --git
a/processing/src/main/java/org/apache/druid/query/filter/DimFilterUtils.java
b/processing/src/main/java/org/apache/druid/query/filter/DimFilterUtils.java
index 00a84fc..c980adb 100644
--- a/processing/src/main/java/org/apache/druid/query/filter/DimFilterUtils.java
+++ b/processing/src/main/java/org/apache/druid/query/filter/DimFilterUtils.java
@@ -52,9 +52,9 @@ public class DimFilterUtils
static final byte COLUMN_COMPARISON_CACHE_ID = 0xD;
static final byte EXPRESSION_CACHE_ID = 0xE;
static final byte TRUE_CACHE_ID = 0xF;
- public static byte BLOOM_DIM_FILTER_CACHE_ID = 0x10;
- public static final byte STRING_SEPARATOR = (byte) 0xFF;
+ public static final byte BLOOM_DIM_FILTER_CACHE_ID = 0x10;
+ public static final byte STRING_SEPARATOR = (byte) 0xFF;
static byte[] computeCacheKey(byte cacheIdKey, List filters)
{
diff --git
a/processing/src/main/java/org/apache/druid/query/filter/TrueDimFilter.java
b/processing/src/main/java/org/apache/druid/query/filter/TrueDimFilter.java
index d10e6d9..2254358 100644
--- a/processing/src/main/java/org/apache/druid/query/filter/TrueDimFilter.java
+++ b/processing/src/main/java/org/apache/druid/query/filter/TrueDimFilter.java
@@ -20,9 +20,9 @@
[GitHub] [druid] himanshug merged pull request #9608: Eliminate common subfilters when converting it to a CNF
himanshug merged pull request #9608: Eliminate common subfilters when converting it to a CNF URL: https://github.com/apache/druid/pull/9608 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] himanshug commented on a change in pull request #9518: AWS RDS token based password provider
himanshug commented on a change in pull request #9518: AWS RDS token based password provider URL: https://github.com/apache/druid/pull/9518#discussion_r403831641 ## File path: extensions-core/druid-aws-rds-extensions/pom.xml ## @@ -0,0 +1,80 @@ + + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + 4.0.0 + + org.apache.druid.extensions + druid-aws-rds-extensions + druid-aws-rds-extensions + druid-aws-rds-extensions + + +org.apache.druid +druid +0.18.0-SNAPSHOT Review comment: updated This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] abhishekrb19 opened a new pull request #9622: Compaction numeric nullhandle
abhishekrb19 opened a new pull request #9622: Compaction numeric nullhandle URL: https://github.com/apache/druid/pull/9622 With `druid.generic.useDefaultValueForNull=false ` set and post compaction, we notice the dimensions with numeric types the original `null` value gets replaced with the default `0` value. However, dimensions of string type work as expected. This would surprise users/systems with null compliance. This issue was reported here: https://github.com/apache/druid/issues/8221 I tested this change set by hand, but I am happy to add a test case if someone could please point me in the right direction. Looks like `CompactionTaskRunTest.java` and `CompactionTaskParallelRunTest.java` are the test files of interest? CC: @clintropolis @himanshug @gianm, thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] als-sdin closed issue #9386: Kafka Indexing Service failed - could not allocate segment for row with timestamp[*]
als-sdin closed issue #9386: Kafka Indexing Service failed - could not allocate segment for row with timestamp[*] URL: https://github.com/apache/druid/issues/9386 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] als-sdin commented on issue #9386: Kafka Indexing Service failed - could not allocate segment for row with timestamp[*]
als-sdin commented on issue #9386: Kafka Indexing Service failed - could not allocate segment for row with timestamp[*] URL: https://github.com/apache/druid/issues/9386#issuecomment-609506646 There were issues with the segments and/or segment metadata for a particular period. Nothing to do with the ingestion. I compacted the segments for the offending periods - creating new segments. The old segments were removed (kill tasks). Everything seems to be working now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] himanshug merged pull request #9615: [Backport] Fix double count ssl connection metrics
himanshug merged pull request #9615: [Backport] Fix double count ssl connection metrics URL: https://github.com/apache/druid/pull/9615 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[druid] branch 0.18.0 updated: Fix double count ssl connection metrics (#9594) (#9615)
This is an automated email from the ASF dual-hosted git repository.
himanshug pushed a commit to branch 0.18.0
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.18.0 by this push:
new 404a558 Fix double count ssl connection metrics (#9594) (#9615)
404a558 is described below
commit 404a5586e44b18784d48204ff3372e561a36589c
Author: Clint Wylie
AuthorDate: Sun Apr 5 15:01:43 2020 -0700
Fix double count ssl connection metrics (#9594) (#9615)
* fix double counted jetty/numOpenConnections metric for ssl connections
* tests
* more better
* style
---
examples/bin/dsql-main | 2 +-
.../docker/tls/generate-good-client-cert.sh| 2 +-
.../tls/generate-server-certs-and-keystores.sh | 2 +-
.../initialization/jetty/JettyServerModule.java| 15 +-
.../druid/server/initialization/BaseJettyTest.java | 70 +++
.../druid/server/initialization/JettyTest.java | 211 -
server/src/test/resources/server.jks | Bin 0 -> 1911 bytes
server/src/test/resources/truststore.jks | Bin 0 -> 1641 bytes
8 files changed, 293 insertions(+), 9 deletions(-)
diff --git a/examples/bin/dsql-main b/examples/bin/dsql-main
index 8dfe882..cf68581 100755
--- a/examples/bin/dsql-main
+++ b/examples/bin/dsql-main
@@ -400,7 +400,7 @@ def main():
parser_fmt.add_argument('--format', type=str, default='table',
choices=('csv', 'tsv', 'json', 'table'), help='Result format')
parser_fmt.add_argument('--header', action='store_true', help='Include
header row for formats "csv" and "tsv"')
parser_fmt.add_argument('--tsv-delimiter', type=str, default='\t',
help='Delimiter for format "tsv"')
- parser_oth.add_argument('--context-option', '-c', type=str, action='append',
help='Set context option for this connection, see
https://docs.imply.io/on-prem/query-data/sql for options')
+ parser_oth.add_argument('--context-option', '-c', type=str, action='append',
help='Set context option for this connection, see
https://druid.apache.org/docs/latest/querying/sql.html#connection-context for
options')
parser_oth.add_argument('--execute', '-e', type=str, help='Execute single
SQL query')
args = parser.parse_args()
diff --git a/integration-tests/docker/tls/generate-good-client-cert.sh
b/integration-tests/docker/tls/generate-good-client-cert.sh
index 895e6c3..0f16c14 100755
--- a/integration-tests/docker/tls/generate-good-client-cert.sh
+++ b/integration-tests/docker/tls/generate-good-client-cert.sh
@@ -58,5 +58,5 @@ openssl x509 -req -days 3650 -in client.csr -CA root.pem
-CAkey root.key -set_se
openssl pkcs12 -export -in client.pem -inkey client.key -out client.p12 -name
druid -CAfile root.pem -caname druid-it-root -password pass:druid123
keytool -importkeystore -srckeystore client.p12 -srcstoretype PKCS12
-destkeystore client.jks -deststoretype JKS -srcstorepass druid123
-deststorepass druid123
-# Create a Java truststore with the imply test cluster root CA
+# Create a Java truststore with the druid test cluster root CA
keytool -import -alias druid-it-root -keystore truststore.jks -file root.pem
-storepass druid123 -noprompt
diff --git
a/integration-tests/docker/tls/generate-server-certs-and-keystores.sh
b/integration-tests/docker/tls/generate-server-certs-and-keystores.sh
index 8f38be3..e26cdac 100755
--- a/integration-tests/docker/tls/generate-server-certs-and-keystores.sh
+++ b/integration-tests/docker/tls/generate-server-certs-and-keystores.sh
@@ -63,7 +63,7 @@ openssl x509 -req -days 3650 -in server.csr -CA root.pem
-CAkey root.key -set_se
openssl pkcs12 -export -in server.pem -inkey server.key -out server.p12 -name
druid -CAfile root.pem -caname druid-it-root -password pass:druid123
keytool -importkeystore -srckeystore server.p12 -srcstoretype PKCS12
-destkeystore server.jks -deststoretype JKS -srcstorepass druid123
-deststorepass druid123
-# Create a Java truststore with the imply test cluster root CA
+# Create a Java truststore with the druid test cluster root CA
keytool -import -alias druid-it-root -keystore truststore.jks -file root.pem
-storepass druid123 -noprompt
# Revoke one of the client certs
diff --git
a/server/src/main/java/org/apache/druid/server/initialization/jetty/JettyServerModule.java
b/server/src/main/java/org/apache/druid/server/initialization/jetty/JettyServerModule.java
index bbb80e4..fc6f93e 100644
---
a/server/src/main/java/org/apache/druid/server/initialization/jetty/JettyServerModule.java
+++
b/server/src/main/java/org/apache/druid/server/initialization/jetty/JettyServerModule.java
@@ -22,6 +22,7 @@ package org.apache.druid.server.initialization.jetty;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider;
import com.fasterxml.jackson.jaxrs.smile.JacksonSmileProvider;
+import com.google.common.annotations.VisibleForTest
[GitHub] [druid] lgtm-com[bot] commented on issue #9621: Add context to AuthenticationResult [WIP]
lgtm-com[bot] commented on issue #9621: Add context to AuthenticationResult [WIP] URL: https://github.com/apache/druid/pull/9621#issuecomment-609426018 This pull request **introduces 3 alerts** when merging 66ac801e348fe30fbfd703f7c1be9ae7bb8e99e8 into 2d999669333fc7bd344e5bec0d74f0fbcd916272 - [view on LGTM.com](https://lgtm.com/projects/g/apache/druid/rev/pr-bf524bddaea0cdb6f68a02cfab280b289fefd436) **new alerts:** * 2 for Dereferenced variable is always null * 1 for Dereferenced variable may be null This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [druid] bolkedebruin opened a new pull request #9621: Add context to AuthenticationResult [WIP]
bolkedebruin opened a new pull request #9621: Add context to AuthenticationResult [WIP] URL: https://github.com/apache/druid/pull/9621 ### Description Looking for feedback. Proper authorization needs to be context aware. Context like IP Address, Proxy addresses etc. This PR adds context obtained from the servlet request and adds this to the authenticationResult. Note: * At the moment I tried to stay away from adjusting AuthenticationResult as much as possible. Hence, it requires the "authenticator" to call enrichContext. Maybe we should consider doing the enrichment in AuthenticationResult # Key changed/added classes in this PR * AuthenticationContextEnricher cc @jon-wei @Fokko This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
