Re: Docker image

On Thu, Aug 22, 2019 at 10:20 AM Driesprong, Fokko wrote: > Hi all, > > For the development of the Druid Scala client library called Scruid > , we're using a Docker image to run CI > tests. The container that we're using in a fork >

Re: dns lookups cached for kafka?

On Thu, 21 Mar 2019 at 11:48, Charles Allen wrote: > Druid assumes the network layer handles whatever tuning is needed regarding > DNS resolution or IP routing. In general this means making sure you have > your java settings correct (see > > https://docs.aws.amazon.com/sdk-for-java/v1/developer-g

dns lookups cached for kafka?

What is the expectation around dns and druid? Specifically, when overlord started, it resolved (correctly) my kafka cluster nodes. A little bit later I made a change which changed their IP. But overlord continues to use that originally resolved IP. Is there a way to force the refresh? Should it be

historical druid.segmentCache error

One of my historical nodes crashes on startup, yielding this error (below). Can someone suggest how to start debugging this? The 2 lines in the config that refer to segmentCache: historical/runtime.properties:druid.segmentCache.locations=[{"path": "var/druid/segment-cache", "maxSize": 320,

Re: docker build

https://github.com/apache/incubator-druid/pull/7296 is now there which removes the mysql but shows how to re-enable should someone wish. On Tue, 19 Mar 2019 at 07:19, Don Bowman wrote: > So if I send a PR removing the mysql connector (which comes from maven > which is itself apache re

Re: docker build

ers in the apache repo", which would mean > not adding in GPL binaries. Which means switching to postgres. I don't > foresee an aggressive legal stance on this issue, meaning it might take a > while as people watch where the industry is going. > > > > On Tue, Mar 5, 201

Re: [VOTE] Release Apache Druid (incubating) 0.14.0 [RC1]

On Fri, 15 Mar 2019 at 21:26, Jonathan Wei wrote: > Hi all, > > I have created a build for Apache Druid (incubating) 0.14.0, release > candidate 1. > > Thanks to everyone who has contributed to this release! You can read the > proposed release notes here: > https://github.com/apache/incubator-dru

Re: docker build

where do we stand on this? the PR is in and accepted, but i feel we need to have this built as part of the release artifacts and on dockerhub to foster adoption. if the only issue is the mysql connector i can remove it in favour of the postgres connector. On Mon, 18 Feb 2019 at 13:58, Don Bowman

Re: docker build

On Mon, 18 Feb 2019 at 19:14, Gaurav Bhatnagar wrote: > I have been thinking if automated scripts can be provided to end users in > Druid for "Additional Dependencies" for user initiated installation and > configuration of optional dependencies to avoid licensing issues. Later > these scripts can

Re: docker build

istance. > > On Mon, Feb 18, 2019 at 1:59 PM Don Bowman wrote: > > > i can just remove the mysql, the postgres works, i was just assuming > folks > > wanted it. > > > > > > On Mon, 18 Feb 2019 at 16:58, Gian Merlino wrote: > > > > > A di

Re: docker build

asks on our > > end are watching LEGAL-437 for advice (or making it moot by removing the > > MySQL jar), asking Infra to set up automated builds once that is sorted > > out, and building some kind of consensus around how we'll label and > promote > > the Docker images. >

Re: docker build

t; I guess there's two open questions I think we should reach a consensus on > > (others feel free to add more!). > > > > - Are we as a community happy to "support" an additional release > artefact? > > I'm happy to try to incorporate this

Re: json logging enable

ur logging system. The easiest way I found was to > piggyback on the jackson format and just annotate a special class with > whatever fields in whichever format you want: > > > https://github.com/metamx/sumologic-log4j2-appender/blob/master/src/main/java/com/sumologic/log4j/core/

Re: json logging enable

n logging infrastructures. > > If you ever do find it worth your while to go down the route of writing a > custom format for your logging system. The easiest way I found was to > piggyback on the jackson format and just annotate a special class with > whatever fields in whichever format

docker build

Now that https://github.com/apache/incubator-druid/pull/6896 is merged (thank you!) who can get this set to build into Dockerhub? Presumably automatically on a 'tag' of the repo. Once that is done it is much more convenient for folks to use this tool. --don

json logging enable

I am finding that the logging produced is very complex to manage. I would like to add logstash json_event ( https://github.com/logstash/log4j-jsonevent-layout). Any comments before I start? In my environment I'm using fluent-bit, others are using logstash. Its inconvenient to have multi-line log m

Re: script, GPL, container question

On Fri, 25 Jan 2019 at 21:00, Gian Merlino wrote: > I did some looking around and found this: > https://issues.apache.org/jira/browse/INFRA-12781. It seems that the > Apache > Infra folks will set up DockerHub configs for projects like us, in such a > way that they build automatically off release

Re: script, GPL, container question

On Fri, 25 Jan 2019 at 16:07, Gian Merlino wrote: > For Q1 - I have the feeling that if we asked the powers that be at Apache > for an opinion on bundling the MySQL connector that they would not be fans > of the idea -- mostly because it is not allowed for binary tarball > releases, and I don't s

Re: script, GPL, container question

On Fri, 25 Jan 2019 at 13:17, Gian Merlino wrote: > For Q1 the legal guidance as I understand it is that we can provide users > with instructions for how to get optional (L)GPL dependencies, but we can't > distribute them ourselves. Putting the mysql-connector in an Docker image > does feel like

script, GPL, container question

Re: PR https://github.com/apache/incubator-druid/pull/6896 I am trying to make a container that works for folks so that we can get the Kubernetes helm chart off the ground. A question has arisen in the PR around the 'mysql-connector-java' which is GPL. Q1. A script checked into the druid contain

Re: Docker image

On Fri, 18 Jan 2019 at 08:19, Don Bowman wrote: > > So I have a Dockerfile which i think will work well. Before I commit and > create the PR can I ask for some direction? > Would we use this as > > a: the Dockerfile is in the source tree, and does a 'COPY .' into th

Re: Docker image

create-home --uid 1000 --gecos '' --gid 1000 --disabled-password druid \ && mkdir -p /opt/druid/var \ && chown -R druid:druid /opt/druid # Use :debug to get a busybox shell so that scripts can run FROM gcr.io/distroless/java:debug MAINTAINER Don Bowman RUN [&quo

Re: Docker image

e. What would your > expectations be of such an image? > > > On Wed, Jan 16, 2019 at 2:35 PM Don Bowman wrote: > > > Is anyone working on a docker image? I mean, there are quite a few out > > there but they have some various issues, usually security based as they > > i

Docker image

Is anyone working on a docker image? I mean, there are quite a few out there but they have some various issues, usually security based as they inherit from non-too-strong bases. I have done one w/ gcr.io/distroless/java as the parent, and it seems working, but not sure if there is a reason or stra