Re: License headers inside Javadoc comments

2024-01-27 Thread Ted Dunning
The right way to get a copyright on every page is to tweak the javadoc command to use a different template (I would think). On Fri, Jan 26, 2024 at 12:00 AM Paul Rogers wrote: > Hi James, > > For some reason, Drill started with the license headers in Javadoc > comments. The (weak) explanation

Re: A deadline for Drill + Daffodil Integration - ApacheCon in Oct.

2023-07-06 Thread Ted Dunning
That's a cool abstract. On Thu, Jul 6, 2023 at 8:29 AM Mike Beckerle wrote: > I decided the only way to force getting this Drill + Daffodil integration > done, or at least started, is to have a deadline. > > So I submitted this abstract below for the upcoming "Community over Code" > (formerly

Re: [DISCUSS] Add schema support for the XML format

2022-04-06 Thread Ted Dunning
XML will never die. The Cobol programmers were reincarnated and built similarly long-lasting generators of XML. If you have a schema, then it is a reasonable format for Drill to parse, if only to turn around and write to another format. On Wed, Apr 6, 2022 at 7:31 PM Paul Rogers wrote: > Hi

Re: [DISCUSS] Add schema support for the XML format

2022-04-06 Thread Ted Dunning
saved as an array even if there is just one occurrence of > in your data. > > -Original Message- > From: Ted Dunning > Sent: Wednesday, April 6, 2022 11:48 AM > To: dev > Cc: u...@drill.apache.org > Subject: Re: [DISCUSS] Add schema support for the XML format > > Ex

Re: [DISCUSS] Add schema support for the XML format

2022-04-06 Thread Ted Dunning
That example: dog > cat can also convert to ["pet":"dog", "pet":"dog'] XML is rife with problems like this. As you say. But worse than can be imagined unless you have been hit by these problems. On Wed, Apr 6, 2022 at 11:39 AM Lee, David wrote: > TO_JSON won't work in cases where.. > >

Re: [VOTE] Adopt the Drill Test Framework from MapR

2022-03-17 Thread Ted Dunning
Big +1 from me!!! We at MapR built that test framework to support aggressive testing of Drill. It would be great to see it continue to meet that need. (... I was CTO at MapR when it was acquired by HPE but I don't have any special position in the Drill community ...) On Thu, Mar 17, 2022 at

thinking of our Ukrainian friends

2022-02-23 Thread Ted Dunning
For commercial historical reasons many of the people who have contributed to Drill live in Ukraine. My heart is with them tonight. I hope they stay safe.

Re: [DISCUSS] Some ideas for Drill 1.21

2022-02-09 Thread Ted Dunning
> > Then, time the plugins of interest. Figure out how to optimize those. > > My guess is that the bottleneck won't turn out to be what we think it is. > It usually isn't. > > - Paul > > On Tue, Feb 8, 2022 at 8:19 AM Ted Dunning wrote: > > >

Re: [DISCUSS] Some ideas for Drill 1.21

2022-02-08 Thread Ted Dunning
ast, and only > after careful consideration. > > [1] https://infra.apache.org/mailing-list-moderation.html > [2] https://www.apache.org/foundation/how-it-works.html#management > [3] https://github.com/apache/drill/pull/2388 > > On 2022/02/07 21:05, Ted Dunning wrote: > > A

Re: [DISCUSS] Some ideas for Drill 1.21

2022-02-07 Thread Ted Dunning
Another option is to store metadata as data in a distributed data store. For static resources, that can scale very well. For highly dynamic resources like conventional databases behind JDBC connections, you can generally delegate metadata to that layer. Performance for delegated metadata won't

Re: [DISCUSS] Lombok - friend or foe?

2022-01-22 Thread Ted Dunning
I agree > with > > Ted, Eclipse (and, I assume IntelliJ), is pretty quick at generating > those > > methods. > > > > Since Lomboc has a cost, and is not a huge win, KISS suggests we avoid > > adding extra dependencies unnecessarily. > > > > That's my 2 c

Re: [DISCUSS] Lombok - friend or foe?

2022-01-21 Thread Ted Dunning
A couple of years ago, I had a dev introduce Lombok into some code without me knowing. That let me be a classic naive user. The result was total confusion on my part. Sooo much code was being automagically generated that I couldn't figure out the code and spent a lot of time chasing my tail and

Re: [DISCUSS] Drill 2 and plug-in organisation

2022-01-17 Thread Ted Dunning
ne know of any good > examples we can learn from? > > Thanks, > > - Paul > > > On Mon, Jan 17, 2022 at 9:40 AM Ted Dunning wrote: > >> >> I don't think that Maven is a forced move just because Drill is in Java. >> It may be a good move, but it isn't a for

Re: [DISCUSS] Drill 2 and plug-in organisation

2022-01-17 Thread Ted Dunning
lso move, if we > > want.) Finally, figure out a solution for Ted's suggestion to make it > easy > > to grab new extensions. Drill is distributed, so adding a new plugin has > to > > happen on all nodes, which is a bit more complex than the typical > > Julia/Python/R

Re: Re: [DISCUSS] Per User Access Controls

2022-01-13 Thread Ted Dunning
GRANT and REVOKE implicitly assumes that the database is king of access control. That works when the database owns the data. In the modern world where data storage is separated from query, it is truly painful to have to manage permissions for each analysis and each query tool and nearly

Re: [DISCUSS] Drill 2 and plug-in organisation

2022-01-13 Thread Ted Dunning
The bigger reason for a separate plug-in world is the enhancement of community. I would recommend looking at the Julia community for examples of effective ways to drive plug in structure. At the core, for any pure julia package, you can simply add a package by referring to the github repository

Re: [DISCUSS] Restarting the Arrow Conversation

2022-01-04 Thread Ted Dunning
place to put this stuff. A "docs" or "design" directory > within the > source tree? > > Thanks, > > - Paul > > On Tue, Jan 4, 2022 at 11:15 AM Ted Dunning wrote: > > > Exactly. I very much had in mind an "On the other hand" kind of document. >

Re: [DISCUSS] Restarting the Arrow Conversation

2022-01-04 Thread Ted Dunning
w that you said as much already. So a collaboratively > edited document? Wiki pages containing a variety of independent views > might turn out something like this collection I suppose > > https://wiki.c2.com/?GarbageCollection > > which isn't bad IMHO. > > On 2022/01/04 16:

Re: [DISCUSS] Restarting the Arrow Conversation

2022-01-04 Thread Ted Dunning
; > https://github.com/apache/drill/issues/2421 > > James > > [1] https://github.com/apache/drill/projects/1 > > On 2022/01/04 09:49, Ted Dunning wrote: > > I wonder if there isn't a better place for this discussion? > > > > As you point out, there are many thread

Re: [DISCUSS] Restarting the Arrow Conversation

2022-01-03 Thread Ted Dunning
t; > row-based format. > > > > > > The upshot is that "value vectors vs. Arrow" is the wrong place to > start > > > the discussion. The right place is "what does our many years of > > experience > > > with Drill suggest is the most efficient format for

Re: [DISCUSS] Restarting the Arrow Conversation

2022-01-03 Thread Ted Dunning
Christian, Your thoughts are very helpful. I find Arrow very nice (I use it in Agstack with Julia and Python). I don't think anybody is saying that Drill wouldn't be well set with a switch to Arrow or even just interfaces to Arrow. But it is a lot of work to make it all happen. On Mon, Jan 3,

Re: [DISCUSS] Restarting the Arrow Conversation

2022-01-03 Thread Ted Dunning
As a little bit of perspective from somebody who *was* at MapR at the time, here are my recollections. Arrow is pretty much the value vectors from Drill with some lessons learned and all dependencies removed so that Arrow can be consumed separately from Drill. The spinout of the Dremio team

Re: Drill Wiki Access

2021-11-24 Thread Ted Dunning
The normal open source action for preventing problems like this is to leave permissions open and monitor all changes. Then respond to problems as they (rarely) arise. On Wed, Nov 24, 2021 at 8:33 AM Rumar, Maksym wrote: > "Any destructive action could be reverted because the wiki is a >

Re: [DISCUSS] Refactoring Drill's CSV (Text) Reader

2021-11-17 Thread Ted Dunning
I think that these would be significant improvements. The current behavior is pretty painful on average. Better defaults and just a bit of deduction could pay off big. I even think that the presence of headers might be pretty reliably inferred. On Wed, Nov 17, 2021 at 4:31 PM Charles Givre

Re: webUI Bug

2021-11-05 Thread Ted Dunning
Nathan, I think you pasted some images into your message. They didn't make it through to the mailing list. Can you select text instead of images? Or push the images to a public place like imgur? On Thu, Nov 4, 2021 at 11:31 PM Caballero, Nathan (Volpe) wrote: > Hello, > > > > I continue to

Re: Parquet compression codecs

2021-09-29 Thread Ted Dunning
A blog is a great idea. I am curious about how much compression costs. On Wed, Sep 29, 2021 at 5:37 AM luoc wrote: > > James, you are doing fine. > Is it possible to post a new blog in the website for this? > > > 在 2021年9月29日,20:27,James Turton 写道: > > > > Hi all > > > > We've got support

Re: New Docker images published automatically

2021-09-20 Thread Ted Dunning
This is great news. Makes me think that these might be the best way to try Drill out as well, especially where containers have low overhead (i.e. on Linux) On Mon, Sep 20, 2021 at 4:32 AM luoc wrote: > Hello James, > Great work. Is it possible to add this NOTICE to Github wiki or docs of >

Re: Query the HBase data in Drill

2021-08-24 Thread Ted Dunning
I know somebody who is querying a very large table and has trouble with pushdown. They are looking for values indexed by primary key with a query like "select * from table where key in s". If s has a very small number of values, this turns into primary key access, but if there are more than just

Re: Strange query crash

2021-08-11 Thread Ted Dunning
issue. I recommend that you upgrade to latest version if is cause by > the schema change. > > > 2021年8月11日 上午4:18,Ted Dunning 写道: > > > > > > I am running a moderate sized data reduction task and getting strange > crash with Drill 1.16. Stack trace is shown below. >

Strange query crash

2021-08-10 Thread Ted Dunning
I am running a moderate sized data reduction task and getting strange crash with Drill 1.16. Stack trace is shown below. The query is this: ``` create table dfs.home.`mrms/grib-07.parquet` partition by (box) as with t1 as ( select value as precip, datetime as t, cast(latitude as double)

Re: [DISCUSS] Drill 1.19.0 release

2021-06-14 Thread Ted Dunning
ld > >>>> >>>> require a > >>>> >>>>>>>>> code change instead of a Guava update. > >>>> >>>>>>>>> > >>>> >>>>>>>>> Since this

Re: [ANNOUNCE] Apache Drill 1.19.0 Released

2021-06-14 Thread Ted Dunning
Congratulations to Laurent as a first time release manager! Well done. On Mon, Jun 14, 2021 at 5:56 PM Laurent Goujon wrote: > On behalf of the Apache Drill community, I am happy to announce the release > of Apache Drill 1.19.0. > > Drill is an Apache open-source SQL query engine for Big

Re: [RESULT] [VOTE] Release Apache Drill 1.19.0 RC1

2021-06-09 Thread Ted Dunning
Thanks. I figured you would be ahead of me on this. On Wed, Jun 9, 2021 at 12:27 PM wrote: > Hello Ted, > > Yes, initially I tried both options. > I have also left a comment on the ticket, hope it will be resolved soon. > > Kind regards, > Volodymyr Vysotskyi > >

Re: [RESULT] [VOTE] Release Apache Drill 1.19.0 RC1

2021-06-09 Thread Ted Dunning
Vova, Gavin responded on INFRA-21981 to the effect that upload should go to the dev side and then svn mv should be used to move to the release side. Is that what you tried to do? On Wed, Jun 9, 2021 at 10:25 AM wrote: > I have some issues, will deploy after >

Re: [VOTE] Release Apache Drill 1.19.0 - RC1

2021-06-05 Thread Ted Dunning
+1 I checked signatures on the source and binary tar files. I extracted the binary tar file and ran some simple queries and validated that the web UI came up when I started embedded (found a dead link in the docs ... filed a JIRA for that). I ran `mvn package` at the root level of the source as

[jira] [Created] (DRILL-7949) documentation error - missing link

2021-06-05 Thread Ted Dunning (Jira)
Ted Dunning created DRILL-7949: -- Summary: documentation error - missing link Key: DRILL-7949 URL: https://issues.apache.org/jira/browse/DRILL-7949 Project: Apache Drill Issue Type: Task

Re: [VOTE] Release Apache Drill 1.19.0 - RC0

2021-06-04 Thread Ted Dunning
The release manager really should be deciding what makes the cut and what does not. That is the origin of the tradition of allowing any committer to be a release manager. If somebody is unhappy about the content going into a release, they can roll another release that meets their particular

Re: Feature/Question

2021-06-01 Thread Ted Dunning
Akshay, One of the major design goals of Drill is to run individual queries in a parallelized fashion. It does that very well, under the right conditions. So the first issue is the installation. You have to have a number of drill-bits that are configured to work together. Your description of

Fwd: [NOTICE] Git web site publishing to be done via .asf.yaml only as of July 1st

2021-05-31 Thread Ted Dunning
Drill is on this list. I think that the fix is relatively trivial, but haven't examined it carefully. -- Forwarded message - From: Daniel Gruno Date: Mon, May 31, 2021 at 6:41 AM Subject: [NOTICE] Git web site publishing to be done via .asf.yaml only as of July 1st To: Users

Re: Feature/Question

2021-05-30 Thread Ted Dunning
Here are some more answers: On Thu, May 27, 2021 at 10:09 AM Akshay Bhasin (BLOOMBERG/ 731 LEX) < abhasi...@bloomberg.net> wrote: > Hi Ted, > > Yes sure - below are the 2 reasons for it - > > 1) If I run 5 drill machines in a cluster, all connected to a single end > point at s3, I'll have to

Re: Feature/Question

2021-05-27 Thread Ted Dunning
Akshay, I don't understand why you can't use Drill to create the parquet files. Can you say more? Is there a language constraint? A process constraint? As I hear it, you are asking "I don't want to use Drill to create parquet, I want to use something else". The problem is that there are tons of

Re: Release and GPG key

2021-05-24 Thread Ted Dunning
Laurent Goujon wrote: > Yes, I was thinking of doing a zoom meeting where I would show proof of id > + key id. Especially because of Covid, that seems the easiest option. > > On Mon, May 24, 2021, 16:08 Ted Dunning wrote: > > > Laurent, > > > > The crit

Re: Release and GPG key

2021-05-24 Thread Ted Dunning
Laurent, The critical question here is how you can substantiate this key. IN person, with a government ID, this would be easy. Do you know a committer personally who could vouch for you? Would you be interested in having a video call where you can present some ID? On Mon, May 24, 2021 at 3:24

Re: known bug in csv header parsing

2021-05-22 Thread Ted Dunning
tching if you are using the latest > version (1.19.0 for better). There are unit tests related to the compliant > text reader (in `drill-java-exec` module, at the > `org.apache.drill.exec.store.easy.text.compliant` package). > > > 2021年5月23日 上午5:19,Ted Dunning 写道: > > > &g

Re: known bug in csv header parsing

2021-05-22 Thread Ted Dunning
Also, where would I find the unit tests for the compliant text reader? I have a simple enough case to write a unit test, but I can't see any reference to the class in question outside of working code. On Thu, May 20, 2021 at 7:40 AM Ted Dunning wrote: > > Luoc, > >

Re: known bug in csv header parsing

2021-05-22 Thread Ted Dunning
e > usage). But this reader does not support your idea. I think we can provide > a few codes to enhance the reader. All the new storage and format plugin > base the EVF, more powerful and stable. > > > 2021年5月20日 下午10:40,Ted Dunning 写道: > > > > Luoc, > > > > How

Re: known bug in csv header parsing

2021-05-20 Thread Ted Dunning
settings to process the errors. And then, We have refactored the > CSV format using the EVF, please see the CompliantTextBatchReader.java > (Complies with the RFC 4180 standard for text/csv files). > > > 在 2021年5月20日,13:49,Ted Dunning 写道: > > > > I have a csv file that ca

known bug in csv header parsing

2021-05-19 Thread Ted Dunning
I have a csv file that causes an exception when read by Drill. The file is slightly mal-formed (but R can read it). Interestingly, if I don't parse the header line, I don't get the exception and the problematic embedded quotes are handled well. Likewise, deleting the first data line (which is

Re: [VOTE] Add Dependabot to Drill

2021-05-16 Thread Ted Dunning
I love dependabot. I do minimal maintenance on several dozen demo projects and having a bot check the dependencies for vulnerabilities is a god-send. There is no downside. Yes, I get a bunch of pull requests when somebody digs up another obscure problem with Jackson, but that isn't a problem. I

Re: Test Apache Drill on Linux ARM64

2021-05-07 Thread Ted Dunning
Martin, This is exciting stuff that you are doing and very useful. My thought is that of the options you describe, it seems like the travis option is a good first step because it is nearly trivial (just add the ci config file with a trivial build and test) Running builds on a remote builder

Re: [DISCUSS] Drill 1.19.0 release

2021-05-03 Thread Ted Dunning
Laurent, I don't have a stake here, so can't really comment about specifics, but the process is looking good. On Mon, May 3, 2021 at 9:23 PM Laurent Goujon wrote: > Thanks for all the answers > > So the issues I found based on the feedback are: > >- DRILL-7878: Fix LGTM Alerts >

Re: Feature/Question

2021-04-27 Thread Ted Dunning
Akshay, That's great news! On Tue, Apr 27, 2021 at 1:10 PM Akshay Bhasin (BLOOMBERG/ 731 LEX) < abhasi...@bloomberg.net> wrote: > Hi Ted, > > Thanks for reaching out. Yes - the below worked successfully. > > I was able to create different objects in s3 like 'XXX/YYY/filename', >

Re: [DISCUSS] Drill 1.19.0 release

2021-04-22 Thread Ted Dunning
Laurent, I don't have any issues that I feel I need in the next release, but I would like to add my encouragement to what you are doing!! On Thu, Apr 22, 2021 at 9:33 AM Laurent Goujon wrote: > Hello everyone, > > It has been more than 6 months since the last release, and I believe this >

Re: Feature/Question

2021-04-21 Thread Ted Dunning
Akshay, Yes. It is possible to do what you want from a few different angles. As you have noted, S3 doesn't have directories. Not really. On the other hand, people simulate this using naming schemes and S3 has some support for this. One of the simplest ways to deal with this is to create a view

Re: Drill Slowness

2021-04-19 Thread Ted Dunning
(adding more) The issue described on stack overflow is also very likely if you have a complex schema. On Mon, Apr 19, 2021 at 8:40 AM Ted Dunning wrote: > > > Dileep, > > As Charles suggested, the problem is probably to do with limitations on > what parts of the query is

Re: Drill Slowness

2021-04-19 Thread Ted Dunning
Dileep, As Charles suggested, the problem is probably to do with limitations on what parts of the query is being pushed down into Oracle. That results in lots of data motion that isn't necessary when the query is executed entirely within Oracle because the Oracle database can use indexes and such

Re: Issue with connecting Apache Drill to S3

2021-04-13 Thread Ted Dunning
WHat happens when you just run a query that reference s3 data instead of trying to do the "use s3" command? On Tue, Apr 13, 2021 at 6:15 AM khushalkj jain wrote: > Hi All, > Need your help in fixing the below Issue. > I am running drill locally on my MAC in embedded mode. > > *Query:* > > >

Re: Requesting a release

2021-04-13 Thread Ted Dunning
> > > > > > Hi Ted, > > > > > > I was led to believe that only a PMC member could perform some of the > > > release tasks, but if not the case, I'm happy to volunteer for the next > > > one. Since it would be my first release, is there any docu

Re: Requesting a release

2021-04-12 Thread Ted Dunning
Apr 12, 2021 at 1:55 PM Ted Dunning wrote: > > > Hey Ray, > > > > Any Drill committer should be able to act as a release manager. > > > > My guess is that you know several Drill committers at Dremio who might be > > able to help with this. > > > >

Re: Requesting a release

2021-04-12 Thread Ted Dunning
Hey Ray, Any Drill committer should be able to act as a release manager. My guess is that you know several Drill committers at Dremio who might be able to help with this. On Mon, Apr 12, 2021 at 12:00 PM Ray Lum wrote: > Hi Drill community, > > Is there a process for requesting a release of

Re: Regular Video Calls?

2021-03-04 Thread Ted Dunning
eople who only listen > to the discussion. GMT+8 > > > 2021年3月4日 上午3:13,Ted Dunning 写道: > > > > I am still around, but not super active lately. Real life has intruded a > > lot over the last two years. > > > > > > > > On Wed, Mar 3, 2021 at 11:

Re: Regular Video Calls?

2021-03-03 Thread Ted Dunning
age: Calendly] > Calendly <https://calendly.com/curtis283/30min> > [image: Data Distillr logo] <https://www.datadistillr.com/> > > > On Wed, Mar 3, 2021 at 12:26 PM Ted Dunning wrote: > > > Curtis, > > > > I think that would be a great thing. The

Re: Regular Video Calls?

2021-03-03 Thread Ted Dunning
Curtis, I think that would be a great thing. The Drill community has changed over the last few years and having periodic events could help people come together in a new way. On Wed, Mar 3, 2021 at 6:35 AM Curtis Lambert wrote: > All, > > I'm still very new to Drill but as I'm getting spun

Re: [DISCUSSION] ARM-based compatibility tests

2021-01-27 Thread Ted Dunning
Cool. On Wed, Jan 27, 2021 at 12:40 PM Ganesh Raju wrote: > Ted, > These hardware would be a proper ARM based datacenter server VM instance > > Ganesh > > On Wed, Jan 27, 2021 at 12:00 PM Ted Dunning > wrote: > > > Yes. The ARM-based macs sound pretty exciting.

Re: [DISCUSSION] ARM-based compatibility tests

2021-01-27 Thread Ted Dunning
Yes. The ARM-based macs sound pretty exciting. My own laptop is about 5 years old so it might be time to think about it. I have two ARMs on my desk and 4 Intel machines. The odds could even up if the wind blows right. On Wed, Jan 27, 2021 at 5:39 AM luoc wrote: > Hi, > > @Ted Dunni

Re: merge-apache-drill-master-onto-mapr-master - Build # 1454 - Still Failing!

2021-01-27 Thread Ted Dunning
Huh. I will look at it. I think that there is a build system that could easily still have old email configured. As you point out, that should be updated. On Wed, Jan 27, 2021, 1:21 AM Niels Basjes wrote: > Hi, > > I got this email below. > Seems like at HPE they still have something running

Re: [DISCUSSION] ARM-based compatibility tests

2021-01-26 Thread Ted Dunning
I did some minimal testing in embedded mode way back, but nothing serious. I saw no issues at all. On Tue, Jan 26, 2021 at 2:53 AM luoc wrote: > Hi all, > > I have some ARM-based machines (Not X86 architecture), and then want to do > ARM-based compatibility tests. I know that Netty must bump

Re: [DISCUSSION] Roles and Privileges, Security, Secrets

2021-01-20 Thread Ted Dunning
I think that pushing too much of this kind of authentication and authorization logic into Drill has a large complexity risk. Anything to do with kerberos magnifies that complexity. I also think that it is a mistake to depend on user identity if authorization tokens are likely to need to be

Re: [VOTE]: James Turton for Committer

2020-11-05 Thread Ted Dunning
I think that looks like a great addition. +1 for James. I don't think that lazy consensus is a great idea, however. Happily, you now have three positives. On Wed, Nov 4, 2020 at 11:00 AM Charles Givre wrote: > Hello all, > I'd like to call a vote for James Turton for committer. James has

Re: SSL auth fails after upgrading Netty version

2020-09-17 Thread Ted Dunning
Hey Alka, I am a little confused about motivations here. Can you say whether you get what you need from 1.18? On Thu, Sep 17, 2020 at 6:49 AM alka kumari wrote: > Hi Team, > > I am using Apache Drill JDBC project. > I built Drill client 1.17 with Netty version 4.1.50.Final. There are some >

Re: Calcite Question for you

2020-06-24 Thread Ted Dunning
Charles, Do you think that suggesting using a non-materialized view would help in this case? On Wed, Jun 24, 2020 at 2:31 PM Charles Givre wrote: > Hi Vova, > I have a Calcite question for you. I’m helping someone debug an issue > they’re having connecting Drill to an OracleDB and they’re

Re: compile issue with MapR repo is being worked

2020-05-04 Thread Ted Dunning
> Hi Ted, > > > > Thanks for your help! It looks like for http protocol this issue was > > resolved. > > > > Kind regards, > > Volodymyr Vysotskyi > > > > > > On Mon, May 4, 2020 at 4:19 AM Charles Givre wrote: > > > >&g

Re: compile issue with MapR repo is being worked

2020-05-03 Thread Ted Dunning
ssue I can say that there were quite a few people compiling > > Drill on a Sunday morning. That bodes well, I think, for community > health. > > > > > > > > On Sun, May 3, 2020 at 11:27 AM Ted Dunning > wrote: > > > >> > >> I just got

Re: Drill with No-SQL [was: Cannot Build Drill "exec/Java Execution Engine"]

2020-05-03 Thread Ted Dunning
last 6 > days, say) with your live DB (for the last one day, say.) > > If you provide a "writer" as well as a "reader", you can use Drill to load > your DB as well as query it. > > > Happy to share whatever else I might have learned if you can describe

Re: compile issue with MapR repo is being worked

2020-05-03 Thread Ted Dunning
think, for community health. On Sun, May 3, 2020 at 11:27 AM Ted Dunning wrote: > > I just got word back that the team is looking at the issue. > > Not surprisingly, their first look indicates that the issue isn't what it > appears to be (i.e. not a bad cert) > > >

compile issue with MapR repo is being worked

2020-05-03 Thread Ted Dunning
I just got word back that the team is looking at the issue. Not surprisingly, their first look indicates that the issue isn't what it appears to be (i.e. not a bad cert)

Re: Cannot Build Drill "exec/Java Execution Engine"

2020-05-03 Thread Ted Dunning
The compile problem is a problem with the MapR repo (I think). I have reported it to the folks who can fix it. Regarding the generic question, I think that Drill is very much a good choice for putting a SQL layer on a noSQL database. It is definitely the case that the community is much broader

Re: More CI Issues

2020-05-03 Thread Ted Dunning
I have asked the MapR team to look into the certificate issue. On Sun, May 3, 2020 at 8:03 AM Charles Givre wrote: > Hello all, > In a recent PR that had nothing to do with this, I saw that the CI is now > failing due to the following error. I did some digging and found that the > URL

Re: Timestamp Issue

2020-02-06 Thread Ted Dunning
That is really frustrating because that timestamp is literally in an ISO 8601 format. https://en.wikipedia.org/wiki/ISO_8601 It would be nice if these formats just worked by default. On Thu, Feb 6, 2020 at 5:05 AM Charles Givre wrote: > Hi Drill Devs > I'm having a small issue interpreting

Re: [DISCUSS]: Thoughts

2020-01-30 Thread Ted Dunning
Igor, Good documentation and first 5-minute experience are very important, but not because a long-term contributor will see it and commit their spare time for the next five years on that basis. It is more about preventing early attrition of contributors who might find the project very exciting

Re: Drill Build Issues

2019-10-31 Thread Ted Dunning
I still don't understand how that came to be. It needs to be made more independent. On Thu, Oct 31, 2019 at 12:31 PM Charles Givre wrote: > It came back on this morning. However it certainly suggests that we > should remove anything that is dependent on MapR infrastructure. > > Sent from my

Re: [Discuss] Minor Release

2019-09-27 Thread Ted Dunning
Yes. On Fri, Sep 27, 2019 at 11:09 AM Charles Givre wrote: > Hello all > There was a recent email to the user group about a blocking issue with > sqlline. The issue was resolved in the latest version of sqlline however > it was preventing a user from executing queries. In a situation like

Re: [DISCUSS]: PCAP Reader Improvements

2019-09-22 Thread Ted Dunning
m pretty sure that the EVF allows for > variant maps so if you have a DNS packet and a ICMP packet, you'd get > different fields in the map. > -- C > > > > > > On Sep 22, 2019, at 11:30 AM, Ted Dunning wrote: > > > > This sounds amazing. > > &

Re: [DISCUSS]: PCAP Reader Improvements

2019-09-22 Thread Ted Dunning
This sounds amazing. Some questions. What is EVF? How can you deal with the problem of variant maps? On Sun, Sep 22, 2019, 7:55 AM Charles Givre wrote: > Hello all, > I'm contemplating some improvements to Drill's PCAP reader. Specifically, > I'd like for Drill to actually be able to parse

Re: Anybody at Apachecon Vegas

2019-09-11 Thread Ted Dunning
o I > wasn't able to attend :-(. Next time! > > > On Sep 11, 2019, at 12:14 PM, Aman Sinha wrote: > > > > Yes, I am here today. See you guys soon. > > > > -Aman > > > > On Tue, Sep 10, 2019 at 9:59 PM Ted Dunning > wrote: > > > >> I

Re: Anybody at Apachecon Vegas

2019-09-10 Thread Ted Dunning
I am here. So is Ellen. I think Aman as well. Come to the Drill track tomorrow. Both Ellen and I have talks. On Tue, Sep 10, 2019 at 2:16 PM Naresh Bhat wrote: > Hi Guys, > > Anybody attending Apachecon at Vegas ? > > Regards > -Naresh >

Re: MapR nexus server

2019-08-13 Thread Ted Dunning
We will have a plan in place to avoid disruption of project builds. On Tue, Aug 13, 2019 at 10:07 AM Catherine Lyman wrote: > Here is a message from Kevin who runs the MapR DevOps team... > > Kevin Cheng > Mon, Aug 12, 1:23 PM (20 hours ago) > to me > We will have it for next 3 months for

Re: complex data structure aggregators?

2019-08-12 Thread Ted Dunning
n my GitHub Wiki. > [1] Aggregate UDFS are covered in [2]. Once we learn the answers to your > specific questions, we can add the info to the Wiki. > > > > Thanks, > > - Paul > > > > [1] > https://github.com/paul-rogers/drill/wiki/UDFs-Background-Information &

Re: complex data structure aggregators?

2019-08-12 Thread Ted Dunning
job done; they > have pretty strong limitations on the surface. > > > > Thanks, > > - Paul > > > > > > > >On Monday, August 12, 2019, 10:59:56 AM PDT, Ted Dunning < > ted.dunn...@gmail.com> wrote: > > > > Is it possible for a UDF t

Re: complex data structure aggregators?

2019-08-12 Thread Ted Dunning
e. > > Thanks, > - Paul > > > > On Monday, August 12, 2019, 10:59:56 AM PDT, Ted Dunning < > ted.dunn...@gmail.com> wrote: > > Is it possible for a UDF to produce multiple scalar results? Can it > produce > a binary result? > > Also, as a nit, stan

Re: complex data structure aggregators?

2019-08-12 Thread Ted Dunning
e > UDFs). A VarChar is simply a special kind of array, and UDFs can emit a > VarChar. > > All this is from memory and so is only approximately accurate. YMMV. > > Thanks, > - Paul > > > > On Monday, August 12, 2019, 07:35:47 AM PDT, Ted Dunning < > ted.dunn.

complex data structure aggregators?

2019-08-12 Thread Ted Dunning
What is the current state of building aggregators that have complex state via UDFs? Is it possible to define multi-level aggregators in a UDF? Can the output of a UDF be a byte array? (these are three different questions)

Re: [DISCUSS]: Drill after MapR

2019-08-08 Thread Ted Dunning
Assuming that HPE won't support continued development is not warranted. There is a risk in all change, but assuming the loss of support isn't correct. That said, the community should be able to release independently of any single company, assuming that Apache has enough resources. On Thu, Aug

Re: Apache Drill Hangout - July 9, 2019

2019-07-10 Thread Ted Dunning
It won't be possible to find a time that works for (Kiev, Germany, US (east and west) and Asia. Traditionally, Drill has mostly had contributors from EU and US so that made it possible to find an overlapping time. I would suggest that Hangout times be varied so that some hit the traditional

Re: Superset and Drill

2019-06-02 Thread Ted Dunning
Nice. I started on this ages ago and got stalled. So very nice that others had more stickem than me and actually followed through on this. On Sun, Jun 2, 2019 at 7:34 AM Charles Givre wrote: > Hello Everyone, > I wanted to send this note to the Drill aliases but as of this weekend, > the

Re: [DISCUSSION] DRILL-7097 Rename MapVector to StructVector

2019-05-31 Thread Ted Dunning
Would it be possible to call the new structure a Dict (following Python's inspiration)? That would avoid the large disruption of renaming Map*. On Fri, May 31, 2019 at 10:10 AM Paul Rogers wrote: > Hi Igor, > > Thank you for finally addressing a long-running irritation: that the Drill > Map

Re: adding insert

2019-05-28 Thread Ted Dunning
writer provided by the storage plugin. The devil, of > course, is always in the details. And in finding resources to do the work... > > Thanks, > - Paul > > > > On Monday, May 27, 2019, 5:28:27 PM PDT, Ted Dunning < > ted.dunn...@gmail.com> wrote: > > I hav

Re: adding insert

2019-05-27 Thread Ted Dunning
works well, and is incredibly simple. The only thing wrong is that I have to bring in a separate tool like spark or python to do the insertions. With sleazyInsert, I could do it all with Drill plus a tiny bit of scripting glue. On Mon, May 27, 2019 at 5:27 PM Ted Dunning wrote: > > I have in mind the

Re: adding insert

2019-05-27 Thread Ted Dunning
pdate-and-delete-data/ > [2] > https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/using-hiveql/content/hive_3_internals.html > > > > > > On Monday, May 27, 2019, 1:15:36 PM PDT, Ted Dunning < > ted.dunn...@gmail.com> wrote: > > I would like to star

adding insert

2019-05-27 Thread Ted Dunning
I would like to start a discussion about how to add insert capabilities to drill. It seems that the basic outline is: 1) making sure Calcite will parse it (almost certain) 2) defining an upsert operator in the logical plan 3) push rules into Drill from the DB driver to allow Drill to push down

  1   2   3   >