dLinkError:
/tmp/libnetty-transport-native-epoll5743269078378802025.so:
/tmp/libnetty-transport-native-epoll5743269078378802025.so: failed to map
segment from shared object: Operation not permitted
Does anyone know of a way to configure Drill to use a different tmp
location?
Thanks,
Scott
Thanks Leon for the suggestion, but do you think this config change will
help with my startup problem? It looks like it changes operations for sort
after startup.
Scott
On Mon, Jul 25, 2016 at 3:55 PM, Leon Clayton <lclay...@maprtech.com> wrote:
>
> I move the /tmp of
Curious what the biggest is. Has anyone configured more than 100
drillbits in a cluster before?
Scott
On 07/14/2016 10:27 AM, Ted Dunning wrote:
On the right distribution, you can restrict the subset of the cluster that
has the data you need to avoid locality variation when Drill only runs
Follow up question: Is there a sweet spot for DRILL_MAX_DIRECT_MEMORY and
DRILL_HEAP settings?
On Wed, Jul 6, 2016 at 2:42 PM, scott <tcots8...@gmail.com> wrote:
> Hello,
> Does anyone know if there is a maximum number of drillbits recommended in
> a Drill cluster? For example
resulting in a plateau or decline of performance?
Thanks,
Scott
On Thu, Jul 7, 2016 at 5:00 PM, Abdel Hakim Deneche <adene...@maprtech.com>
wrote:
> I'm not sure you'll get any performance improvement from running more than
> a single drillbit per cluster node.
>
> On Thu, Jul
together?
Thanks,
Scott
Thanks Jacques. I think I have solved my immediate problem. I added
-Djava.io.tmpdir=/newtmp to the DRILL_JAVA_OPTS variable in drill-env.sh. I
am now able to start drillbit without error.
Scott
On Fri, Jul 29, 2016 at 4:29 PM, Jacques Nadeau <jacq...@dremio.com> wrote:
> Unfo
I misunderstanding how user
impersonation works?
Thanks for your time,
Scott
Impersonation using the default dfs configuration is not supported? The
documentation for Impersonation Support says that File System is a
supported Storage Plugin, and that only HBase is not supported.
If this is true, do you know if there is a Jira ticket to add this feature?
Scott
On Thu, Jun
Am I able to open a Jira ticket on this? Or, is this something a developer
has to do?
Scott
On Thu, Jun 30, 2016 at 5:17 PM, scott <tcots8...@gmail.com> wrote:
> Impersonation using the default dfs configuration is not supported? The
> documentation for Impersonation Support sa
Great explanation, Ted. I think I will still open the ticket, if nothing
else to address the gaps in documentation.
Thanks,
Scott
On Fri, Jul 1, 2016 at 2:30 PM, Ted Dunning <ted.dunn...@gmail.com> wrote:
> You can certainly open a ticket for this.
>
> The problem is resol
Chun,
You were right. Thanks for pointing me in the right direction. One of my
drillbits didn't get my configuration change, and by chance the zookeeper
connection didn't assign it to me. I'll keep a close eye out for this
situation in the future.
Scott
On Mon, Aug 15, 2016 at 9:38 PM, Chun
of the
drillbits using port 31010 with same login credentials. This time, I had
full access to everything in maprfs, as if I were logged in as the mapr
user.
Has anyone else experienced this or has an explanation for it?
Thanks,
Scott
I found the issue. Drill was fine, but I had one of the new nodes which had
IP tables blocking some ports. Good lesson learned here for me.
Thanks,
Scott
On Wed, Aug 31, 2016 at 3:55 PM, Jinfeng Ni <j...@apache.org> wrote:
> Can you please check the query profiles for both cases
tell me if there are some configuration settings that can improve
this? Or, is there some point where Drill performance decreases when the
size of the cluster is too large?
Thanks,
Scott
Hello,
I'm wondering if it is possible to connect two Drill clusters? For
instance, could I add a JDBC storage plugin in one cluster that connects to
another Drill cluster?
Thanks,
Scott
hat do you think?
On Mon, Aug 27, 2018 at 11:58 PM Paul Rogers
wrote:
> Hi Scott,
>
> I created a file, "test.json", using the data from your e-mail:
>
> [ { "var1": "foo", "var2":"bar"},{"var1": "fo", "
Paul, Thank you for the workaround, that worked in my case perfectly !!
Scott
On Tue, Aug 28, 2018 at 12:20 PM Lee, David wrote:
> This is a pretty ugly json file.. 568 megs for 7227 records..
>
> => ls -l test.jsonl
> -rw-r--r-- 1 my_login users 568693075 Aug 28 1
lution since this was reported
4 years back. Does anyone have a solution or workaround to this problem?
Thanks,
Scott
Paul,
I'm using version 1.12. Can you tell me what version you think that was
fixed in? The ticket I referenced is still open, with no comments.
Scott
On Mon, Aug 27, 2018 at 5:47 PM Paul Rogers
wrote:
> Hi David,
>
> JSON files are never splittable: there is no single-character wa
the weaker ones.
>
> This increases overheads, but it might help you out.
>
>
>
> On Tue, Aug 21, 2018 at 1:48 PM scott wrote:
>
> > Hi community,
> > I am trying to find a way to tune Drill so that weaker drillbits get less
> > data to work on so that the
: -Xms8G -Xmx8G
-XX:MaxDirectMemorySize=10G on a server that has 48G RAM.
Has anyone else experienced this? Are there any workarounds you can suggest?
Thanks for your time,
Scott
appreciated if you built one.
Scott
On 04/19/2018 11:21 AM, Timothy Farkas wrote:
Rahul / Chris,
The easiest way to do this would be to create a docker container for Drill.
Also it looks like Docker leverage's Windows 10's linux subsystem so docker
images don't run in a VM
https://www.hanselman.com
http://mufeed.livejournal.com
--
*Jim Scott*
Director, Enterprise Strategy Architecture
+1 (347) 746-9281
http://www.mapr.com/
[image: MapR Technologies] http://www.mapr.com
jdbc:drill:zk=127.0.0.1:31010` without success.
Thanks
Davide
--
*Jim Scott*
Director, Enterprise Strategy Architecture
+1 (347) 746-9281
http://www.mapr.com/
[image: MapR Technologies] http://www.mapr.com
Now Available - Free Hadoop On-Demand Training
http://www.mapr.com/training
Can anyone point me to a configuration setting (not maxwidth) that can
prevent having the drill output in the CLI from overwriting data that is
already in a row? (i.e. the data is not wrapping to a new line)
I have a query that when I select all the columns, the data in the columns
at the end
.
Please let me know, any info on the above points would be helpful.
Thanks,
Naveen
--
*Jim Scott*
Director, Enterprise Strategy Architecture
+1 (347) 746-9281
@kingmesal https://twitter.com/kingmesal
http://www.mapr.com/
[image: MapR Technologies] http://www.mapr.com
Now Available - Free
work on
the Java SE JDK 8 versions?
Thanks,
Joe
--
*Jim Scott*
Director, Enterprise Strategy Architecture
+1 (347) 746-9281
@kingmesal https://twitter.com/kingmesal
http://www.mapr.com/
[image: MapR Technologies] http://www.mapr.com
Now Available - Free Hadoop On-Demand
to work around this.
On Jun 25, 2015 1:13 PM, Jim Scott jsc...@maprtech.com wrote:
Can anyone point me to a configuration setting (not maxwidth) that can
prevent having the drill output in the CLI from overwriting data that is
already in a row? (i.e. the data is not wrapping to a new line)
I
, 2015 at 2:39 PM, Jim Scott jsc...@maprtech.com wrote:
an IPv6 address is actually two longs. Depending on the type of analysis
you are doing you may prefer to store them that way.
e.g. the range on the left side is a home / location and the range on the
right side are sub values
That functionality does not yet exist. There is a JIRA open for that
functionality to be added. But has not yet been prioritized high enough to
get completed.
Is this something you are interested in contributing toward?
In the mean time, your options are:
* Alias (select columns[0] as field
/asdrill) now also
includes a IPv4 conversion (One may exist already) .
Regards,
-Stefan
--
*Jim Scott*
Director, Enterprise Strategy Architecture
+1 (347) 746-9281
http://www.mapr.com/
[image: MapR Technologies] http://www.mapr.com
Now Available - Free Hadoop On-Demand Training
http
--
*Jim Scott*
Director, Enterprise Strategy Architecture
+1 (347) 746-9281
http://www.mapr.com/
[image: MapR Technologies] http://www.mapr.com
Now Available - Free Hadoop On-Demand Training
http://www.mapr.com/training?utm_source=Emailutm_medium=Signatureutm_campaign=Free%20available
el
> like there are also other things pressing so I probably won't come back to
> it for a bit. If someone else has the time to move it forward, that would
> be great.
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Sun, Oct 18, 2015 at 3:21 PM, Jim Scott <jsc...@maprt
the aws command line tool for sanity check.
SCott
> On Nov 17, 2015, at 1:36 AM, Andries Engelbrecht <aengelbre...@maprtech.com>
> wrote:
>
> For large cluster I normally use clush (cluster shell), edit the config files
> on one node and then copy them to all using clu
arate window and never comes back.
> >>>>
> >>>>
> >>>>
> >>>> I have tried to configure the settings in Drill webpage - Storage
> with the following settings:
> >>>>
> >>>>
> >>>>
> >>
I have had problems with hyphens in the s3 names in the past when used with
drill. try removing the hyphen.
SCott
> On Oct 15, 2015, at 11:03 AM, Jason Jho <jason...@gmail.com> wrote:
>
> I've enabled JetS3t and configured a storage plugin for S3, but I'm having
> trouble c
ich...@lancetdatasciences.com> wrote:
> Looking for documentation, example or syntax to read xml files with Drill.
>
> Is there a way to invoke an xml parser in Drill?
>
> Thanks-
> hrich...@lancetdatasciences.com
>
>
--
*Jim Scott*
Director, Enterprise Strategy & Archite
tc", "USER AGENT"]
>
> I'm having this response (3columns):
> ["XXX.XXX.XXX.XXX", "200", "GET / ... etc\" \"USER AGENT\""]
>
> But if I edit the file with comma delimiter a the configuration, it's
> working fine.
> I
somebody else have a good idea for this?
>
>
> On Fri, Oct 2, 2015 at 7:17 AM, Jim Scott <jsc...@maprtech.com> wrote:
>
> > John,
> >
> > You may want to ask this question on the dev list as well.
> >
> > I think, logically, this could be accompli
comma as a delimiter, I always ALWAYS get bit in the rear. So I
will switch to using a different delimiter within the log file - one that does
NOT compete with the commas typically found inside JSON.
You guys came through.
Thank you.
SCott
> On Sep 23, 2015, at 9:53 AM, Andries Engelbre
artifacts or if you want me to turn on a debug/verbose
flag.
Thanks,
SCott
——>>>>>>1. First Attempt (use s3n) <<<<<<<<<<<<<<
with the connections string doing the “s3n” prefix - null pointer errors
Here is the co
.
The only xml that I saw was the core-site.xml and it does not have that kind of
information within it ….
Thanks in advance.
SCott
> On Sep 22, 2015, at 10:47 AM, Andries Engelbrecht <aengelbre...@maprtech.com>
> wrote:
>
> Scott,
>
> In your SP configuration c
of my hadoop_excludes.txt
5. View of the end of the sqlline.log
Please let me know what I should change?
Thanks,
SCott
Scott C. Cote
scottcc...@gmail.com <mailto:scottcc...@gmail.com>
972.672.6484
=>>>>> Part 1 <<<<=
What I see when I request for a list of da
Congrats Ellen.
To the point, we wouldn’t have had Drill as a topic at @DfwDataScience and
@DallasMongoDB if she had not evangelized with us. :)
SCott
> On Nov 22, 2015, at 6:55 PM, Andrew Brust
> <andrew.br...@bluebadgeinsights.com> wrote:
>
> Excellent news!
>
&g
around them in some way?
Thanks all
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
proprietary information and ma
Nevermind, this is because I changed the directory structure.
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
proprietary information
ultInputFormat": "json"
},
...
And i get the same thing.
I do not have hadoop installed. do i need it?
Thanks,
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments c
Yep, I disabled all other storage plugins (default dfs, cp and custom s3
plugin) and the alter session took .5 sec as opposed to 169.
Thank you!
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail
, ResponseStatus: Forbidden, XML Error Message: AccessDeniedAccess
Denied95BB171A886F3C27NLoKTo5jzzBS3j0mIRUeVwF8tGvTFUh/4jJrkPr+0U6kjo1e3t7uX3wv2Thkb1G/+/ZDODJXbVE=
What am I doing wrong?
Thanks all,
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.28
My post is dated also :)
Eventually got around it. but I don’t remember how at the moment.
No worries
SCott
> On May 26, 2016, at 4:44 PM, Nathan Griffith <ngriff...@dremio.com> wrote:
>
> Hi, Scott.
>
> That article is a bit dated. Try setting up your core-site.xml fil
il.com> wrote:
>
> Niels Basjes and Jim Scott worked on something related to this recently.
>
> The idea was to make the output of Niels' log-parser package directly
> queryable in Drill.
>
> I don't know the current status.
>
>
> On Sat, Feb 6, 2016 at 4:25 PM, David A
Anyone working on a DrillBit that can poke into an ignite grid?
SCott
> On Jan 26, 2016, at 11:58 AM, Jacques Nadeau <jacq...@dremio.com> wrote:
>
> https://plus.google.com/hangouts/_/dremio.com/drillhangout?authuser=0
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
almost always beets the pants off of
pulling data from storage ….
Reach out to them u...@ignite.apache.org <mailto:u...@ignite.apache.org>
Might be a nice mashup for Dremio and GridGain :)
Hope this helps you guys grow.
SCott
> On Jan 29, 2016, at 11:39 AM, Stefán Baxter <ste...@
Hello,
Is there a way to convert other date formats into the DATE type? I would like
to cast a field that contains a date in the format mm/dd/, i.e
"01/25/2016", into DATE type.
Thanks,
Scott Wilburn
;
> On Tue, Feb 23, 2016 at 10:41 AM, Wilburn, Scott <
> scott.wilb...@verizonwireless.com.invalid> wrote:
>
> > Hello,
> > Is there a way to convert other date formats into the DATE type? I
> > would like to cast a field that contains a date in the forma
Hello,
Is there a way to specify multiple delimiters when configuring a storage plugin
record format? For example, could I split records into fields by comma or by
semicolon characters.
Thanks,
Scott Wilburn
Scott,
You would need a format defined for each file type. e.g. csv has commas,
tsv has tabs, so on
If you are looking for multiple delimiters within the same file or
potentially with a single file extension that isn't supported.
Jim
On Thu, Feb 18, 2016 at 5:43 PM, Wilburn, Scott <
scott.w
] = aab
columns[5] = 99
columns[6] = 66
columns[7] = aab
In this example, the record contains 8 fields when split by comma and by
semicolon.
Is something like this possible?
Thanks,
Scott Wilburn
-Original Message-
From: Jim Scott [mailto:jsc...@maprtech.com]
Sent: Thursday, February 18
Magnus,
Great suggestions. I would like to try to extend the functionality of the
"text" format type, used for csv files today. Do you know if there are any
published instructions for this process?
Thanks,
Scott Wilburn
-Original Message-
From: Magnus Pierre [m
, key and value, I really want to
print three columns: filename,key,value .
Any help would be appreciated.
Thanks,
Scott Wilburn
It is working now. Thanks for the suggestions. It was a matter of getting the
user credentials working for my database. I only wish the error messages were
more descriptive, and specific to the failure reason. Glad you opened that
ticket, Chris.
Thanks,
Scott Wilburn
-Original Message
Hello,
Is there a way to index or create primary keys for fields in Drill? I'm
interested in indexing for performance and also for creating unique
constraints. If there is no way to create indexes, then is there another way to
generate a unique constraint?
Thanks,
Scott Wilburn
Hello,
Is there a way to use Drill to search through binary files? I have some binary
files that require a special compiled tool to convert into CSV format. Can
Drill execute an external tool to convert each file into ASCII before searching?
Thanks,
Scott Wilburn
The compiled tool I use to read the files today is written in C. It converts
the files to ASCII CSV and prints to STDOUT. Would that work in a format plugin?
Thanks,
Scott Wilburn
-Original Message-
From: Jason Altekruse [mailto:altekruseja...@gmail.com]
Sent: Tuesday, March 08, 2016
All,
I didn't get any response on this question. Does anyone have a comment or
advice for me? I didn't see anywhere in the Drill online docs that explain
indexes or unique constraints. Did I miss something?
Thanks,
Scott Wilburn
-Original Message-
From: Wilburn, Scott
Thanks. I've started a track on that. Do you know of any workaround in the
meantime?
Thanks,
Scott Wilburn
-Original Message-
From: Abhishek Girish [mailto:abhishek.gir...@gmail.com]
Sent: Monday, March 21, 2016 02:40 PM
To: user
Subject: [E] Re: dfs filename
This is currently
Node already supports ODBC.
Drill also has a ReST interface.
Is there something more specific you had in mind?
On Fri, Apr 29, 2016 at 6:47 AM, Jeff Gunther <jeff.gunt...@gmail.com>
wrote:
> Are there any plans for a Node.js driver?
>
> Jeff Gunther
>
--
*Jim Scott*
Di
-parsing the data?
Thanks,
Scott Wilburn
Information Technology
> Internet Based Communication Networks and Services (IBCN)
> Ghent University - iMinds
> Technologiepark Zwijnaarde 15, B-9052 Gent, Belgium
> E: leandro.ordo...@intec.ugent.be, leandro.ordoneza...@ugent.be
> W: www.ibcn.intec.UGent.be
>
>
--
*Jim Scott*
to provide a clean
> interface and depiction of valuable schemas to my user via workspaces, and
> this small flag, I believe would be a low impact way to do that.
>
> I would love discussion on this, if others would find this valuable, I will
> happily make a JIRA.
>
> Jo
Parth is a nice “bit” to add to the tool kit :)
SCott
> On May 25, 2016, at 11:15 AM, Abdel Hakim Deneche <adene...@maprtech.com>
> wrote:
>
> Congrats Parth !
>
> On Wed, May 25, 2016 at 9:15 AM, Zelaine Fong <zf...@maprtech.com> wrote:
>
>> C
etting the json all_text_mode and read_numbers_as_double.
I have a drill-embeded instance that seems to pick these json setting from the
drill-override file just fine.
Can you see what I'm doing wrong?
____
Scott Kinney | DevOps
stem <http://www.stem.com/> |
Yep, I didn't set the workspace 'writable': true
user error as usual
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
proprietary information
it's still not picking up the store.json* config changes
The only way I can see to set these is with running ALTER SYSTEM query after
drill api is up.
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail
=0)
dfsl is just a copy of the default dfs storage plugin.
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
propr
will have out data in s3 in
a name/year/month/day and I dont want to have to put this schema file in every
directory is s3. that seems unmanageable.
Is there a way to set the schema from a file before making a query via the REST
api?
Scott Kinney | DevOps
stem <h
No i haven't.
I'll give that a try.
Thanks.
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
proprietary information and material for the sole
Can I have drill open gzipped json files who's names do not end in .json.gz?
We have a spark job generating these files and it just dosn't want to change
the name or append the .json.gz.
?
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.28
>From what I can tell the s3 plugin must be configured via the web interface.
>We really need drill to be able to scale and cluster. Has anyone found a way
>to configure drill to use s3 with xml (or whatever) files rather than the web
>interface?
?
__
Well that's a bummer but I believe it setting "defaultInputFormat": "json"
doesn't seem to have any effect.
____
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachmen
Ah, ok. Thats something.
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
proprietary information and material for the sole use of the intended
Hm, i must have set those another way in embeded mode. I can't see where. Those
settings persist between drill restarts.
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain
.
thanks!
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
proprietary information and material for the sole use of the intended
recipient(s). Any
There is a key 'foo': [ 99, 99.1, 99.8 ].
Is there a way get drill to handle this? Maybe treat all ints as floats?
____
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any att
!
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
proprietary information and material for the sole use of the intended
recipient(s). Any review, use or distribution that has
That looks promising but didn't work.
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and
proprietary information and material for the sole use
Not that I know of but I'm new to drill.
I've done 'alter system' for json all_text_mode & read_numbers_as_double.
Do you know of a setting that might cause something like this?
____
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Mill
When i query from sqlline i can see all the data, very complicated / nested
json structure but when i query with the api or the web ui a lot of the data is
missing.
?
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.282.1299
100 Rollin
cate
> data through both an original csv and converted Parquet file at the same
> time.
>
> Is there a common practice to making data available once converted, in
> something similar to a transactional batch of "convert then (re)move source
> csv files" ?
>
--
*Jim Scot
Sounds like an opportunity for drill triggers
Sent from my iPhone
> On Sep 22, 2015, at 2:18 PM, Edmon Begoli wrote:
>
> Yes, that is my current thinking and most likely course of action.
>
>> On Tue, Sep 22, 2015 at 3:13 PM, Paul Ilechko wrote:
>>
91 matches
Mail list logo