Creating a single parquet or csv file using CTAS command?

Hi, is there a way to force drill to create a single file when performing a CTAS command (or some other method). Right now, I'm creating CSV files, and then have to perform and extra step to stitch 1_0_0.parquet 1_1_0.parquet 1_2_0.parquet etc. together into a single file. Thank you. Peder

Re: Creating a single parquet or csv file using CTAS command?

On Thu, Feb 4, 2016 at 11:15 AM, Andries Engelbrecht < aengelbre...@maprtech.com> wrote: > Is there a reason to create a single file? Typically you may want more > files to improve parallel operation on distributed systems like drill. > Good question. I'm not actually using Drill for "big

Re: Creating a single parquet or csv file using CTAS command?

optimum config for your use > case. > > --Andries > > > On Feb 4, 2016, at 10:12 AM, Peder Jakobsen | gmail <pjakob...@gmail.com> > wrote: > > > > Sorry, bad typo: I have 50GB of data, NOT 500GB ;). And I usually only > > query a 1 GB subset of this data

Re: Creating a single parquet or csv file using CTAS command?

Sorry, bad typo: I have 50GB of data, NOT 500GB ;). And I usually only query a 1 GB subset of this data using Drill. On Thu, Feb 4, 2016 at 1:04 PM, Peder Jakobsen | gmail <pjakob...@gmail.com> wrote: > On Thu, Feb 4, 2016 at 11:15 AM, Andries Engelbrecht < > aengelbre.

Re: Creating a single parquet or csv file using CTAS command?

part from just a > > basic hosting solution? As an example you will find MapR with Drill > > available on demand on Amazon AWS and Azure. You may want to look at that > > to spin up a node(s), load/process/download your data, and then spin it > > down. Might be worth a l

Can Drill connect to an MS Access Database file using ODBC?

I have an .mdb (Access file) that ideally I'd like for Drill to see as just another data source. Is this possible? It seems like the MapR driver is for connecting to Drill via ODBC only, which doesn't cover my use case. Thanks :)

Re: SQL Lookup table - how to mimic?

gt; 10 rows selected (0.905 seconds) > 0: jdbc:drill:> select * from maprfs.cmatta.`country_data.csv` limit 10; > ++ > | columns | > +-

Re: Does drill recognize new line correctly?

Hi, if you are working in Unix, use *iconv* to remove newlines, and other things like BOMS, converting to UTF-8 etc. Perhaps Google "iconv remove newlines from csv"..? iconv is quick, you can process Gibabytes of nested csv files in minutes. If you are using Windows, I'm not sure. Peder On

Re: SQL Lookup table - how to mimic?

t;,"AUD","Dollar","61","AU","AUS","36",".au\r"] | > > +-+ > > 10 rows selected (0.905 seconds) > > 0: jdbc:drill:> select * from maprfs.cmatta.`country_data.csv` limit 10; > > +-

SQL Lookup table - how to mimic?

Does drill have something analogous to SQL CONSTRAINT and REFERENCES for doing table type lookups? I have a large data that uses numerical ISO country codes; they need to be replaced with country names that are mapped in a small CSV file (iso_code, country_name). Any suggestions? Thanks

Re: ODBC in Virtual Box connects to Drill on host Unix system?

io.com> wrote: > Peder, > > What BI tool are you running on Debian? > > Thanks, > Tomer > > On Thu, Dec 31, 2015 at 8:40 AM, Peder Jakobsen | gmail < > pjakob...@gmail.com > > wrote: > > > Hi Norris, > > > > Just discovered that the ODBC driver

Documentation update: 64bit Windows is a basic requirement for running Drill on Windows?

It doesn't seem possible to run Drill on 32 bit versions of Windows, as implied in the instructions on installing the ODBC driver here. https://drill.apache.org/docs/installing-the-driver-on-windows/ I'd like to update the documentation to make this more explicit via a Github pull request. I

Re: MS Windows: Hadoop binaries required to run drill?

t it doesn't > indicate why Drill is failing to initialize these settings. > > Can you possibly try this under the Administrator account if one exists? > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail <

Re: MS Windows: Hadoop binaries required to run drill?

oesn't > indicate why Drill is failing to initialize these settings. > > Can you possibly try this under the Administrator account if one exists? > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail < > pj

Re: Drill 1.4 fails to start on Windows 7, JDK 1.7.0 - "Unable to deserialize"

Best, > Nathan > > On Tue, Dec 29, 2015 at 2:04 PM, Peder Jakobsen | gmail > <pjakob...@gmail.com> wrote: > > C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local" > > DRILL_ARGS - " -u jdbc:drill:zk=local" > > HADOOP_HOME n

Re: ODBC in Virtual Box connects to Drill on host Unix system?

orr...@simba.com> wrote: > Hey Peder, > > Yep, I tried using Tableau on a Windows VM (VMWare) connecting to Drill > running on a Host Linux system and it worked perfectly fine. > > Norris > > -Original Message- > From: Peder Jakobsen | gmail [mailto:pjakob...@gma

MS Windows: Hadoop binaries required to run drill?

Hi, does hadoop have to be installed in order to run apache drill on windows? So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and still no luck. Once I figure it out, I will update the documentation for windows if you like. Thanks, Peder J.

Re: MS Windows: Hadoop binaries required to run drill?

e to deserialize "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0) On Thu, Dec 31, 2015 at 4:33 PM, Ted Dunning <ted.dunn...@gmail.com> wrote: > Can you say a bit more about your platform and environment? > > What about the errors you are seeing? > >

Re: MS Windows: Hadoop binaries required to run drill?

r Shiran <tshi...@dremio.com> wrote: > > > No. Drill does not depend on Hadoop > > > > Can you try these instructions and see if they work: > > > > http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/ > > > > > > > On

Re: MS Windows: Hadoop binaries required to run drill?

com> wrote: > Hey Peder, > > What's in your C:\Windows\Temp directory? Is there something that > looks like it's from Drill? If there is, try deleting it and running > again. > > --Nathan > > On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail > <pjakob...@gma

Re: MS Windows: Hadoop binaries required to run drill?

elping with your case? > > On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail > <pjakob...@gmail.com> wrote: > > OK, at startup, Drill creates an empty file called hive.sys.drill that's > > located in C:\tmp\drill\sys.storage_plugins > > > > Perhaps it's not s

Re: MS Windows: Hadoop binaries required to run drill?

Administrator account if one exists? > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail < > pjakob...@gmail.com > > wrote: > > > I spoke too soon, perhaps (but it must be a simple I/O issue on s

Re: MS Windows: Hadoop binaries required to run drill?

appears to be empty. On Linux, lots of stuff is included in this drill path: profiles, sys.options & sys.storage_plugins Hope this helps P. On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <pjakob...@gmail.com > wrote: > I deleted everything in C:\Windows\Temp. Note, when I

Fixed: Apache Drill won't start on windows 7

This appears to be a common issue that was addressed here: https://issues.apache.org/jira/browse/DRILL-1907 Adam Gilmore added a comment - 08/Jan/15 05:30 Fixed it - basically it's a problem that, by default, Drill tries

Where is the Drill Explorer and / or docs on installing it?

Hi, The docs refer to the Drill Explorer, which for OS X is supposed to be located in the Applications folder: https://drill.apache.org/docs/connecting-drill-explorer-to-data/ But there is no mention in the documentation on how to install this tool (I think)? Are the docs referring to and old

Learning to use the C++ client libraries

Hi, Coming from the Python/Ruby /bash shell world, I find working SQLLine a hard pill to swallow; it's it's very "DOS" like, aka almost featureless. For this reason, I'd like to explore writing a python wrapper around the Drill C++ client libraries. What's the best way to learn to use this