Re: Looking for workaround to Schema detection problems

Hi Alexander, Please try with turning on the union type: ALTER SESSION SET `exec.enable_union_type` = true; Kind regards Vitalii 2016-07-08 10:50 GMT+00:00 Holy Alexander : > My JSON data looks - simplified - like this > > {"ID":1,"a":"some text"} >

Implement "DROP TABLE IIF EXISTS" statement

Hi all! I'm going to implement "DROP TABLE IIF EXISTS" and "DROP VIEW IIF EXISTS" statements in Drill (DRILL-4673 ). The reason of using "IIF" is inability of adding "IF" keyword to non-reserved words list (due to SQL:2011 standard which calcite

Re: Reading Drill generated Timestamp from spark

Hi Rahul, According to the parquet specification the primitive datatype for the TIMESTAMP logical type is INT64 (INT96 has a deprecated label). That's why Drill has not a mechanism to generate such values.

Re: Issue with time zone

Hi Kostyantyn, * Without zookeeper you can just run drillbit in embedded mode: *bin/drill-embedded* * Also "*timeofday()*" Drill function can help you to identify a timezone used by your drillbit: "start your sql engine" 0: jdbc:drill:zk=local> SELECT TIMEOFDAY() FROM (VALUES(1));

Re: Error reading int96 fields

ster/result.tar.gz > > Regards, > Rahul > > On Tue, Dec 5, 2017 at 5:57 PM, Vitalii Diravka <vitalii.dira...@gmail.com > > > wrote: > > > Hi Rahul, > > > > It looks like a bug. > > Could you please open the jira ticket and provide the query and d

Re: Error reading int96 fields

Hi Rahul, It looks like a bug. Could you please open the jira ticket and provide the query and dataset to reproduce the issue? Thanks Kind regards Vitalii On Tue, Dec 5, 2017 at 2:01 PM, Rahul Raj wrote: > I am getting the error - SYSTEM ERROR :

Re: How to deal with Parquet files containing no rows without Drill errors?

Hi Dave, The issue is not in joining, Drill can join empty schemaless table (for example empty JSON file or empty directory). DRILL-4517 is exactly describes the issue. You can add your test case with data to that jira ticket. Regarding workarounds, I am not aware of any. Kind regards Vitalii

Re: Unable to run Drill queries on Drill1.13

Hi Peter, Do you use User Authentication [1]? What kind of? Could you share please drill-override.conf file and connection string, which you are using to connect to the drillbit? anonymous user has not access to perform your query. You should specify username in connecting string [2], which has

Re: running embedded mode under windows

Can you share also the error from the log file? Kind regards Vitalii On Thu, May 31, 2018 at 3:47 AM Divya Gehlot wrote: > Can share the steps you followed ? > > On Thu, 31 May 2018 at 11:54 AM, Bo Qiang wrote: > > > Hi, > > > > Sorry for this newbie question. I have been following the

Re: unit tests

Hi Padma, Looks like you have wrong some hostname or ip in your /etc/hosts. Please find out more here [1], Sorabh has already answered a similar question :) [1] https://lists.apache.org/thread.html/%3che1pr07mb33068e59f257a4d78f29304d84...@he1pr07mb3306.eurprd07.prod.outlook.com%3E Kind regards

Re: Drill error

Hi Nitin, It happens in the process of reallocation of the size of buffers in the memory. It isn't a User Exception, so it looks like a bug, if you get it in some existed plugin. But to say you exactly, please describe your case. What kind of query did you perform, any UDF's, which data source?

Re: Drill Hangout tomorrow 06/26

Lately Drill Travis Build fails more often because of Travis job time expires. The right way is to accelerate Drill execution :) Nevertheless I believe we should consider excluding some more tests from Travis Build. We can add all TPCH tests ( TestTpchLimit0, TestTpchExplain, TestTpchPlanning,

Re: Drill 1.12 query hive transactional orc table

Hi, Thanks for your question. Drill started supporting queries on Hive ACID tables past DRILL-1.13.0 version[1]. Please do upgrade onto the last Drill version, then you will able to perform queries for Hive transnational tables. [1] https://drill.apache.org/docs/hive-storage-plugin/ Kind

Re: question about views

Ted, The rules are enabled and DRILL-3855 [1] is resolved. Please try your queries with latest Drill master version. [1] https://issues.apache.org/jira/browse/DRILL-3855 Kind regards Vitalii On Mon, Apr 30, 2018 at 4:31 PM Nicolas Paris wrote: > Hi > > This looks an

Re: Code too large

Hi all! Looks like the above issue is connected to much expressions in the generated code. I believe it can be resolved by reducing the value for "exec.java.compiler.exp_in_method_size" option on the session or system level [1]. If it is not working, it means that logic is not implemented for

Re: Requesting guidance. Having trouble generating parquet files from jdbc connection to PostgreSQL. "java.lang.OutOfMemoryError: GC overhead limit exceeded"

ommit. > > > > [1] https://issues.apache.org/jira/browse/DRILL-4177 > > > > > > > > > so, something along the lines of > > > > > > ..snip.. > > > "url": "jdbc:postgresql:// > myhost.mydomain.com/ateb?useCursorF

Re: Requesting guidance. Having trouble generating parquet files from jdbc connection to PostgreSQL. "java.lang.OutOfMemoryError: GC overhead limit exceeded"

> ERROR: A column you queried has a data type that is not currently > supported by the JDBC storage plugin. The column's name was actionjson > and its JDBC data type was OTHER. > > > thanks, > reid > > On Wed, 2018-08-15 at 14:44 +0300, Vitalii Diravka wrote: > > [EXTERNAL

Re: distributed drill on local file system

Hi Mehran, This is a question for user mailing list. Looks like there are no issues with it, you can run Drill in distributed mode on Windows, Linux or MacOS based machines. It necessary to specify *zk.connect* for Zookeeper hostname and port number in *drill-override.conf* file and to run

Re: [ANNOUNCE] New PMC member: Boaz Ben-Zvi

Congrats Boaz! Kind regards Vitalii On Fri, Aug 17, 2018 at 12:51 PM Arina Ielchiieva wrote: > I am pleased to announce that Drill PMC invited Boaz Ben-Zvi to the PMC and > he has accepted the invitation. > > Congratulations Boaz and thanks for your contributions! > > - Arina > (on behalf of

Re: Drill Hangout tomorrow 08/21

est Regards, > Alex > > On Mon, Aug 20, 2018 at 10:50 PM Volodymyr Vysotskyi > > wrote: > > > Hi all, > > > > I and Vitalii Diravka want to give the presentation with our ideas > > connected with Drill Metadata management project (DRILL-6552 > > <ht

Re: Drill High availability

Hi Satish! I think it really depends on your system, cluster configurations and use cases for which you are using Apache Drill. Please specify them, to do more accurate suggestions for you. Kind regards Vitalii On Wed, Aug 22, 2018 at 5:00 AM pujari Satish wrote: > Hi Team, > > Good

Re: Google Sheets plugin

Hi Pedro, Currently there is no any Excel plugin in Drill, but you can export the data from Excel file into CSV file and query it by Drill. There is a ticket for Excel files plugin [1]. You can create a similar one for Google sheets. [1] https://issues.apache.org/jira/browse/DRILL-3738 Kind

Re: Google Sheets plugin

h. > > > > Pedro > > Like Vitalii suggested, please create a JIRA. This helps gain visibility > within the community and someone (including yourself) can consider writing > a Storage Plugin for it, > > > > Kunal > > On 7/19/2018 12:56:24 PM, Vitalii Diravka > w

Re: Source for drill-calcite

Hi Rahul, Updating of Calcite onto 1.16.0 version is on review. You can test it if you wish, but it requires some changes in Drill code. Please find the PR here: https://github.com/mapr/incubator-calcite/pull/18 Kind regards Vitalii On Wed, Mar 28, 2018 at 4:28 PM, Kunal Khatua

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

formation :- > Apache Hadoop 2.7.3Apache Hive 2.1.1 > @Vitalii, For testing i can setup upgrade hive but upgrading hive will > take time > on our production server. Don't you think it should be back compatible? > > > > > On Tue, Mar 20, 2018 4:33 PM, Vitalii Diravka vital

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

Hi Anup, Maybe we missed something after updating onto hive-2.3 client versions. Could you provide the following info: * What is your hive-server and metastore versions? If it is not 2.3 version could you update onto this? * What is your hadoop distribution? Kind regards Vitalii On Tue, Mar 20,

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

Anup, "get_table_req" method is present in ThriftHiveMetastore header of Apache Hive 2.3 version. I believe Hive upgrade will help you. Probably it is Hive's back compatibility issue. Please let us know, whether upgrade helps. Kind regards Vitalii On Tue, Mar 20, 2018 at 12:56 P

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

bles > via drill. > > 2. Added capability property in hive-site.xml and restarted metastore and > tried > to access hive tables via drill. > > Both ways didn't work. So does that mean Drill 1.13.0 version is > compatible with > Hive 2.3 and above? > > > &

Re: Drill error

> > >>> > > >>> > > >>> > On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar < > nitinpawar...@gmail.com> > >>> > wrote: > >>> > > >>> > > Hi Vitalii, > >>> > > > >>> &g

Re: Drill error

Thank you, Nitin Kind regards Vitalii On Thu, Jun 28, 2018 at 6:04 PM Nitin Pawar wrote: > created https://issues.apache.org/jira/browse/DRILL-6551 > > On Thu, Jun 28, 2018 at 7:43 PM, Vitalii Diravka < > vitalii.dira...@gmail.com> > wrote: > > > Hi Nit

Re: Drill JDBC Plugin limit queries

Hi Rahul, Drill has *DrillPushLimitToScanRule* [1] rule, which should do this optimization, whether the GroupScan supports Limit Push Down. Also you can verify in debug mode whether this rule is fired. Possibly for some external DB (like MapR-DB) Drill should have the separate class for this

Re: Drill JDBC Plugin limit queries

; > JdbcPrel(sql=[SELECT * FROM "u_g001"."executioniteration" WHERE > > "id" > 36050 ]): rowcount = 50.0, cumulative cost = {50.0 rows, 50.0 > cpu > > > > Regarding the second point, its the java heap getting filled with jdbc >

Re: Drill JDBC Plugin limit queries

ill-user/201808.mbox/%3C0d36e0e6e8dc1e77bbb67bbfde5f5296e290c075.camel%40omnicell.com%3E On Sat, Oct 13, 2018 at 3:56 PM Rahul Raj wrote: > Should I create tickets to track these issues or should I create a ticket > to update the documentation? > > Rahul > > On Sat, Oct 13, 20

Re: Drill JDBC Plugin limit queries

Raj wrote: > Vitalii, > > Created documentation ticket DRILL-6794 > > How do we proceed on extending the scan operators to support JDBC plugins? > > Regards, > Rahul > > On Sat, Oct 13, 2018 at 6:47 PM Vitalii Diravka > wrote: > > > To update the do

Re: [ANNOUNCE] New Committer: Hanumath Rao Maduri

Congratulations! Kind regards Vitalii On Thu, Nov 1, 2018 at 5:43 PM salim achouche wrote: > Congrats Hanu! > > On Thu, Nov 1, 2018 at 6:05 AM Arina Ielchiieva wrote: > > > The Project Management Committee (PMC) for Apache Drill has invited > > Hanumath > > Rao Maduri to become a committer,

Re: Drill JDBC Plugin limit queries

100.0 cpu, 0.0 io, 0.0 network, 0.0 > memory}, id = 164 > > Regards, > Rahul > > On Fri, Oct 19, 2018 at 8:47 PM Rahul Raj wrote: > > > I will make the changes and update you. > > > > Regards, > > Rahul > > > > On Fri, Oct 19,

Re: November Apache Drill board report

+1 Does it make sense to add "Support Transitive Closure during Filter Push Down and Partition Pruning" feature DRILL-6173 [1]? This is one of the places where we are farther than Spark, for example [2], [3]. [1] https://issues.apache.org/jira/browse/DRILL-6173 [2]

Re: [ANNOUNCE] New Committer: Chunhui Shi

Congrats Chunhui and welcome to the Drill comittership! On Fri, Sep 28, 2018 at 11:23 PM Timothy Farkas wrote: > Congrats! > > On Fri, Sep 28, 2018 at 1:17 PM Sorabh Hamirwasia > wrote: > > > Congratulations Chunhui!! > > > > Thanks, > > Sorabh > > > > On Fri, Sep 28, 2018 at 12:56 PM Paul

Re: [ANNOUNCE] New PMC member: Charles Givre

Congrats Charles! And thank you for your enthusiasm and work on Drill On Mon, Sep 3, 2018 at 4:22 PM Arina Ielchiieva wrote: > I am pleased to announce that Drill PMC invited Charles Givre to the PMC > and he has accepted the invitation. > > Congratulations Charles and welcome to PMC squad :) >

Re: How does apache drill support DateTimeOffset type?

Hi, There is no screenshot in this mail. Please upload it to Google Drive and share here. Also what do you mean DateTimeOffset type? There is no DateTimeOffset logical Date/Time datatype in parquet [1]. Please use "parquet-tools" to define the schema of your parquet file [2]. [1]

Re: N4}[48H$DHWIOXNQQB%~$GD

Hi! Do you use UNION type (is "exec.enable_union_type" enabled)? This feature is experimental: https://drill.apache.org/docs/json-data-model/#experimental-feature-heterogeneous-types Kind regards Vitalii On Thu, Jan 17, 2019 at 9:52 PM 陈。 wrote: > > > > -- 原始邮件

Re: [VOTE] Apache Drill release 1.15.0 - RC0

://issues.apache.org/jira/browse/DRILL-6916 [2] https://github.com/apache/drill/pull/1579 Kind regards Vitalii On Wed, Dec 19, 2018 at 2:18 PM Vitalii Diravka wrote: > @Aman Sinha I am investigating, which maven plugin > causes creating this dir. > > I guess this sinks rc0. I'll prepare

Re: Query on tables

ster so that > my queries are executed fast. Currently one CTAS command and partition by > commands are taking hours to process GBs of data. > > I can provide statistics of elapsed time of query execution if required, > for your analysis. > > Kind Regards > Kiran > >

Re: [ANNOUNCE] New Committer: Salim Achouche

Congratulations Salim! Well deserved! Kind regards Vitalii On Mon, Dec 17, 2018 at 12:40 PM Arina Ielchiieva wrote: > The Project Management Committee (PMC) for Apache Drill has invited Salim > Achouche to become a committer, and we are pleased to announce that he has > accepted. > > Salim

Re: Query on tables

Hello! Can you filter the newly coming data and run CTAS only for those data? It will allow to avoid extra work. Kind regards Vitalii On Mon, Dec 24, 2018 at 6:53 PM Idea Everywhere wrote: > Hi Team, > > My current situation: > I have apache drill installed in AWS EC2 (M4.4x large) instances

Re: Drill “VALIDATION ERROR: A table or view with given name already exists in schema” for empty directory

Hi Reed, It looks like a bug. Could you please create a jira ticket with an above description? https://issues.apache.org/jira/projects/DRILL/issues Kind regards Vitalii On Wed, Dec 5, 2018 at 6:57 PM Reed Villanueva wrote: > After upgrading drill on our cluster to drill-1.12.0-mapr, testing

Re: Drill “VALIDATION ERROR: A table or view with given name already exists in schema” for empty directory

Khurram > > On Wed, Dec 5, 2018 at 4:51 PM Vitalii Diravka wrote: > > > Hi Reed, > > > > It looks like a bug. Could you please create a jira ticket with an above > > description? > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apa

Re: Drill “VALIDATION ERROR: A table or view with given name already exists in schema” for empty directory

Vitalii On Thu, Dec 6, 2018 at 5:17 PM Vitalii Diravka wrote: > @Khurram Thank you for pointing out the Jira ticket. It differs and it is > no more the issue. I have resolved the ticket. > > @Reed I looked to your case and it looks like it is expected > behavior. Empty directory

Re: Running drill in distributed mode

Hi! You can run drillbit on one machine and configure FileSystem Storage Plugin to specify "Connection" to the remote fs [1]. Do you need to run drillbit on every machine and coordinate them via Zookeeper? Not sure that it is efficient for local file system. [1]

Hangout Discussion Topics

Hi All, Does anyone have any topics to discuss during the hangout tomorrow? Kind regards Vitalii

Re: Problem when using files with differents schemas in the same SELECT

Hi Benj, This is a known issue with a column data type without values, by default it is INT:OPTIONAL (for your last query) and when the meaningful data is came that INT:OPTIONAL is treated to the new datatype. It was discussed very frequently in the different topics. Paul raised this topic in

Re: Drill Hangout tomorrow 08/21

had in Drill. > > > Further metadata discussion will be continued in the next hangout. > > -Hanu > > On Tue, Aug 21, 2018 at 9:53 AM Vitalii Diravka > > wrote: > > > Hi Alex, > > > > The issues pointed by you really exist. And using of HMS is

Re: Long running query succeeds but UI times out?

Hi James, This is the mail for user mailing list. There is no attachment, please upload it to Google Drive, for instance, and give us the link. Did you try to use Drill SqlLine? Kind regards Vitalii On Sat, Sep 15, 2018 at 7:45 PM James Barney wrote: > Hey, > I've had pretty great success

Re: [VOTE] Apache Drill release 1.15.0 - RC0

> drwxrwxr-x vitalii/vitalii 0 2018-12-18 03:48 > > apache-drill-1.15.0-src/${project.*basedir*}/src/site/resources/repo/ > > > > On Tue, Dec 18, 2018 at 10:58 AM Vitalii Diravka > > wrote: > > > > > Hi all, > > > > > > I'd like to p

Re: Drill and OData

Hi Kevin! It is feasible. Any Drill Storage or Format Plugin can be as an example for you. Somewhere I saw initial work for Drill REST Plugin. Earlier Charles Givre posted an email about REST plugin. I will add info from that mail here. Charles could share your findings in this area with Kevin?

Re: Import drill sources in eclipse

Hi Angelo, Welcome to Drill community. Most of Drill devs use Intellij Idea. When I used eclipse for Drill project, I haven't this issue. But looks like there are several ways how to solve it: * You can compile the project externally and the import it into Eclipse. * You can fix it in Drill root

Re: Cyrillic names for views

Hi Nick, Looks like Cyrillic View name works fine for me: 0: jdbc:drill:zk=local> create view МояВьюха as select employee_id, full_name from cp.`employee.json` limit 2; +---+---+ | ok | summary

Re: January Apache Drill board report

+1 Kind regards Vitalii On Thu, Jan 31, 2019 at 6:18 PM Aman Sinha wrote: > Thanks for putting this together, Arina. > The Drill Developer Day and Meetup were separate events, so you can split > them up. > - A half day Drill Developer Day was held on Nov 14. A variety of > technical design

Re: [DISCUSS] Format plugins in contrib module

Absolutely agree with Arina. I think the core Format Plugins for Parquet, Json and CSV, TSV, PSV files (which are used for creating Drill tables) can be left in current config file and the rest ones should be factored out to the separate config files along with creating separate modules in Drill

Re: [VOTE] Apache Drill Release 1.16.0 - RC0

Arina/Anton thanks for catching DRILL-7186 Sorabh, agree with you DRILL-7190 should be resolved and included to 1.16.0 Drill release. I will provide PR shortly. Kind regards Vitalii On Fri, Apr 19, 2019 at 8:05 PM SorabhApache wrote: > Hi Anton/Arina, > Thanks for finding the issue and

Re: [ANNOUNCE] New Committer: Anton Gozhyi

My congratulations Anton! You deserved it! Kind regards Vitalii On Mon, Jul 29, 2019 at 1:13 PM Arina Ielchiieva wrote: > Congratulations Anton! Thanks for your contributions. > > Kind regards, > Arina > > On Mon, Jul 29, 2019 at 12:55 PM Павел Семенов > > wrote: > > > Congratulations Anton

Re: [ANNOUNCE] New Committer: Bohdan Kazydub

Congrats Bohdan! Well deserved! Kind regards Vitalii On Mon, Jul 15, 2019 at 6:48 PM Paul Rogers wrote: > Congrats Bohdan! > - Paul > > > > On Monday, July 15, 2019, 01:08:04 AM PDT, Arina Ielchiieva < > ar...@apache.org> wrote: > > The Project Management Committee (PMC) for Apache Drill

Re: [ANNOUNCE] New Committer: Denys Ordynskiy

Congrats Denys! Glad to know you became a committer! Happy New Year :) Kind regards Vitalii On Fri, Jan 3, 2020 at 10:11 AM Ellen Friedman wrote: > Great way to start 2020! Congratulations Denys and thank you for all your > contributions to Drill. > > Best wishes, > Ellen > > On Mon, Dec 30,

[DISCUSSION] Roles and Privileges, Security, Secrets

Hi Dev and User, Drill has a very important feature - Roles and Privileges [1], but it has really weak functionality. There are only two roles (admin and user) and admin can't really give any user permissions to set query options for all their sessions or to allow configure storage plugin in

Re: [DISCUSSION] Roles and Privileges, Security, Secrets

> Is there a reason that most or all of this couldn't be handled by storing > the configuration in files? That would allow file permissions to naturally > allow or disallow these operations. > > Also, what are the specific goals here? > > > > On Wed, Jan 20, 2021 a

Re: Slow query execution in Drill Embedded

Hello Jonathan, Did you try to execute both logical and physical plans? Does the query involve any aggregation, join or sort operators? Could you provide the query here, please? Did you try to exclude some fields from the query (to identify what fields cause the delay). Kind regards Vitalii On

Re: Json Schma change error

Hi Prabhakar Bhosaale. Could you provide the test case with a data source to reproduce? I am going to try it with a new JSON loader Kind regards Vitalii On Sun, Sep 26, 2021 at 4:12 PM Prabhakar Bhosaale wrote: > Thanks Luoc. will check on it. > > > Regards > Prabhakar > > On Sun, Sep 26,