Drill Query Problem

Hello, I am getting below error while running big query. === Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]', Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large

Is it possible to configure a drillbit to use disk when necessary?

Hi, I was trying to dump into a parquet file data contained in a very big MySQL table. After setting the store.format to parquet and the compression to snappy, I used a query like: Create table dfs.tmp.`file.parquet` as (select ... from mysq.schema.table); The problem that I found is that the

Re: Is it possible to configure a drillbit to use disk when necessary?

Try looking into the the "sort.external.spill.directories" and "sort.external.spill.fs" settings under drill.exec. These "may" help with that memory pressure, but I am not an expert. John On Fri, Dec 4, 2015 at 7:22 AM, Daniel Garcia < daniel.gar...@eurotaxglass.com> wrote: > Hi, > > > > I was

Re: Announcing new committer: Kristine Hahn

Congratulations and welcome, Kris! > On Dec 4, 2015, at 9:19 AM, Jacques Nadeau wrote: > > The Apache Drill PMC is very pleased to announce Kristine Hahn as a new > committer. > > Kris has worked tirelessly on creating and improving the Drill > documentation. She has been

Announcing new committer: Kristine Hahn

The Apache Drill PMC is very pleased to announce Kristine Hahn as a new committer. Kris has worked tirelessly on creating and improving the Drill documentation. She has been extraordinary in her engagement with the community and has greatly accelerated the speed to resolution of doc issues and

Re: Drill Query Problem

Hi Nirav, can you give us more information to help reproduce this issue ? thanks On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah wrote: > Hello, > > I am getting below error while running big query. > > === > > Error: SYSTEM ERROR:

Re: Announcing new committer: Kristine Hahn

Congratulations Kristine :) On Fri, Dec 4, 2015 at 9:43 AM, Abdel Hakim Deneche wrote: > Congrats Kristine :D > > On Fri, Dec 4, 2015 at 9:36 AM, Sudheesh Katkam > wrote: > > > Congratulations and welcome, Kris! > > > > > On Dec 4, 2015, at 9:19 AM,

Re: Announcing new committer: Kristine Hahn

Congratulations Kris! On Fri, Dec 4, 2015 at 9:46 AM, Jason Altekruse wrote: > Congrats Kris! Well deserved, the docs are looking great! > > On Fri, Dec 4, 2015 at 9:36 AM, Sudheesh Katkam > wrote: > > > Congratulations and welcome, Kris! > > > >

Re: Announcing new committer: Kristine Hahn

Congrats Kris! Well deserved, the docs are looking great! On Fri, Dec 4, 2015 at 9:36 AM, Sudheesh Katkam wrote: > Congratulations and welcome, Kris! > > > On Dec 4, 2015, at 9:19 AM, Jacques Nadeau wrote: > > > > The Apache Drill PMC is very pleased

Re: Is it possible to configure a drillbit to use disk when necessary?

Disk spill is not used for CTAS operations directly, unless the CTAS statement includes aggregations, sorts and some join operations. I'm not familiar with the JDBC plugin, but when creating tables with parquet Drill direct memory should be the primary memory not the heap memory. The GC

Re: Drill Query Problem

If JDK is not available, according to [1], there would be a warning in drillbit.log. [1] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/compile/JDKClassCompiler.java#L47 On Fri, Dec 4, 2015 at 11:11 AM, Jason Altekruse

Re: Drill Query Problem

I think we might want to do more than just log in this case. Here enabling the option is a good suggestion for possibly fixing the issue, but if a JDK is not installed it looks like there will now just be a warning logged and the same error will be produced. It would probably be worth it to just

Re: CSV Reader on 1.3

Added that it's available in Drill 1.4 and later: http://drill.apache.org/docs/plugin-configuration-basics/#using-the-formats-attributes-as-table-function-parameters Thanks, Bridget On Thu, Dec 3, 2015 at 9:55 PM, Jacques Nadeau wrote: > One note, this feature is in the

Re: CSV Reader on 1.3

Thanks for the fix Bridget. I just took a look at the posted version an noticed that the example query is missing some necessary backticks Julien had tried to include an escaped backtick in his PR, but it doesn't look like this is the right way to include a backtick in this type of text. The

Re: CSV Reader on 1.3

Thanks for pointing that out, Jason. Oversight on my part. I escaped the backticks so they appear around the file path: http://drill.apache.org/docs/plugin-configuration-basics/#using-the-formats-attributes-as-table-function-parameters The code text in the MD file looks like this: ``select a, b

Re: Drill Query Problem

I'm guessing this is due to an extremely large number of columns, complicated expressions or extremely long column names. We test up to a fairly large size here [1] but you may have exceeded what we test. Can you share the query? [1]

Re: Drill Query Problem

You may consider switching the java_compiler from DEFALT to JDK, by : alter session set `exec.java_compiler` = 'JDK'; All Drill will automatically switch fro janino compiler to JDK, when the source code length is beyond certain limit. If your query happens to have smaller code than that limit,

Re: Drill Query Problem

Do we have something that checks a JDK is installed when launching Drill? Based on this JIRA it looks like we will fall back to janino if we cannot find a JDK. It is possible that this option does nothing if a JDK is not available, or it may fail with an error. We should test this out in various