Hi all,
I am guessing that each of your EMR nodes are quite large? EMR nodes are:
r4.2xlarge ('vcpu': 8, 'memory': 61)
Property "planner.width.max_per_node" is set to = 6
What is the system memory and what are the allocations for heap and direct?
System Memory: 61GB (EMR nodes above)
drill_mem_heap: 12G
drill_mem_max: 48G
The view is simple: SELECT * FROM s3://myparquet.parquet (14GB)
planner.memory.max_query_memory_per_node = 10479720202
Drillbit.log attached (I think I have the correct selection included).
Thanks
On Fri, Jan 26, 2018 at 2:41 PM, Kunal Khatua <[email protected]> wrote:
> What is the system memory and what are the allocations for heap and
> direct? The memory crash might be occurring due to insufficient heap. The
> limits parameter applies to the direct memory and not Heap.
>
> Can you share details in the logs from the crash?
>
> -----Original Message-----
> From: Timothy Farkas [mailto:[email protected]]
> Sent: Thursday, January 25, 2018 2:58 PM
> To: [email protected]
> Subject: Re: Creating a Tableau extracts with Drill 1.12 uses unlimited
> memory
>
> Hi Francis,
>
> I am guessing that each of your EMR nodes are quite large (32 or 64
> vcpus). On large machines Drill's planner over parallelizes and over
> allocates memory. There is a property "planner.width.max_per_node" which
> limits the number of operators that can simultaneously execute on a
> Drillbit for a query. If you configure the width per node to something like
> 5 or 10 (you may have to play around with it) things should start working.
>
> Thanks,
> Tim
>
> ________________________________
> From: Francis McGregor-Macdonald <[email protected]>
> Sent: Thursday, January 25, 2018 1:58:22 PM
> To: [email protected]
> Subject: Creating a Tableau extracts with Drill 1.12 uses unlimited memory
>
> Creating a creating a Tableau (with 10.3, 10.5 desktop) extract from a
> Drill (1.12 on EMR) cluster memory appears not to adhere to the limits set
> by planner.memory.max_query_memory_per_node.
>
> The extract query consumes all memory and then crashes drill.
>
> Running the same query as a create table memory behaves as expected.
>
> The query complexity is trivial:
> select * from view only a single parquet with no calculated fields.
>
> Has anyone else observed this behavior?
>
>
2018-01-26 05:58:37,904 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query id
25953c71-9933-e3ad-680b-d65363d46f0d: SELECT 1 AS `Number of Records`,
`nielsen_cl_1`.`dat_fact` AS `dat_fact`,
`nielsen_cl_1`.`dat_market` AS `dat_market`,
`nielsen_cl_1`.`dat_period` AS `dat_period`,
`nielsen_cl_1`.`dat_product` AS `dat_product`,
`nielsen_cl_1`.`dat_value` AS `dat_value`,
`nielsen_cl_1`.`dataset` AS `dataset`,
`nielsen_cl_1`.`mar_ccy` AS `mar_ccy`,
`nielsen_cl_1`.`mar_country` AS `mar_country`,
`nielsen_cl_1`.`mar_long_desc` AS `mar_long_desc`,
`nielsen_cl_1`.`mar_sequence` AS `mar_sequence`,
`nielsen_cl_1`.`mar_short_desc` AS `mar_short_desc`,
`nielsen_cl_1`.`mar_tag` AS `mar_tag`,
`nielsen_cl_1`.`per_list` AS `per_list`,
`nielsen_cl_1`.`per_long_desc` AS `per_long_desc`,
`nielsen_cl_1`.`per_sequence` AS `per_sequence`,
`nielsen_cl_1`.`per_short_desc` AS `per_short_desc`,
`nielsen_cl_1`.`per_tag` AS `per_tag`,
`nielsen_cl_1`.`pro_aroma/fragancia/sabor` AS `pro_aroma/fragancia/sabor`,
`nielsen_cl_1`.`pro_atributo` AS `pro_atributo`,
`nielsen_cl_1`.`pro_barcode` AS `pro_barcode`,
`nielsen_cl_1`.`pro_categoria` AS `pro_categoria`,
`nielsen_cl_1`.`pro_category` AS `pro_category`,
`nielsen_cl_1`.`pro_char08` AS `pro_char08`,
`nielsen_cl_1`.`pro_char10` AS `pro_char10`,
`nielsen_cl_1`.`pro_char16` AS `pro_char16`,
`nielsen_cl_1`.`pro_char17` AS `pro_char17`,
`nielsen_cl_1`.`pro_color` AS `pro_color`,
`nielsen_cl_1`.`pro_consistencia` AS `pro_consistencia`,
`nielsen_cl_1`.`pro_empaque/envase` AS `pro_empaque/envase`,
`nielsen_cl_1`.`pro_empaque` AS `pro_empaque`,
`nielsen_cl_1`.`pro_envasado/granel` AS `pro_envasado/granel`,
`nielsen_cl_1`.`pro_envase` AS `pro_envase`,
`nielsen_cl_1`.`pro_fabricante` AS `pro_fabricante`,
`nielsen_cl_1`.`pro_formato` AS `pro_formato`,
`nielsen_cl_1`.`pro_gasificacion` AS `pro_gasificacion`,
`nielsen_cl_1`.`pro_item` AS `pro_item`,
`nielsen_cl_1`.`pro_level` AS `pro_level`,
`nielsen_cl_1`.`pro_long_desc` AS `pro_long_desc`,
`nielsen_cl_1`.`pro_marca` AS `pro_marca`,
`nielsen_cl_1`.`pro_marcas` AS `pro_marcas`,
`nielsen_cl_1`.`pro_natural/sabor` AS `pro_natural/sabor`,
`nielsen_cl_1`.`pro_rangos` AS `pro_rangos`,
`nielsen_cl_1`.`pro_reg/diet` AS `pro_reg/diet`,
`nielsen_cl_1`.`pro_regular/light` AS `pro_regular/light`,
`nielsen_cl_1`.`pro_regular_/_light_-_diet` AS `pro_regular_/_light_-_diet`,
`nielsen_cl_1`.`pro_sabor` AS `pro_sabor`,
`nielsen_cl_1`.`pro_sal` AS `pro_sal`,
`nielsen_cl_1`.`pro_segmento` AS `pro_segmento`,
`nielsen_cl_1`.`pro_segmento_1` AS `pro_segmento_1`,
`nielsen_cl_1`.`pro_segmentos` AS `pro_segmentos`,
`nielsen_cl_1`.`pro_sequence` AS `pro_sequence`,
`nielsen_cl_1`.`pro_short_desc` AS `pro_short_desc`,
`nielsen_cl_1`.`pro_submarca` AS `pro_submarca`,
`nielsen_cl_1`.`pro_submarcas` AS `pro_submarcas`,
`nielsen_cl_1`.`pro_tag` AS `pro_tag`,
`nielsen_cl_1`.`pro_tamano` AS `pro_tamano`,
`nielsen_cl_1`.`pro_tipo` AS `pro_tipo`,
`nielsen_cl_1`.`pro_tipo_ii` AS `pro_tipo_ii`,
`nielsen_cl_1`.`pro_total_fam_e_ind` AS `pro_total_fam_e_ind`,
`nielsen_cl_1`.`pro_variedad` AS `pro_variedad`,
`nielsen_cl_1`.`pro_variedad_2` AS `pro_variedad_2`,
`nielsen_cl_1`.`publisher` AS `publisher`,
`nielsen_cl_1`.`version` AS `version`
FROM `s3-mycustomer-data.atomic`.`nielsen_cl_1` `nielsen_cl_1`
2018-01-26 05:58:41,703 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:41,760 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Took 44 ms to read directories from
directory cache file
2018-01-26 05:58:42,644 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - No directories were modified. Took 883 ms
to check modification time of 11 directories
2018-01-26 05:58:42,644 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.createFromDirectories() took
0 ms
2018-01-26 05:58:42,737 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:42,777 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:42,897 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:42,897 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:43,380 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 125
2018-01-26 05:58:43,380 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 125
2018-01-26 05:58:43,380 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 125
2018-01-26 05:58:43,380 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 125
2018-01-26 05:58:43,380 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 125
2018-01-26 05:58:43,380 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 125
2018-01-26 05:58:43,380 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 125
2018-01-26 05:58:43,474 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:43,512 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:43,632 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:43,632 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:44,291 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:44,291 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:44,291 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:44,291 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:44,291 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:44,291 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:44,291 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:44,387 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:44,424 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:44,546 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:44,547 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:45,625 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:45,625 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:45,625 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:45,625 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:45,625 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:45,625 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:45,625 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:45,727 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:45,764 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:45,935 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:45,935 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:46,628 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:46,628 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:46,628 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:46,628 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:46,628 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:46,628 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:46,628 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:46,734 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:46,772 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:46,901 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:46,901 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:47,973 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:47,973 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:47,973 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:47,973 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:47,973 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:47,973 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:47,973 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:48,073 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:48,108 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:48,228 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:48,229 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:49,360 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:49,360 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:49,360 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:49,360 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:49,361 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:49,361 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:49,361 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:49,464 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:49,504 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:49,630 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:49,630 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:50,664 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:50,664 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:50,664 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:50,664 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:50,664 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:50,664 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:50,664 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 70
2018-01-26 05:58:50,804 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:50,838 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:50,962 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:50,962 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:51,661 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:51,661 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:51,661 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:51,661 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:51,661 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:51,661 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:51,661 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 75
2018-01-26 05:58:51,759 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:51,793 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:51,910 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:51,910 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:52,510 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:52,510 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:52,510 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:52,510 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:52,510 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:52,510 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:52,510 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 100
2018-01-26 05:58:52,610 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:52,704 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:52,825 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:52,825 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:53,868 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 111
2018-01-26 05:58:53,868 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 111
2018-01-26 05:58:53,868 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 111
2018-01-26 05:58:53,868 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 111
2018-01-26 05:58:53,868 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 111
2018-01-26 05:58:53,868 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 111
2018-01-26 05:58:53,868 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 111
2018-01-26 05:58:53,971 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:54,013 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:54,136 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:54,136 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-01-26 05:58:55,250 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 98
2018-01-26 05:58:55,250 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 98
2018-01-26 05:58:55,250 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 98
2018-01-26 05:58:55,250 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 98
2018-01-26 05:58:55,250 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 98
2018-01-26 05:58:55,250 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 98
2018-01-26 05:58:55,250 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 98
2018-01-26 05:58:55,267 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:58:55,268 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:58:55,268 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:58:55,805 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
536 ms within batchIndex: 0 with # of partitions : 4
2018-01-26 05:58:55,805 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 4 partitions down to 1
2018-01-26 05:58:55,805 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 537 ms
2018-01-26 05:58:55,805 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:58:55,805 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:58:55,806 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:58:56,852 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
1046 ms within batchIndex: 0 with # of partitions : 10
2018-01-26 05:58:56,852 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 10 partitions down to 2
2018-01-26 05:58:56,853 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 1047 ms
2018-01-26 05:58:56,853 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:58:56,853 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:58:56,853 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:58:57,908 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
1053 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:58:57,908 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:58:57,908 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 1054 ms
2018-01-26 05:58:57,908 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:58:57,908 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:58:57,909 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:58:58,476 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
567 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:58:58,476 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:58:58,476 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 568 ms
2018-01-26 05:58:58,477 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:58:58,477 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:58:58,477 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:58:59,189 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
711 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:58:59,189 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:58:59,189 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 712 ms
2018-01-26 05:58:59,190 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:58:59,190 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:58:59,190 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:58:59,821 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
630 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:58:59,821 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:58:59,821 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 631 ms
2018-01-26 05:58:59,822 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:58:59,822 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:58:59,822 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:59:00,407 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
584 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:59:00,407 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:59:00,408 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 586 ms
2018-01-26 05:59:00,409 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:59:00,409 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:59:00,409 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:59:00,993 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
584 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:59:00,993 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:59:00,994 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 584 ms
2018-01-26 05:59:00,994 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:59:00,994 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:59:00,995 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:59:01,644 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
649 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:59:01,644 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:59:01,645 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 650 ms
2018-01-26 05:59:01,645 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:59:01,645 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:59:01,645 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:59:02,408 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
761 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:59:02,408 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:59:02,408 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 763 ms
2018-01-26 05:59:02,408 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:59:02,408 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:59:02,409 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:59:04,154 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
1745 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:59:04,154 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:59:04,154 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 1746 ms
2018-01-26 05:59:04,155 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pruning, pruning
class:
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-01-26 05:59:04,155 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total elapsed time to build and analyze
filter tree: 0 ms
2018-01-26 05:59:04,155 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time to populate partitioning
column vectors: 0 ms within batchIndex: 0
2018-01-26 05:59:04,731 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Elapsed time in interpreter evaluation:
575 ms within batchIndex: 0 with # of partitions : 5
2018-01-26 05:59:04,731 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Pruned 5 partitions down to 1
2018-01-26 05:59:04,731 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.e.p.l.partition.PruneScanRule - Total pruning elapsed time: 576 ms
2018-01-26 05:59:05,966 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 42 out of
42 using 16 threads. Time: 551ms total, 169.348213ms avg, 549ms max.
2018-01-26 05:59:05,966 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 42 out of
42 using 16 threads. Earliest start: 164.659000 ?s, Latest start: 356660.893000
?s, Average start: 148623.283833 ?s .
2018-01-26 05:59:06,819 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 31 out of
31 using 16 threads. Time: 429ms total, 173.783424ms avg, 298ms max.
2018-01-26 05:59:06,819 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 31 out of
31 using 16 threads. Earliest start: 189.389000 ?s, Latest start: 270288.711000
?s, Average start: 86323.202613 ?s .
2018-01-26 05:59:07,771 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 20 out of
20 using 16 threads. Time: 567ms total, 237.829148ms avg, 566ms max.
2018-01-26 05:59:07,772 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 20 out of
20 using 16 threads. Earliest start: 193.612000 ?s, Latest start: 194597.912000
?s, Average start: 34668.430500 ?s .
2018-01-26 05:59:08,232 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 15 out of
15 using 15 threads. Time: 217ms total, 178.978427ms avg, 216ms max.
2018-01-26 05:59:08,232 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 15 out of
15 using 15 threads. Earliest start: 165.297000 ?s, Latest start: 1153.126000
?s, Average start: 621.814267 ?s .
2018-01-26 05:59:08,916 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 14 out of
14 using 14 threads. Time: 383ms total, 141.045754ms avg, 382ms max.
2018-01-26 05:59:08,916 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 14 out of
14 using 14 threads. Earliest start: 168.772000 ?s, Latest start: 1013.144000
?s, Average start: 605.558429 ?s .
2018-01-26 05:59:10,564 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 20 out of
20 using 16 threads. Time: 335ms total, 150.231104ms avg, 334ms max.
2018-01-26 05:59:10,564 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 20 out of
20 using 16 threads. Earliest start: 183.286000 ?s, Latest start: 147405.498000
?s, Average start: 27566.986700 ?s .
2018-01-26 05:59:11,000 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 15 out of
15 using 15 threads. Time: 191ms total, 134.639427ms avg, 190ms max.
2018-01-26 05:59:11,000 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 15 out of
15 using 15 threads. Earliest start: 176.329000 ?s, Latest start: 1097.354000
?s, Average start: 613.923533 ?s .
2018-01-26 05:59:11,466 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 14 out of
14 using 14 threads. Time: 229ms total, 155.019690ms avg, 228ms max.
2018-01-26 05:59:11,466 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 14 out of
14 using 14 threads. Earliest start: 185.812000 ?s, Latest start: 1004.909000
?s, Average start: 588.068143 ?s .
2018-01-26 05:59:12,027 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 20 out of
20 using 16 threads. Time: 252ms total, 157.420970ms avg, 229ms max.
2018-01-26 05:59:12,027 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 20 out of
20 using 16 threads. Earliest start: 181.736000 ?s, Latest start: 147288.552000
?s, Average start: 27262.380300 ?s .
2018-01-26 05:59:12,424 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 15 out of
15 using 15 threads. Time: 160ms total, 118.440523ms avg, 159ms max.
2018-01-26 05:59:12,424 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 15 out of
15 using 15 threads. Earliest start: 185.062000 ?s, Latest start: 1136.499000
?s, Average start: 649.409800 ?s .
2018-01-26 05:59:14,194 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 83 out of
83 using 16 threads. Time: 670ms total, 110.131512ms avg, 403ms max.
2018-01-26 05:59:14,195 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 83 out of
83 using 16 threads. Earliest start: 176.280000 ?s, Latest start: 506285.708000
?s, Average start: 261875.661193 ?s .
2018-01-26 05:59:14,291 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Took 80 ms to read metadata from cache file
2018-01-26 05:59:14,791 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 498 ms,
numFiles: 2
2018-01-26 05:59:14,791 [25953c71-9933-e3ad-680b-d65363d46f0d:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 2
2018-01-26 05:59:17,593 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:9] INFO
o.a.d.e.w.fragment.FragmentExecutor - 25953c71-9933-e3ad-680b-d65363d46f0d:1:9:
State change requested AWAITING_ALLOCATION --> RUNNING
2018-01-26 05:59:17,593 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:9] INFO
o.a.d.e.w.f.FragmentStatusReporter - 25953c71-9933-e3ad-680b-d65363d46f0d:1:9:
State to report: RUNNING
2018-01-26 05:59:18,234 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:1] INFO
o.a.d.e.w.fragment.FragmentExecutor - 25953c71-9933-e3ad-680b-d65363d46f0d:1:1:
State change requested AWAITING_ALLOCATION --> RUNNING
2018-01-26 05:59:18,234 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:1] INFO
o.a.d.e.w.f.FragmentStatusReporter - 25953c71-9933-e3ad-680b-d65363d46f0d:1:1:
State to report: RUNNING
2018-01-26 05:59:18,239 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:13] INFO
o.a.d.e.w.fragment.FragmentExecutor -
25953c71-9933-e3ad-680b-d65363d46f0d:1:13: State change requested
AWAITING_ALLOCATION --> RUNNING
2018-01-26 05:59:18,239 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:13] INFO
o.a.d.e.w.f.FragmentStatusReporter - 25953c71-9933-e3ad-680b-d65363d46f0d:1:13:
State to report: RUNNING
2018-01-26 05:59:18,932 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:5] INFO
o.a.d.e.w.fragment.FragmentExecutor - 25953c71-9933-e3ad-680b-d65363d46f0d:1:5:
State change requested AWAITING_ALLOCATION --> RUNNING
2018-01-26 05:59:18,932 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:5] INFO
o.a.d.e.w.f.FragmentStatusReporter - 25953c71-9933-e3ad-680b-d65363d46f0d:1:5:
State to report: RUNNING
2018-01-26 05:59:23,829 [25953c99-7e37-fc27-98d1-2cb87b038dfc:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 25953c99-7e37-fc27-98d1-2cb87b038dfc:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2018-01-26 05:59:23,829 [25953c99-7e37-fc27-98d1-2cb87b038dfc:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 25953c99-7e37-fc27-98d1-2cb87b038dfc:0:0:
State to report: RUNNING
2018-01-26 05:59:23,829 [25953c99-7e37-fc27-98d1-2cb87b038dfc:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 25953c99-7e37-fc27-98d1-2cb87b038dfc:0:0:
State change requested RUNNING --> CANCELLATION_REQUESTED
2018-01-26 05:59:23,829 [25953c99-7e37-fc27-98d1-2cb87b038dfc:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 25953c99-7e37-fc27-98d1-2cb87b038dfc:0:0:
State to report: CANCELLATION_REQUESTED
2018-01-26 05:59:23,829 [25953c99-7e37-fc27-98d1-2cb87b038dfc:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 25953c99-7e37-fc27-98d1-2cb87b038dfc:0:0:
State change requested CANCELLATION_REQUESTED --> FINISHED
2018-01-26 05:59:23,829 [25953c99-7e37-fc27-98d1-2cb87b038dfc:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 25953c99-7e37-fc27-98d1-2cb87b038dfc:0:0:
State to report: CANCELLED
2018-01-26 05:59:41,005 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:13] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:41,006 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:13] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`dipro_levelr0`, returning null instance.
2018-01-26 05:59:41,619 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:9] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:41,620 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:9] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`dipro_levelr0`, returning null instance.
2018-01-26 05:59:41,664 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:5] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:41,664 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:5] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`dipro_levelr0`, returning null instance.
2018-01-26 05:59:42,469 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:1] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:42,470 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:1] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`dipro_levelr0`, returning null instance.
2018-01-26 05:59:43,416 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:13] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:44,124 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:9] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:44,467 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:5] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:45,331 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:1] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:45,795 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:13] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:46,329 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:9] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:47,161 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:5] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:48,262 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:1] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:48,635 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:13] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:49,418 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:9] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:50,219 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:5] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:51,400 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:1] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`pro_regular_/_light_-_diet`, returning null instance.
2018-01-26 05:59:54,207 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 25953c71-9933-e3ad-680b-d65363d46f0d:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2018-01-26 05:59:54,207 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 25953c71-9933-e3ad-680b-d65363d46f0d:0:0:
State to report: RUNNING
2018-01-26 05:59:56,650 [25953c71-9933-e3ad-680b-d65363d46f0d:frag:1:13] INFO
o.a.d.e.c.ClassCompilerSelector - Java compiler policy: DEFAULT, Debug option:
true
Fri Jan 26 06:04:31 UTC 2018 Starting drillbit on ip-172-30-0-146
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 245565
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 245565
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
2018-01-26 06:04:32,292 [main] INFO o.a.drill.exec.util.GuavaPatcher -
Google's Stopwatch patched for old HBase Guava version.
2018-01-26 06:04:32,304 [main] INFO o.a.drill.exec.util.GuavaPatcher -
Google's Closeables patched for old HBase Guava version.
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: zookeeper.version=3.4.6-1569965, built on 02/20/2014
09:09 GMT
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: host.name=ip-172-30-0-146.ap-southeast-2.compute.internal
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: java.version=1.8.0_151
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: java.vendor=Oracle Corporation
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment:
java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-1.b12.35.amzn1.x86_64/jre
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment:
java.class.path=/opt/drill/conf:/opt/drill/jars/drill-storage-hbase-1.12.0.jar:/opt/drill/jars/drill-storage-kafka-1.12.0.jar:/opt/drill/jars/drill-hive-exec-shaded-1.12.0.jar:/opt/drill/jars/drill-memory-base-1.12.0.jar:/opt/drill/jars/vector-1.12.0.jar:/opt/drill/jars/drill-gis-1.12.0.jar:/opt/drill/jars/drill-rpc-1.12.0.jar:/opt/drill/jars/drill-jdbc-1.12.0.jar:/opt/drill/jars/drill-logical-1.12.0.jar:/opt/drill/jars/drill-storage-hive-core-1.12.0.jar:/opt/drill/jars/drill-mongo-storage-1.12.0.jar:/opt/drill/jars/drill-jdbc-storage-1.12.0.jar:/opt/drill/jars/drill-kudu-storage-1.12.0.jar:/opt/drill/jars/drill-common-1.12.0.jar:/opt/drill/jars/drill-protocol-1.12.0.jar:/opt/drill/jars/drill-java-exec-1.12.0.jar:/opt/drill/jars/tpch-sample-data-1.12.0.jar:/opt/drill/jars/ext/zookeeper-3.4.6.jar:/opt/drill/jars/3rdparty/parquet-format-2.3.0-incubating.jar:/opt/drill/jars/3rdparty/hadoop-auth-2.7.1.jar:/opt/drill/jars/3rdparty/parquet-generator-1.8.1-drill-r0.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.logback.appender.multiplex-classic-0.9.44.jar:/opt/drill/jars/3rdparty/httpclient-4.2.5.jar:/opt/drill/jars/3rdparty/hbase-protocol-1.1.3.jar:/opt/drill/jars/3rdparty/bcprov-jdk15on-1.52.jar:/opt/drill/jars/3rdparty/objenesis-1.0.jar:/opt/drill/jars/3rdparty/protostuff-json-1.0.8.jar:/opt/drill/jars/3rdparty/velocity-1.7.jar:/opt/drill/jars/3rdparty/paranamer-2.5.6.jar:/opt/drill/jars/3rdparty/kerb-client-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/parquet-common-1.8.1-drill-r0.jar:/opt/drill/jars/3rdparty/hadoop-hdfs-2.7.1.jar:/opt/drill/jars/3rdparty/kerb-admin-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/commons-codec-1.10.jar:/opt/drill/jars/3rdparty/kerb-identity-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/api-util-1.0.0-M20.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.logback.converter-classic-0.9.44.jar:/opt/drill/jars/3rdparty/mongo-java-driver-3.2.0.jar:/opt/drill/jars/3rdparty/json-simple-1.1.1.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.logback.classic-0.9.44.jar:/opt/drill/jars/3rdparty/parquet-column-1.8.1-drill-r0.jar:/opt/drill/jars/3rdparty/eigenbase-properties-1.1.5.jar:/opt/drill/jars/3rdparty/jetty-http-9.1.5.v20140505.jar:/opt/drill/jars/3rdparty/curator-recipes-2.7.1.jar:/opt/drill/jars/3rdparty/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:/opt/drill/jars/3rdparty/parquet-encoding-1.8.1-drill-r0.jar:/opt/drill/jars/3rdparty/xalan-2.7.1.jar:/opt/drill/jars/3rdparty/htrace-core-3.1.0-incubating.jar:/opt/drill/jars/3rdparty/parquet-hadoop-1.8.1-drill-r0.jar:/opt/drill/jars/3rdparty/hbase-common-1.1.3.jar:/opt/drill/jars/3rdparty/commons-pool2-2.1.jar:/opt/drill/jars/3rdparty/serializer-2.7.1.jar:/opt/drill/jars/3rdparty/curator-client-2.7.1.jar:/opt/drill/jars/3rdparty/dom4j-1.6.1.jar:/opt/drill/jars/3rdparty/jackson-annotations-2.7.9.jar:/opt/drill/jars/3rdparty/metrics-core-3.0.1.jar:/opt/drill/jars/3rdparty/hbase-hadoop-compat-1.1.3.jar:/opt/drill/jars/3rdparty/okhttp-3.3.0.jar:/opt/drill/jars/3rdparty/javassist-3.16.1-GA.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.data.logging-0.9.44.jar:/opt/drill/jars/3rdparty/netty-buffer-4.0.48.Final.jar:/opt/drill/jars/3rdparty/slf4j-api-1.7.6.jar:/opt/drill/jars/3rdparty/xz-1.0.jar:/opt/drill/jars/3rdparty/hbase-annotations-1.1.3.jar:/opt/drill/jars/3rdparty/httpdlog-parser-2.4.jar:/opt/drill/jars/3rdparty/validation-api-1.1.0.Final.jar:/opt/drill/jars/3rdparty/calcite-linq4j-1.4.0-drill-r23.jar:/opt/drill/jars/3rdparty/kafka-clients-0.11.0.1.jar:/opt/drill/jars/3rdparty/commons-beanutils-1.8.3.jar:/opt/drill/jars/3rdparty/sqljdbc42.jar:/opt/drill/jars/3rdparty/xercesImpl-2.11.0.jar:/opt/drill/jars/3rdparty/kerb-common-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/parquet-jackson-1.8.1-drill-r0.jar:/opt/drill/jars/3rdparty/jackson-mapper-asl-1.9.13.jar:/opt/drill/jars/3rdparty/antlr-2.7.7.jar:/opt/drill/jars/3rdparty/netty-transport-4.0.48.Final.jar:/opt/drill/jars/3rdparty/commons-compiler-2.7.6.jar:/opt/drill/jars/3rdparty/datanucleus-rdbms-3.2.9.jar:/opt/drill/jars/3rdparty/mockito-core-1.9.5.jar:/opt/drill/jars/3rdparty/java-sizeof_2.11-0.1.jar:/opt/drill/jars/3rdparty/jul-to-slf4j-1.7.6.jar:/opt/drill/jars/3rdparty/retrofit-2.1.0.jar:/opt/drill/jars/3rdparty/jetty-servlets-9.1.5.v20140505.jar:/opt/drill/jars/3rdparty/config-1.0.0.jar:/opt/drill/jars/3rdparty/xml-apis-1.4.01.jar:/opt/drill/jars/3rdparty/jsr305-3.0.0.jar:/opt/drill/jars/3rdparty/jsch-0.1.42.jar:/opt/drill/jars/3rdparty/jackson-databind-2.7.9.1.jar:/opt/drill/jars/3rdparty/stringtemplate-3.2.1.jar:/opt/drill/jars/3rdparty/jdo-api-3.0.1.jar:/opt/drill/jars/3rdparty/de.huxhorn.sulky.codec-0.9.17.jar:/opt/drill/jars/3rdparty/netty-codec-4.0.48.Final.jar:/opt/drill/jars/3rdparty/jetty-continuation-9.1.1.v20140108.jar:/opt/drill/jars/3rdparty/guava-18.0.jar:/opt/drill/jars/3rdparty/freemarker-2.3.26-incubating.jar:/opt/drill/jars/3rdparty/kerb-crypto-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/metrics-jvm-3.0.1.jar:/opt/drill/jars/3rdparty/httpcore-4.2.4.jar:/opt/drill/jars/3rdparty/jetty-xml-9.1.1.v20140108.jar:/opt/drill/jars/3rdparty/gson-2.2.4.jar:/opt/drill/jars/3rdparty/hadoop-yarn-client-2.7.1.jar:/opt/drill/jars/3rdparty/kafka_2.11-0.11.0.1.jar:/opt/drill/jars/3rdparty/metrics-json-3.0.1.jar:/opt/drill/jars/3rdparty/commons-configuration-1.6.jar:/opt/drill/jars/3rdparty/jline-2.10.jar:/opt/drill/jars/3rdparty/janino-2.7.6.jar:/opt/drill/jars/3rdparty/leveldbjni-all-1.8.jar:/opt/drill/jars/3rdparty/commons-beanutils-core-1.8.0.jar:/opt/drill/jars/3rdparty/hppc-0.7.1.jar:/opt/drill/jars/3rdparty/hadoop-yarn-common-2.7.1.jar:/opt/drill/jars/3rdparty/okio-1.8.0.jar:/opt/drill/jars/3rdparty/netty-transport-native-epoll-4.0.48.Final-linux-x86_64.jar:/opt/drill/jars/3rdparty/findbugs-annotations-1.3.9-1.jar:/opt/drill/jars/3rdparty/apacheds-i18n-2.0.0-M15.jar:/opt/drill/jars/3rdparty/protostuff-api-1.0.8.jar:/opt/drill/jars/3rdparty/curator-x-discovery-2.7.1.jar:/opt/drill/jars/3rdparty/protobuf-java-2.5.0.jar:/opt/drill/jars/3rdparty/commons-digester-1.8.1.jar:/opt/drill/jars/3rdparty/jetty-security-9.1.5.v20140505.jar:/opt/drill/jars/3rdparty/sqlline-1.1.9-drill-r7.jar:/opt/drill/jars/3rdparty/hamcrest-core-1.3.jar:/opt/drill/jars/3rdparty/hbase-common-1.1.3-tests.jar:/opt/drill/jars/3rdparty/jcodings-1.0.8.jar:/opt/drill/jars/3rdparty/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/drill/jars/3rdparty/interface-annotations-1.3.0.jar:/opt/drill/jars/3rdparty/hadoop-mapreduce-client-common-2.7.1.jar:/opt/drill/jars/3rdparty/libthrift-0.9.2.jar:/opt/drill/jars/3rdparty/libfb303-0.9.2.jar:/opt/drill/jars/3rdparty/datanucleus-api-jdo-3.2.6.jar:/opt/drill/jars/3rdparty/commons-math-2.2.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.logback.appender.multiplex-core-0.9.44.jar:/opt/drill/jars/3rdparty/commons-pool-1.5.4.jar:/opt/drill/jars/3rdparty/netty-handler-4.0.48.Final.jar:/opt/drill/jars/3rdparty/jna-4.5.0.jar:/opt/drill/jars/3rdparty/esri-geometry-api-2.0.0.jar:/opt/drill/jars/3rdparty/univocity-parsers-1.3.0.jar:/opt/drill/jars/3rdparty/hadoop-mapreduce-client-shuffle-2.7.1.jar:/opt/drill/jars/3rdparty/asm-debug-all-5.0.3.jar:/opt/drill/jars/3rdparty/commons-cli-1.2.jar:/opt/drill/jars/3rdparty/jcl-over-slf4j-1.7.6.jar:/opt/drill/jars/3rdparty/derby-10.10.2.0.jar:/opt/drill/jars/3rdparty/calcite-avatica-1.4.0-drill-r23.jar:/opt/drill/jars/3rdparty/jackson-core-asl-1.9.13.jar:/opt/drill/jars/3rdparty/foodmart-data-json-0.4.jar:/opt/drill/jars/3rdparty/bonecp-0.8.0.RELEASE.jar:/opt/drill/jars/3rdparty/metrics-healthchecks-3.0.1.jar:/opt/drill/jars/3rdparty/javax.inject-1.jar:/opt/drill/jars/3rdparty/commons-net-3.6.jar:/opt/drill/jars/3rdparty/jsp-api-2.1.jar:/opt/drill/jars/3rdparty/calcite-core-1.4.0-drill-r23.jar:/opt/drill/jars/3rdparty/api-asn1-api-1.0.0-M20.jar:/opt/drill/jars/3rdparty/stax-api-1.0-2.jar:/opt/drill/jars/3rdparty/jopt-simple-5.0.3.jar:/opt/drill/jars/3rdparty/commons-io-2.4.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.data.eventsource-0.9.44.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.data.converter-0.9.44.jar:/opt/drill/jars/3rdparty/jetty-util-9.1.5.v20140505.jar:/opt/drill/jars/3rdparty/avro-mapred-1.7.7.jar:/opt/drill/jars/3rdparty/kerby-asn1-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/jetty-webapp-9.1.1.v20140108.jar:/opt/drill/jars/3rdparty/commons-collections-3.2.1.jar:/opt/drill/jars/3rdparty/commons-math3-3.1.1.jar:/opt/drill/jars/3rdparty/scala-library-2.11.0.jar:/opt/drill/jars/3rdparty/kerb-util-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/joda-time-2.9.jar:/opt/drill/jars/3rdparty/commons-dbcp-1.4.jar:/opt/drill/jars/3rdparty/jcommander-1.30.jar:/opt/drill/jars/3rdparty/hadoop-mapreduce-client-core-2.7.1.jar:/opt/drill/jars/3rdparty/converter-jackson-2.1.0.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.data.logging.protobuf-0.9.44.jar:/opt/drill/jars/3rdparty/hadoop-aws-2.7.1.jar:/opt/drill/jars/3rdparty/avro-ipc-1.7.7-tests.jar:/opt/drill/jars/3rdparty/jackson-module-afterburner-2.7.9.jar:/opt/drill/jars/3rdparty/jackson-core-2.7.9.jar:/opt/drill/jars/3rdparty/de.huxhorn.sulky.io-0.9.17.jar:/opt/drill/jars/3rdparty/datanucleus-core-3.2.10.jar:/opt/drill/jars/3rdparty/aws-java-sdk-1.7.4.jar:/opt/drill/jars/3rdparty/de.huxhorn.sulky.formatting-0.9.17.jar:/opt/drill/jars/3rdparty/jackson-jaxrs-base-2.7.9.jar:/opt/drill/jars/3rdparty/jackson-jaxrs-json-provider-2.7.9.jar:/opt/drill/jars/3rdparty/joni-2.1.2.jar:/opt/drill/jars/3rdparty/kerby-config-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/jta-1.1.jar:/opt/drill/jars/3rdparty/hive-contrib-1.2.1.jar:/opt/drill/jars/3rdparty/jackson-xc-1.9.13.jar:/opt/drill/jars/3rdparty/antlr-runtime-3.4.jar:/opt/drill/jars/3rdparty/hbase-prefix-tree-1.1.3.jar:/opt/drill/jars/3rdparty/avro-ipc-1.7.7.jar:/opt/drill/jars/3rdparty/kudu-client-1.3.0.jar:/opt/drill/jars/3rdparty/bcpkix-jdk15on-1.52.jar:/opt/drill/jars/3rdparty/netty-common-4.0.48.Final.jar:/opt/drill/jars/3rdparty/jackson-jaxrs-1.9.13.jar:/opt/drill/jars/3rdparty/parser-core-2.4.jar:/opt/drill/jars/3rdparty/hive-hbase-handler-1.2.1.jar:/opt/drill/jars/3rdparty/kerb-simplekdc-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/log4j-over-slf4j-1.7.6.jar:/opt/drill/jars/3rdparty/zkclient-0.10.jar:/opt/drill/jars/3rdparty/hadoop-client-2.7.1.jar:/opt/drill/jars/3rdparty/hadoop-mapreduce-client-app-2.7.1.jar:/opt/drill/jars/3rdparty/avro-1.7.7.jar:/opt/drill/jars/3rdparty/libpam4j-1.8-rev1.jar:/opt/drill/jars/3rdparty/protostuff-core-1.0.8.jar:/opt/drill/jars/3rdparty/jpam-1.1.jar:/opt/drill/jars/3rdparty/metrics-core-2.2.0.jar:/opt/drill/jars/3rdparty/curator-framework-2.7.1.jar:/opt/drill/jars/3rdparty/disruptor-3.3.0.jar:/opt/drill/jars/3rdparty/kerby-util-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/jetty-servlet-9.1.5.v20140505.jar:/opt/drill/jars/3rdparty/hbase-hadoop2-compat-1.1.3.jar:/opt/drill/jars/3rdparty/hadoop-annotations-2.7.1.jar:/opt/drill/jars/3rdparty/async-1.4.1.jar:/opt/drill/jars/3rdparty/commons-validator-1.4.1.jar:/opt/drill/jars/3rdparty/hadoop-mapreduce-client-jobclient-2.7.1.jar:/opt/drill/jars/3rdparty/kerb-server-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/lz4-1.3.0.jar:/opt/drill/jars/3rdparty/jetty-server-9.1.5.v20140505.jar:/opt/drill/jars/3rdparty/hbase-procedure-1.1.3.jar:/opt/drill/jars/3rdparty/commons-lang3-3.1.jar:/opt/drill/jars/3rdparty/hadoop-yarn-server-common-2.7.1.jar:/opt/drill/jars/3rdparty/xmlenc-0.52.jar:/opt/drill/jars/3rdparty/scala-parser-combinators_2.11-1.0.4.jar:/opt/drill/jars/3rdparty/hbase-server-1.1.3.jar:/opt/drill/jars/3rdparty/snappy-java-1.1.2.6.jar:/opt/drill/jars/3rdparty/metrics-servlets-3.0.1.jar:/opt/drill/jars/3rdparty/msgpack-0.6.6.jar:/opt/drill/jars/3rdparty/jackson-module-jaxb-annotations-2.7.9.jar:/opt/drill/jars/3rdparty/kerby-pkix-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/hbase-client-1.1.3.jar:/opt/drill/jars/3rdparty/commons-lang-2.6.jar:/opt/drill/jars/3rdparty/hive-metastore-1.2.1.jar:/opt/drill/jars/3rdparty/de.huxhorn.lilith.sender-0.9.44.jar:/opt/drill/jars/3rdparty/commons-compress-1.4.1.jar:/opt/drill/jars/3rdparty/commons-httpclient-3.1.jar:/opt/drill/jars/3rdparty/kerb-core-1.0.0-RC2.jar:/opt/drill/jars/3rdparty/hadoop-common-2.7.1.jar:/opt/drill/jars/3rdparty/jetty-io-9.1.5.v20140505.jar:/opt/drill/jars/classb/jersey-container-jetty-servlet-2.8.jar:/opt/drill/jars/classb/javax.inject-2.2.0.jar:/opt/drill/jars/classb/logback-classic-1.0.13.jar:/opt/drill/jars/classb/jersey-server-2.8.jar:/opt/drill/jars/classb/jersey-mvc-freemarker-2.8.jar:/opt/drill/jars/classb/jersey-guava-2.8.jar:/opt/drill/jars/classb/reflections-0.9.8.jar:/opt/drill/jars/classb/jersey-client-2.8.jar:/opt/drill/jars/classb/jersey-media-json-jackson-2.8.jar:/opt/drill/jars/classb/jetty-util-6.1.26.jar:/opt/drill/jars/classb/jersey-mvc-2.8.jar:/opt/drill/jars/classb/hk2-locator-2.2.0.jar:/opt/drill/jars/classb/mimepull-1.9.3.jar:/opt/drill/jars/classb/javax.servlet-api-3.1.0.jar:/opt/drill/jars/classb/codemodel-2.6.jar:/opt/drill/jars/classb/jersey-common-2.8.jar:/opt/drill/jars/classb/jaxb-api-2.2.2.jar:/opt/drill/jars/classb/hk2-api-2.2.0.jar:/opt/drill/jars/classb/jetty-6.1.26.jar:/opt/drill/jars/classb/jersey-container-servlet-core-2.8.jar:/opt/drill/jars/classb/aopalliance-repackaged-2.2.0.jar:/opt/drill/jars/classb/javax.annotation-api-1.2.jar:/opt/drill/jars/classb/jersey-container-jetty-http-2.8.jar:/opt/drill/jars/classb/activation-1.1.jar:/opt/drill/jars/classb/hk2-utils-2.2.0.jar:/opt/drill/jars/classb/jersey-container-servlet-2.8.jar:/opt/drill/jars/classb/jersey-media-multipart-2.8.jar:/opt/drill/jars/classb/logback-core-1.0.13.jar:/opt/drill/jars/classb/javax.ws.rs-api-2.0.jar:/opt/drill/jars/classb/osgi-resource-locator-1.0.1.jar:/opt/drill/jars/3rdparty/linux/netty-tcnative-2.0.1.Final-linux-x86_64.jar
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment:
java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: java.io.tmpdir=/tmp
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: java.compiler=<NA>
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: os.name=Linux
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: os.arch=amd64
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: os.version=4.9.51-10.52.amzn1.x86_64
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: user.name=root
2018-01-26 06:04:32,310 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: user.home=/root
2018-01-26 06:04:32,311 [main] INFO o.apache.drill.exec.server.Drillbit -
Drillbit environment: user.dir=/mnt/home/hadoop
2018-01-26 06:04:32,490 [main] INFO o.a.drill.common.config.DrillConfig -
Configuration and plugin file(s) identified in 140ms.
Base Configuration:
- jar:file:/opt/drill/jars/drill-common-1.12.0.jar!/drill-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
-
jar:file:/opt/drill/jars/drill-storage-kafka-1.12.0.jar!/drill-module.conf
-
jar:file:/opt/drill/jars/drill-memory-base-1.12.0.jar!/drill-module.conf
-
jar:file:/opt/drill/jars/drill-kudu-storage-1.12.0.jar!/drill-module.conf
- jar:file:/opt/drill/jars/drill-common-1.12.0.jar!/drill-module.conf
-
jar:file:/opt/drill/jars/drill-hive-exec-shaded-1.12.0.jar!/drill-module.conf
-
jar:file:/opt/drill/jars/drill-storage-hive-core-1.12.0.jar!/drill-module.conf
- jar:file:/opt/drill/jars/drill-java-exec-1.12.0.jar!/drill-module.conf
-
jar:file:/opt/drill/jars/drill-mongo-storage-1.12.0.jar!/drill-module.conf
-
jar:file:/opt/drill/jars/drill-jdbc-storage-1.12.0.jar!/drill-module.conf
- jar:file:/opt/drill/jars/drill-logical-1.12.0.jar!/drill-module.conf
- jar:file:/opt/drill/jars/drill-gis-1.12.0.jar!/drill-module.conf
-
jar:file:/opt/drill/jars/drill-storage-hbase-1.12.0.jar!/drill-module.conf
Override File: file:/opt/drill/conf/drill-override.conf
2018-01-26 06:04:33,008 [main] INFO o.a.d.common.scanner.BuildTimeScan -
Loaded prescanned packages [org.apache.drill.exec.store.mock,
org.apache.drill.common.logical, org.apache.drill.storage,
org.apache.drill.exec.store.mock, org.apache.drill.common.logical,
org.apache.drill.exec.expr, org.apache.drill.exec.physical,
org.apache.drill.exec.store, org.apache.drill.exec.rpc.user.security,
org.apache.drill.exec.rpc.security, org.apache.drill.storage,
org.apache.drill.exec.store.mock, org.apache.drill.common.logical] from
locations
[jar:file:/opt/drill/jars/drill-common-1.12.0.jar!/META-INF/drill-module-scan/registry.json,
jar:file:/opt/drill/jars/drill-logical-1.12.0.jar!/META-INF/drill-module-scan/registry.json,
jar:file:/opt/drill/jars/drill-java-exec-1.12.0.jar!/META-INF/drill-module-scan/registry.json]
2018-01-26 06:04:33,836 [main] INFO o.a.d.c.scanner.ClassPathScanner -
Scanning packages [org.apache.drill.exec.store.kafka,
org.apache.drill.exec.store.kudu, org.apache.drill.exec.store.mock,
org.apache.drill.common.logical, org.apache.hadoop.hive,
org.apache.hadoop.hive, org.apache.drill.exec.fn.hive,
org.apache.drill.exec.expr, org.apache.drill.exec.physical,
org.apache.drill.exec.store, org.apache.drill.exec.rpc.user.security,
org.apache.drill.exec.rpc.security, org.apache.drill.exec.store.mongo,
org.apache.drill.exec.store.jdbc, org.apache.drill.exec.store.hbase,
org.apache.drill.exec.expr.fn.impl.conv] in locations
[jar:file:/opt/drill/jars/drill-jdbc-storage-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-storage-hive-core-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-storage-hbase-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-hive-exec-shaded-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-gis-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-storage-kafka-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-memory-base-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-mongo-storage-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-kudu-storage-1.12.0.jar!/] took 824ms
2018-01-26 06:04:34,591 [main] INFO o.a.d.e.s.s.PersistentStoreRegistry -
Using the configured PStoreProvider class:
'org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider'.
2018-01-26 06:04:34,758 [main] INFO o.apache.drill.exec.server.Drillbit - Upto
1000 latest query profiles will be retained in-memory
2018-01-26 06:04:34,881 [main] INFO o.apache.drill.exec.server.Drillbit -
Construction completed (1041 ms).
2018-01-26 06:04:35,076 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
16 classes for org.apache.drill.common.logical.data.LogicalOperator took 5ms
2018-01-26 06:04:35,081 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
11 classes for org.apache.drill.common.logical.StoragePluginConfig took 4ms
2018-01-26 06:04:35,089 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 6ms
2018-01-26 06:04:35,188 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
66 classes for org.apache.drill.exec.physical.base.PhysicalOperator took 56ms
2018-01-26 06:04:35,257 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
38 classes for org.apache.drill.exec.physical.impl.BatchCreator took 27ms
2018-01-26 06:04:35,262 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
5 classes for org.apache.drill.exec.physical.impl.RootCreator took 3ms
2018-01-26 06:04:35,618 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
1 classes for org.apache.drill.exec.expr.fn.PluggableFunctionRegistry took 4ms
2018-01-26 06:04:36,123 [main] INFO o.a.d.c.scanner.ClassPathScanner -
Scanning packages [org.apache.drill.exec.store.kafka,
org.apache.drill.exec.store.kudu, org.apache.drill.exec.store.mock,
org.apache.drill.common.logical, org.apache.hadoop.hive,
org.apache.hadoop.hive, org.apache.drill.exec.fn.hive,
org.apache.drill.exec.expr, org.apache.drill.exec.physical,
org.apache.drill.exec.store, org.apache.drill.exec.rpc.user.security,
org.apache.drill.exec.rpc.security, org.apache.drill.exec.store.mongo,
org.apache.drill.exec.store.jdbc, org.apache.drill.exec.store.hbase,
org.apache.drill.exec.expr.fn.impl.conv] in locations
[jar:file:/opt/drill/jars/drill-jdbc-storage-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-storage-hive-core-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-storage-hbase-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-hive-exec-shaded-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-gis-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-storage-kafka-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-memory-base-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-mongo-storage-1.12.0.jar!/,
jar:file:/opt/drill/jars/drill-kudu-storage-1.12.0.jar!/] took 504ms
2018-01-26 06:04:36,181 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
114 classes for org.apache.hadoop.hive.ql.udf.generic.GenericUDF took 56ms
2018-01-26 06:04:36,496 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
68 classes for org.apache.hadoop.hive.ql.exec.UDF took 190ms
2018-01-26 06:04:36,511 [main] INFO o.a.d.e.e.f.FunctionImplementationRegistry
- Function registry loaded. 459 functions loaded in 1240 ms.
2018-01-26 06:04:36,519 [main] INFO o.a.d.e.e.f.FunctionImplementationRegistry
- Created and validated local udf directory [/tmp/drill/udf/udf/local]
2018-01-26 06:04:36,524 [main] INFO o.a.drill.exec.compile.CodeCompiler -
Plain java code generation preferred: false
2018-01-26 06:04:36,694 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
11 classes for org.apache.drill.exec.store.StoragePlugin took 20ms
2018-01-26 06:04:38,861 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
7 classes for org.apache.drill.exec.store.dfs.FormatPlugin took 27ms
2018-01-26 06:04:38,875 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,941 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
7 classes for org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2018-01-26 06:04:38,943 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,944 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,944 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,959 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
7 classes for org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2018-01-26 06:04:38,960 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,960 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,961 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,961 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,976 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
7 classes for org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2018-01-26 06:04:38,976 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,977 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,977 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:38,977 [main] INFO o.a.d.c.s.persistence.ScanResult - loading
8 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2018-01-26 06:04:39,011 [main] INFO o.a.d.e.e.f.r.RemoteFunctionRegistry -
Preparing three remote udf areas: staging, registry and tmp.
2018-01-26 06:04:39,036 [main] INFO o.a.d.e.e.f.r.RemoteFunctionRegistry -
Created remote udf area [/root/drill/udf/registry] on file system [file:///]
2018-01-26 06:04:39,038 [main] INFO o.a.d.e.e.f.r.RemoteFunctionRegistry -
Created remote udf area [/root/drill/udf/staging] on file system [file:///]
2018-01-26 06:04:39,040 [main] INFO o.a.d.e.e.f.r.RemoteFunctionRegistry -
Created remote udf area [/root/drill/udf/tmp] on file system [file:///]
2018-01-26 06:04:39,293 [main] INFO o.a.drill.exec.server.rest.WebServer -
Setting up HTTP connector for web server
2018-01-26 06:04:39,990 [main] INFO o.apache.drill.exec.server.Drillbit -
Startup completed (5108 ms).
