hadu created PARQUET-1066:
-----------------------------
Summary: GC overhead limit exceeded while run a hive task with
parquet format hdfs file
Key: PARQUET-1066
URL: https://issues.apache.org/jira/browse/PARQUET-1066
Project: Parquet
Issue Type: Bug
Reporter: hadu
I run a very simple hive sql task on hive console client, the original table
tmp_adi_kl_pm_crowd_dev has two partitions, which the total size is over 220GB。
I have tried so many times, but finally end with the GC overhead limit exceeded
error like this:
hive> dfs -du -h /user/da/da_hivesrc/haitao/haitao_open/tmp_adi_kl_pm_crowd_dev;
186.5 G
/user/da/da_hivesrc/haitao/haitao_open/tmp_adi_kl_pm_crowd_dev/day=1970-01-02
39.0 G
/user/da/da_hivesrc/haitao/haitao_open/tmp_adi_kl_pm_crowd_dev/day=2017-07-20
hive> select
> user_id,
> concat_ws(',',collect_set(crowd_id))
> from
> haitao_open.tmp_adi_kl_pm_crowd_dev
> where
> (day='2017-07-20' or day='1970-01-02')
> group by user_id;
Query ID = da_20170724095641_d52f8717-d1c0-4445-863a-7669090e9781
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 300
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:55 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:56 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:57 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:58 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:56:59 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:00 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:01 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:02 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:03 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:14 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:15 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:16 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:17 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:18 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:19 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:20 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:21 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:22 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 24, 2017 9:57:23 AM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Jul 24, 2java.lang.OutOfMemoryError: GC overhead limit exceeded
at
org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:595)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:855)
at
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:877)
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1097)
at org.apache.hadoop.hdfs.DFSClient$Conf.<init>(DFSClient.java:310)
at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:157)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:602)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:547)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:139)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:365)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:97)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:242)
at
org.apache.hadoop.hive.shims.Hadoop23Shims$1.listStatus(Hadoop23Shims.java:146)
at
org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:217)
at
org.apache.hadoop.mapred.lib.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:75)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:309)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getCombineSplits(CombineHiveInputFormat.java:470)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:571)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:520)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:512)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
FAILED: Execution Error, return code -101 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask. GC overhead limit exceeded
someone can help to solve the problem?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
