Has anyone seen a failure as shown below my signature? I have seen it with
Drill 1.3.0 and 1.2.0. The DECIMAL values are encoded in big-endian byte
order, as converted from java.math.BigInteger.
Dave Oshinsky
Commvault Systems
Here is a successful query on a UTF8 field in the same parquet file:
0: jdbc:drill:zk=local> select count(*) from
dfs.`c:/dao/DBArchivePredictor/acct_mstr.parquet` where opr_mode='JO';
+-----------+
| EXPR$0 |
+-----------+
| 37050847 |
+-----------+
1 row selected (9.934 seconds)
0: jdbc:drill:zk=local>
Here is a query that fails, on a DECIMAL field acct_no:
0: jdbc:drill:zk=local> select count(*) from
dfs.`c:/dao/DBArchivePredictor/acct_mstr.parquet` where acct_no=70000020;
........
Error: SYSTEM ERROR: ClassCastException: org.apache.drill.exec.vector.Decimal28S
parseVector cannot be cast to org.apache.drill.exec.vector.VariableWidthVector
Fragment 1:0
[Error Id: bfde2a91-2c86-48b3-b188-7a7bf3d8e6ac on DaveOshinsky-PC.gp.cv.commvau
lt.com:31010] (state=,code=0)
0: jdbc:drill:zk=local>
The output is quite lengthy from the failure, including the following:
org.apache.drill.common.exceptions.DrillRuntimeException: Error in parquet recor
d reader.
Message: Failure in setting up reader
Parquet Metadata: ParquetMetaData{FileMetaData{schema: message my_table {
required binary ACCT_NO (DECIMAL(20,0));
optional binary SF_NO (UTF8);
optional binary LF_NO (UTF8);
optional binary BRANCH_NO (DECIMAL(20,0));
optional binary INTRO_CUST_NO (DECIMAL(20,0));
optional binary INTRO_ACCT_NO (DECIMAL(20,0));
optional binary INTRO_SIGN (UTF8);
optional binary TYPE (UTF8);
optional binary OPR_MODE (UTF8);
optional binary CUR_ACCT_TYPE (UTF8);
optional binary TITLE (UTF8);
optional binary CORP_CUST_NO (DECIMAL(20,0));
optional binary APLNDT (UTF8);
optional binary OPNDT (UTF8);
optional binary VERI_EMP_NO (DECIMAL(20,0));
optional binary VERI_SIGN (UTF8);
optional binary MANAGER_SIGN (UTF8);
optional binary CURBAL (DECIMAL(8,2));
optional binary STATUS (UTF8);
}
......
blocks: [BlockMetaData{12710100, 1108111133 [ColumnMetaData{SNAPPY [ACCT_NO]
BINA
RY [BIT_PACKED, PLAIN], 4}, ColumnMetaData{SNAPPY [SF_NO] BINARY [RLE, BIT_PAC
KED, PLAIN_DICTIONARY], 56990517}, ColumnMetaData{SNAPPY [LF_NO] BINARY [RLE, B
IT_PACKED, PLAIN_DICTIONARY], 57125602}, ColumnMetaData{SNAPPY [BRANCH_NO] BINAR
Y [RLE, BIT_PACKED, PLAIN_DICTIONARY], 57233332}, ColumnMetaData{SNAPPY [INTRO_
CUST_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN], 57281915}, ColumnMe
taData{SNAPPY [INTRO_ACCT_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 89933
829}, ColumnMetaData{SNAPPY [INTRO_SIGN] BINARY [RLE, BIT_PACKED, PLAIN_DICTION
ARY], 110250177}, ColumnMetaData{SNAPPY [TYPE] BINARY [RLE, BIT_PACKED, PLAIN_D
ICTIONARY], 110291904}, ColumnMetaData{SNAPPY [OPR_MODE] BINARY [RLE, BIT_PACKE
D, PLAIN_DICTIONARY], 110346542}, ColumnMetaData{SNAPPY [CUR_ACCT_TYPE] BINARY
[RLE, BIT_PACKED, PLAIN_DICTIONARY], 110406593}, ColumnMetaData{SNAPPY [TITLE] B
INARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 110467110}, ColumnMetaData{SNAPPY [C
ORP_CUST_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN], 110628878}, Col
umnMetaData{SNAPPY [APLNDT] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN],
143290986}, ColumnMetaData{SNAPPY [OPNDT] BINARY [RLE, BIT_PACKED, PLAIN_DICTIO
NARY, PLAIN], 158585878}, ColumnMetaData{SNAPPY [VERI_EMP_NO] BINARY [RLE, BIT_
PACKED, PLAIN_DICTIONARY, PLAIN], 173880774}, ColumnMetaData{SNAPPY [VERI_SIGN]
BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 205251396}, ColumnMetaData{SNAPPY [
MANAGER_SIGN] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 205294552}, ColumnMet
aData{SNAPPY [CURBAL] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN], 205337
708}, ColumnMetaData{SNAPPY [STATUS] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY]
, 268041364}]}, BlockMetaData{16740100, 877627388 [ColumnMetaData{SNAPPY [ACCT_N
O] BINARY [BIT_PACKED, PLAIN], 268084308}, ColumnMetaData{SNAPPY [SF_NO] BINARY
[RLE, BIT_PACKED, PLAIN_DICTIONARY], 339879714}, ColumnMetaData{SNAPPY [LF_NO]
BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 340048465}, ColumnMetaData{SNAPPY
[BRANCH_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 340178908}, ColumnMetaD
ata{SNAPPY [INTRO_CUST_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 34023528
0}, ColumnMetaData{SNAPPY [INTRO_ACCT_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIO
NARY], 361297427}, ColumnMetaData{SNAPPY [INTRO_SIGN] BINARY [RLE, BIT_PACKED,
PLAIN_DICTIONARY], 382437967}, ColumnMetaData{SNAPPY [TYPE] BINARY [RLE, BIT_PA
CKED, PLAIN_DICTIONARY], 382492898}, ColumnMetaData{SNAPPY [OPR_MODE] BINARY [R
LE, BIT_PACKED, PLAIN_DICTIONARY], 382563576}, ColumnMetaData{SNAPPY [CUR_ACCT_T
YPE] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 382632643}, ColumnMetaData{SNA
PPY [TITLE] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 382703353}, ColumnMetaD
ata{SNAPPY [CORP_CUST_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 383347416
}, ColumnMetaData{SNAPPY [APLNDT] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PL
AIN], 404409563}, ColumnMetaData{SNAPPY [OPNDT] BINARY [RLE, BIT_PACKED, PLAIN_
DICTIONARY, PLAIN], 418195496}, ColumnMetaData{SNAPPY [VERI_EMP_NO] BINARY [RLE
, BIT_PACKED, PLAIN_DICTIONARY], 431981127}, ColumnMetaData{SNAPPY [VERI_SIGN] B
INARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 451668116}, ColumnMetaData{SNAPPY [M
ANAGER_SIGN] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 451723163}, ColumnMeta
Data{SNAPPY [CURBAL] BINARY [RLE, BIT_PACKED, PLAIN], 451779553}, ColumnMetaDat
a{SNAPPY [STATUS] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 535811598}]}, Blo
ckMetaData{15390100, 1010393203 [ColumnMetaData{SNAPPY [ACCT_NO] BINARY [BIT_PA
CKED, PLAIN], 535866529}, ColumnMetaData{SNAPPY [SF_NO] BINARY [RLE, BIT_PACKED
, PLAIN_DICTIONARY], 603471354}, ColumnMetaData{SNAPPY [LF_NO] BINARY [RLE, BIT
_PACKED, PLAIN_DICTIONARY], 603636501}, ColumnMetaData{SNAPPY [BRANCH_NO] BINARY
[RLE, BIT_PACKED, PLAIN_DICTIONARY], 603756535}, ColumnMetaData{SNAPPY [INTRO_
CUST_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 603807077}, ColumnMetaData
{SNAPPY [INTRO_ACCT_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 623171740},
ColumnMetaData{SNAPPY [INTRO_SIGN] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY],
642536380}, ColumnMetaData{SNAPPY [TYPE] BINARY [RLE, BIT_PACKED, PLAIN_DICTIO
NARY], 642586922}, ColumnMetaData{SNAPPY [OPR_MODE] BINARY [RLE, BIT_PACKED, PL
AIN_DICTIONARY], 642652002}, ColumnMetaData{SNAPPY [CUR_ACCT_TYPE] BINARY [RLE,
BIT_PACKED, PLAIN_DICTIONARY], 642717093}, ColumnMetaData{SNAPPY [TITLE] BINARY
[RLE, BIT_PACKED, PLAIN_DICTIONARY], 642782179}, ColumnMetaData{SNAPPY [CORP_C
UST_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 651988784}, ColumnMetaData{
SNAPPY [APLNDT] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN], 671353447},
ColumnMetaData{SNAPPY [OPNDT] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN]
, 689248030}, ColumnMetaData{SNAPPY [VERI_EMP_NO] BINARY [RLE, BIT_PACKED, PLAI
N_DICTIONARY], 707142273}, ColumnMetaData{SNAPPY [VERI_SIGN] BINARY [RLE, BIT_P
ACKED, PLAIN_DICTIONARY], 726506776}, ColumnMetaData{SNAPPY [MANAGER_SIGN] BINAR
Y [RLE, BIT_PACKED, PLAIN_DICTIONARY], 726558408}, ColumnMetaData{SNAPPY [CURBA
L] BINARY [RLE, BIT_PACKED, PLAIN], 726610040}, ColumnMetaData{SNAPPY [STATUS]
BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 803868149}]}, BlockMetaData{1487010
0, 997915332 [ColumnMetaData{SNAPPY [ACCT_NO] BINARY [BIT_PACKED, PLAIN], 80391
9875}, ColumnMetaData{SNAPPY [SF_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY]
, 869264876}, ColumnMetaData{SNAPPY [LF_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICT
IONARY], 869417004}, ColumnMetaData{SNAPPY [BRANCH_NO] BINARY [RLE, BIT_PACKED,
PLAIN_DICTIONARY], 869532661}, ColumnMetaData{SNAPPY [INTRO_CUST_NO] BINARY [R
LE, BIT_PACKED, PLAIN_DICTIONARY], 876919008}, ColumnMetaData{SNAPPY [INTRO_ACCT
_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 895628500}, ColumnMetaData{SNA
PPY [INTRO_SIGN] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 915616293}, Column
MetaData{SNAPPY [TYPE] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 915665431},
ColumnMetaData{SNAPPY [OPR_MODE] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 91
5728109}, ColumnMetaData{SNAPPY [CUR_ACCT_TYPE] BINARY [RLE, BIT_PACKED, PLAIN_
DICTIONARY], 915790793}, ColumnMetaData{SNAPPY [TITLE] BINARY [RLE, BIT_PACKED,
PLAIN_DICTIONARY], 915853477}, ColumnMetaData{SNAPPY [CORP_CUST_NO] BINARY [RL
E, BIT_PACKED, PLAIN_DICTIONARY], 924595366}, ColumnMetaData{SNAPPY [APLNDT] BIN
ARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN], 943304858}, ColumnMetaData{SNAP
PY [OPNDT] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN], 960868676}, Colum
nMetaData{SNAPPY [VERI_EMP_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 9784
32494}, ColumnMetaData{SNAPPY [VERI_SIGN] BINARY [RLE, BIT_PACKED, PLAIN_DICTIO
NARY], 997142542}, ColumnMetaData{SNAPPY [MANAGER_SIGN] BINARY [RLE, BIT_PACKED
, PLAIN_DICTIONARY], 997192537}, ColumnMetaData{SNAPPY [CURBAL] BINARY [RLE, BI
T_PACKED, PLAIN], 997242440}, ColumnMetaData{SNAPPY [STATUS] BINARY [RLE, BIT_P
ACKED, PLAIN_DICTIONARY], 1071891224}]}, BlockMetaData{7154452, 447096168 [Colum
nMetaData{SNAPPY [ACCT_NO] BINARY [BIT_PACKED, PLAIN], 1071941219}, ColumnMetaD
ata{SNAPPY [SF_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 1103384569}, Col
umnMetaData{SNAPPY [LF_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 11034553
36}, ColumnMetaData{SNAPPY [BRANCH_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONAR
Y], 1103509961}, ColumnMetaData{SNAPPY [INTRO_CUST_NO] BINARY [RLE, BIT_PACKED,
PLAIN_DICTIONARY], 1107598352}, ColumnMetaData{SNAPPY [INTRO_ACCT_NO] BINARY [
RLE, BIT_PACKED, PLAIN_DICTIONARY], 1116601592}, ColumnMetaData{SNAPPY [INTRO_SI
GN] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 1125605021}, ColumnMetaData{SNA
PPY [TYPE] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 1125628519}, ColumnMetaD
ata{SNAPPY [OPR_MODE] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 1125658015},
ColumnMetaData{SNAPPY [CUR_ACCT_TYPE] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY
], 1125687511}, ColumnMetaData{SNAPPY [TITLE] BINARY [RLE, BIT_PACKED, PLAIN_DI
CTIONARY], 1125717007}, ColumnMetaData{SNAPPY [CORP_CUST_NO] BINARY [RLE, BIT_P
ACKED, PLAIN_DICTIONARY], 1130010136}, ColumnMetaData{SNAPPY [APLNDT] BINARY [R
LE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN], 1139013376}, ColumnMetaData{SNAPPY [OP
NDT] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY, PLAIN], 1146672810}, ColumnMeta
Data{SNAPPY [VERI_EMP_NO] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY], 115433224
4}, ColumnMetaData{SNAPPY [VERI_SIGN] BINARY [RLE, BIT_PACKED, PLAIN_DICTIONARY
], 1163335628}, ColumnMetaData{SNAPPY [MANAGER_SIGN] BINARY [RLE, BIT_PACKED, P
LAIN_DICTIONARY], 1163359126}, ColumnMetaData{SNAPPY [CURBAL] BINARY [RLE, BIT_
PACKED, PLAIN], 1163382624}, ColumnMetaData{SNAPPY [STATUS] BINARY [RLE, BIT_PA
CKED, PLAIN_DICTIONARY], 1199297307}]}]}
at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader
.handleAndRaise(ParquetRecordReader.java:346)
at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader
.setup(ParquetRecordReader.java:339)
at org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:1
01)
at org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(
ParquetScanBatchCreator.java:168)
at org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(
ParquetScanBatchCreator.java:56)
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCr
eator.java:151)
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreat
or.java:174)
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCr
eator.java:131)
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreat
or.java:174)
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCr
eator.java:131)
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreat
or.java:174)
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCr
eator.java:131)
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreat
or.java:174)
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCr
eator.java:131)
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreat
or.java:174)
at org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreat
or.java:105)
at org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.j
ava:79)
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExec
utor.java:230)
at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable
.java:38)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassCastException: org.apache.drill.exec.vector.Decimal28S
parseVector cannot be cast to org.apache.drill.exec.vector.VariableWidthVector
at org.apache.drill.exec.store.parquet.columnreaders.VarLengthValuesColu
mn.<init>(VarLengthValuesColumn.java:44)
at org.apache.drill.exec.store.parquet.columnreaders.VarLengthColumnRead
ers$Decimal28Column.<init>(VarLengthColumnReaders.java:52)
at org.apache.drill.exec.store.parquet.columnreaders.ColumnReaderFactory
.getReader(ColumnReaderFactory.java:178)
at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader
.setup(ParquetRecordReader.java:319)
... 20 more
Error: SYSTEM ERROR: ClassCastException: org.apache.drill.exec.vector.Decimal28S
parseVector cannot be cast to org.apache.drill.exec.vector.VariableWidthVector
Fragment 1:0
[Error Id: bfde2a91-2c86-48b3-b188-7a7bf3d8e6ac on DaveOshinsky-PC.gp.cv.commvau
lt.com:31010] (state=,code=0)
0: jdbc:drill:zk=local>
***************************Legal Disclaimer***************************
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
**********************************************************************
