[
https://issues.apache.org/jira/browse/HAWQ-314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dong Li updated HAWQ-314:
-------------------------
Description:
{code}
newperf=# create table os_order_item_1 as select * from os_order_item;
ERROR: VarBlock is not valid. Valid block check error 10, detail 'offset 0 at
index 0 is bad -- must equal 8 (bytes_0_3 0x81007c60, bytes_4_7 0x000001a1)'
(appendonlyam.c:747) (seg15 ****:5532 pid=561713) (dispatcher.c:1701)
DETAIL:
Append-Only storage Small Content header: smallcontent_bytes_0_3 0x1906840F,
smallcontent_bytes_4_7 0xF5400000, headerKind = 1, executorBlockKind = 1,
rowCount = 417, usingChecksums = false, header checksum 0x0, block checksum
0x0, dataLength 32682, compressedLength 0, overallBlockLen 32696
Scan of Append-Only Row-Oriented relation 'os_order_item'. Append-Only segment
file 'hdfs://***:9000/hawq/hawq-1451203332/16385/26367/26435/139', block header
offset in file = 16088, bufferCount 4103
{code}
Debug information
{code}
(gdb) p ( * (AppendOnlyScanDescData
*)0x33df218)->storageRead->bufferedRead->largeReadLen
$24 = 48928
{code}
The bufferedRead->largeReadLen should be 65536(bufferedRead->maxLargeReadLen).
I find the bug is in cdbbufferedread.c:158
{code}
int32 real_fileLen = fileLen - bufferedRead->largeReadPosition;
if (real_fileLen > 0)
{
/*
* Do the first read.
*/
if (real_fileLen > bufferedRead->maxLargeReadLen)
bufferedRead->largeReadLen =
bufferedRead->maxLargeReadLen;
else
bufferedRead->largeReadLen = (int32)real_fileLen;
BufferedReadIo(bufferedRead);
}
{code}
real_fileLen should be int64 as fileLen is a int64 number and
bufferedRead->largeReadPosition is also a int64 number.
According to the code, a int64 number is changed to int32 which make the number
wrong.
To *fix* this, just define the real_fileLen as a int64 number.
{code}
int64 real_fileLen = fileLen - bufferedRead->largeReadPosition;
{code}
was:
{code}
newperf=# create table os_order_item_1 as select * from os_order_item;
ERROR: VarBlock is not valid. Valid block check error 10, detail 'offset 0 at
index 0 is bad -- must equal 8 (bytes_0_3 0x81007c60, bytes_4_7 0x000001a1)'
(appendonlyam.c:747) (seg15 gva-w10:5532 pid=561713) (dispatcher.c:1701)
DETAIL:
Append-Only storage Small Content header: smallcontent_bytes_0_3 0x1906840F,
smallcontent_bytes_4_7 0xF5400000, headerKind = 1, executorBlockKind = 1,
rowCount = 417, usingChecksums = false, header checksum 0x0, block checksum
0x0, dataLength 32682, compressedLength 0, overallBlockLen 32696
Scan of Append-Only Row-Oriented relation 'os_order_item'. Append-Only segment
file 'hdfs://gva-mst2:9000/hawq/hawq-1451203332/16385/26367/26435/139', block
header offset in file = 16088, bufferCount 4103
{code}
Debug information
{code}
(gdb) p ( * (AppendOnlyScanDescData
*)0x33df218)->storageRead->bufferedRead->largeReadLen
$24 = 48928
{code}
The bufferedRead->largeReadLen should be 65536(bufferedRead->maxLargeReadLen).
I find the bug is in cdbbufferedread.c:158
{code}
int32 real_fileLen = fileLen - bufferedRead->largeReadPosition;
if (real_fileLen > 0)
{
/*
* Do the first read.
*/
if (real_fileLen > bufferedRead->maxLargeReadLen)
bufferedRead->largeReadLen =
bufferedRead->maxLargeReadLen;
else
bufferedRead->largeReadLen = (int32)real_fileLen;
BufferedReadIo(bufferedRead);
}
{code}
real_fileLen should be int64 as fileLen is a int64 number and
bufferedRead->largeReadPosition is also a int64 number.
According to the code, a int64 number is changed to int32 which make the number
wrong.
To *fix* this, just define the real_fileLen as a int64 number.
{code}
int64 real_fileLen = fileLen - bufferedRead->largeReadPosition;
{code}
> AO read error due to wrong init of bufferedRead->largeReadLen
> -------------------------------------------------------------
>
> Key: HAWQ-314
> URL: https://issues.apache.org/jira/browse/HAWQ-314
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Storage
> Reporter: Dong Li
> Assignee: Lei Chang
>
> {code}
> newperf=# create table os_order_item_1 as select * from os_order_item;
> ERROR: VarBlock is not valid. Valid block check error 10, detail 'offset 0
> at index 0 is bad -- must equal 8 (bytes_0_3 0x81007c60, bytes_4_7
> 0x000001a1)' (appendonlyam.c:747) (seg15 ****:5532 pid=561713)
> (dispatcher.c:1701)
> DETAIL:
> Append-Only storage Small Content header: smallcontent_bytes_0_3 0x1906840F,
> smallcontent_bytes_4_7 0xF5400000, headerKind = 1, executorBlockKind = 1,
> rowCount = 417, usingChecksums = false, header checksum 0x0, block checksum
> 0x0, dataLength 32682, compressedLength 0, overallBlockLen 32696
> Scan of Append-Only Row-Oriented relation 'os_order_item'. Append-Only
> segment file 'hdfs://***:9000/hawq/hawq-1451203332/16385/26367/26435/139',
> block header offset in file = 16088, bufferCount 4103
> {code}
> Debug information
> {code}
> (gdb) p ( * (AppendOnlyScanDescData
> *)0x33df218)->storageRead->bufferedRead->largeReadLen
> $24 = 48928
> {code}
> The bufferedRead->largeReadLen should be 65536(bufferedRead->maxLargeReadLen).
> I find the bug is in cdbbufferedread.c:158
> {code}
> int32 real_fileLen = fileLen - bufferedRead->largeReadPosition;
> if (real_fileLen > 0)
> {
> /*
> * Do the first read.
> */
> if (real_fileLen > bufferedRead->maxLargeReadLen)
> bufferedRead->largeReadLen =
> bufferedRead->maxLargeReadLen;
> else
> bufferedRead->largeReadLen = (int32)real_fileLen;
> BufferedReadIo(bufferedRead);
> }
> {code}
> real_fileLen should be int64 as fileLen is a int64 number and
> bufferedRead->largeReadPosition is also a int64 number.
> According to the code, a int64 number is changed to int32 which make the
> number wrong.
> To *fix* this, just define the real_fileLen as a int64 number.
> {code}
> int64 real_fileLen = fileLen - bufferedRead->largeReadPosition;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
