[
https://issues.apache.org/jira/browse/HDDS-7235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17606593#comment-17606593
]
mingchao zhao edited comment on HDDS-7235 at 9/20/22 3:08 AM:
--------------------------------------------------------------
Looking at the stack is the problem with using SequenceFile. The implementation
of this part in Hive is as follows.
!screenshot-2.png!
I use this case tested the HDFS and found the execution was successful, as
shown below:
!image-2022-09-20-10-55-29-649.png!
But using the above test in Ozone throws an EOFException. So the behavior of
Ozone O3Fs here should be consistent with that of HDFS.
The problem mainly lies in the check of pos in BlockInputStream and
ChunkInputStream. BlockInputStream and ChunkInputStream need to be consistent
with KeyInputStream.
!image-2022-09-19-21-12-04-228.png!
!image-2022-09-19-21-15-59-053.png!
was (Author: micahzhao):
Looking at the stack is the problem with using SequenceFile. The implementation
of this part in Hive is as follows.
!screenshot-2.png!
I use this case tested the HDFS and found it to be normal.
!image-2022-09-20-10-55-29-649.png!
But using the above test in Ozone throws an EOFException. So the behavior of
Ozone O3Fs here should be consistent with that of HDFS.
The problem mainly lies in the check of pos in BlockInputStream and
ChunkInputStream. BlockInputStream and ChunkInputStream need to be consistent
with KeyInputStream.
!image-2022-09-19-21-12-04-228.png!
!image-2022-09-19-21-15-59-053.png!
> EOFException occurs when executing TPC-DS using o3fs
> ----------------------------------------------------
>
> Key: HDDS-7235
> URL: https://issues.apache.org/jira/browse/HDDS-7235
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: mingchao zhao
> Assignee: mingchao zhao
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-09-19-21-12-04-228.png,
> image-2022-09-19-21-15-59-053.png, image-2022-09-20-10-55-29-649.png,
> screenshot-2.png, screenshot-3.png, screenshot-5.png, syslog
>
>
> When use o3fs to generate TPCDS test data, the following exceptions will
> occur during the process of converting text to ORC.
> See the full log of
> https://issues.apache.org/jira/secure/attachment/13049447/syslog
> !screenshot-5.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
