[
https://issues.apache.org/jira/browse/HIVE-17458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16236716#comment-16236716
]
Eugene Koifman edited comment on HIVE-17458 at 11/2/17 10:35 PM:
-----------------------------------------------------------------
1. acid 2.0 is the only acid supported by hive 3.0. The old data is still
readable after the upgrade process from 2.x to 3.0. (Either way this is no way
affected by this patch)
2. it is necessary. say you are running an update on a table converted to acid
but not having gone through major compaction.
was (Author: ekoifman):
1. acid 2.0 is the only acid supported by hive 3.0. The old data is still
readable after the upgrade process from 2.x to 3.0.
2. it is necessary. say you are running an update on a table converted to acid
but not having gone through major compaction.
> VectorizedOrcAcidRowBatchReader doesn't handle 'original' files
> ---------------------------------------------------------------
>
> Key: HIVE-17458
> URL: https://issues.apache.org/jira/browse/HIVE-17458
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 2.2.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Critical
> Attachments: HIVE-17458.01.patch, HIVE-17458.02.patch,
> HIVE-17458.03.patch, HIVE-17458.04.patch, HIVE-17458.05.patch,
> HIVE-17458.06.patch, HIVE-17458.07.patch, HIVE-17458.07.patch,
> HIVE-17458.08.patch, HIVE-17458.09.patch, HIVE-17458.10.patch,
> HIVE-17458.11.patch, HIVE-17458.12.patch, HIVE-17458.12.patch,
> HIVE-17458.13.patch, HIVE-17458.14.patch, HIVE-17458.15.patch
>
>
> VectorizedOrcAcidRowBatchReader will not be used for original files. This
> will likely look like a perf regression when converting a table from non-acid
> to acid until it runs through a major compaction.
> With Load Data support, if large files are added via Load Data, the read ops
> will not vectorize until major compaction.
> There is no reason why this should be the case. Just like
> OrcRawRecordMerger, VectorizedOrcAcidRowBatchReader can look at the other
> files in the logical tranche/bucket and calculate the offset for the RowBatch
> of the split. (Presumably getRecordReader().getRowNumber() works the same in
> vector mode).
> In this case we don't even need OrcSplit.isOriginal() - the reader can infer
> it from file path... which in particular simplifies
> OrcInputFormat.determineSplitStrategies()
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
