[
https://issues.apache.org/jira/browse/HIVE-26734?focusedWorklogId=825856&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-825856
]
ASF GitHub Bot logged work on HIVE-26734:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Nov/22 18:07
Start Date: 14/Nov/22 18:07
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3758:
URL: https://github.com/apache/hive/pull/3758#discussion_r1021897067
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/writer/HiveIcebergDeleteWriter.java:
##########
@@ -41,21 +41,31 @@
class HiveIcebergDeleteWriter extends HiveIcebergWriterBase {
private final GenericRecord rowDataTemplate;
+ private final boolean isWriteDeleteRow;
HiveIcebergDeleteWriter(Schema schema, Map<Integer, PartitionSpec> specs,
FileWriterFactory<Record> writerFactory, OutputFileFactory fileFactory,
FileIO io,
- long targetFileSize) {
+ long targetFileSize, boolean isWriteDeleteRow) {
super(schema, specs, io,
new ClusteredPositionDeleteWriter<>(writerFactory, fileFactory, io,
targetFileSize));
rowDataTemplate = GenericRecord.create(schema);
+ this.isWriteDeleteRow = isWriteDeleteRow;
}
@Override
public void write(Writable row) throws IOException {
Record rec = ((Container<Record>) row).get();
PositionDelete<Record> positionDelete =
IcebergAcidUtil.getPositionDelete(rec, rowDataTemplate);
int specId = IcebergAcidUtil.parseSpecId(rec);
- writer.write(positionDelete, specs.get(specId),
partition(positionDelete.row(), specId));
+ if (isWriteDeleteRow) {
+ writer.write(positionDelete, specs.get(specId),
partition(positionDelete.row(), specId));
Review Comment:
Could we refactor to:
````
Record rowData = positionDelete.row();
if (skipOriginalRow) {
positionDelete.set(positionDelete.path(), positionDelete.pos(), null);
}
writer.write(positionDelete, specs.get(specId), partition(rowData, specId));
````
Issue Time Tracking
-------------------
Worklog Id: (was: 825856)
Time Spent: 50m (was: 40m)
> Iceberg: Add an option to allow positional delete files without actual row
> data
> -------------------------------------------------------------------------------
>
> Key: HIVE-26734
> URL: https://issues.apache.org/jira/browse/HIVE-26734
> Project: Hive
> Issue Type: Improvement
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Allow an option to have actual row data in the Iceberg PositionalDelete file
> as optional, to avoid reading and writing huge amount of actual row data
> during query executions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
