[jira] [Work logged] (HIVE-26734) Iceberg: Add an option to allow positional delete files without actual row data

[ASF GitHub Bot (Jira)]

     [ 
https://issues.apache.org/jira/browse/HIVE-26734?focusedWorklogId=826018&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-826018
 ]

ASF GitHub Bot logged work on HIVE-26734:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 15/Nov/22 07:39
            Start Date: 15/Nov/22 07:39
    Worklog Time Spent: 10m 
      Work Description: ayushtkn commented on code in PR #3758:
URL: https://github.com/apache/hive/pull/3758#discussion_r1022434349


##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/writer/HiveIcebergDeleteWriter.java:
##########
@@ -41,21 +41,28 @@
 class HiveIcebergDeleteWriter extends HiveIcebergWriterBase {
 
   private final GenericRecord rowDataTemplate;
+  private final boolean skipRowData;
 
   HiveIcebergDeleteWriter(Schema schema, Map<Integer, PartitionSpec> specs,
       FileWriterFactory<Record> writerFactory, OutputFileFactory fileFactory, 
FileIO io,
-      long targetFileSize) {
+      long targetFileSize, boolean skipRowData) {
     super(schema, specs, io,
         new ClusteredPositionDeleteWriter<>(writerFactory, fileFactory, io, 
targetFileSize));
     rowDataTemplate = GenericRecord.create(schema);
+    this.skipRowData = skipRowData;
   }
 
   @Override
   public void write(Writable row) throws IOException {
     Record rec = ((Container<Record>) row).get();
     PositionDelete<Record> positionDelete = 
IcebergAcidUtil.getPositionDelete(rec, rowDataTemplate);
     int specId = IcebergAcidUtil.parseSpecId(rec);
-    writer.write(positionDelete, specs.get(specId), 
partition(positionDelete.row(), specId));
+    Record rowData = positionDelete.row();
+    if (skipRowData) {
+      // Set null as the row data as we intend to avoid writing the actual row 
data in the delete file.
+      positionDelete.set(positionDelete.path(), positionDelete.pos(), null);

Review Comment:
   That row content is required for the partition details below in the writer 
`` partition(rowData, specId));``, so that we need anyway irrespective of 
whether we write the data or not





Issue Time Tracking
-------------------

    Worklog Id:     (was: 826018)
    Time Spent: 2h 10m  (was: 2h)

> Iceberg: Add an option to allow positional delete files without actual row 
> data
> -------------------------------------------------------------------------------
>
>                 Key: HIVE-26734
>                 URL: https://issues.apache.org/jira/browse/HIVE-26734
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Ayush Saxena
>            Assignee: Ayush Saxena
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Allow an option to have actual row data in the Iceberg PositionalDelete file 
> as optional, to avoid reading and writing huge amount of actual row data 
> during query executions.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)