JeonDaehong commented on code in PR #3670:
URL: https://github.com/apache/iceberg-rust/pull/3670#discussion_r4599962292


##########
crates/iceberg/src/spec/manifest/reader.rs:
##########
@@ -51,8 +55,15 @@ impl ManifestReader {
             None => input_file.read().await?,
         };
 
-        let (metadata, mut entries) =
-            Manifest::try_from_avro_bytes(&bytes, 
Some(&manifest_file.manifest_path))?;
+        let location = manifest_file.manifest_path.clone();
+        let (metadata, mut entries) = self
+            .runtime
+            .cpu()
+            .spawn(async move { Manifest::try_from_avro_bytes(&bytes, 
Some(&location)) })

Review Comment:
   Only the parse moves here, and the gain shrinks as manifests get larger 
(numbers in the review body), so the per-entry work after the parse is probably 
what's left on the calling task.



##########
crates/iceberg/src/spec/manifest/reader.rs:
##########
@@ -51,8 +55,15 @@ impl ManifestReader {
             None => input_file.read().await?,
         };
 
-        let (metadata, mut entries) =
-            Manifest::try_from_avro_bytes(&bytes, 
Some(&manifest_file.manifest_path))?;
+        let location = manifest_file.manifest_path.clone();
+        let (metadata, mut entries) = self
+            .runtime
+            .cpu()
+            .spawn(async move { Manifest::try_from_avro_bytes(&bytes, 
Some(&location)) })
+            .await
+            // A `JoinError` (the parse panicked, or the cpu runtime shut down)
+            // names no manifest; a parse error already does.
+            .map_err(|err| err.with_context("manifest", 
&manifest_file.manifest_path))??;

Review Comment:
   A panicking parse used to unwind through the planning task and now comes 
back as `Unexpected` with the manifest path. That seems like the better 
behavior, and `test_read_names_manifest_when_parse_is_cancelled` covers the 
same `JoinError` path.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to