pjfanning commented on code in PR #1582:
URL: https://github.com/apache/poi/pull/1582#discussion_r4647932369
##########
poi-ooxml/src/main/java/org/apache/poi/ooxml/POIXMLDocumentPart.java:
##########
@@ -769,11 +775,42 @@ protected void read(POIXMLFactory factory,
Map<PackagePart, POIXMLDocumentPart>
}
}
+ childRelationships = null;
+ childRelationshipsByTarget = null;
+
for (POIXMLDocumentPart childPart : readLater) {
childPart.read(factory, context);
}
}
+ /**
+ * Returns the relationship of this part to a child part, while {@link
#read} creates the child parts.
+ * {@link POIXMLFactory#getPackageRelationship} is called for each new
child part, so the relationships
+ * are indexed by their target once instead of being searched for each
child part.
+ *
+ * @param part the child part
+ * @return the first relationship that targets the part (ignoring case),
or {@code null} if there is none
+ * or the child parts aren't being read
+ */
+ /* package */ PackageRelationship findChildRelationship(PackagePart part) {
+ if (childRelationships == null) {
+ return null;
+ }
+ if (childRelationshipsByTarget == null) {
+ childRelationshipsByTarget = new HashMap<>();
Review Comment:
The index includes every relationship, including `TargetMode.EXTERNAL` and
hyperlink relationships, which can never be the target of a child
`PackagePart`. With many external links (as in the test) we resolve, lowercase
and store all of them just to look up a handful of internal parts. Only
indexing internal relationships would keep time and memory proportional to the
real children.
##########
poi-ooxml/src/main/java/org/apache/poi/ooxml/POIXMLDocumentPart.java:
##########
@@ -713,6 +718,7 @@ protected void read(POIXMLFactory factory, Map<PackagePart,
POIXMLDocumentPart>
PackageRelationshipCollection rels = packagePart.getRelationships();
List<POIXMLDocumentPart> readLater = new ArrayList<>();
+ childRelationships = rels;
Review Comment:
`read()` already holds the exact `rel` when it calls
`factory.createDocumentPart(this, p)`. Rather than building a target ->
relationship map to find it again, could we just remember that relationship (a
single field set right before `createDocumentPart`, or an overload that passes
it through) and have `getPackageRelationship` return it after checking the
target matches the part? That's O(1) with no map, and it would also make the
`finally`, external-link and liveness-check comments below moot.
##########
poi-ooxml/src/main/java/org/apache/poi/ooxml/POIXMLDocumentPart.java:
##########
@@ -769,11 +775,42 @@ protected void read(POIXMLFactory factory,
Map<PackagePart, POIXMLDocumentPart>
}
}
+ childRelationships = null;
+ childRelationshipsByTarget = null;
+
for (POIXMLDocumentPart childPart : readLater) {
childPart.read(factory, context);
}
}
+ /**
+ * Returns the relationship of this part to a child part, while {@link
#read} creates the child parts.
+ * {@link POIXMLFactory#getPackageRelationship} is called for each new
child part, so the relationships
+ * are indexed by their target once instead of being searched for each
child part.
+ *
+ * @param part the child part
+ * @return the first relationship that targets the part (ignoring case),
or {@code null} if there is none
+ * or the child parts aren't being read
+ */
+ /* package */ PackageRelationship findChildRelationship(PackagePart part) {
+ if (childRelationships == null) {
+ return null;
+ }
+ if (childRelationshipsByTarget == null) {
+ childRelationshipsByTarget = new HashMap<>();
+ for (PackageRelationship rel : childRelationships) {
+ try {
+
childRelationshipsByTarget.putIfAbsent(rel.getTargetURI().toASCIIString().toLowerCase(Locale.ROOT),
rel);
Review Comment:
Swallowing `IllegalArgumentException` here is a behaviour change: the old
linear search iterated in id order and would throw on an unresolvable target
that came before the matching relationship, failing the load. Now that entry is
silently skipped and the document loads. Probably fine either way, but it
should be a deliberate choice; if we want to keep the old behaviour, let it
propagate.
##########
poi-ooxml/src/main/java/org/apache/poi/ooxml/POIXMLDocumentPart.java:
##########
@@ -769,11 +775,42 @@ protected void read(POIXMLFactory factory,
Map<PackagePart, POIXMLDocumentPart>
}
}
+ childRelationships = null;
Review Comment:
These are only reset on the normal path. If anything in the loop throws
(e.g. `createDocumentPart` rethrowing a `POIXMLException`, or `createPartName`
throwing `InvalidFormatException`), the part keeps the collection plus the full
`HashMap` (100k entries in the test's scenario), and later
`getPackageRelationship` calls are served from the stale index. Please move the
reset into a `try`/`finally` around the loop.
##########
poi-ooxml/src/test/java/org/apache/poi/ooxml/TestPOIXMLDocument.java:
##########
@@ -293,6 +299,53 @@ void dontParseEmbeddedDocuments() throws IOException {
}
}
+ /**
+ * The relationship of each new child part was looked up by searching all
relationships of
+ * the parent, which made loading quadratic in the number of child parts
and relationships
+ */
+ @Test
+ void testManyChildPartsAndRelationships() throws IOException {
+ final int childParts = 900;
+ final String relNs =
"http://schemas.openxmlformats.org/package/2006/relationships";
+ UnsynchronizedByteArrayOutputStream bos =
UnsynchronizedByteArrayOutputStream.builder().get();
+ try (ZipOutputStream zos = new ZipOutputStream(bos)) {
+ addEntry(zos, "[Content_Types].xml",
+ "<Types
xmlns=\"http://schemas.openxmlformats.org/package/2006/content-types\">"
+ + "<Default Extension=\"rels\"
ContentType=\"application/vnd.openxmlformats-package.relationships+xml\"/>"
+ + "<Default Extension=\"xml\"
ContentType=\"application/xml\"/></Types>");
+ addEntry(zos, "_rels/.rels", "<Relationships xmlns=\"" + relNs +
"\">"
+ + "<Relationship Id=\"rId1\" Type=\"" +
PackageRelationshipTypes.CORE_DOCUMENT + "\" Target=\"main.xml\"/>"
+ + "</Relationships>");
+ addEntry(zos, "main.xml", "<main/>");
+ StringBuilder rels = new StringBuilder("<Relationships xmlns=\"" +
relNs + "\">");
+ for (int i = 0; i < childParts; i++) {
+ rels.append("<Relationship Id=\"rIdP").append(i)
+ .append("\" Type=\"urn:child\"
Target=\"child/p").append(i).append(".xml\"/>");
+ addEntry(zos, "child/p" + i + ".xml", "<p/>");
+ }
+ for (int i = 0; i < 50_000; i++) {
+ rels.append("<Relationship Id=\"rIdE").append(i)
+ .append("\" Type=\"urn:link\" TargetMode=\"External\"
Target=\"https://example.com/").append(i).append("\"/>");
+ }
+ addEntry(zos, "_rels/main.xml.rels",
rels.append("</Relationships>").toString());
+ }
+
Review Comment:
A 20 s wall-clock limit is likely to be flaky on slow/overloaded CI runners,
and `assertTimeoutPreemptively` runs the body on a different thread, which
loses any thread-local state (e.g. per-thread IOUtils / zip limits). Could this
drop the timeout (or use a much smaller input with a generous bound) and rely
on the functional assertions instead?
##########
poi-ooxml/src/test/java/org/apache/poi/ooxml/TestPOIXMLDocument.java:
##########
@@ -293,6 +299,53 @@ void dontParseEmbeddedDocuments() throws IOException {
}
}
+ /**
+ * The relationship of each new child part was looked up by searching all
relationships of
+ * the parent, which made loading quadratic in the number of child parts
and relationships
+ */
+ @Test
+ void testManyChildPartsAndRelationships() throws IOException {
+ final int childParts = 900;
+ final String relNs =
"http://schemas.openxmlformats.org/package/2006/relationships";
+ UnsynchronizedByteArrayOutputStream bos =
UnsynchronizedByteArrayOutputStream.builder().get();
+ try (ZipOutputStream zos = new ZipOutputStream(bos)) {
+ addEntry(zos, "[Content_Types].xml",
+ "<Types
xmlns=\"http://schemas.openxmlformats.org/package/2006/content-types\">"
+ + "<Default Extension=\"rels\"
ContentType=\"application/vnd.openxmlformats-package.relationships+xml\"/>"
+ + "<Default Extension=\"xml\"
ContentType=\"application/xml\"/></Types>");
+ addEntry(zos, "_rels/.rels", "<Relationships xmlns=\"" + relNs +
"\">"
+ + "<Relationship Id=\"rId1\" Type=\"" +
PackageRelationshipTypes.CORE_DOCUMENT + "\" Target=\"main.xml\"/>"
+ + "</Relationships>");
+ addEntry(zos, "main.xml", "<main/>");
+ StringBuilder rels = new StringBuilder("<Relationships xmlns=\"" +
relNs + "\">");
+ for (int i = 0; i < childParts; i++) {
+ rels.append("<Relationship Id=\"rIdP").append(i)
+ .append("\" Type=\"urn:child\"
Target=\"child/p").append(i).append(".xml\"/>");
+ addEntry(zos, "child/p" + i + ".xml", "<p/>");
+ }
+ for (int i = 0; i < 50_000; i++) {
+ rels.append("<Relationship Id=\"rIdE").append(i)
+ .append("\" Type=\"urn:link\" TargetMode=\"External\"
Target=\"https://example.com/").append(i).append("\"/>");
+ }
+ addEntry(zos, "_rels/main.xml.rels",
rels.append("</Relationships>").toString());
+ }
+
+ assertTimeoutPreemptively(Duration.ofSeconds(20), () -> {
+ try (OPCPackage pkg = OPCPackage.open(bos.toInputStream());
+ OPCParser doc = new OPCParser(pkg)) {
+ doc.parse(new TestFactory());
Review Comment:
This only checks the number of relations. A bug that mapped a child to the
wrong relationship would still pass. Please also assert each child gets the
right relationship, e.g. `doc.getRelationId(child)` equals `"rIdP" + i` for the
part `child/p<i>.xml`.
##########
poi-ooxml/src/main/java/org/apache/poi/ooxml/POIXMLDocumentPart.java:
##########
@@ -769,11 +775,42 @@ protected void read(POIXMLFactory factory,
Map<PackagePart, POIXMLDocumentPart>
}
}
+ childRelationships = null;
+ childRelationshipsByTarget = null;
+
for (POIXMLDocumentPart childPart : readLater) {
childPart.read(factory, context);
}
}
+ /**
+ * Returns the relationship of this part to a child part, while {@link
#read} creates the child parts.
+ * {@link POIXMLFactory#getPackageRelationship} is called for each new
child part, so the relationships
+ * are indexed by their target once instead of being searched for each
child part.
+ *
+ * @param part the child part
+ * @return the first relationship that targets the part (ignoring case),
or {@code null} if there is none
+ * or the child parts aren't being read
+ */
+ /* package */ PackageRelationship findChildRelationship(PackagePart part) {
+ if (childRelationships == null) {
+ return null;
+ }
+ if (childRelationshipsByTarget == null) {
+ childRelationshipsByTarget = new HashMap<>();
+ for (PackageRelationship rel : childRelationships) {
+ try {
+
childRelationshipsByTarget.putIfAbsent(rel.getTargetURI().toASCIIString().toLowerCase(Locale.ROOT),
rel);
+ } catch (IllegalArgumentException e) {
+ // the target can't be resolved, a part can't be found by
it
+ }
+ }
+ }
+ PackageRelationship rel =
childRelationshipsByTarget.get(part.getPartName().getName().toLowerCase(Locale.ROOT));
+ // only return a relationship the part still has
+ return (rel != null && rel ==
getPackagePart().getRelationship(rel.getId())) ? rel : null;
Review Comment:
The `rel == getPackagePart().getRelationship(rel.getId())` check can't fail
while the index only exists inside `read()`'s loop, since nothing adds or
removes relationships there. It costs an extra lookup per child and suggests
the index can outlive the loop. With the `finally` fix it can be dropped (along
with the 'a relationship the part still has' wording).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]