Colin Chambers created IVY-1673:
-----------------------------------
Summary: Install from Maven repository fails if POM has DOCTYPE
declaration
Key: IVY-1673
URL: https://issues.apache.org/jira/browse/IVY-1673
Project: Ivy
Issue Type: Bug
Components: Maven Compatibility
Affects Versions: 2.6.0
Environment: Ubuntu 24.04.2 LTS
openjdk version "21.0.7" 2025-04-15 LTS
Reporter: Colin Chambers
Attachments: PomReader.java
An attempt to use the Ivy install task to install a module from a Maven
repository where the module POM has a DOCTYPE declaration fails.
The following request is an example demonstrating the failure:
<ivy:install organisation="com.vaadin.addon"
module="vaadin-charts"
revision="3.3.4"
from="vaadin-addons"
to="third-party-local"
transitive="false"
overwrite="true"/>
This produces the following failure:
[ivy:install] :: installing com.vaadin.addon#vaadin-charts;3.3.4 ::
[ivy:install] :: resolving dependencies ::
[ivy:install] [Fatal Error] ivy-3.3.4.xml.original:4:10: Already seen
doctype.
[ivy:install] :: downloading artifacts to cache ::
[ivy:install] :: installing in third-party-local ::
[ivy:install] :: install resolution report ::
[ivy:install] :: resolution report :: resolve 0ms :: artifacts dl 0ms
---------------------------------------------------------------------
| | modules || artifacts
|
| conf | number| search|dwnlded|evicted||
number|dwnlded|
---------------------------------------------------------------------
| default | 1 | 0 | 0 | 0 || 0 | 0
|
---------------------------------------------------------------------
[ivy:install]
[ivy:install] :: problems summary ::
[ivy:install] :::: WARNINGS
[ivy:install] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:install] :: UNRESOLVED DEPENDENCIES ::
[ivy:install] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:install] :: com.vaadin.addon#vaadin-charts;3.3.4:
java.text.ParseException: Already seen
doctype.
[ivy:install] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:install] :::: ERRORS
[ivy:install] Already seen doctype.
[ivy:install]
[ivy:install] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
BUILD FAILED
The "vaadin-addons" resolver is defined as:
<ibiblio name="vaadin-addons" usepoms="true" useMavenMetadata="true"
m2compatible="true" root="https://maven.vaadin.com/vaadin-addons"/>
DEFECT DIAGNOSIS
Running with "-debug" shows following as root cause exception:
[ivy:install] Caused by: org.xml.sax.SAXParseException; systemId:
file:/home/colin.chambers/.ivy2/cache/com.vaadin.addon/vaadin-charts/ivy-3.3.4.xml.original;
lineNumber: 4; columnNumber: 10; Already seen doctype.
[ivy:install] at
java.xml/com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:262)
[ivy:install] at
java.xml/com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:342)
[ivy:install] at
org.apache.ivy.util.XMLHelper.parseToDom(XMLHelper.java:256)
[ivy:install] at
org.apache.ivy.plugins.parser.m2.PomReader.<init>(PomReader.java:129)
[ivy:install] at
org.apache.ivy.plugins.parser.m2.PomModuleDescriptorParser.parseDescriptor(PomModuleDescriptorParser.java:140)
The file "ivy-3.3.4.xml.original" referenced in the cache starts with:
1: <?xml version="1.0" encoding="UTF-8"?>
2: <!DOCTYPE project>
3: <project xmlns="https://maven.apache.org/POM/4.0.0";
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance";
4: xsi:schemaLocation="https://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd";>
5:
6: <parent>
7: <groupId>com.vaadin.addon</groupId>
8: <artifactId>vaadin-charts-parent</artifactId>
9: <version>3.3.4</version>
10: </parent>
Note that line 4 is *not* a DOCTYPE declaration.
This file may also be observed in the "vaadin-addons" repository at URL
https://maven.vaadin.com/vaadin-addons/com/vaadin/addon/vaadin-charts/3.3.4/vaadin-charts-3.3.4.pom
(Use "view source" to see the prolog and DOCTYPE declaration.)
The source code around line number 129 of PomReader (from stack trace above) is
as follows:
public PomReader(final URL descriptorURL, final Resource res) throws
IOException, SAXException {
InputStream stream = new AddDTDFilterInputStream(
URLHandlerRegistry.getDefault().openStream(descriptorURL));
InputSource source = new InputSource(stream);
source.setSystemId(XMLHelper.toSystemId(descriptorURL));
try {
129: Document pomDomDoc = XMLHelper.parseToDom(source, new
EntityResolver() {
public InputSource resolveEntity(String publicId, String
systemId)
throws SAXException, IOException {
if (systemId != null &&
systemId.endsWith("m2-entities.ent")) {
// IVY-921: return an InputSource for our local
packaged m2-entities.ent file
return new InputSource(
PomReader.class.getResourceAsStream("m2-entities.ent"));
}
return null;
}
}, true, XMLHelper.ExternalResources.IGNORE);
Line 129 invokes an XML parser on the source. Just above this the InputStream
can be seen
to be filtered by class AddDTDFilterInputStream. This class is defined in
PomReader and
the code up to and including the constructor definition is as follows (line 797
on):
private static final class AddDTDFilterInputStream extends
FilterInputStream {
private static final int MARK = 10000;
private static final String DOCTYPE = "<!DOCTYPE project SYSTEM
\"m2-entities.ent\">\n";
private int count;
private byte[] prefix = DOCTYPE.getBytes();
private AddDTDFilterInputStream(InputStream in) throws IOException {
super(new BufferedInputStream(in));
this.in.mark(MARK);
// TODO: we should really find a better solution for this...
// maybe we could use a FilterReader instead of a FilterInputStream?
int byte1 = this.in.read();
int byte2 = this.in.read();
int byte3 = this.in.read();
if (byte1 == 239 && byte2 == 187 && byte3 == 191) {
// skip the UTF-8 BOM
this.in.mark(MARK);
} else {
this.in.reset();
}
824: int bytesToSkip = 0;
LineNumberReader reader = new LineNumberReader(new
InputStreamReader(this.in, StandardCharsets.UTF_8),
100);
String firstLine = reader.readLine();
if (firstLine != null) {
String trimmed = firstLine.trim();
if (trimmed.startsWith("<?xml ")) {
int endIndex = trimmed.indexOf("?>");
String xmlDecl = trimmed.substring(0, endIndex + 2);
prefix = (xmlDecl + "\n" + DOCTYPE).getBytes();
bytesToSkip = xmlDecl.getBytes().length;
}
} else {
prefix = new byte[0];
}
this.in.reset();
for (int i = 0; i < bytesToSkip; i++) {
this.in.read();
}
}
...
>From line 824 ("int bytesToSkip = 0") onwards, if the document starts with an
>"<?xml"
prolog then a prefix is constructed containg the prolog and a DOCTYPE
declaration
(defined on the first line above). The DOCTYPE declaration is inserted whether
or
not the document already contains a DOCTYPE declaration. The code below (not
shown)
inserts the prefix in the stream returned by the "read()" methods.
Therefore if the document already contained a DOCTYPE declaration (as in the
example)
then the document returned by this filter class contains two DOCTYPE
declarations,
thus causing the "Already seen doctype" ParseException.
The following is the start of the output from this code for the example POM
above:
1: <?xml version="1.0" encoding="UTF-8"?>
2: <!DOCTYPE project SYSTEM "m2-entities.ent">
3:
4: <!DOCTYPE project>
5: <project xmlns="https://maven.apache.org/POM/4.0.0";
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance";
6: xsi:schemaLocation="https://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd";>
7:
8: <parent>
9: <groupId>com.vaadin.addon</groupId>
10: <artifactId>vaadin-charts-parent</artifactId>
11: <version>3.3.4</version>
12: </parent>
Note the blank line 3 (caused by inaccurate calculation of bytes_to_skip) and
the second DOCTYPE on line 4 (as reported in the failure).
PROPOSED FIX
See attached "PomReader.java". This is a modified copy of
"src/java/org/apache/ivy/plugins/parser/m2/PomReader.java" from
latest "https://gitbox.apache.org/repos/asf/ant-ivy.git";.
The class "AddDTDFilterInputStream" in this file has been modified
to parse the POM XML prolog with a recursive-descent parser and separate
lexical analyser. It inserts the POM DOCTYPE and removes an existing
DOCTYPE, if present. The parser uses grammar production rules taken
directly from the W3C "Extensible Markup Language (XML) 1.0 (Fifth Edition)"
document [https://www.w3.org/TR/REC-xml/] with some simplifying assumptions.
This is more general than the original code.
A completely general solution would require executing a full-blown
XML parser on the POM XML prolog.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
