[
https://issues.apache.org/jira/browse/NIFI-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922473#comment-16922473
]
Nico Verwer edited comment on NIFI-6498 at 9/10/19 6:47 AM:
------------------------------------------------------------
This response has been updated with refactored code fragments on 2019-09-10.
Saxon uses an ErrorListener when compiling a stylesheet. The
net.sf.saxon.lib.StandardErrorListener writes messages to System.err, if
nothing else is specified. Where they end up is unclear to me.
To send the error messages to Nifi's log-file, change the following in
TransformXML.java:
{code:java}
private Templates newTemplates(final ProcessContext context, final String
path) throws TransformerConfigurationException, LookupFailureException {
final ComponentLog logger = getLogger();
final Boolean secureProcessing =
context.getProperty(SECURE_PROCESSING).asBoolean();
TransformerFactory factory = TransformerFactory.newInstance();
final boolean isFilename = context.getProperty(XSLT_FILE_NAME).isSet();
if (secureProcessing) {
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
// don't be overly DTD-unfriendly forcing
http://apache.org/xml/features/disallow-doctype-decl
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-parameter-entities";,
false);
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-general-entities";,
false);
} // Set all dynamic properties that are boolean-valued Saxon
features,
// for example "http://saxon.sf.net/feature/allow-external-functions";.
for (final Map.Entry<PropertyDescriptor, String> entry :
context.getProperties().entrySet()) {
if (entry.getKey().isDynamic() &&
entry.getKey().getName().startsWith("http://saxon.sf.net/";)) {
factory.setFeature(entry.getKey().getName(),
Boolean.parseBoolean(entry.getValue()));
}
} ErrorListener errorListener = new ErrorListener() {
@Override
public void warning(TransformerException exception) throws
TransformerException {
logger.warn(exception.getMessageAndLocation());
}
@Override
public void error(TransformerException exception) throws
TransformerException {
logger.error(exception.getMessageAndLocation());
}
@Override
public void fatalError(TransformerException exception) throws
TransformerException {
logger.log(LogLevel.FATAL, exception.getMessageAndLocation());
}
};
factory.setErrorListener(errorListener); if(isFilename) {
return factory.newTemplates(new StreamSource(path));
} else {
final String coordinateKey =
lookupService.get().getRequiredKeys().iterator().next();
final Optional<String> attributeValue =
lookupService.get().lookup(Collections.singletonMap(coordinateKey, path));
if (attributeValue.isPresent() &&
StringUtils.isNotBlank(attributeValue.get())) {
return factory.newTemplates(new StreamSource(new
ByteArrayInputStream(attributeValue.get().getBytes(StandardCharsets.UTF_8))));
} else {
throw new TransformerConfigurationException("No XSLT definition
is associated to " + path + " in the lookup controller service.");
}
}
}
{code}
The important part is the addition of an ErrorListener. Then, exceptions are
caught and the collected messages are added to the exception.
I have also added the possibility to set binary Saxon properties like
"http://saxon.sf.net/feature/allow-external-functions"; from the processor's
properties.
The above will send compile-time messages to the Nifi logging, but we need to
do something similar for run-time messages emitted by <xsl:message>. This
requires the following change inside the onTrigger() method:
{code:java}
public void process(final InputStream rawIn, final OutputStream
out) throws IOException {
try (final InputStream in = new BufferedInputStream(rawIn))
{
final Templates templates;
if (cache != null) {
templates = cache.get(path);
} else {
templates = newTemplates(context, path);
} final Transformer transformer
= templates.newTransformer();
transformer.setOutputProperty(OutputKeys.INDENT,
(indentOutput ? "yes" : "no"));
setMessageLogger(transformer); // Pass all dynamic
properties to the transformer
for (final Map.Entry<PropertyDescriptor, String> entry
: context.getProperties().entrySet()) {
if (entry.getKey().isDynamic() &&
!entry.getKey().getName().startsWith("http://saxon.sf.net/";)) {
String value =
context.newPropertyValue(entry.getValue()).evaluateAttributeExpressions(original).getValue();
transformer.setParameter(entry.getKey().getName(), value);
}
} // use a StreamSource with
Saxon
StreamSource source = new StreamSource(in);
StreamResult result = new StreamResult(out);
transformer.transform(source, result);
} catch (final Exception e) {
throw new IOException(e);
}
}
{code}
The setMessageLogger method is:
{code:java}
/**
* The Saxon transformer sends messages from <xsl:message> to a
messageEmitter, making them disappear.
* Use Nifi logging for these messages instead.
* @param logger
* @param transformer
*/
private void setMessageLogger(final Transformer transformer) {
final ComponentLog logger = getLogger();
if (transformer instanceof net.sf.saxon.jaxp.TransformerImpl) {
net.sf.saxon.jaxp.TransformerImpl saxonTransformer =
(net.sf.saxon.jaxp.TransformerImpl)transformer;
Receiver messageReceiver = new net.sf.saxon.serialize.XMLEmitter() {
boolean terminate = false;
@Override
public void startDocument(int properties) throws XPathException {
setWriter(new StringWriter());
terminate = (properties & ReceiverOptions.TERMINATE) != 0;
super.startDocument(properties);
}
@Override
public void endDocument() throws XPathException {
String message = getWriter().toString();
if (terminate) {
XPathException xpe = new XPathException(message);
xpe.setErrorCode("XTMM9000");
throw xpe;
} else {
logger.warn(message);
}
}
@Override
public void close() {}
};
saxonTransformer.getUnderlyingController().setMessageEmitter(messageReceiver);
}
}
{code}
I would make a pull request if I could, but I have not been able to get Nifi
working in Eclipse on my PC (XPS15, i7, 16GB RAM). Eclipse just keeps mavening
things forever. :(
Maybe someone can test this code fragment and add it to Nifi?
was (Author: nverwer):
Saxon uses an ErrorListener when compiling a stylesheet. The
net.sf.saxon.lib.StandardErrorListener writes messages to System.err, if
nothing else is specified. Where they end up is unclear to me.
To send the error messages to Nifi's log-file, change the following in
TransformXML.java:
{code:java}
private Templates newTemplates(final ProcessContext context, final String
path) throws TransformerConfigurationException, LookupFailureException {
final Boolean secureProcessing =
context.getProperty(SECURE_PROCESSING).asBoolean();
TransformerFactory factory = TransformerFactory.newInstance();
final boolean isFilename = context.getProperty(XSLT_FILE_NAME).isSet();
StringBuffer messages = new StringBuffer(); if
(secureProcessing) {
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
// don't be overly DTD-unfriendly forcing
http://apache.org/xml/features/disallow-doctype-decl
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-parameter-entities";,
false);
factory.setFeature("http://saxon.sf.net/feature/parserFeature?uri=http://xml.org/sax/features/external-general-entities";,
false);
}
ErrorListener errorListener = new ErrorListener() {
@Override
public void warning(TransformerException exception) throws
TransformerException {
messages.append("Warning:
"+exception.getMessageAndLocation()+"\n");
}
@Override
public void error(TransformerException exception) throws
TransformerException {
messages.append("Error:
"+exception.getMessageAndLocation()+"\n");
}
@Override
public void fatalError(TransformerException exception) throws
TransformerException {
messages.append("Fatal Error:
"+exception.getMessageAndLocation()+"\n");
}
};
factory.setErrorListener(errorListener);
try {
if(isFilename) {
return factory.newTemplates(new StreamSource(path));
} else {
final String coordinateKey =
lookupService.get().getRequiredKeys().iterator().next();
final Optional<String> attributeValue =
lookupService.get().lookup(Collections.singletonMap(coordinateKey, path));
if (attributeValue.isPresent() &&
StringUtils.isNotBlank(attributeValue.get())) {
return factory.newTemplates(new StreamSource(new
ByteArrayInputStream(attributeValue.get().getBytes(StandardCharsets.UTF_8))));
} else {
throw new TransformerConfigurationException("No XSLT
definition is associated to " + path + " in the lookup controller service.");
}
}
} catch (TransformerConfigurationException tce) {
throw new TransformerConfigurationException("Errors compiling XSLT:
"+messages.toString(), tce);
}
}
{code}
The important part is the addition of an ErrorListener. Then, exceptions are
caught and the collected messages are added to the exception.
I would make a pull request if I could, but I have not been able to get Nifi
working in Eclipse on my PC (XPS15, i7, 16GB RAM). Eclipse just keeps mavening
things forever. :(
Maybe someone can test this code fragment and add it to Nifi?
> XMLTransform processor: XSLT doesn't give detail error message with line
> number
> -------------------------------------------------------------------------------
>
> Key: NIFI-6498
> URL: https://issues.apache.org/jira/browse/NIFI-6498
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.9.2
> Reporter: Raymond
> Priority: Major
>
> When something is wrong with the XSLT in the XMLTransform processor you get
> an error, but this is only a general error without any details or the line
> number where the parsing error occurs. For example when using Saxon from
> example I get the following error:
> Error on line 17 column 5 SXXP0003: Error reported by XML parser: The element
> type "xsl:tomplate" must be terminated by the matching end-tag
> "</xsl:tomplate>".
> When using NiFi the following error is printed:
> 11:07:43 CEST
> ERROR
> 2b4c1dab-3a87-175f-f817-3f3c72af4102
> TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102] Unable to transform
> StandardFlowFileRecord[uuid=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1564391260760-1, container=default,
> section=1], offset=0,
> length=6380],offset=0,name=c4dde8e5-89b0-4f29-8f2d-a97a009cf843,size=6380]
> due to IOException thrown from
> TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]: java.io.IOException:
> java.util.concurrent.ExecutionException:
> javax.xml.transform.TransformerConfigurationException:
> net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error
> reported: org.apache.nifi.processor.exception.ProcessException: IOException
> thrown from TransformXml[id=2b4c1dab-3a87-175f-f817-3f3c72af4102]:
> java.io.IOException: java.util.concurrent.ExecutionException:
> javax.xml.transform.TransformerConfigurationException:
> net.sf.saxon.s9api.SaxonApiException: Stylesheet compilation failed: 1 error
> reported
> The error in NiFi gives almost no information for debugging. Would be a big
> improvement if the Saxon error message is catched and add to the NiFi error
> message (bulletin board).
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
