Jan and I had a couple of email exchange on this. It turns out that the compliance check at the beginning of the processing was not working when there is no xml-declaration in the input. Consequently, when your input message has no xml-declaration and you have sjxp parser, you will get incorrectly extracted tokens. I created CAMEL-8164 to fix this.
2014-12-17 19:35 GMT+01:00 Aki Yoshida <elak...@gmail.com>: > @Jan, > the failing test itself is expected if you don't have woodstox. The > question is how it is failing, whether it stops at the beginning and > throws the exception like in the quoted console output in my previous > reply or after extracting the bad token and when comparing the token? > You mentioned that you observed this second behavior and I would like > to know where I can reproduce it. > > thanks. > > > 2014-12-17 16:40 GMT+01:00 Jan Matèrne (jhm) <apa...@materne.de>: >> Java(TM) SE Runtime Environment (build 1.7.0_21-b11) >> Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode) >> by Oracle (according to the copyright file) >> - without woodstox: fail >> - with woodstox: pass >> >> >> Same for >> Java(TM) SE Runtime Environment (build 1.6.0_43-b01) >> Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed mode) >> (also Oracle) >> >> >> Same for >> Java(TM) SE Runtime Environment (build 1.8.0-ea-b87) >> Java HotSpot(TM) 64-Bit Server VM (build 25.0-b28, mixed mode) >> (also Oracle) >> >> >> Same for >> Java(TM) SE Runtime Environment (build 1.9.0-ea-b06) >> Java HotSpot(TM) 64-Bit Server VM (build 25.0-b62, mixed mode) >> (also Oracle) >> >> >> All on Win7 64bit. >> >> >> Jan >> >> >> >>> -----Ursprüngliche Nachricht----- >>> Von: Aki Yoshida [mailto:elak...@gmail.com] >>> Gesendet: Mittwoch, 17. Dezember 2014 15:49 >>> An: users@camel.apache.org >>> Cc: Jan Matèrne (jhm) >>> Betreff: Re: AW: camel - xsd >>> >>> @Jan >>> I just ran the camel-core's unit test XMLTokenExpressionIteratorTest on >>> jdk8 without woodstox (use profile deactivation -P!woodstox to disable >>> woodstox), the tokenizer correctly reports the error at the beginning >>> without proceeding to the tokenizing step. (on both OSX and Ubuntu with >>> Oracle JDK8). >>> >>> testExtractSomeUnqualifiedChild(org.apache.camel.support.XMLTokenExpres >>> sionIteratorTest) >>> Time elapsed: 0.001 sec <<< ERROR! >>> javax.xml.stream.XMLStreamException: reader not supporting Location at >>> org.apache.camel.support.XMLTokenExpressionIterator$XMLTokenIterator.<i >>> nit>(XMLTokenExpressionIterator.java:219) >>> >>> Can you tell me which JDK you had to get the incorrectly extracted >>> tokens? >>> >>> thanks >>> >>> 2014-12-16 14:27 GMT+01:00 Aki Yoshida <elak...@gmail.com>: >>> > @Jan, >>> > I just missed your previous replies. >>> > I just saw you also have provided the JDK info already. >>> > thanks. >>> > >>> > 2014-12-16 14:25 GMT+01:00 Aki Yoshida <elak...@gmail.com>: >>> >> It's good to hear it is working in your environment. >>> >> >>> >> by the way, I saw your camael doc update regarding the concrete >>> error >>> >> example you saw. >>> >> We need to formulate that differently. >>> >> There is a small sanity check before starting the tokenization (not >>> >> comparing the impl name itself but trying to infer its conformance >>> by >>> >> its initial behavior). That check was working for Oracle JDK 7 on >>> OSX. >>> >> I would like to know which JDK that you use. In any case, we can >>> >> either make the initial sanity check to detect this incompatible >>> >> behavior and detect it later if it is not detected at the beginning. >>> >> In either way, we can throw an exception at some point. >>> >> >>> >> regards, aki >>> >> >>> >> 2014-12-16 10:18 GMT+01:00 Jan Matèrne (jhm) <apa...@materne.de>: >>> >>> Sorry for my late answer, I was ill :( >>> >>> >>> >>> Thanks for your sample project, I investigated into it now. >>> >>> All tests passed. (Camel 2.14.0) >>> >>> >>> >>> But your test used the XMLTokenExpressionIterator directly I >>> >>> migrated my own test class (using the RouteBuilder) into your >>> project. >>> >>> - added the test class >>> >>> - added the xsd >>> >>> - added the dependency to commons-io + camel-test-spring >>> >>> - changed the paths in my test class to fit your directory layout >>> >>> Now this also passed. >>> >>> >>> >>> It's good to see that there is no bug in Camel. ;) >>> >>> >>> >>> >>> >>> Because my example still fails in my other environment I check that >>> ... >>> >>> >>> >>> >>> >>> Jan >>> >>> >>> >>> >>> >>>> -----Ursprüngliche Nachricht----- >>> >>>> Von: Aki Yoshida [mailto:elak...@gmail.com] >>> >>>> Gesendet: Mittwoch, 10. Dezember 2014 23:57 >>> >>>> An: users@camel.apache.org >>> >>>> Betreff: Re: AW: camel - xsd >>> >>>> >>> >>>> i meant woodstox-core-asl (e.g., woodstox-core-asl-4.4.1.jar). >>> >>>> its maven coordinate is >>> >>>> >>> >>>> mvn:org.codehaus.woodstox/woodstox-core-asl/4.4.1 >>> >>>> >>> >>>> you need to have woodstox or some other parser that reliably >>> >>>> reports the offset location at each parse event. >>> >>>> Sjsxp (sun/oracle implementation included in JDK) doesn't do that, >>> >>>> so you can't use it. >>> >>>> >>> >>>> jmtest.tar.gz contains a maven project that can be executed >>> >>>> directly by typing mvn test at the console or can be imported into >>> >>>> your eclipse IDE. >>> >>>> >>> >>>> 2014-12-10 13:13 GMT+01:00 Aki Yoshida <elak...@gmail.com>: >>> >>>> > i just created a test that uses your data and verified that it >>> is >>> >>>> working fine. >>> >>>> > please take a look at this file at my dropbox. >>> >>>> > https://www.dropbox.com/s/pfgvs9si9bvujxo/jmtest.tar.gz?dl=0 >>> >>>> > just extract the files into camel-core and run JMNewsTest. >>> >>>> > and see if that works in your environment, (i believe it does). >>> >>>> > >>> >>>> > in that case, we need to find out the difference to your case. >>> >>>> > do you have woodstox-api in your class path? >>> >>>> > >>> >>>> > >>> >>>> > 2014-12-10 12:15 GMT+01:00 Aki Yoshida <elak...@gmail.com>: >>> >>>> >> okay. >>> >>>> >> let me take a look. >>> >>>> >> >>> >>>> >> >>> >>>> >> 2014-12-10 12:02 GMT+01:00 Jan Matèrne (jhm) >>> <apa...@materne.de>: >>> >>>> >>> No, input is valid xml. >>> >>>> >>> After invoking xtokenize() the xml is invalid. >>> >>>> >>> >>> >>>> >>> CAMEL-8106 is very .... short. I can't see whether it is >>> related. >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> I posted an example on the user list >>> >>>> >>> http://mail-archives.apache.org/mod_mbox/camel- >>> >>>> users/201412.mbox/%3C >>> >>>> >>> 002b01d01452%24e8ee27a0%24baca76e0%24%40de%3E >>> >>>> >>> >>> >>>> >>> I had done a 'workaround' for this special example >>> >>>> >>> // Workaround of a bug?? in XMLTokenizerExpression >>> >>>> >>> .setBody(simple("${body.replace('</</news:Newsletter>', >>> >>>> '</news:Newsletter>')}")) >>> >>>> >>> .setBody(simple("${body.replace('<</news:Newsletter>', >>> >>>> >>> '</news:Newsletter>')}")) >>> >>>> >>> >>> >>>> >>> Without that the split messages have wrong end tags. >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> Jan >>> >>>> >>> >>> >>>> >>> >>> >>>> >>>> -----Ursprüngliche Nachricht----- >>> >>>> >>>> Von: Aki Yoshida [mailto:elak...@gmail.com] >>> >>>> >>>> Gesendet: Mittwoch, 10. Dezember 2014 10:34 >>> >>>> >>>> An: users@camel.apache.org >>> >>>> >>>> Betreff: Re: AW: camel - xsd >>> >>>> >>>> >>> >>>> >>>> you are talking about the invalid-xml parsing bug in >>> xtokenzier. >>> >>>> >>>> that has been fixe with CAMEL-8106. It should be part of the >>> >>>> >>>> new 2.14.1, which is to be released shortly. >>> >>>> >>>> >>> >>>> >>>> regards, aki >>> >>>> >>>> >>> >>>> >>>> 2014-12-10 9:25 GMT+01:00 Jan Matèrne (jhm) >>> <apa...@materne.de>: >>> >>>> >>>> > I played a little bit. >>> >>>> >>>> > >>> >>>> >>>> > Jan >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > public class XmlTest extends CamelTestSupport { >>> >>>> >>>> > >>> >>>> >>>> > @EndpointInject(uri="mock:valid") >>> >>>> >>>> > MockEndpoint valid; >>> >>>> >>>> > >>> >>>> >>>> > @EndpointInject(uri="mock:validationError") >>> >>>> >>>> > MockEndpoint validationError; >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > @Test >>> >>>> >>>> > public void validNewsfeed() throws Exception { >>> >>>> >>>> > // Newsfeed contains 2 news >>> >>>> >>>> > valid.expectedMessageCount(2); >>> >>>> >>>> > valid.expectedMessagesMatches( >>> >>>> >>>> > // Use helper methods from the static imported >>> >>>> >>>> PredicateBuilder >>> >>>> >>>> > and( >>> >>>> >>>> > >>> >>>> >>>> > header("newsfeed.date").isEqualTo("2014.12.09 >>> >>>> >>>> 14:15"), >>> >>>> >>>> > header("news.author").isEqualTo("Jan"), >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> >>> >>>> >>> body().contains("xmlns:news=\"http://www.materne.de/camel/test/xml/ >>> >>>> >>>> com >>> >>>> >>>> > plex/\ >>> >>>> >>>> > "") >>> >>>> >>>> > ) >>> >>>> >>>> > ); >>> >>>> >>>> > // no error expected >>> >>>> >>>> > validationError.expectedMessageCount(0); >>> >>>> >>>> > >>> >>>> >>>> > // Read xml from classpath and send to Camel route >>> >>>> >>>> > String xml = >>> >>>> >>>> > >>> >>>> >>>> >>> >>>> >>> IOUtils.toString(getClass().getResourceAsStream("/de/materne/camel/ >>> >>>> >>>> tes >>> >>>> >>>> > t/xml/ >>> >>>> >>>> > complex/validNewsfeed.xml")); >>> >>>> >>>> > sendBody("direct:in", xml); >>> >>>> >>>> > >>> >>>> >>>> > // 'execute' all tests >>> >>>> >>>> > assertMockEndpointsSatisfied(); >>> >>>> >>>> > } >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > @Test >>> >>>> >>>> > public void invalidXml() throws Exception { >>> >>>> >>>> > valid.expectedMessageCount(0); >>> >>>> >>>> > validationError.expectedMessageCount(1); >>> >>>> >>>> > >>> >>>> >>>> > String xml = "<xml/>"; >>> >>>> >>>> > sendBody("direct:in", xml); >>> >>>> >>>> > >>> >>>> >>>> > assertMockEndpointsSatisfied(); >>> >>>> >>>> > } >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > @Override >>> >>>> >>>> > protected RouteBuilder createRouteBuilder() throws >>> >>>> >>>> > Exception >>> >>>> { >>> >>>> >>>> > return new RouteBuilder() { >>> >>>> >>>> > @Override >>> >>>> >>>> > public void configure() throws Exception { >>> >>>> >>>> > // Our XML uses namespaces, so we have to >>> >>>> >>>> > deal with >>> >>>> >>>> that. >>> >>>> >>>> > Namespaces ns = new Namespaces("news", >>> >>>> >>>> > "http://www.materne.de/camel/test/xml/complex/";); >>> >>>> >>>> > >>> >>>> >>>> > // XSD-invalid data goes to this endpoint >>> >>>> >>>> > onException(ValidationException.class) >>> >>>> >>>> > .to("mock:validationError"); >>> >>>> >>>> > >>> >>>> >>>> > from("direct:in") >>> >>>> >>>> > // XSD-validation >>> >>>> >>>> > >>> >>>> >>>> > >>> .to("validator:de/materne/camel/test/xml/complex/newsfeed.xs >>> >>>> >>>> > d") >>> >>>> >>>> > >>> >>>> >>>> > // Store newsfeed data in the header >>> >>>> >>>> > before >>> >>>> >>>> split, >>> >>>> >>>> > so we haven't to do that on each >>> >>>> >>>> > // splittet news-message. >>> >>>> >>>> > .setHeader("newsfeed.date", >>> >>>> >>>> > ns.xpath("/news:Newsletter/@date";, String.class)) >>> >>>> >>>> > >>> >>>> >>>> > // >>> http://camel.apache.org/splitter.html >>> >>>> >>>> > // xtokenize() is available since Camel >>> >>>> 2.14. >>> >>>> >>>> > // Use the 'wrap'-mode so we keep the >>> >>>> >>>> > Newsletter- >>> >>>> >>>> Header >>> >>>> >>>> > >>> >>>> >>>> > .split().xtokenize("/news:Newsletter/News";, >>> >>>> >>>> > 'w', >>> >>>> >>>> > ns) >>> >>>> >>>> > >>> >>>> >>>> > // Workaround of a bug?? in >>> >>>> >>>> XMLTokenizerExpression >>> >>>> >>>> > >>> >>>> >>>> > .setBody(simple("${body.replace('</</news:Newsletter>', >>> >>>> >>>> > '</news:Newsletter>')}")) >>> >>>> >>>> > >>> >>>> >>>> > .setBody(simple("${body.replace('<</news:Newsletter>', >>> >>>> >>>> > '</news:Newsletter>')}")) >>> >>>> >>>> > >>> >>>> >>>> > // Get some data from the splittet news >>> >>>> >>>> > .setHeader("news.date", >>> >>>> >>>> > ns.xpath("/news:Newsletter/News/@date";, String.class)) >>> >>>> >>>> > .setHeader("news.author", >>> >>>> >>>> > ns.xpath("/news:Newsletter//News/@author";, String.class)) >>> >>>> >>>> > >>> >>>> >>>> > .to("mock:valid"); >>> >>>> >>>> > } >>> >>>> >>>> > }; >>> >>>> >>>> > } >>> >>>> >>>> > >>> >>>> >>>> > } >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > <?xml version="1.0" encoding="UTF-8"?> <schema >>> >>>> >>>> > xmlns="http://www.w3.org/2001/XMLSchema"; >>> >>>> >>>> > >>> >>>> xmlns:news="http://www.materne.de/camel/test/xml/complex/"; >>> >>>> >>>> > >>> >>>> >>>> > >>> targetNamespace="http://www.materne.de/camel/test/xml/comple >>> >>>> >>>> > x/"> >>> >>>> >>>> > >>> >>>> >>>> > <element name="Newsletter"> >>> >>>> >>>> > <complexType> >>> >>>> >>>> > <sequence> >>> >>>> >>>> > <element name="News" minOccurs="1" >>> >>>> >>>> > maxOccurs="unbounded"> >>> >>>> >>>> > <complexType> >>> >>>> >>>> > >>> >>>> >>>> > <simpleContent> >>> >>>> >>>> > >>> >>>> >>>> > <extension base="string"> >>> >>>> >>>> > >>> >>>> >>>> > <attribute name="date" type="string"/> >>> >>>> >>>> > >>> >>>> >>>> > <attribute name="author" type="string"/> >>> >>>> >>>> > >>> >>>> </extension> >>> >>>> >>>> > >>> </simpleContent> >>> >>>> >>>> > </complexType> >>> >>>> >>>> > </element> >>> >>>> >>>> > </sequence> >>> >>>> >>>> > <attribute name="date" type="string"/> >>> >>>> >>>> > </complexType> >>> >>>> >>>> > </element> >>> >>>> >>>> > >>> >>>> >>>> > </schema> >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > <news:Newsletter >>> >>>> >>>> > >>> >>>> >>>> xmlns:news="http://www.materne.de/camel/test/xml/complex/"; >>> >>>> >>>> > >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema- >>> >>>> instance" >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> xsi:schemaLocation="http://www.materne.de/camel/test/xml/complex/ >>> >>>> >>>> > newsfeed.xsd " >>> >>>> >>>> > date="2014.12.09 14:15"> >>> >>>> >>>> > <News date="2014.12.09 14:15" author="Jan"> >>> >>>> >>>> > Here is a demo news. >>> >>>> >>>> > </News> >>> >>>> >>>> > <News date="2014.12.09 14:13" author="Jan"> >>> >>>> >>>> > Second news paragraph. >>> >>>> >>>> > </News> >>> >>>> >>>> > </news:Newsletter> >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> > >>> >>>> >>>> >> -----Ursprüngliche Nachricht----- >>> >>>> >>>> >> Von: smilevasu6 [mailto:srinivas.thu...@gmail.com] >>> >>>> >>>> >> Gesendet: Dienstag, 9. Dezember 2014 15:12 >>> >>>> >>>> >> An: users@camel.apache.org >>> >>>> >>>> >> Betreff: Re: AW: camel - xsd >>> >>>> >>>> >> >>> >>>> >>>> >> Don't mind can you give me sample code instead of link >>> >>>> >>>> >> >>> >>>> >>>> >> >>> >>>> >>>> >> >>> >>>> >>>> >> -- >>> >>>> >>>> >> View this message in context: >>> >>>> >>>> >> http://camel.465427.n5.nabble.com/camel- >>> >>>> >>>> >> xsd-tp5760376p5760396.html >>> >>>> >>>> >> Sent from the Camel - Users mailing list archive at >>> Nabble.com. >>> >>>> >>>> > >>> >>>> >>> >>> >>> >>
