[
https://issues.apache.org/jira/browse/CAMEL-14085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955639#comment-16955639
]
Robert Half commented on CAMEL-14085:
-------------------------------------
Here I created a reproducer test, which fails with the error below.
The problem with the fixed length records and variable encoding like UTF-8 is
that the format as far as I'm concerned and which is the case for the format
I'm processing, is based on records given in bytes - not characters. So if I
know the first record is 10 bytes, I can only read it with bindy if every
character takes 1 byte.
StacktraceStacktrace---------------------------------------------------------------------------------------------------------------------------------------java.lang.IllegalArgumentException:
Size of the record: 9 is not equal to the value provided in the model: 10 at
org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.createModel(BindyFixedLengthDataFormat.java:295)
at
org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.unmarshal(BindyFixedLengthDataFormat.java:242)2019-10-20T10:49:27.873
| -1 | main | INFO | --- --- --- | com.mypackage.CamelBindyReproducerTest
| -
{code:java}
import org.apache.camel.EndpointInject;
import org.apache.camel.Exchange;
import org.apache.camel.Produce;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.dataformat.bindy.annotation.DataField;
import org.apache.camel.dataformat.bindy.annotation.FixedLengthRecord;
import org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat;
import org.apache.camel.spring.javaconfig.SingleRouteCamelConfiguration;
import org.apache.camel.test.spring.CamelSpringDelegatingTestContextLoader;
import org.apache.camel.test.spring.CamelSpringRunner;
import org.apache.camel.test.spring.CamelTestContextBootstrapper;
import org.apache.camel.test.spring.MockEndpoints;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.context.annotation.Configuration;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.BootstrapWith;
import org.springframework.test.context.ContextConfiguration;
import java.nio.charset.StandardCharsets;
import static org.junit.Assert.assertThat;
@RunWith(CamelSpringRunner.class)
@BootstrapWith(CamelTestContextBootstrapper.class)
@ContextConfiguration(
classes = CamelBindyReproducerTest.TestConfig.class,
loader = CamelSpringDelegatingTestContextLoader.class)
@MockEndpoints
@DirtiesContext(
classMode = DirtiesContext.ClassMode.AFTER_CLASS)
public class CamelBindyReproducerTest {
@Produce(uri = "direct:in")
private ProducerTemplate producer;
@EndpointInject(uri = "mock:result")
private MockEndpoint result;
@Test
/*
Let's assume we want to read the content of a 10 bytes record from an UTF-8
encoded file.
test string takes 10 bytes with 9 characters (2 char = 3 bytes content +
padding).
I assume to be able to read 9 char string from this 10 bytes fixed length
record with bindy.
*/
public void reproduceBindyErrorTest() throws InterruptedException {
String test = "aß ";
byte[] testAsBytes = test.getBytes(StandardCharsets.UTF_8);
Assert.assertEquals("aß takes 3 bytes, because ß takes 2", 10,
testAsBytes.length);
result.expectedMessagesMatches(exchange ->
"aß".equals(exchange.getIn().getBody(TestRecord.class).field1));
producer.sendBody(test);
result.assertIsSatisfied();
}
@Configuration
static class TestConfig extends SingleRouteCamelConfiguration {
@Override
public RouteBuilder route() {
return new RouteBuilder() {
@Override
public void configure() throws Exception {
from("direct:in")
.setHeader(Exchange.CHARSET_NAME, constant(StandardCharsets.UTF_8.name()))
.unmarshal(new BindyFixedLengthDataFormat(TestRecord.class))
.to("mock:result");
}
};
}
}
@FixedLengthRecord(length = 10, paddingChar = ' ')
public static class TestRecord {
public TestRecord() {
}
public String getField1() {
return field1;
}
public void setField1(String field1) {
this.field1 = field1;
}
@DataField(pos = 1, length = 10)
private String field1;
}
}{code}
> Camel Bindy @FixedLengthRecord cannot parse UTF-8 with multibyte characters
> ---------------------------------------------------------------------------
>
> Key: CAMEL-14085
> URL: https://issues.apache.org/jira/browse/CAMEL-14085
> Project: Camel
> Issue Type: Bug
> Components: camel-bindy
> Affects Versions: 2.21.5
> Reporter: Robert Half
> Priority: Major
> Fix For: 2.25.0, 3.x
>
>
> We are using Camel Bindy to parse fixed length data.
> I came across the issue, that it fails to read data containing characters
> like ß, which takes 2 bytes. The character set is properly set on the
> exchange to UTF-8, the record length is 1568 bytes, but since there are 2xß
> characters, it throws the error:
> Size of the record: 1566 is not equal to the value provided in the model:
> 1568, StackTrace: java.lang.IllegalArgumentException: Size of the record:
> 1566 is not equal to the value provided in the model: 1568Size of the record:
> 1566 is not equal to the value provided in the model: 1568, StackTrace:
> java.lang.IllegalArgumentException: Size of the record: 1566 is not equal to
> the value provided in the model: 1568 at
> org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.createModel(BindyFixedLengthDataFormat.java:295)
> here:
> BindyFixedLengthDataFormat#createModel
> {code:java}
> if ((myLine.length() < factory.recordLength()
> && !factory.isIgnoreMissingChars()) || (myLine.length() >
> factory.recordLength())) {
> throw new java.lang.IllegalArgumentException("Size of the record: " +
> myLine.length()
> + " is not equal to the value provided in the model: " +
> factory.recordLength());
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
