[
https://issues.apache.org/jira/browse/BEAM-7427?focusedWorklogId=272828&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-272828
]
ASF GitHub Bot logged work on BEAM-7427:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Jul/19 11:11
Start Date: 06/Jul/19 11:11
Worklog Time Spent: 10m
Work Description: zouabimourad commented on issue #8757: [BEAM-7427] Fix
JmsCheckpointMark Avro Encoding
URL: https://github.com/apache/beam/pull/8757#issuecomment-508917576
> @zouabimourad just for some extra context my comment on testing
encoding/decoding comes from the fact that `CheckpointMark`s should be able to
be converted from/to bytes. However the current implementation is trying to do
that for a list of not yet processed `javax.jms.Message`s which is probably not
a good idea because we cannot guarantee that those will be correctly serialized
with or without Avro. So probably a better approach (and if I understand
correctly is what @jbonofre proposes) is to contain a list of `JmsRecord` in
`JmsCheckpointMark` because we know those are `Serializable`.
I agree with you `javax.jms.Message` could not be deserialized since it's an
interface .. i got errors after using
`CoderProperties.coderDecodeEncodeEqual(avroCoder, checkpointMark);` in test.
@jbonofre has a better fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 272828)
Time Spent: 1h 40m (was: 1.5h)
> JmsCheckpointMark Avro Serialisation issue with UnboundedSource
> ---------------------------------------------------------------
>
> Key: BEAM-7427
> URL: https://issues.apache.org/jira/browse/BEAM-7427
> Project: Beam
> Issue Type: Bug
> Components: io-java-jms
> Affects Versions: 2.12.0
> Environment: Message Broker : solace
> JMS Client (Over AMQP) : "org.apache.qpid:qpid-jms-client:0.42.0
> Reporter: Mourad
> Assignee: Mourad
> Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> I get the following exception when reading from unbounded JMS Source:
>
> {code:java}
> Caused by: org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1151)
> at org.apache.avro.Schema.access$200(Schema.java:81)
> at org.apache.avro.Schema$Field.<init>(Schema.java:403)
> at org.apache.avro.Schema$Field.<init>(Schema.java:396)
> at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:622)
> at org.apache.avro.reflect.ReflectData.createFieldSchema(ReflectData.java:740)
> at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:604)
> at org.apache.avro.specific.SpecificData$2.load(SpecificData.java:218)
> at org.apache.avro.specific.SpecificData$2.load(SpecificData.java:215)
> at
> avro.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
> at
> avro.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
> at
> avro.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
> at
> avro.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
> {code}
>
> The exception is thrown by Avro when introspecting {{JmsCheckpointMark}} to
> generate schema.
> JmsIO config :
>
> {code:java}
> PCollection<DFAMessage> messages = pipeline.apply("read messages from the
> events broker", JmsIO.<DFAMessage>readMessage()
> .withConnectionFactory(jmsConnectionFactory) .withTopic(options.getTopic())
> .withMessageMapper(new DFAMessageMapper())
> .withCoder(AvroCoder.of(DFAMessage.class)));
> {code}
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
