[
https://issues.apache.org/jira/browse/STORM-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Morrigan Jones updated STORM-1854:
----------------------------------
Description:
In the process of upgrading our Storm code from 0.10.0 to 1.0.0, I've run into
an issue with TransactionalTridentKafkaSpout. When running one of our
topologies I'm getting the following exception:
{code}
Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast to
java.lang.Integer
at
org.apache.storm.trident.spout.PartitionedTridentSpoutExecutor$Coordinator.initializeTransaction(PartitionedTridentSpoutExecutor.java:55)
~[storm-core-1.0.0.jar:1.0.0]
at
org.apache.storm.trident.spout.PartitionedTridentSpoutExecutor$Coordinator.initializeTransaction(PartitionedTridentSpoutExecutor.java:43)
~[storm-core-1.0.0.jar:1.0.0]
at
org.apache.storm.trident.spout.TridentSpoutCoordinator.execute(TridentSpoutCoordinator.java:70)
~[storm-core-1.0.0.jar:1.0.0]
at
org.apache.storm.topology.BasicBoltExecutor.execute(BasicBoltExecutor.java:50)
~[storm-core-1.0.0.jar:1.0.0]
{code}
The issue appears to be caused by a change in PartitionedTridentSpoutExecutor
between the two versions, specifically this method:
1.0.0 -
https://github.com/apache/storm/blob/v1.0.0/storm-core/src/jvm/org/apache/storm/trident/spout/PartitionedTridentSpoutExecutor.java#L51
{code}
public Integer initializeTransaction(long txid, Integer prevMetadata, Integer
currMetadata) {
if(currMetadata!=null) {
return currMetadata;
} else {
return _coordinator.getPartitionsForBatch();
}
}
{code}
0.10.0 -
https://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/storm/trident/spout/PartitionedTridentSpoutExecutor.java#L51
{code}
public Object initializeTransaction(long txid, Object prevMetadata, Object
currMetadata) {
if(currMetadata!=null) {
return currMetadata;
} else {
return _coordinator.getPartitionsForBatch();
}
}
{code}
This was introduced by:
https://github.com/apache/storm/commit/9e4c3df17ffbc737210e606d3d8a9cdae8f86634
TransactionalTridentKafkaSpout uses List<GlobalPartitionInformation> for its
metadata. Generally, transactional spouts should have metadata that is more
complex than just an Integer. OpaquePartitionedTridentSpoutExecutor uses Object
for its metadata and correctly handles the metadata used by
OpaqueTridentKafkaSpout (List<GlobalPartitionInformation>).
It looks like reverting the metadata type for transactional spouts in
PartitionedTridentSpoutExecutor should work, but I haven't tried this yet.
was:
In the process of upgrading our Storm code from 0.10.0 to 1.0.0, I've run into
an issue with TransactionalTridentKafkaSpout. When running one of our
topologies I'm getting the following exception:
Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast to
java.lang.Integer
at
org.apache.storm.trident.spout.PartitionedTridentSpoutExecutor$Coordinator.initializeTransaction(PartitionedTridentSpoutExecutor.java:55)
~[storm-core-1.0.0.jar:1.0.0]
at
org.apache.storm.trident.spout.PartitionedTridentSpoutExecutor$Coordinator.initializeTransaction(PartitionedTridentSpoutExecutor.java:43)
~[storm-core-1.0.0.jar:1.0.0]
at
org.apache.storm.trident.spout.TridentSpoutCoordinator.execute(TridentSpoutCoordinator.java:70)
~[storm-core-1.0.0.jar:1.0.0]
at
org.apache.storm.topology.BasicBoltExecutor.execute(BasicBoltExecutor.java:50)
~[storm-core-1.0.0.jar:1.0.0]
The issue appears to be caused by a change in PartitionedTridentSpoutExecutor
between the two versions, specifically this method:
1.0.0 -
https://github.com/apache/storm/blob/v1.0.0/storm-core/src/jvm/org/apache/storm/trident/spout/PartitionedTridentSpoutExecutor.java#L51
public Integer initializeTransaction(long txid, Integer prevMetadata, Integer
currMetadata) {
if(currMetadata!=null) {
return currMetadata;
} else {
return _coordinator.getPartitionsForBatch();
}
}
0.10.0 -
https://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/storm/trident/spout/PartitionedTridentSpoutExecutor.java#L51
public Object initializeTransaction(long txid, Object prevMetadata, Object
currMetadata) {
if(currMetadata!=null) {
return currMetadata;
} else {
return _coordinator.getPartitionsForBatch();
}
}
This was introduced by:
https://github.com/apache/storm/commit/9e4c3df17ffbc737210e606d3d8a9cdae8f86634
TransactionalTridentKafkaSpout uses List<GlobalPartitionInformation> for its
metadata. Generally, transactional spouts should have metadata that is more
complex than just an Integer. OpaquePartitionedTridentSpoutExecutor uses Object
for its metadata and correctly handles the metadata used by
OpaqueTridentKafkaSpout (List<GlobalPartitionInformation>).
It looks like reverting the metadata type for transactional spouts in
PartitionedTridentSpoutExecutor should work, but I haven't tried this yet.
> Trident transactional spouts are broken in 1.0.x
> ------------------------------------------------
>
> Key: STORM-1854
> URL: https://issues.apache.org/jira/browse/STORM-1854
> Project: Apache Storm
> Issue Type: Bug
> Components: trident
> Affects Versions: 1.0.0, 1.0.1
> Reporter: Morrigan Jones
>
> In the process of upgrading our Storm code from 0.10.0 to 1.0.0, I've run
> into an issue with TransactionalTridentKafkaSpout. When running one of our
> topologies I'm getting the following exception:
> {code}
> Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast
> to java.lang.Integer
> at
> org.apache.storm.trident.spout.PartitionedTridentSpoutExecutor$Coordinator.initializeTransaction(PartitionedTridentSpoutExecutor.java:55)
> ~[storm-core-1.0.0.jar:1.0.0]
> at
> org.apache.storm.trident.spout.PartitionedTridentSpoutExecutor$Coordinator.initializeTransaction(PartitionedTridentSpoutExecutor.java:43)
> ~[storm-core-1.0.0.jar:1.0.0]
> at
> org.apache.storm.trident.spout.TridentSpoutCoordinator.execute(TridentSpoutCoordinator.java:70)
> ~[storm-core-1.0.0.jar:1.0.0]
> at
> org.apache.storm.topology.BasicBoltExecutor.execute(BasicBoltExecutor.java:50)
> ~[storm-core-1.0.0.jar:1.0.0]
> {code}
> The issue appears to be caused by a change in PartitionedTridentSpoutExecutor
> between the two versions, specifically this method:
> 1.0.0 -
> https://github.com/apache/storm/blob/v1.0.0/storm-core/src/jvm/org/apache/storm/trident/spout/PartitionedTridentSpoutExecutor.java#L51
> {code}
> public Integer initializeTransaction(long txid, Integer prevMetadata, Integer
> currMetadata) {
> if(currMetadata!=null) {
> return currMetadata;
> } else {
> return _coordinator.getPartitionsForBatch();
> }
> }
> {code}
> 0.10.0 -
> https://github.com/apache/storm/blob/v0.10.0/storm-core/src/jvm/storm/trident/spout/PartitionedTridentSpoutExecutor.java#L51
> {code}
> public Object initializeTransaction(long txid, Object prevMetadata, Object
> currMetadata) {
> if(currMetadata!=null) {
> return currMetadata;
> } else {
> return _coordinator.getPartitionsForBatch();
> }
> }
> {code}
> This was introduced by:
> https://github.com/apache/storm/commit/9e4c3df17ffbc737210e606d3d8a9cdae8f86634
> TransactionalTridentKafkaSpout uses List<GlobalPartitionInformation> for its
> metadata. Generally, transactional spouts should have metadata that is more
> complex than just an Integer. OpaquePartitionedTridentSpoutExecutor uses
> Object for its metadata and correctly handles the metadata used by
> OpaqueTridentKafkaSpout (List<GlobalPartitionInformation>).
> It looks like reverting the metadata type for transactional spouts in
> PartitionedTridentSpoutExecutor should work, but I haven't tried this yet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
