John Roesler created KAFKA-8080:
-----------------------------------
Summary: Remove streams_eos_test system test
Key: KAFKA-8080
URL: https://issues.apache.org/jira/browse/KAFKA-8080
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: John Roesler
After KAFKA-7944 / [https://github.com/apache/kafka/pull/6382] , the system
test streams_eos_test.py is mostly redundant.
Quoting my analysis from
[https://github.com/apache/kafka/pull/6382#discussion_r263536548,]
{quote}Ok, so the smoke test and the eos test are similar, but not identical.
The smoke test application has more features in it, though. So, we have more
feature coverage under eos when we test with the smoke test.
The eos test evaluates two topologies one with no repartition, and one with a
repartition. The smoke test topology contains repartitions, so it only tests
_with_ repartition. I think that it should be sufficient to test only _with_
repartition.
The eos test verification specifically checks that "all transactions finished"
({{org.apache.kafka.streams.tests.EosTestDriver#verifyAllTransactionFinished}}).
I'm not clear on exactly what we're looking for here. It looks like we create
a transactional producer and send a record to each partition and then expect to
get all those records back, without seeing any other records. But I'm not sure
why we're doing this. If we want to drop the eos test, then we might need to
add this to the smoke test verification.
{quote}
And [~guozhang]'s reply:
{quote}{{verifyAllTransactionFinished}} aimed to avoid a situation that some
dangling txn is open forever, without committing or aborting. Because the
consumer needs to guarantee offset ordering when returning data, with
read-committed they will also be blocked on those open txn's data forever (this
usually indicates a broker-side issue, not streams though, but still).
I think we should still retain this check if we want to merge in the EosTest to
SmokeTest.
{quote}
As described above, the task is simply to add a similar check to the end of the
verification logic in the SmokeTestDriver. Then, we can remove the EOS system
test, as well as all the Java code that supports it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
