[
https://issues.apache.org/jira/browse/BEAM-8513?focusedWorklogId=336542&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336542
]
ASF GitHub Bot logged work on BEAM-8513:
----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Oct/19 23:54
Start Date: 30/Oct/19 23:54
Worklog Time Spent: 10m
Work Description: drobert commented on pull request #9937: [BEAM-8513]
Allow reads from exchange-bound queue without declaring the exchange
URL: https://github.com/apache/beam/pull/9937#discussion_r340910446
##########
File path:
sdks/java/io/rabbitmq/src/test/java/org/apache/beam/sdk/io/rabbitmq/RabbitMqIOTest.java
##########
@@ -185,20 +234,154 @@ public void testReadExchange() throws Exception {
publisher.join();
} finally {
if (channel != null) {
- channel.close();
+ // channel may have already been closed automatically due to protocol
failure
+ try {
+ channel.close();
+ } catch (Exception e) {
+ /* ignored */
+ }
}
if (connection != null) {
- connection.close();
+ // connection may have already been closed automatically due to
protocol failure
+ try {
+ connection.close();
+ } catch (Exception e) {
+ /* ignored */
+ }
}
}
}
+ private void doExchangeTest(ExchangeTestPlan testPlan) throws Exception {
+ doExchangeTest(testPlan, false);
+ }
+
+ @Test(timeout = ONE_MINUTE_MS)
+ public void testReadDeclaredFanoutExchange() throws Exception {
+ doExchangeTest(
+ new ExchangeTestPlan(
+ RabbitMqIO.read().withExchange("DeclaredFanoutExchange", "fanout",
"ignored"), 10));
+ }
+
+ @Test(timeout = ONE_MINUTE_MS)
+ public void testReadDeclaredTopicExchangeWithQueueDeclare() throws Exception
{
+ doExchangeTest(
+ new ExchangeTestPlan(
+ RabbitMqIO.read()
+ .withExchange("DeclaredTopicExchangeWithQueueDeclare",
"topic", "#")
+ .withQueue("declared-queue")
+ .withQueueDeclare(true),
+ 10));
+ }
+
+ @Test(timeout = ONE_MINUTE_MS)
+ public void testReadDeclaredTopicExchange() throws Exception {
+ final int numRecords = 10;
+ RabbitMqIO.Read read =
+ RabbitMqIO.read().withExchange("DeclaredTopicExchange", "topic",
"user.create.#");
+
+ final Supplier<String> publishRoutingKeyGen =
+ new Supplier<String>() {
+ private AtomicInteger counter = new AtomicInteger(0);
+
+ @Override
+ public String get() {
+ int count = counter.getAndIncrement();
+ if (count % 2 == 0) {
+ return "user.create." + count;
+ }
+ return "user.delete." + count;
+ }
+ };
+
+ ExchangeTestPlan plan =
+ new ExchangeTestPlan(read, numRecords / 2, numRecords) {
+ @Override
+ public Supplier<String> publishRoutingKeyGen() {
+ return publishRoutingKeyGen;
+ }
+
+ @Override
+ public List<String> expectedResults() {
+ return IntStream.range(0, numRecords)
+ .filter(i -> i % 2 == 0)
+ .mapToObj(RabbitMqTestUtils::generateRecord)
+ .map(RabbitMqTestUtils::recordToString)
+ .collect(Collectors.toList());
+ }
+ };
+
+ doExchangeTest(plan);
+ }
+
+ @Test(timeout = ONE_MINUTE_MS)
+ public void testDeclareIncompatibleExchangeFails() throws Exception {
+ RabbitMqIO.Read read =
+ RabbitMqIO.read().withExchange("IncompatibleExchange", "direct",
"unused");
+ try {
+ doExchangeTest(new ExchangeTestPlan(read, 1), true);
+ fail("Expected to have failed to declare an incompatible exchange");
+ } catch (Exception e) {
+ Throwable cause = Throwables.getRootCause(e);
+ if (cause instanceof ShutdownSignalException) {
+ ShutdownSignalException sse = (ShutdownSignalException) cause;
+ Method reason = sse.getReason();
+ if (reason instanceof com.rabbitmq.client.AMQP.Connection.Close) {
+ com.rabbitmq.client.AMQP.Connection.Close close =
+ (com.rabbitmq.client.AMQP.Connection.Close) reason;
+ assertEquals("Expected failure is 530: not-allowed", 530,
close.getReplyCode());
+ } else {
+ fail(
+ "Unexpected ShutdownSignalException reason. Expected
Connection.Close. Got: "
+ + reason);
+ }
+ } else {
+ fail("Expected to fail with ShutdownSignalException. Instead failed
with " + cause);
+ }
+ }
+ }
+
+ @Test(timeout = ONE_MINUTE_MS)
+ public void testUseCorrelationIdSucceedsWhenIdsPresent() throws Exception {
Review comment:
This test and the one below retroactively add testing support for
https://github.com/apache/beam/pull/9782 (BEAM-8390) now that it's easier to
write this test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 336542)
Time Spent: 1h (was: 50m)
> RabbitMqIO: Allow reads from exchange-bound queue without declaring the
> exchange
> --------------------------------------------------------------------------------
>
> Key: BEAM-8513
> URL: https://issues.apache.org/jira/browse/BEAM-8513
> Project: Beam
> Issue Type: Improvement
> Components: io-java-rabbitmq
> Environment: testing with DirectRunner
> Reporter: Nick Aldwin
> Priority: Critical
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The RabbitMqIO always declares an exchange if it is configured to read from
> it. This is problematic with pre-existing exchanges (a relatively common
> pattern), as there's no provided configuration for the exchange beyond
> exchange type. (We stumbled on this because RabbitMqIO always declares a
> non-durable exchange, which fails if the exchange already exists as a durable
> exchange)
>
> A solution to this would be to allow RabbitMqIO to read from an exchange
> without declaring it. This pattern is already available for queues via the
> `queueDeclare` flag. I propose an `exchangeDeclare` flag which preserves
> existing behavior except for skipping the call to `exchangeDeclare` before
> binding the queue to the exchange.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
