[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1080989821
##
fineract-provider/src/main/java/org/apache/fineract/infrastructure/event/external/config/ExternalEventJMSConfiguration.java:
##
@@ -60,4 +61,13 @@ public ActiveMQTopic activeMqTopic() {
public ActiveMQQueue activeMqQueue() {
return new
ActiveMQQueue(fineractProperties.getEvents().getExternal().getProducer().getJms().getEventQueueName());
}
+
+@Bean("externalEventJmsProducerExecutor")
+public ThreadPoolTaskExecutor externalEventJmsProducerExecutor() {
+ThreadPoolTaskExecutor threadPoolTaskExecutor = new
ThreadPoolTaskExecutor();
+threadPoolTaskExecutor.setCorePoolSize(10);
Review Comment:
Would it make sense to move these into application.properties?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1072383085
##
fineract-provider/src/main/java/org/apache/fineract/infrastructure/event/external/repository/ExternalEventRepository.java:
##
@@ -19,19 +19,25 @@
package org.apache.fineract.infrastructure.event.external.repository;
import java.time.LocalDate;
+import java.time.OffsetDateTime;
import java.util.List;
import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEvent;
import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEventStatus;
+import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEventView;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
public interface ExternalEventRepository extends JpaRepository {
-List findByStatusOrderById(ExternalEventStatus status,
Pageable batchSize);
+List findByStatusOrderById(ExternalEventStatus status,
Pageable batchSize);
@Modifying(flushAutomatically = true)
@Query("delete from ExternalEvent e where e.status = :status and
e.businessDate <= :dateForPurgeCriteria")
void deleteOlderEventsWithSentStatus(ExternalEventStatus status, LocalDate
dateForPurgeCriteria);
+
+@Modifying
Review Comment:
Can it happen to fetch the "not yet sent" events again from the DB before
this async event sending logic got executed for all of the events?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1072301485
##
fineract-provider/src/main/java/org/apache/fineract/infrastructure/event/external/repository/ExternalEventRepository.java:
##
@@ -19,19 +19,25 @@
package org.apache.fineract.infrastructure.event.external.repository;
import java.time.LocalDate;
+import java.time.OffsetDateTime;
import java.util.List;
import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEvent;
import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEventStatus;
+import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEventView;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
public interface ExternalEventRepository extends JpaRepository {
-List findByStatusOrderById(ExternalEventStatus status,
Pageable batchSize);
+List findByStatusOrderById(ExternalEventStatus status,
Pageable batchSize);
@Modifying(flushAutomatically = true)
@Query("delete from ExternalEvent e where e.status = :status and
e.businessDate <= :dateForPurgeCriteria")
void deleteOlderEventsWithSentStatus(ExternalEventStatus status, LocalDate
dateForPurgeCriteria);
+
+@Modifying
Review Comment:
Would autoflush make sense here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1072301485
##
fineract-provider/src/main/java/org/apache/fineract/infrastructure/event/external/repository/ExternalEventRepository.java:
##
@@ -19,19 +19,25 @@
package org.apache.fineract.infrastructure.event.external.repository;
import java.time.LocalDate;
+import java.time.OffsetDateTime;
import java.util.List;
import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEvent;
import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEventStatus;
+import
org.apache.fineract.infrastructure.event.external.repository.domain.ExternalEventView;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
public interface ExternalEventRepository extends JpaRepository {
-List findByStatusOrderById(ExternalEventStatus status,
Pageable batchSize);
+List findByStatusOrderById(ExternalEventStatus status,
Pageable batchSize);
@Modifying(flushAutomatically = true)
@Query("delete from ExternalEvent e where e.status = :status and
e.businessDate <= :dateForPurgeCriteria")
void deleteOlderEventsWithSentStatus(ExternalEventStatus status, LocalDate
dateForPurgeCriteria);
+
+@Modifying
Review Comment:
autoflush?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1072297910
##
fineract-provider/src/main/java/org/apache/fineract/infrastructure/event/external/producer/jms/JMSMultiExternalEventProducer.java:
##
@@ -0,0 +1,151 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.fineract.infrastructure.event.external.producer.jms;
+
+import static
org.apache.fineract.infrastructure.core.service.MeasuringUtil.measure;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import javax.jms.Connection;
+import javax.jms.Destination;
+import javax.jms.JMSException;
+import javax.jms.MessageProducer;
+import javax.jms.Session;
+import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.activemq.ActiveMQConnectionFactory;
+import org.apache.fineract.infrastructure.core.config.FineractProperties;
+import org.apache.fineract.infrastructure.core.messaging.jms.MessageFactory;
+import org.apache.fineract.infrastructure.core.service.HashingService;
+import
org.apache.fineract.infrastructure.event.external.exception.AcknowledgementTimeoutException;
+import
org.apache.fineract.infrastructure.event.external.producer.ExternalEventProducer;
+import org.springframework.beans.factory.InitializingBean;
+import org.springframework.beans.factory.annotation.Qualifier;
+import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
+import org.springframework.core.task.AsyncTaskExecutor;
+import org.springframework.stereotype.Component;
+
+@Component
+@Slf4j
+@RequiredArgsConstructor
+@ConditionalOnProperty(value =
"fineract.events.external.producer.jms.enabled", havingValue = "true")
+public class JMSMultiExternalEventProducer implements ExternalEventProducer,
InitializingBean {
+
+@Qualifier("eventDestination")
+private final Destination destination;
+
+private final ActiveMQConnectionFactory connectionFactory;
Review Comment:
We will support only the ActiveMQ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1072285436
##
fineract-provider/src/main/java/org/apache/fineract/infrastructure/event/external/jobs/SendAsynchronousEventsTasklet.java:
##
@@ -69,20 +79,66 @@ private boolean isDownstreamChannelEnabled() {
return
fineractProperties.getEvents().getExternal().getProducer().getJms().isEnabled();
}
-private List getQueuedEventsBatch() {
+private List getQueuedEventsBatch() {
int readBatchSize = getBatchSize();
Pageable batchSize = PageRequest.ofSize(readBatchSize);
-return
repository.findByStatusOrderById(ExternalEventStatus.TO_BE_SENT, batchSize);
+return measure(() ->
repository.findByStatusOrderById(ExternalEventStatus.TO_BE_SENT, batchSize),
Review Comment:
I guess for production we dont need this :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1072285436
##
fineract-provider/src/main/java/org/apache/fineract/infrastructure/event/external/jobs/SendAsynchronousEventsTasklet.java:
##
@@ -69,20 +79,66 @@ private boolean isDownstreamChannelEnabled() {
return
fineractProperties.getEvents().getExternal().getProducer().getJms().isEnabled();
}
-private List getQueuedEventsBatch() {
+private List getQueuedEventsBatch() {
int readBatchSize = getBatchSize();
Pageable batchSize = PageRequest.ofSize(readBatchSize);
-return
repository.findByStatusOrderById(ExternalEventStatus.TO_BE_SENT, batchSize);
+return measure(() ->
repository.findByStatusOrderById(ExternalEventStatus.TO_BE_SENT, batchSize),
Review Comment:
I guess for production we dont need this :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1072171238
##
fineract-provider/src/main/java/org/apache/fineract/infrastructure/core/messaging/jms/ActiveMQMessageFactory.java:
##
@@ -16,21 +16,20 @@
* specific language governing permissions and limitations
* under the License.
*/
-package org.apache.fineract.infrastructure.event.external.config;
+package org.apache.fineract.infrastructure.core.messaging.jms;
-import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
-import org.springframework.context.annotation.Bean;
-import org.springframework.context.annotation.Configuration;
-import org.springframework.integration.channel.DirectChannel;
-import org.springframework.integration.config.EnableIntegration;
+import javax.jms.BytesMessage;
+import javax.jms.JMSException;
+import org.apache.activemq.command.ActiveMQBytesMessage;
+import org.springframework.stereotype.Component;
-@Configuration
-@EnableIntegration
-@ConditionalOnProperty(value = "fineract.events.external.enabled", havingValue
= "true")
-public class ExternalEventProducerConfiguration {
+@Component
Review Comment:
Dont we need condition whether we wanna have ActiveMQMessageFactory?
Or will it be the de facto implementation and others might override it only?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [fineract] adamsaghy commented on a diff in pull request #2891: FINERACT-1724: Implemented support for multiple parallel JMS producers for external events
adamsaghy commented on code in PR #2891:
URL: https://github.com/apache/fineract/pull/2891#discussion_r1072168430
##
fineract-provider/src/main/java/org/apache/fineract/cob/loan/LoanAccountsStayedLockedBusinessEvent.java:
##
@@ -39,4 +39,9 @@ public String getType() {
public String getCategory() {
return CATEGORY;
}
+
+@Override
Review Comment:
Null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fineract.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
