We have more than 100 senders and the problem with JDBC I see always going 1 message in size, do we have any parameter of broker.xml which we can increase to get better performance.
PFA for reference with issue highlighted. Thanks, Shivang -----Original Message----- From: Justin Bertram <jbert...@apache.org> Sent: Thursday, September 14, 2023 1:14 AM To: users@activemq.apache.org Subject: Re: Artemis File Storage Persistence vs JDBC Persistence To be clear, my previous explanation assumes that nothing outside the broker is removing the persistent data. If you are operating in a containerized/virtual/cloud environment where the persistent data is being deleted then clearly no data will be available when the broker restarts. If this is the case then you need to use a persistent volume to store the file-based journal as mentioned by another user on this thread. Justin On Wed, Sep 13, 2023 at 12:29 PM Justin Bertram <jbert...@apache.org> wrote: > If you are sending one or more durable messages in your transaction > then the broker will attempt to write them to storage when they > arrive. If the broker fails to write the messages, for whatever > reason, then the sender will receive an exception when it attempts to > commit the transaction. If the broker succeeds in writing the messages > to storage but the broker stops for whatever reason before the > messages are consumed then when the broker restarts those messages > will be loaded from storage and made available to consumers. > > This is the same whether you're using the file-based journal or a > database to store the messages. > > > Justin > > On Wed, Sep 13, 2023 at 11:57 AM Shivang Modi > <sm...@provenir.com.invalid> > wrote: > >> Hi Justin, >> >> We are using Artemis docker image and start kubernetes pods with it. >> We have one sender which will write messages on queue and one >> receiver which will read messages queue. >> Now due to any reason, kubernetes queue pod gets restarted so before >> restarts whatever transactions gets enqueued by sender but not read >> by receiver, will that persisted with file storage and If yes, in any >> scenario file storage, chances of losing transactions? >> >> Thanks, >> Shivang. >> >> -----Original Message----- >> From: Justin Bertram <jbert...@apache.org> >> Sent: Wednesday, September 13, 2023 10:23 PM >> To: users@activemq.apache.org >> Subject: Re: Artemis File Storage Persistence vs JDBC Persistence >> >> I'm not really sure what you're asking. Are you asking whether you >> should use the file-based journal or a database if you have 100k >> transactions? >> >> To be clear, what is "best" in one situation is often not "best" in >> another. >> Everything depends on the specifics of your particular use-case. >> >> >> Justin >> >> On Wed, Sep 13, 2023 at 11:47 AM Shivang Modi >> <sm...@provenir.com.invalid >> > >> wrote: >> >> > If scenario is no loss transactions 100% if queue goes down >> > whatever transactions gets enqueued, should get dequeued once queue >> > comes up, we have 100k transactions or more need to flow up via >> > queue. What would be best in such scenarios? >> > >> > Thanks, >> > Shivang >> > >> > -----Original Message----- >> > From: Justin Bertram <jbert...@apache.org> >> > Sent: Wednesday, September 13, 2023 8:38 PM >> > To: users@activemq.apache.org >> > Subject: Re: Artemis File Storage Persistence vs JDBC Persistence >> > >> > When deciding between the file-based journal on local storage >> > versus a remote database I think the three main considerations are: >> > >> > - Performance >> > - Infrastructure >> > - Reliability >> > >> > The file-based journal on local storage will be faster than a >> > database for a few reasons: >> > - The storage is local so there's no network latency to deal with. >> > - The file-based journal was specifically written and heavily >> > optimized for the message broker use-case. >> > >> > The file-based journal on local storage requires less >> > infrastructure than a database since most servers already come with >> > local storage. >> > Using a database requires provisioning additional hardware as well >> > as installing and maintaining a distinct piece of software. This >> > can be costly both in terms of money and man-power. >> > >> > Generally speaking, local storage is always going to be more >> > reliable than a remote database simply because it's much simpler >> > (i.e. no network, no database with its own maintenance requirements, >> > etc.). >> > This simplicity tends to reduce downtime. >> > >> > In my experience the only folks who choose to use a database are >> > those in an environment where there's already been a substantial >> > investment in an enterprise database and stuff like automated >> > backups, redundant networking, data replications, etc. are available. >> > >> > No matter which option you choose, the broker is written so that >> > you should >> > *never* lose messages. >> > >> > >> > Justin >> > >> > >> > >> > On Wed, Sep 13, 2023 at 7:14 AM Shivang Modi >> > <sm...@provenir.com.invalid> >> > wrote: >> > >> > > Hi Team, >> > > >> > > >> > > >> > > Can anyone share pros and cons in depth between both. I see only >> > > file storage is faster than JDBC storage. Is there any >> > > disadvantage of File Storage like losing the enqueued data or >> > > anything? >> > > >> > > >> > > >> > > Thanks, >> > > >> > > Shivang. >> > > >> > > -- >> > > *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION >> > > intended solely for the use of the addressee(s). If you are not >> > > the intended recipient, please notify the sender by e-mail and >> > > delete the original message. Further, you are not to copy, >> > > disclose, or distribute this e-mail or its contents to any other >> > > person and any such actions maybe unlawful*. >> > > This e-mail may contain viruses. Provenir has taken every >> > > reasonable precaution to minimize this risk, but is not liable >> > > for any damage you may sustain as a result of any virus in this >> > > e-mail. You should carry out your own virus checks before opening >> > > the e-mail or >> attachment. >> > > Provenir reserves the right to monitor and review the content of >> > > all messages sent to or from this e-mail address. Messages sent >> > > to or from this e-mail address may be stored on the Provenir e-mail >> > > system. >> > > >> > >> > -- >> > *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION >> > intended solely for the use of the addressee(s). If you are not the >> > intended recipient, please notify the sender by e-mail and delete >> > the original message. Further, you are not to copy, disclose, or >> > distribute this e-mail or its contents to any other person and any >> > such actions maybe unlawful*. >> > This e-mail may contain viruses. Provenir has taken every >> > reasonable precaution to minimize this risk, but is not liable for >> > any damage you may sustain as a result of any virus in this e-mail. >> > You should carry out your own virus checks before opening the e-mail or >> > attachment. >> > Provenir reserves the right to monitor and review the content of >> > all messages sent to or from this e-mail address. Messages sent to >> > or from this e-mail address may be stored on the Provenir e-mail >> > system. >> > >> > >> >> -- >> *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION >> intended solely for the use of the addressee(s). If you are not the >> intended recipient, please notify the sender by e-mail and delete the >> original message. Further, you are not to copy, disclose, or >> distribute this e-mail or its contents to any other person and any >> such actions maybe unlawful*. >> This e-mail may contain viruses. Provenir has taken every reasonable >> precaution to minimize this risk, but is not liable for any damage >> you may sustain as a result of any virus in this e-mail. You should >> carry out your own virus checks before opening the e-mail or >> attachment. Provenir reserves the right to monitor and review the >> content of all messages sent to or from this e-mail address. Messages >> sent to or from this e-mail address may be stored on the Provenir >> e-mail system. >> >> -- *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions maybe unlawful*. This e-mail may contain viruses. Provenir has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Provenir reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Provenir e-mail system.
