Thanks Artem. We have SMTP logging in many other services with the
threshold set to ERROR so not sure why this would be an issue in this
specific application.
One observation: we haven't set a threshold for the *requestLog* portion
which is a deviation from the other services so it is using the default
threshold.
I pasted the smtp logging config below
server:
applicationConnectors:
- type: http
port: 8080
adminConnectors:
- type: http
port: 8081
requestLog:
timeZone: UTC
appenders:
- type: file
logFormat: null
currentLogFilename: /var/log/rtr/dfs/requests.log
archive: true
archivedLogFilenamePattern: /var/log/rtr/dfs/archived/requests-%d.log.gz
archivedFileCount: 5
threshold: INFO
logging:
appenders:
- type: file
threshold: INFO
logFormat: "%-6level [%d{HH:mm:ss.SSS}] [%t] %logger{5} - %X{code} %msg %n"
currentLogFilename: /var/log/rtr/dfs/dfs.log
archivedLogFilenamePattern: /var/log/rtr/dfs/dfs-%d.log.gz
archivedFileCount: 5
timeZone: UTC
smtp-logger:
enabled: true
server: ...
sender: ...
recipients: ...
subject: "%X{hostname} [%level, %X{env}, %X{code}] \"%msg\""
body: "%X{context}%n%n"
threshold: ERROR
On Tuesday, April 18, 2017 at 4:57:02 PM UTC-4, [email protected]
wrote:
>
> Hi Dropwizard Users,
>
> We recently upgraded one of our query services to Dropwizard 0.8.4.
> Previously we were on Dropwizard6.
> With an increased load on the service, the # of waiting threads continued
> to increase beyond the set default limit of 1024. Eventually our
> application tipped over and resulted in a *java.lang.OutOfMemoryError:
> unable to create new native thread*. This was expected as the dw-* thread
> count was increasing when I checked via curl *http://localhost
> <http://localhost>:<admin port>/threads *and the number of dw-* threads
> and WAITING threads increased.
>
> I have shared the thread dumps below..
>
> 1. https://drive.google.com/open?id=0B2IzsO0HbvslYzhBT3RvUG5WOWc
>
> 2. https://drive.google.com/open?id=0B2IzsO0HbvslODVlNUFweGpsREk
>
> It seems like quite a few threads are waiting on the log.info statement
> and we suspect it could be the culprit here.
>
> at ch.qos.logback.classic.Logger.info(Logger.java:608)
>
>
> Has anyone else come across this issue with an upgrade? It looks http
> request specific (and not really database related) Any insights or tips
> would greatly help us as we continue to investigate the issue.
> Another observation - don't see any 500 http error codes in the stage
> requests.log. In the QA environment, we do see many 500s as expected for
> bad requests.
>
> Thanks
> Sonya
>
--
This email message, including any attachment(s), is intended only for the
named recipient(s) and may contain confidential, proprietary or legally
privileged information. Unauthorized individuals or entities are not
permitted access to this information. Any dissemination, distribution,
disclosure, or copying of this information is unauthorized and strictly
prohibited. If you have received this message in error, please advise the
sender by reply email, and delete this message and any attachments.
--
You received this message because you are subscribed to the Google Groups
"dropwizard-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
