Re: Application silently shuts down by itself after running for some hours

[Herwig Hochleitner]

2017-03-08 9:00 GMT+01:00 JokkeB :

> Now tracking down if I'm simply too low on memory or if the app has a
> memory leak, but that's a different topic.
>

Your app may well be fine. The OOM killer removes processes in descending
order of memory usage, when RAM runs out. As java processes are usually
large, those are amongst the first victims.
This happens even when your app has plenty of free heap, because the java
garbage collector reserves a lot of memory before allocating it, and you
only get OutOfMemoryExceptions, when the GC hits its limit (set by -Xmx).

To fix the OOM killer issue, simply add more swap space.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[JokkeB]

Thanks to all the replies. Yes, the reason indeed was linux killing the 
process. cat /var/log/kern.log verified this. There was no exception logged 
because there was no exception/error. Now tracking down if I'm simply too 
low on memory or if the app has a memory leak, but that's a different 
topic. 

Thanks again!

perjantai 3. maaliskuuta 2017 17.57.20 UTC+2 JokkeB kirjoitti:
>
> I have an application which I run with "lein run". The main looks 
> something like this
>
> (defn -main []
>  (log/info "start")
>  (log/info "channel returned" (async/ ) (websocket-server 9122
>  (log/info "quit"))
>
> start-server returns a channel created with async/reduce which shouldn't 
> ever produce a value. I see the "start" logged, the application runs for a 
> day or two (not sure if the time is always the same) and then it closes 
> without any errors or without logging "channel returned" or "quit". 
>
> If there is an error in my code and the program quits I should see the 
> "quit" message. If there is a memory leak or something I would assume I'd 
> get an error message. Has anyone experienced anything similar?
>
> Could the reason be running it with lein (I haven't tried a jar)? If so, 
> why?
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[Mark Nutter]

I see this thread is going on for a while, and I'll say up front that I
don't have any helpful insights. I did see an interesting talk at
Abstractions, though, about how to use dtrace to get to the bottom of weird
problems like this. The video for that event doesn't seem to be online
anywhere, but the speaker gave a similar talk at !!Con, and that one is
online. If you're still having issues you might want to check this out, in
case it gives you an extra tool to use for tracking this down.

https://www.youtube.com/watch?v=1OMX69KOhGg

Good luck.


On Fri, Mar 3, 2017 at 10:57 AM, JokkeB  wrote:

> I have an application which I run with "lein run". The main looks
> something like this
>
> (defn -main []
>  (log/info "start")
>  (log/info "channel returned" (async/ ) (websocket-server 9122
>  (log/info "quit"))
>
> start-server returns a channel created with async/reduce which shouldn't
> ever produce a value. I see the "start" logged, the application runs for a
> day or two (not sure if the time is always the same) and then it closes
> without any errors or without logging "channel returned" or "quit".
>
> If there is an error in my code and the program quits I should see the
> "quit" message. If there is a memory leak or something I would assume I'd
> get an error message. Has anyone experienced anything similar?
>
> Could the reason be running it with lein (I haven't tried a jar)? If so,
> why?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[lawrence . krubner]

To catch OutOfMemoryError s: 

catch(OutOfMemoryError e)




On Tuesday, March 7, 2017 at 5:18:44 PM UTC-5, JokkeB wrote:
>
> I'm under the impression that setDefaultExceptionHandler still catches OOM 
> errors. Some googling suggests this too. If not, how should I try to catch 
> it?
>
> I am running the app on a virtual server with 512mb ram. free -m is 
> showing 30mb free. Lack of memory can be the issue. What would be the best 
> way to confirm this if I'm not able to log the error?
>
> Sent from my iPhone
>
> On 7 Mar 2017, at 23.25, piast...@gmail.com  wrote:
>
> I asked the same question a year ago. The problem was that I was getting 
> an OutOfMemoryError. This is an Error but it is not an Exception. If you 
> catch all Exceptions, you will still not catch the OutOfMemoryError. You 
> have to catch that too. 
>
>
>
> On Tuesday, March 7, 2017 at 2:54:26 PM UTC-5, Kevin Corcoran wrote:
>>
>> On Mon, Mar 6, 2017 at 11:56 PM, JokkeB  wrote:
>>
>>> After adding this, I still can't see an exception before the app dies.
>>>
>>
>> I've encountered this before when the Linux "OOM killer" kicks in, which 
>> is especially likely if you are running your application on a 
>> resource-constrained system (say, a VM with a low RAM allocation) or your 
>> application is competing with other programs for memory.
>>
>> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to a topic in the 
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/clojure/zhdcNMC7fQ8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[Johannes Ahvenniemi]

I'm under the impression that setDefaultExceptionHandler still catches OOM 
errors. Some googling suggests this too. If not, how should I try to catch it?

I am running the app on a virtual server with 512mb ram. free -m is showing 
30mb free. Lack of memory can be the issue. What would be the best way to 
confirm this if I'm not able to log the error?

Sent from my iPhone

> On 7 Mar 2017, at 23.25, piastkra...@gmail.com wrote:
> 
> I asked the same question a year ago. The problem was that I was getting an 
> OutOfMemoryError. This is an Error but it is not an Exception. If you catch 
> all Exceptions, you will still not catch the OutOfMemoryError. You have to 
> catch that too. 
> 
> 
> 
>> On Tuesday, March 7, 2017 at 2:54:26 PM UTC-5, Kevin Corcoran wrote:
>>> On Mon, Mar 6, 2017 at 11:56 PM, JokkeB  wrote:
>>> After adding this, I still can't see an exception before the app dies.
>> 
>> I've encountered this before when the Linux "OOM killer" kicks in, which is 
>> especially likely if you are running your application on a 
>> resource-constrained system (say, a VM with a low RAM allocation) or your 
>> application is competing with other programs for memory.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to a topic in the Google 
> Groups "Clojure" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/clojure/zhdcNMC7fQ8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[Alex Miller]

If you are getting an OOME there are some JVM flags that can help dump an 
error file or heap dump, or run arbitrary commands when an error occurs:

-XX:ErrorFile=./hs_err_pid.log
-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=./java_pid.hprof
-XX:OnError=";"
-XX:OnOutOfMemoryError=";"

That might help you determine the cause if it is an error condition.

On Tuesday, March 7, 2017 at 3:25:52 PM UTC-6, piastkra...@gmail.com wrote:
>
> I asked the same question a year ago. The problem was that I was getting 
> an OutOfMemoryError. This is an Error but it is not an Exception. If you 
> catch all Exceptions, you will still not catch the OutOfMemoryError. You 
> have to catch that too. 
>
>
>
> On Tuesday, March 7, 2017 at 2:54:26 PM UTC-5, Kevin Corcoran wrote:
>>
>> On Mon, Mar 6, 2017 at 11:56 PM, JokkeB  wrote:
>>
>>> After adding this, I still can't see an exception before the app dies.
>>>
>>
>> I've encountered this before when the Linux "OOM killer" kicks in, which 
>> is especially likely if you are running your application on a 
>> resource-constrained system (say, a VM with a low RAM allocation) or your 
>> application is competing with other programs for memory.
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[piastkrakow]

I asked the same question a year ago. The problem was that I was getting an 
OutOfMemoryError. This is an Error but it is not an Exception. If you catch 
all Exceptions, you will still not catch the OutOfMemoryError. You have to 
catch that too. 



On Tuesday, March 7, 2017 at 2:54:26 PM UTC-5, Kevin Corcoran wrote:
>
> On Mon, Mar 6, 2017 at 11:56 PM, JokkeB  
> wrote:
>
>> After adding this, I still can't see an exception before the app dies.
>>
>
> I've encountered this before when the Linux "OOM killer" kicks in, which 
> is especially likely if you are running your application on a 
> resource-constrained system (say, a VM with a low RAM allocation) or your 
> application is competing with other programs for memory.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[Kevin Corcoran]

On Mon, Mar 6, 2017 at 11:56 PM, JokkeB  wrote:

> After adding this, I still can't see an exception before the app dies.
>

I've encountered this before when the Linux "OOM killer" kicks in, which is
especially likely if you are running your application on a
resource-constrained system (say, a VM with a low RAM allocation) or your
application is competing with other programs for memory.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[JokkeB]

After adding this, I still can't see an exception before the app dies.

maanantai 6. maaliskuuta 2017 12.04.16 UTC+2 JokkeB kirjoitti:
>
> Thanks for the reply.
>
> I haven't tried catching any or all errors. Is it a false assumption that 
> an error should be logged if it comes from the main thread? Can an 
> exception in another thread kill the app? Anyways, I now added this piece 
> of code in main, like instructed in 
> https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions and let's 
> see what happens. Is this what you meant?
>
>   (Thread/setDefaultUncaughtExceptionHandler
>(reify Thread$UncaughtExceptionHandler
>  (uncaughtException [_ thread ex]
>(log/error ex "Uncaught exception on" (.getName thread)
>
>
>
> lauantai 4. maaliskuuta 2017 13.02.53 UTC+2 Scott Bauer kirjoitti:
>>
>> Out of curiosity, have you tried catching any and all errors as opposed 
>> to expecting to see an error message logged upon the system dying?
>>
>> While searching for possible causes, I did come across this similar post 
>>  
>> from a few years ago... not sure if it will help, but it may.
>>
>>
>> On Friday, March 3, 2017 at 10:57:20 AM UTC-5, JokkeB wrote:
>>>
>>> I have an application which I run with "lein run". The main looks 
>>> something like this
>>>
>>> (defn -main []
>>>  (log/info "start")
>>>  (log/info "channel returned" (async/>> 9121) (websocket-server 9122
>>>  (log/info "quit"))
>>>
>>> start-server returns a channel created with async/reduce which shouldn't 
>>> ever produce a value. I see the "start" logged, the application runs for a 
>>> day or two (not sure if the time is always the same) and then it closes 
>>> without any errors or without logging "channel returned" or "quit". 
>>>
>>> If there is an error in my code and the program quits I should see the 
>>> "quit" message. If there is a memory leak or something I would assume I'd 
>>> get an error message. Has anyone experienced anything similar?
>>>
>>> Could the reason be running it with lein (I haven't tried a jar)? If so, 
>>> why?
>>>
>>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[JokkeB]

Thanks for the reply.

I haven't tried catching any or all errors. Is it a false assumption that 
an error should be logged if it comes from the main thread? Can an 
exception in another thread kill the app? Anyways, I now added this piece 
of code in main, like instructed 
in https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions and 
let's see what happens. Is this what you meant?

  (Thread/setDefaultUncaughtExceptionHandler
   (reify Thread$UncaughtExceptionHandler
 (uncaughtException [_ thread ex]
   (log/error ex "Uncaught exception on" (.getName thread)



lauantai 4. maaliskuuta 2017 13.02.53 UTC+2 Scott Bauer kirjoitti:
>
> Out of curiosity, have you tried catching any and all errors as opposed to 
> expecting to see an error message logged upon the system dying?
>
> While searching for possible causes, I did come across this similar post 
>  
> from a few years ago... not sure if it will help, but it may.
>
>
> On Friday, March 3, 2017 at 10:57:20 AM UTC-5, JokkeB wrote:
>>
>> I have an application which I run with "lein run". The main looks 
>> something like this
>>
>> (defn -main []
>>  (log/info "start")
>>  (log/info "channel returned" (async/> 9121) (websocket-server 9122
>>  (log/info "quit"))
>>
>> start-server returns a channel created with async/reduce which shouldn't 
>> ever produce a value. I see the "start" logged, the application runs for a 
>> day or two (not sure if the time is always the same) and then it closes 
>> without any errors or without logging "channel returned" or "quit". 
>>
>> If there is an error in my code and the program quits I should see the 
>> "quit" message. If there is a memory leak or something I would assume I'd 
>> get an error message. Has anyone experienced anything similar?
>>
>> Could the reason be running it with lein (I haven't tried a jar)? If so, 
>> why?
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Application silently shuts down by itself after running for some hours

[Scott Bauer]

Out of curiosity, have you tried catching any and all errors as opposed to 
expecting to see an error message logged upon the system dying?

While searching for possible causes, I did come across this similar post 
 
from a few years ago... not sure if it will help, but it may.


On Friday, March 3, 2017 at 10:57:20 AM UTC-5, JokkeB wrote:
>
> I have an application which I run with "lein run". The main looks 
> something like this
>
> (defn -main []
>  (log/info "start")
>  (log/info "channel returned" (async/ ) (websocket-server 9122
>  (log/info "quit"))
>
> start-server returns a channel created with async/reduce which shouldn't 
> ever produce a value. I see the "start" logged, the application runs for a 
> day or two (not sure if the time is always the same) and then it closes 
> without any errors or without logging "channel returned" or "quit". 
>
> If there is an error in my code and the program quits I should see the 
> "quit" message. If there is a memory leak or something I would assume I'd 
> get an error message. Has anyone experienced anything similar?
>
> Could the reason be running it with lein (I haven't tried a jar)? If so, 
> why?
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.