Re: [sage-devel] Overcommitted memory and error handling
On 2016-10-18 12:13, Jeroen Demeyer wrote: I think it crashes for a different reason: there are some unchecked allocations in realloc(). This has nothing to do with overcommit. Fixed at https://trac.sagemath.org/ticket/21720 -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Overcommitted memory and error handling
I think it crashes for a different reason: there are some unchecked allocations in realloc(). This has nothing to do with overcommit. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Overcommitted memory and error handling
On 2016-10-17 17:40, Jori Mäntysalo wrote: Is it even theoretically possible to handle something like Graph(10^10)? That will crash at the line memset(self.vertices, NULL, nverts * self.hash_length * sizeof(SparseGraphBTNode *)) Are you *sure* that it crashes at the above line? It seems to me that there is integer overflow involved in the number of vertices which is of type "int" so 10^10 may be too large for it. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Overcommitted memory and error handling
On Tue, 18 Oct 2016, Jeroen Demeyer wrote: Setting (as root) the value of /proc/sys/vm/overcommit_memory will not help Are you sure? With vm.overcommit_memory set to "2", the error should happen during allocation. Yes, I tested that too. Just add print-function before and after the line I sent, and you can see that it will fail in memset(). (But, as said, with different error message.) I used 64-bit Ubuntu 16.04 LTS with most recent kernel from Ubuntu packages. This kernel should not have any tweaks. -- Jori Mäntysalo
Re: [sage-devel] Overcommitted memory and error handling
On 2016-10-17 17:40, Jori Mäntysalo wrote: Setting (as root) the value of /proc/sys/vm/overcommit_memory will not help Are you sure? With vm.overcommit_memory set to "2", the error should happen during allocation. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Overcommitted memory and error handling
On 2016-10-17 17:40, Jori Mäntysalo wrote: but due to overcommit there is no exception raised. That's really the fault of the OS, not the fault of Sage. Side comment: don't use x = malloc(n) memset(x, 0, n) There is calloc for that. And *certainly* don't use NULL, use 0 for that. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
[sage-devel] Overcommitted memory and error handling
Is it even theoretically possible to handle something like Graph(10^10)?
That will crash at the line
memset(self.vertices, NULL, nverts * self.hash_length *
sizeof(SparseGraphBTNode *))
Before that we have
sig_malloc(nverts * self.hash_length * . . .
if not . . .
raise RuntimeError("Failure allocating memory.")
but due to overcommit there is no exception raised. Setting (as root) the
value of /proc/sys/vm/overcommit_memory will not help, but may change
error message given.
--
Jori Mäntysalo
