Latest SVN version, no alloccol set, still crashing a lot. I don't use [<- or $<-, the only times I modify a data.table are with := or by doing DT=merge(DT,blah).
Any more info I can provide? On 15 December 2011 08:32, Matthew Dowle <[email protected]> wrote: > Great fingers and toes crossed. If you could unset alloccol option just to > be sure please, that would be great. You're our best hope of confirming > it's fixed since it was biting you several times an hour. If you use [<- > or $<- syntax then R will copy via *tmp* and at that point the *tmp* > data.table is similar to a data.table loaded from disk in that it isn't > over-allocated anymore, I realised. Also a copy() will lose > over-allocation until the next column addition. That 'should' all be fine > now in both <=2.13.2 and >=2.14.0, although the bug was something simpler. > > 1.7.7 is on CRAN now and been built for windows so if CRAN check results > tick over from "ERROR" to "OK" later today (for both windows and mac > old-rel), and, you're ok too, then it's fixed. > > >> I've updated to the latest SVN version, and I'll be sure to let you >> know if it still crashes (however I do have the alloccol option set to >> 1000, so I shouldn't be bumping into reallocation very often). Thanks >> for finding the bug so fast! >> >> On 14 December 2011 19:56, Matthew Dowle <[email protected]> wrote: >>> >>> Hm. Sounds like it could be a different problem then if it was in R >>> 2.14. There have been quite a few fixes since 1.7.4 so if you can >>> reproduce with 1.7.7 would be great. Or, we've sometimes seen that just >>> after a package upgrade that a clean re-install can often fix things. >>> Perhaps if the .so was in use by another R process or a zombie, or >>> something. R seems to report data.table v1.7.4 (say) but it hasn't fully >>> installed it properly and is still (perhaps partially) at 1.7.3. So quit >>> all R (reboot to clear zombies too perhaps) and try reinstalling using R >>> CMD INSTALL. Next time it happens I mean. Can also run test.data.table() >>> to check the install. >>> >>> On Wed, 2011-12-14 at 17:40 +0000, Timothée Carayol wrote: >>>> Hi -- >>>> >>>> I have been having many unreproducible bugs with R 2.14, data.table >>>> 1.7.4 and ubuntu 64 bits about 10 days ago. Data was getting >>>> corrupted, and then R crashed. I had to go back to data.frame for the >>>> bits of code affected. I was doing a lot of rather unsafe >>>> manipulations with row names, rbind and cbinds. >>>> I didn't file a report, nor signal it, as it was occurring seemingly >>>> at random, and I was doing operations which aren't really what >>>> data.table was made for (tons of little manipulations on small data); >>>> still I guess I should now signal that 2.14 didn't fix everything for >>>> me. I do not know whether bugs subsist on post-1.7.4 versions. >>>> >>>> t >>>> >>>> On Wed, Dec 14, 2011 at 5:31 PM, Matthew Dowle <[email protected]> >>>> wrote: >>>> > >>>> > Maybe, worth a try. Are you loading any data.table objects from disk? >>>> > >>>> >> 64 bit 2.12.1 linux. >>>> >> >>>> >> Is there an option I can set in my session in order to work around >>>> the >>>> >> truelength issue? I don't care if I lose some of the over-allocation >>>> >> niceties if it stops things from crashing. Looking at the truelength >>>> >> help, would just doing: >>>> >> >>>> >> options(datatable.alloc=quote(1000)) >>>> >> >>>> >> stop this? I never have more than about 50 columns at a time. >>>> >> >>>> >> On 14 December 2011 11:43, Matthew Dowle <[email protected]> >>>> wrote: >>>> >>> >>>> >>> You're R < 2.14.0, right? I'm really struggling in R < 2.14.0 to >>>> make >>>> >>> over-allocation work because R only started to initialize >>>> truelength to >>>> >>> 0 >>>> >>> in R 2.14.0+. Before that it's unitialized (random). Trouble is my >>>> >>> attempts in R < 2.14.0 to work around that work fine for me in >>>> linux >>>> >>> 32bit >>>> >>> when I test in R 2.13.2, and I even test in 2.12.0 too. I test on >>>> 64bit >>>> >>> too but just 2.14.0. CRAN is also showing errors on 2.13.2 >>>> (old-rel) >>>> >>> for >>>> >>> both mac and windows. >>>> >>> >>>> >>> So, this is a pre-2.14.0 (only) problem that I'll continue to try >>>> and >>>> >>> fix. >>>> >>> >>>> >>> Are you 64bit pre-2.14.0? Which OS? If you are 64bit linux then it >>>> adds >>>> >>> weight to me installing pre-2.14.0 on my 64bit instance in an >>>> effort to >>>> >>> reproduce. >>>> >>> >>>> >>> >>>> >>>> This will be a crappy help request because I can't seem to >>>> reproduce >>>> >>>> it, but the past few days I've been getting a lot of segfaults. >>>> The >>>> >>>> only common thing between every crash is that it happens when I do >>>> >>>> >>>> >>>> DT[, z := x] >>>> >>>> >>>> >>>> where z was not a column that existed in DT before, and x is >>>> either an >>>> >>>> existing column of DT or a separate variable, doesn't matter. >>>> Beyond >>>> >>>> that I can't reproduce a set of steps that gets R to crash. This >>>> is >>>> >>>> with the latest SVN version. >>>> >>>> >>>> >>>> Is there more information I can provide to help track this down? >>>> >>>> _______________________________________________ >>>> >>>> datatable-help mailing list >>>> >>>> [email protected] >>>> >>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >>>> >>>> >>>> >>> >>>> >>> >>>> >> >>>> > >>>> > >>>> > _______________________________________________ >>>> > datatable-help mailing list >>>> > [email protected] >>>> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >>> >>> >>> _______________________________________________ >>> datatable-help mailing list >>> [email protected] >>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >> > > _______________________________________________ datatable-help mailing list [email protected] https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
