Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package moarvm for openSUSE:Factory checked in at 2021-07-25 20:09:17 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/moarvm (Old) and /work/SRC/openSUSE:Factory/.moarvm.new.1899 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "moarvm" Sun Jul 25 20:09:17 2021 rev:31 rq:908137 version:2021.07 Changes: -------- --- /work/SRC/openSUSE:Factory/moarvm/moarvm.changes 2021-07-10 22:55:19.767375600 +0200 +++ /work/SRC/openSUSE:Factory/.moarvm.new.1899/moarvm.changes 2021-07-25 20:09:26.755431475 +0200 @@ -1,0 +2,17 @@ +Sat Jul 24 21:25:38 CEST 2021 - n...@detonation.org + +- update to version 2021.07 + ++ Fix "Collectable in a gen2 freelist accessed" when deserializing a method cache ++ Fix possible acces to a freed SC during deserialization ++ Convert more realloc+memset(0) to recalloc ++ Fix some bugs in the various `hash_fsck` functions ++ Fix read buffer overflow in bytecode dumper ++ Fix overflows on slight reallocations of fixed size alloced buffers ++ Fix error handling of mkdir ++ Assign the result of a socket(2) call immediately ++ Fix memory corruption by accessing freed spesh stats ++ Fix ShiftJIS decoder bug where a replacement is used + and a decoding error is encountered after '\r' character + +------------------------------------------------------------------- Old: ---- MoarVM-2021.06.tar.gz New: ---- MoarVM-2021.07.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ moarvm.spec ++++++ --- /var/tmp/diff_new_pack.hrWigZ/_old 2021-07-25 20:09:27.307430860 +0200 +++ /var/tmp/diff_new_pack.hrWigZ/_new 2021-07-25 20:09:27.307430860 +0200 @@ -16,7 +16,7 @@ # -%global mvrel 2021.06 +%global mvrel 2021.07 Name: moarvm Version: %mvrel Release: 2.1 ++++++ MoarVM-2021.06.tar.gz -> MoarVM-2021.07.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/Configure.pl new/MoarVM-2021.07/Configure.pl --- old/MoarVM-2021.06/Configure.pl 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/Configure.pl 2021-07-24 12:24:40.000000000 +0200 @@ -594,13 +594,6 @@ $config{dasm_flags} = ''; } - -if ($config{cc} eq 'cl') { - $config{install} .= "\t\$(MKPATH) \"\$(DESTDIR)\$(PREFIX)/include/msinttypes\"\n" - . "\t\$(CP) 3rdparty/msinttypes/*.h \"\$(DESTDIR)\$(PREFIX)/include/msinttypes\"\n"; - push @hllincludes, 'msinttypes'; -} - if ($^O eq 'aix' && $config{ptr_size} == 4) { $config{ldflags} = join(',', $config{ldflags}, '-bmaxdata:0x80000000'); } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/README.markdown new/MoarVM-2021.07/README.markdown --- old/MoarVM-2021.06/README.markdown 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/README.markdown 2021-07-24 12:24:40.000000000 +0200 @@ -89,7 +89,7 @@ Contributions by pull request are accepted. Commit bits are given to those who contribute quality work. If you are interested in contributing, drop by the -`#moarvm` channel on freenode.org. +`#moarvm` channel on libera.chat. See the LICENSE file in the root directory for information on the license of the source code in the MoarVM repository. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/VERSION new/MoarVM-2021.07/VERSION --- old/MoarVM-2021.06/VERSION 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/VERSION 2021-07-24 12:24:40.000000000 +0200 @@ -1 +1 @@ -2021.06 +2021.07 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/docs/ChangeLog new/MoarVM-2021.07/docs/ChangeLog --- old/MoarVM-2021.06/docs/ChangeLog 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/docs/ChangeLog 2021-07-24 12:24:40.000000000 +0200 @@ -1,3 +1,33 @@ +New in 2021.07 + +6model: ++ [21f8cadd] Fix "Collectable in a gen2 freelist accessed" when deserializing a method cache ++ [d7a07bf3] Fix possible acces to a freed SC during deserialization ++ [8f513d55] Convert more `realloc+memset(0)` to `recalloc` + +Core: ++ [736154d2] Fix some bugs in the various `hash_fsck` functions ++ [1180c988] Fix read buffer overflow in bytecode dumper ++ [a27efd30] Fix overflows on slight reallocations of fixed size alloced buffers + +IO: ++ [8e9afcf5] Fix error handling of `mkdir` ++ [97948c2a] Assign the result of a `socket(2)` call immediately ++ [af001c96,3a3eb7f9] Fix possible deadlock when waiting for mutex_threads + +Profiler: ++ [ab2a0100,f8d5f300] Simplify code needed for GCC to avoid a warning + +Spesh: ++ [a4d59492] Fix memory corruption by accessing freed spesh stats + +Strings: ++ [4fea2f9a] Fix ShiftJIS decoder bug where a replacement is used + and a decoding error is encountered after '\r' character + +Tooling/Build: ++ [42bbff95] Remove msinttypes from `$PREFIX\include` as no longer needed + New in 2021.06 6model: @@ -9,7 +39,7 @@ Tooling/Build: + [9b7ab7c4] Put include paths needed for HLL builds in config -+ [91b1ccf6,bb767ece,192ea6bb] Define _GNU_SOURCE for GNU builds ++ [91b1ccf6,bb767ece,192ea6bb] Define `_GNU_SOURCE` for GNU builds + [12cbafce] Disable GCC pragmas when using MSVC + [4751ca6d] Add note about macro confusion between gcc and clang diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/docs/contributing.org new/MoarVM-2021.07/docs/contributing.org --- old/MoarVM-2021.06/docs/contributing.org 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/docs/contributing.org 2021-07-24 12:24:40.000000000 +0200 @@ -5,7 +5,7 @@ * Community MoarVM is a fairly small community. We communicate primarily on the -#moarvm channel on freenode, and to a lesser extent via github issues. +#moarvm channel on libera.chat, and to a lesser extent via github issues. We use IRC bots to keep up to date with new commits that are published; but those don't always work well, unfortunately. @@ -13,9 +13,6 @@ The [[https://rakudoweekly.blog/][Rakudo Weekly News]] newsletter publishes progress in MoarVM, and there's also the =/r/rakulang= subreddit. -Apparently you can connect to freenode via discord, but the author of -this text wouldn't know how. - * Code Standards Some things to keep in mind are: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/docs/reveal.md new/MoarVM-2021.07/docs/reveal.md --- old/MoarVM-2021.06/docs/reveal.md 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/docs/reveal.md 2021-07-24 12:24:40.000000000 +0200 @@ -163,4 +163,4 @@ ## Where is... * The Git repository: https://github.com/MoarVM/MoarVM -* The IRC channel: #moarvm on freenode.org +* The IRC channel: #moarvm on libera.chat diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/ports/macports/README.md new/MoarVM-2021.07/ports/macports/README.md --- old/MoarVM-2021.06/ports/macports/README.md 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/ports/macports/README.md 2021-07-24 12:24:40.000000000 +0200 @@ -47,6 +47,6 @@ * **port**: `MoarVM` * Make sure to attach the diff. - * Hang out in `#macports` on freenode and mention the ticket or write to the macports-dev mailing list with URL to the ticket and a descriptive subject if there is no response for a while. + * Hang out in `#macports` on libera.chat and mention the ticket or write to the macports-dev mailing list with URL to the ticket and a descriptive subject if there is no response for a while. 5. Push the updated `Portfile` also to git. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/6model/reprs/CArray.c new/MoarVM-2021.07/src/6model/reprs/CArray.c --- old/MoarVM-2021.06/src/6model/reprs/CArray.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/6model/reprs/CArray.c 2021-07-24 12:24:40.000000000 +0200 @@ -214,8 +214,7 @@ const size_t old_size = body->allocated * repr_data->elem_size; const size_t new_size = next_size * repr_data->elem_size; - body->storage = MVM_realloc(body->storage, new_size); - memset((char *)body->storage + old_size, 0, new_size - old_size); + body->storage = MVM_recalloc(body->storage, old_size, new_size); } is_complex = (repr_data->elem_kind == MVM_CARRAY_ELEM_KIND_CARRAY @@ -229,8 +228,7 @@ const size_t old_size = body->allocated * sizeof(MVMObject *); const size_t new_size = next_size * sizeof(MVMObject *); - body->child_objs = (MVMObject **) MVM_realloc(body->child_objs, new_size); - memset((char *)body->child_objs + old_size, 0, new_size - old_size); + body->child_objs = (MVMObject **) MVM_recalloc(body->child_objs, old_size, new_size); } body->allocated = next_size; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/6model/sc.c new/MoarVM-2021.07/src/6model/sc.c --- old/MoarVM-2021.06/src/6model/sc.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/6model/sc.c 2021-07-24 12:24:40.000000000 +0200 @@ -315,10 +315,8 @@ sc->body->alloc_stables += 32; if (sc->body->alloc_stables < (MVMuint64)idx + 1) sc->body->alloc_stables = idx + 1; - sc->body->root_stables = MVM_realloc(sc->body->root_stables, + sc->body->root_stables = MVM_recalloc(sc->body->root_stables, orig_size, sc->body->alloc_stables * sizeof(MVMSTable *)); - memset(sc->body->root_stables + orig_size, 0, - (sc->body->alloc_stables - orig_size) * sizeof(MVMSTable *)); } MVM_ASSIGN_REF(tc, &(sc->common.header), sc->body->root_stables[idx], st); sc->body->num_stables = idx + 1; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/6model/serialization.c new/MoarVM-2021.07/src/6model/serialization.c --- old/MoarVM-2021.06/src/6model/serialization.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/6model/serialization.c 2021-07-24 12:24:40.000000000 +0200 @@ -3006,12 +3006,14 @@ MVM_gc_allocate_gen2_default_set(tc); /* Stub the object. */ - stub_object(tc, sr, idx); + MVMROOT(tc, sc, { + stub_object(tc, sr, idx); - /* Add to worklist and process as needed. */ - worklist_add_index(tc, &(sr->wl_objects), idx); - if (sr->working == 1) - work_loop(tc, sr); + /* Add to worklist and process as needed. */ + worklist_add_index(tc, &(sr->wl_objects), idx); + if (sr->working == 1) + work_loop(tc, sr); + }); /* Clear up. */ MVM_gc_allocate_gen2_default_clear(tc); @@ -3061,12 +3063,14 @@ MVM_gc_allocate_gen2_default_set(tc); /* Stub the STable. */ - stub_stable(tc, sr, idx); + MVMROOT(tc, sc, { + stub_stable(tc, sr, idx); - /* Add to worklist and process as needed. */ - worklist_add_index(tc, &(sr->wl_stables), idx); - if (sr->working == 1) - work_loop(tc, sr); + /* Add to worklist and process as needed. */ + worklist_add_index(tc, &(sr->wl_stables), idx); + if (sr->working == 1) + work_loop(tc, sr); + }); /* Clear up. */ MVM_gc_allocate_gen2_default_clear(tc); @@ -3095,12 +3099,14 @@ sr->working++; MVM_gc_allocate_gen2_default_set(tc); - /* Deserialize the code object. */ - deserialize_closure(tc, sr, idx - sr->num_static_codes); + MVMROOT(tc, sc, { + /* Deserialize the code object. */ + deserialize_closure(tc, sr, idx - sr->num_static_codes); - /* Add to worklist and process as needed. */ - if (sr->working == 1) - work_loop(tc, sr); + /* Add to worklist and process as needed. */ + if (sr->working == 1) + work_loop(tc, sr); + }); /* Clear up. */ MVM_gc_allocate_gen2_default_clear(tc); @@ -3169,8 +3175,10 @@ /* Deserialize what we need. */ cache = MVM_serialization_read_ref(tc, sr); - if (sr->working == 1) - work_loop(tc, sr); + MVMROOT3(tc, st, sc, cache, { /* Keep cache from getting freed prematurely */ + if (sr->working == 1) + work_loop(tc, sr); + }); MVM_ASSIGN_REF(tc, &(st->header), st->method_cache, cache); /* Clear up. */ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/core/bytecodedump.c new/MoarVM-2021.07/src/core/bytecodedump.c --- old/MoarVM-2021.06/src/core/bytecodedump.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/core/bytecodedump.c 2021-07-24 12:24:40.000000000 +0200 @@ -9,10 +9,15 @@ va_list args; va_start(args, str); - vsnprintf(string, line_length, str, args); + int append_len = vsnprintf(string, line_length, str, args); va_end(args); + if (append_len >= line_length) { /* str got truncated by vsnprintf */ + string[line_length - 1] = '\0'; + len = line_length; + } + else + len = strlen(string) + 1; /* include terminating \0 so we copy that, too */ - len = strlen(string); if (*length + len > *size) { while (*length + len > *size) *size = *size * 2; @@ -20,7 +25,7 @@ } memcpy(*out + *length, string, len); - *length = *length + len; + *length = *length + len - 1; /* exclude \0 so we append before it */ } static const char * get_typename(MVMuint16 type) { @@ -105,7 +110,7 @@ /* allocate a line buffer */ s = 200; l = 0; - o = MVM_calloc(s, sizeof(char)); + o = MVM_malloc(s * sizeof(char)); lineloc = cur_op - bytecode_start; /* mark that this line starts at this point in the bytestream */ @@ -370,7 +375,7 @@ MVMuint32 s = 1024; MVMuint32 l = 0; MVMuint32 i, j, k; - char *o = MVM_calloc(s, sizeof(char)); + char *o = MVM_malloc(s * sizeof(char)); char ***frame_lexicals = MVM_malloc(sizeof(char **) * cu->body.num_frames); a("\nMoarVM dump of binary compilation unit:\n\n"); @@ -493,7 +498,7 @@ void MVM_dump_bytecode_of(MVMThreadContext *tc, MVMFrame *frame, MVMSpeshCandidate *maybe_candidate) { MVMuint32 s = 1024; MVMuint32 l = 0; - char *o = MVM_calloc(s, sizeof(char)); + char *o = MVM_malloc(s * sizeof(char)); MVMuint8 *addr; if (!frame) { @@ -516,7 +521,7 @@ void MVM_dump_bytecode_staticframe(MVMThreadContext *tc, MVMStaticFrame *frame) { MVMuint32 s = 1024; MVMuint32 l = 0; - char *o = MVM_calloc(s, sizeof(char)); + char *o = MVM_malloc(s * sizeof(char)); bytecode_dump_frame_internal(tc, frame, NULL, NULL, NULL, &o, &s, &l); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/core/fixedsizealloc.c new/MoarVM-2021.07/src/core/fixedsizealloc.c --- old/MoarVM-2021.06/src/core/fixedsizealloc.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/core/fixedsizealloc.c 2021-07-24 12:24:40.000000000 +0200 @@ -228,12 +228,12 @@ #else MVMuint32 old_bin = bin_for(old_bytes); MVMuint32 new_bin = bin_for(new_bytes); - if (old_bin == new_bin) { + if (old_bin < MVM_FSA_BINS && old_bin == new_bin) { return p; } else if (old_bin < MVM_FSA_BINS || new_bin < MVM_FSA_BINS) { void *allocd = MVM_fixed_size_alloc(tc, al, new_bytes); - memcpy(allocd, p, new_bin > old_bin ? old_bytes : new_bytes); + memcpy(allocd, p, new_bytes > old_bytes ? old_bytes : new_bytes); MVM_fixed_size_free(tc, al, old_bytes, p); return allocd; } @@ -253,12 +253,12 @@ #else MVMuint32 old_bin = bin_for(old_bytes); MVMuint32 new_bin = bin_for(new_bytes); - if (old_bin == new_bin) { + if (old_bin < MVM_FSA_BINS && old_bin == new_bin) { return p; } else { void *allocd = MVM_fixed_size_alloc(tc, al, new_bytes); - memcpy(allocd, p, new_bin > old_bin ? old_bytes : new_bytes); + memcpy(allocd, p, new_bytes > old_bytes ? old_bytes : new_bytes); MVM_fixed_size_free_at_safepoint(tc, al, old_bytes, p); return allocd; } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/core/fixkey_hash_table.c new/MoarVM-2021.07/src/core/fixkey_hash_table.c --- old/MoarVM-2021.06/src/core/fixkey_hash_table.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/core/fixkey_hash_table.c 2021-07-24 12:24:40.000000000 +0200 @@ -418,7 +418,7 @@ ++metadata; entry_raw -= sizeof(MVMString ***); } - if (*metadata != 1) { + if (*metadata != 0) { ++errors; if (display) { fprintf(stderr, "%s %02x!\n", prefix_hashes, *metadata); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/core/str_hash_table.c new/MoarVM-2021.07/src/core/str_hash_table.c --- old/MoarVM-2021.06/src/core/str_hash_table.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/core/str_hash_table.c 2021-07-24 12:24:40.000000000 +0200 @@ -796,10 +796,8 @@ if (offset < 1) { wrong_order = '<'; } else if (offset > control->max_probe_distance) { - ++errors; wrong_order = '>'; } else if (offset > prev_offset + 1) { - ++errors; wrong_order = '!'; } else { wrong_order = ' '; @@ -859,7 +857,7 @@ ++metadata; entry_raw -= control->entry_size; } - if (*metadata != 1) { + if (*metadata != 0) { ++errors; if (display) { fprintf(stderr, "%s %02x!\n", prefix_hashes, *metadata); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/core/uni_hash_table.c new/MoarVM-2021.07/src/core/uni_hash_table.c --- old/MoarVM-2021.06/src/core/uni_hash_table.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/core/uni_hash_table.c 2021-07-24 12:24:40.000000000 +0200 @@ -373,10 +373,8 @@ if (offset < 1) { wrong_order = '<'; } else if (offset > control->max_probe_distance) { - ++errors; wrong_order = '>'; } else if (offset > prev_offset + 1) { - ++errors; wrong_order = '!'; } else { wrong_order = ' '; @@ -395,7 +393,7 @@ ++metadata; entry_raw -= sizeof(struct MVMUniHashEntry); } - if (*metadata != 1) { + if (*metadata != 0) { ++errors; if (display) { fprintf(stderr, "%s %02x!\n", prefix_hashes, *metadata); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/core/vector.h new/MoarVM-2021.07/src/core/vector.h --- old/MoarVM-2021.06/src/core/vector.h 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/core/vector.h 2021-07-24 12:24:40.000000000 +0200 @@ -36,8 +36,7 @@ #define MVM_VECTOR_GROW(x, size) do {\ size_t _s = (size); \ - (x) = MVM_realloc(x, _s*sizeof(*x)); \ - memset((x) + (x ## _alloc), 0, (_s - (x ## _alloc)) * sizeof(*(x))); \ + (x) = MVM_recalloc(x, MVM_VECTOR_SIZE(x), _s*sizeof(*x)); \ (x ## _alloc) = _s; \ } while (0) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/io/dirops.c new/MoarVM-2021.07/src/io/dirops.c --- old/MoarVM-2021.06/src/io/dirops.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/io/dirops.c 2021-07-24 12:24:40.000000000 +0200 @@ -44,15 +44,11 @@ ch = *p; *p = '\0'; #ifdef _WIN32 - if (CreateDirectoryW(pathname, NULL)) { - created = 1; - } + created = CreateDirectoryW(pathname, NULL); #else - if (uv_fs_stat(NULL, &req, pathname, NULL) <= 0) { - if (mkdir(pathname, mode) != -1) { - created = 1; - } - } + created = (uv_fs_stat(NULL, &req, pathname, NULL) == 0 + && S_ISDIR(req.statbuf.st_mode) + || mkdir(pathname, mode) == 0); #endif if (!(*p = ch)) break; } @@ -102,7 +98,7 @@ MVM_free(wpathname); #else - if (mkdir_p(tc, pathname, mode) == -1 && errno != EEXIST) { + if (mkdir_p(tc, pathname, mode) == -1) { int mkdir_error = errno; MVM_free(pathname); MVM_exception_throw_adhoc(tc, "Failed to mkdir: %s", strerror(mkdir_error)); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/io/procops.c new/MoarVM-2021.07/src/io/procops.c --- old/MoarVM-2021.06/src/io/procops.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/io/procops.c 2021-07-24 12:24:40.000000000 +0200 @@ -1251,9 +1251,9 @@ /* Allow MVM_io_eventloop_start to restart the thread if necessary */ instance->event_loop_thread = NULL; - MVM_gc_mark_thread_blocked(tc); + /* Do not mark thread blocked as the GC also tries to acquire + * mutex_threads and it's held only briefly by all holders anyway */ uv_mutex_lock(&instance->mutex_threads); - MVM_gc_mark_thread_unblocked(tc); /* Check if we are single threaded and if true, fork() */ if (MVM_thread_cleanup_threads_list(tc, &instance->threads) == 1) { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/io/syncsocket.c new/MoarVM-2021.07/src/io/syncsocket.c --- old/MoarVM-2021.06/src/io/syncsocket.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/io/syncsocket.c 2021-07-24 12:24:40.000000000 +0200 @@ -394,7 +394,7 @@ struct sockaddr *dest = MVM_io_resolve_host_name(tc, host, port, family, MVM_SOCKET_TYPE_STREAM, MVM_SOCKET_PROTOCOL_ANY, 0); int r; - MVMSocket s = socket(dest->sa_family , SOCK_STREAM , 0); + MVMSocket s = data->handle = socket(dest->sa_family , SOCK_STREAM , 0); if (MVM_IS_SOCKET_ERROR(s)) { MVM_free(dest); MVM_telemetry_interval_stop(tc, interval_id, "syncsocket connect"); @@ -411,8 +411,6 @@ MVM_telemetry_interval_stop(tc, interval_id, "syncsocket connect"); throw_error(tc, s, "connect socket"); } - - data->handle = s; } else { MVM_telemetry_interval_stop(tc, interval_id, "syncsocket didn't connect"); @@ -426,7 +424,7 @@ struct sockaddr *dest = MVM_io_resolve_host_name(tc, host, port, family, MVM_SOCKET_TYPE_STREAM, MVM_SOCKET_PROTOCOL_ANY, 1); int r; - MVMSocket s = socket(dest->sa_family , SOCK_STREAM , 0); + MVMSocket s = data->handle = socket(dest->sa_family , SOCK_STREAM , 0); if (MVM_IS_SOCKET_ERROR(s)) { MVM_free(dest); throw_error(tc, s, "create socket"); @@ -453,8 +451,6 @@ r = listen(s, (int)backlog); if (MVM_IS_SOCKET_ERROR(r)) throw_error(tc, s, "start listening on socket"); - - data->handle = s; } else { MVM_exception_throw_adhoc(tc, "Socket is already bound or connected"); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/profiler/heapsnapshot.c new/MoarVM-2021.07/src/profiler/heapsnapshot.c --- old/MoarVM-2021.06/src/profiler/heapsnapshot.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/profiler/heapsnapshot.c 2021-07-24 12:24:40.000000000 +0200 @@ -93,8 +93,7 @@ void **store = (void **)store_ptr; if (*num == *alloc) { *alloc = *alloc ? 2 * *alloc : 32; - *store = MVM_realloc(*store, *alloc * size); - memset(((char *)*store) + *num * size, 0, (*alloc - *num) * size); + *store = MVM_recalloc(*store, *num * size, *alloc * size); } } @@ -907,24 +906,18 @@ { char namebuf[8]; - /* Yes, this is a lot of boiler plate to silence a bogus warning :( + /* Yes, this is a lot of boiler plate to silence a bogus gcc warning (but not clang) :( * Unfortunately, this is a real edge case. Using memcpy would lead to - * a buffer overflow if name is shorter than 8 bytes. The compiler + * a buffer overflow if name is shorter than 8 bytes. The gcc compiler * warning aside, strncpy seems like exactly the right tool for the job * as we want at most 8 bytes and don't care for any trailing \0, but * are OK with zero padding at the end. */ -#ifndef _MSC_VER -#pragma GCC diagnostic push -#pragma GCC diagnostic ignored "-Wunknown-pragmas" -#pragma clang diagnostic push -#pragma clang diagnostic ignored "-Wunknown-warning-option" +#if defined(__GNUC__) && !defined(__clang__) #pragma GCC diagnostic push #pragma GCC diagnostic ignored "-Wstringop-truncation" #endif strncpy(namebuf, name, 8); -#ifndef _MSC_VER -#pragma GCC diagnostic pop -#pragma clang diagnostic pop +#if defined(__GNUC__) && !defined(__clang__) #pragma GCC diagnostic pop #endif fwrite(namebuf, 8, 1, fh); @@ -1013,8 +1006,8 @@ char typename[8] = "strings\0"; MVMuint64 size = 0; - MVMuint64 size_position = 0; - MVMuint64 end_position = 0; + MVMuint64 size_position = 0; + MVMuint64 end_position = 0; while (i < col->num_strings) { char *str = col->strings[i]; @@ -1680,7 +1673,7 @@ entry->collectables_size += s->num_collectables * i + 4 + sizeof(MVMuint64) * 2; - + #if DUMP_EVERYTHING_RAW gzFile kind_fh, tofi_fh, collsize_fh, unmansize_fh, refstart_fh, refcount_fh; kind_fh = open_coll_file(col, "kind"); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/spesh/stats.c new/MoarVM-2021.07/src/spesh/stats.c --- old/MoarVM-2021.06/src/spesh/stats.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/spesh/stats.c 2021-07-24 12:24:40.000000000 +0200 @@ -657,26 +657,59 @@ * updated in a while, clears them out. */ void MVM_spesh_stats_cleanup(MVMThreadContext *tc, MVMObject *check_frames) { MVMint64 elems = MVM_repr_elems(tc, check_frames); - MVMint64 insert_pos = 0; - MVMint64 i; - for (i = 0; i < elems; i++) { - MVMStaticFrame *sf = (MVMStaticFrame *)MVM_repr_at_pos_o(tc, check_frames, i); - MVMStaticFrameSpesh *spesh = sf->body.spesh; - MVMSpeshStats *ss = spesh->body.spesh_stats; - if (!ss) { - /* No stats; already destroyed, don't keep this frame under - * consideration. */ - } - else if (tc->instance->spesh_stats_version - ss->last_update > MVM_SPESH_STATS_MAX_AGE) { - MVM_spesh_stats_destroy(tc, ss); - MVM_free(spesh->body.spesh_stats); - spesh->body.spesh_stats = NULL; - } - else { - MVM_repr_bind_pos_o(tc, check_frames, insert_pos++, (MVMObject *)sf); + MVMROOT(tc, check_frames, { + MVMint64 insert_pos = 0; + MVMint64 i; + for (i = 0; i < elems; i++) { + MVMStaticFrame *sf = (MVMStaticFrame *)MVM_repr_at_pos_o(tc, check_frames, i); + MVMROOT(tc, sf, { + MVMStaticFrameSpesh *spesh = sf->body.spesh; + MVMSpeshStats *ss = spesh->body.spesh_stats; + if (!ss) { + /* No stats; already destroyed, don't keep this frame under + * consideration. */ + } + else if (tc->instance->spesh_stats_version - ss->last_update > MVM_SPESH_STATS_MAX_AGE) { + /* Do not mark thread blocked as the GC also tries to acquire + * mutex_threads and it's held only briefly by all holders anyway */ + uv_mutex_lock(&tc->instance->mutex_threads); + + MVMThread *current = tc->instance->threads; + int found = 0; + while (current && !found) { + MVMThreadContext *cur_tc = current->body.tc; + if (cur_tc) { + MVMSpeshSimStack *sims = cur_tc->spesh_sim_stack; + if (sims) { + for (MVMuint32 j = 0; j < sims->used; j++) { + MVMSpeshSimStackFrame *simf = &sims->frames[j]; + if (simf->ss == ss) { + found = 1; + break; + } + } + } + } + current = current->body.next; + } + + uv_mutex_unlock(&tc->instance->mutex_threads); + + if (!found) { + MVM_spesh_stats_destroy(tc, ss); + MVM_free_null(spesh->body.spesh_stats); + } + else { + MVM_repr_bind_pos_o(tc, check_frames, insert_pos++, (MVMObject *)sf); + } + } + else { + MVM_repr_bind_pos_o(tc, check_frames, insert_pos++, (MVMObject *)sf); + } + }); } - } - MVM_repr_pos_set_elems(tc, check_frames, insert_pos); + MVM_repr_pos_set_elems(tc, check_frames, insert_pos); + }); } void MVM_spesh_stats_gc_mark(MVMThreadContext *tc, MVMSpeshStats *ss, MVMGCWorklist *worklist) { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/MoarVM-2021.06/src/strings/shiftjis.c new/MoarVM-2021.07/src/strings/shiftjis.c --- old/MoarVM-2021.06/src/strings/shiftjis.c 2021-06-19 16:36:27.000000000 +0200 +++ new/MoarVM-2021.07/src/strings/shiftjis.c 2021-07-24 12:24:40.000000000 +0200 @@ -221,25 +221,40 @@ MVMGrapheme32 *buffer = MVM_malloc(sizeof(MVMGrapheme32) * result_size); result_graphs = 0; - while (pos < num_bytes || repl_pos) { + int start_repl = 0; + while (pos < num_bytes || start_repl) { MVMGrapheme32 graph; MVMGrapheme32 codepoint; MVMuint8 byte; - int graph_is_set = 0; - if (repl_pos) { + if (start_repl) { + /* Except for the case of the previous grapheme being \r and the replacement + * starting with \n, we shouldn't get any normalization or codepoints combining + * to make graphemes */ graph = MVM_string_get_grapheme_at_nocheck(tc, replacement, repl_pos++); - graph_is_set = 1; - if (repl_length <= repl_pos) repl_pos = 0; - } - else if (is_prepended) { - byte = prepended; - is_prepended = 0; + if (last_was_cr) { + if (graph == '\n') { + graph = MVM_nfg_crlf_grapheme(tc); + } + else { + graph = '\r'; + repl_pos--; + } + last_was_cr = 0; + } + if (repl_length <= repl_pos) { + repl_pos = 0; + start_repl = 0; + } } + /* Main path. There was no replacement */ else { - byte = bytes[pos++]; - } - /* graph_is_set will be 0 unless we just grabbed a replacement grapheme */ - if (!graph_is_set) { + if (is_prepended) { + byte = prepended; + is_prepended = 0; + } + else { + byte = bytes[pos++]; + } int handler_rtrn = decoder_handler(tc, &Shift_JIS_lead, byte, &codepoint); if (handler_rtrn == DECODE_CODEPOINT) { graph = codepoint; @@ -252,11 +267,8 @@ * a replacement is used. */ Shift_JIS_lead = 0x00; if (replacement) { - graph = MVM_string_get_grapheme_at_nocheck(tc, replacement, repl_pos); - /* If the replacement is more than one grapheme we need - * to set repl_pos++ so we will grab the next grapheme on - * the next loop */ - if (1 < repl_length) repl_pos++; + start_repl = 1; + continue; } else { MVM_free(buffer); @@ -286,7 +298,10 @@ } last_was_cr = 0; } - else if (graph == '\r') { + /* Only follow this path if we aren't in a replacement + * Any replacement will already be NFG, and if we had a \r + * in the replacement, this will cause a loop */ + else if (graph == '\r' && !start_repl) { last_was_cr = 1; continue; } @@ -325,6 +340,7 @@ MVMuint32 reached_stopper; MVMStringIndex repl_length = ds->replacement ? MVM_string_graphs(tc, ds->replacement) : 0; MVMStringIndex repl_pos = 0; + int start_repl = 0; MVMuint8 Shift_JIS_lead = 0x00; MVMuint8 prepended = 0; int is_prepended = 0; @@ -348,26 +364,40 @@ /* Process this buffer. */ MVMint32 pos = cur_bytes == ds->bytes_head ? ds->bytes_head_pos : 0; MVMuint8 *bytes = (MVMuint8 *)cur_bytes->bytes; - while (pos < cur_bytes->length || repl_pos) { + while (pos < cur_bytes->length || start_repl) { MVMGrapheme32 graph; MVMCodepoint codepoint; MVMuint8 byte; - int graph_is_set = 0; int handler_rtrn = 0; - if (repl_pos) { + if (start_repl) { + /* Except for the case of the previous grapheme being \r and the replacement + * starting with \n, we shouldn't get any normalization or codepoints combining + * to make graphemes */ graph = MVM_string_get_grapheme_at_nocheck(tc, ds->replacement, repl_pos++); - graph_is_set = 1; - if (repl_length <= repl_pos) repl_pos = 0; - } - else if (is_prepended) { - byte = prepended; - is_prepended = 0; + if (last_was_cr) { + if (graph == '\n') { + graph = MVM_nfg_crlf_grapheme(tc); + } + else { + graph = '\r'; + repl_pos--; + } + last_was_cr = 0; + } + if (repl_length <= repl_pos) { + repl_pos = 0; + start_repl = 0; + } } + /* Main path. There was no replacement */ else { - byte = bytes[pos++]; - } - /* graph_is_set will be 0 unless we just grabbed a replacement grapheme */ - if (!graph_is_set) { + if (is_prepended) { + byte = prepended; + is_prepended = 0; + } + else { + byte = bytes[pos++]; + } handler_rtrn = decoder_handler(tc, &Shift_JIS_lead, byte, &codepoint); if (handler_rtrn == DECODE_CODEPOINT) { graph = codepoint; @@ -380,11 +410,8 @@ * a replacement is used. */ Shift_JIS_lead = 0x00; if (ds->replacement) { - graph = MVM_string_get_grapheme_at_nocheck(tc, ds->replacement, repl_pos); - /* If the replacement is more than one grapheme we need - * to set repl_pos++ so we will grab the next grapheme on - * the next loop */ - if (1 < repl_length) repl_pos++; + start_repl = 1; + continue; } else { /* Throw if it's unmapped */ @@ -414,7 +441,10 @@ } last_was_cr = 0; } - else if (graph == '\r') { + /* Only follow this path if we aren't in a replacement + * Any replacement will already be NFG, and if we had a \r + * in the replacement, this will cause a loop */ + else if (graph == '\r' && !start_repl) { last_was_cr = 1; continue; }