Thanks, Alon - very helpful! Having unsigned 32bit ints would help, but not necessarily a lot. We process video files that can occasionally be huge, especially when dealing with poorly compressed video streams such as motion JPEGs. The fact that off_t is declared as a 32 bit int (signed or not) strikes me as odd. Is that a musl limitation?
On Monday, 5 March 2018 04:58:55 UTC+10, Alon Zakai wrote: > > About the 31 bit issue, there's a chance the issue is that the asm.js FFI > boundary is treated as signed (an asm.js function returning a 32-bit > integer will use | 0). In that case, what might be the bug is that when JS > calls a function returning an unsigned value it should use >>> 0. Another > possibility is that the loads/stores of that struct value (makeSetValue > etc.) may need to be marked as unsigned. > > On Sun, Mar 4, 2018 at 10:55 AM, Alon Zakai <[email protected] > <javascript:>> wrote: > >> C_STRUCTS is generated from the headers in gen_struct_info.py, basically >> by compiling small C programs to see what the offsets are. I believe it >> does not look at sizes, though (except for __size__ which is computed for >> the entire struct). The numbers there are the offsets, not the sizes. So >> st_size is at offset 36. >> >> The stat.h says >> >> off_t st_size; >> blksize_t st_blksize; >> >> I'm not sure how to easily find the definition of off_t, but looking in >> the offsets, st_size is 36 and st_blksize which is after it is 40, so the >> size must be 4. So it's not big enough if you need more then 32 bits, off_t >> would need to be redefined. (Do you really need more than 32 bits, though?) >> >> A separate question is if 32-bit values work - I think you said 31 bits >> seems to be the limit. That could be due to treating the value as signed >> somewhere ( | 0 will do that). If 32 unsigned bits are enough for you, >> finding that bug might be practical. >> >> >> On Sat, Mar 3, 2018 at 12:31 AM, Soeren Balko <[email protected] >> <javascript:>> wrote: >> >>> In struct_info.compiled.json, the "stat" struct is declared like so: >>> >>> "stat":{ >>> >>> "st_rdev":28, >>> "st_mtim":{ >>> "tv_sec":56, >>> "tv_nsec":60, >>> "__size__":8 >>> }, >>> "st_blocks":44, >>> "st_atim":{ >>> "tv_sec":48, >>> "tv_nsec":52, >>> "__size__":8 >>> }, >>> "st_nlink":16, >>> "__st_ino_truncated":8, >>> "st_ctim": { >>> >>> "tv_sec":64, >>> "tv_nsec":68, >>> "__size__":8 >>> }, >>> "st_mode":12, >>> "st_blksize":40, >>> "__st_dev_padding":4, >>> "st_dev":0, >>> "st_size":36, >>> "st_gid":24, >>> "__st_rdev_padding":32, >>> "st_uid":20, >>> "st_ino":72, >>> "__size__":76 >>> } >>> >>> >>> I assume the properties are the bit widths of the various fields (?). >>> According to this, st_size is 36 bits, which is enough to cater even for >>> very large files. >>> >>> Can you please confirm, Alon? >>> >>> >>> On Saturday, March 3, 2018 at 6:18:09 PM UTC+10, Soeren Balko wrote: >>> >>>> Thanks, Alon! This does indeed seem to be the issue. In >>>> library_syscall.js, the "st_size" member is considered am i32 (see below). >>>> I do not yet fully understand how C_STRUCTS is generated. I can see that >>>> compiler.js receives a JSON object STRUCT_INFO that contains the type >>>> definitions. Is this generated from the musl headers? >>>> >>>> doStat: function(func, path, buf) { >>>> try { >>>> var stat = func(path); >>>> } catch (e) { >>>> if (e && e.node && PATH.normalize(path) !== PATH.normalize(FS.getPath(e >>>> .node))) { >>>> // an error occurred while trying to look up the path; we should just >>>> report ENOTDIR >>>> return -ERRNO_CODES.ENOTDIR; >>>> } >>>> throw e; >>>> } >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_dev, 'stat.dev', 'i32') }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.__st_dev_padding, '0', 'i32') >>>> }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.__st_ino_truncated, 'stat.ino', >>>> 'i32') }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_mode, 'stat.mode', 'i32') >>>> }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_nlink, 'stat.nlink', 'i32') >>>> }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_uid, 'stat.uid', 'i32') }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_gid, 'stat.gid', 'i32') }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_rdev, 'stat.rdev', 'i32') >>>> }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.__st_rdev_padding, '0', 'i32') >>>> }}}; >>>> *{{{ makeSetValue('buf', C_STRUCTS.stat.st_size, 'stat.size', 'i32') >>>> }}};* >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_blksize, '4096', 'i32') }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_blocks, 'stat.blocks', 'i32') >>>> }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_atim.tv_sec, >>>> '(stat.atime.getTime() >>>> / 1000)|0', 'i32') }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_atim.tv_nsec, '0', 'i32') >>>> }}}; >>>> {{{ makeSetValue('buf', C_STRUCTS.stat.st_mtim.tv_sec, >>>> '(stat.mtime.getTime() >>>> / 1000)|0', 'i32') }}}; >>>> <td id="LC61" class="blob-code blob-code-inner js-file-line" >>>> style="text-align: left; box-sizing: border-box; padding-right: 10px; >>>> padding-lef >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] >>> <javascript:>. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
