On Mon, Nov 13, 2017 at 09:33:49PM +0100, Christian Brabandt wrote: > > On So, 12 Nov 2017, Tim Chase wrote: > > > Taking on the challenge listed here > > > > https://dev.to/jorinvo/csv-challenge-1al > > > > it provides a .json file link to mung into date-named files (never > > mind that these happen to all be the same date), I came up with this > > vim solution: > > > > :g/.*name":"\([^"]*\)".*card":"\([^"]*\)".*/let > > s=substitute(getline('.'), '.*stamp":"\(\d\+\)-\(\d\+\)-\(\d\+\).*', > > '\1\2\3','').'.txt'|s//\1,\2/|exec ".w!>>".s > > > > which works. But for some reason, it is *painfully* slow on my > > machine. The basic intent is that it identifies rows with a (fake) > > credit-card number, snapshots the YYYYMMMDD.txt filename, then turns > > the row into a CSV entry ("name, CC"), and writes (appending) that one > > line into the output filename. > > > > The whole thing should happen in a blink. Maybe a second at worst. > > But this was walk-away-do-something-and-come-back slow. > > > > If I change the "exec" to an "echo", it's as fast as I expect. > > > > Any idea what might be making the exec so slow? > > No idea and I bet running it under gdb will make it even slower :( > > Nevertheless can you try this with a debug Vim and hit <Ctrl>-C > > This should give us a clue, which codepath is slow. Please try this > several times, so we are sure, that we hit the slow codepath. > > Of course, easier would be to provide a correct profiling information :)
I profiled Vim while running this command, but my gprof-fu isn't strong enough to see the problem. I tried different combinations of old/new regexp engine, enabling/disabling autocmds, :silent, but these didn't make a discernible difference. The gprof logs can be found at http://unnovative.net/json_challenge/, if anybody smarter than I cares to take a look. The odd thing that I noticed is that, according to htop, the Vim process spends lots of cycles in IOWait. Here's what iotop shows during a typical run: Total DISK READ : 0.00 B/s | Total DISK WRITE : 1372.27 K/s Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 2.68 M/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 304 be/3 root 0.00 B/s 0.00 B/s 0.00 % 80.22 % [jbd2/nvme0n1p2-] 17734 be/4 fadein 0.00 B/s 1372.27 K/s 0.00 % 0.89 % vim -c source slow.vim data.json That's a lot of writing to my hard disk nvme0n1p2. It looks like Vim thrashing a temp file somewhere, but I didn't see anything that looked likely under /proc/$(pgrep vim)/fd, and lsof -p $(pgrep vim) didn't turn up anything that looked out of the ordinary. Just to be sure I also tried disabling the swap file :set noswf but still, Vim thrashes. -- Erik Falor Registered Linux User #445632 http://unnovative.net -- -- You received this message from the "vim_use" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_use" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
signature.asc
Description: PGP signature
