Re: Memory is used up when I replace \t0\n to \n

2009-09-15 Fir de Conversatie Yakov Lerner
On Fri, Sep 11, 2009 at 06:57, Matt Wozniski m...@drexel.edu wrote:


 On Thu, Sep 10, 2009 at 4:13 AM, John Beckett wrote:
 
  Aleafs wrote:
  :1,$  s/\t0\n/\n/g

 Note that :% is a shortcut for :1,$

  In a substitute, \n means two different things:
  - In the pattern, it refers to a newline.
  - In the replacement, it refers to a null byte (8 zero bits).
 
  You can see this at ':help :s' by following the first two links.
 
  In a replacement, '\r' inserts a newline. Yes, it's strange, and
  no, I didn't try to work out why this issue caused the problem
  you report.

 My guess: Vim is really, really bad at handling extremely long lines,
 and if the goal was to delete the last column of a text file (I'm
 assuming every line ended with \t0) then this command told vim to
 turn the file into one 700kb long line.


1. I don't understand why vim grows here (800MB and up)without giving memory
overallocation error based on 'maxmem' and 'maxmemtot'.
'maxmem' and 'maxmemtot' have values of of 150524 (that's KB, or 150MB).
It's undo memory causes memory overuse here.

2. This problem of memory use here (and speed)is is with 'noswapfile' and
'undolevels', and it's solved by:

:set noswapfile undolevel=-1,

Then vim finishes the replacement (wait 1/2 minutes).
And vim opens 700k-long file consisting of single line and many 0-bytes, in
the first place, without problem.

Testcase:

1) 700k file consisting of single line and many 0-bytes
perl -e 'printf %8d %8d %8d %8d\n, $_,$_,$_,$_ for(1..2)' /tmp/1
tr '\n' '\000' /tmp/1 /tmp/2
vim /tmp/2 # no problem

2) vim repalces newlines to 0-bytes without growing in memory, 700k file 20k
lines:
perl -e 'printf %8d %8d %8d %8d\n, $_,$_,$_,$_ for(1..2)' /tmp/1
vim /tmp/1
:set noswapfile undolevel=-1
:%s/\n/\n/finished in 30 sec, resul tis single 700k-long
line

3) without 'et noswapfile undolevel=-1', blows up vim to pieces:
perl -e 'printf %8d %8d %8d %8d\n, $_,$_,$_,$_ for(1..2)' /tmp/1
vim /tmp/1
:%s/\n/\n/  grows beyond 1G in size and killed by OS, if you have patience
Killed

In scenario #3, I thought vim would report error based on 'maxmem' and
'maxmemtot', but
it did not. My values of 'maxmem' and 'maxmemtot' are 150254 (=150MB), but
why
vim grew over 800MB in vm size without giving memory size error ?

Yakov

--~--~-~--~~~---~--~~
You received this message from the vim_dev maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---



RE: Memory is used up when I replace \t0\n to \n

2009-09-10 Fir de Conversatie John Beckett

Aleafs wrote:
 :1,$  s/\t0\n/\n/g

In a substitute, \n means two different things:
- In the pattern, it refers to a newline.
- In the replacement, it refers to a null byte (8 zero bits).

You can see this at ':help :s' by following the first two links.

In a replacement, '\r' inserts a newline. Yes, it's strange, and
no, I didn't try to work out why this issue caused the problem
you report.

John


--~--~-~--~~~---~--~~
You received this message from the vim_dev maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---



Re: Memory is used up when I replace \t0\n to \n

2009-09-10 Fir de Conversatie Aleafs

Thanks a lot. Expecting your conclusion


On 9月10日, 下午4时13分, John Beckett johnb.beck...@gmail.com wrote:
 Aleafs wrote:
  :1,$  s/\t0\n/\n/g

 In a substitute, \n means two different things:
 - In the pattern, it refers to a newline.
 - In the replacement, it refers to a null byte (8 zero bits).

 You can see this at ':help :s' by following the first two links.

 In a replacement, '\r' inserts a newline. Yes, it's strange, and
 no, I didn't try to work out why this issue caused the problem
 you report.

 John
--~--~-~--~~~---~--~~
You received this message from the vim_dev maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---



Re: Memory is used up when I replace \t0\n to \n

2009-09-10 Fir de Conversatie Matt Wozniski

On Thu, Sep 10, 2009 at 4:13 AM, John Beckett wrote:

 Aleafs wrote:
 :1,$  s/\t0\n/\n/g

Note that :% is a shortcut for :1,$

 In a substitute, \n means two different things:
 - In the pattern, it refers to a newline.
 - In the replacement, it refers to a null byte (8 zero bits).

 You can see this at ':help :s' by following the first two links.

 In a replacement, '\r' inserts a newline. Yes, it's strange, and
 no, I didn't try to work out why this issue caused the problem
 you report.

My guess: Vim is really, really bad at handling extremely long lines,
and if the goal was to delete the last column of a text file (I'm
assuming every line ended with \t0) then this command told vim to
turn the file into one 700kb long line.

~Matt

--~--~-~--~~~---~--~~
You received this message from the vim_dev maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---