Re: memory leak in bash only during boot up in Bash-3.2.48

[Chet Ramey]

On 12/22/18 3:47 AM, ABHISHEK PALIWAL wrote:
> Hi Chet,
> 
>>What is the hard limit on the number of processes for a process started in
>>this environment? (The value of `ulimit -n'.)
> 
> Here are the Hard and soft limits for open files and max user processes.
> root@localhost:/root> ulimit -Hn
> 1024
> root@localhost:/root> ulimit -Hu
> 516046
> root@localhost:/root> ulimit -Su
> 1024
> root@localhost:/root> ulimit -S> 1024

These are probably not the same values that are available to processes run
as services.

Look at http://lists.gnu.org/archive/html/bug-bash/2018-12/msg00084.html
for a discussion of the issue and a fix.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: memory leak in bash only during boot up in Bash-3.2.48

[ABHISHEK PALIWAL]

Hi Chet,

>What is the hard limit on the number of processes for a process started in
>this environment? (The value of `ulimit -n'.)

Here are the Hard and soft limits for open files and max user processes.
root@localhost:/root> ulimit -Hn
1024
root@localhost:/root> ulimit -Hu
516046
root@localhost:/root> ulimit -Su
1024
root@localhost:/root> ulimit -Sn
1024

Regards,
Abhishek

On Fri, Dec 21, 2018 at 10:20 PM Chet Ramey  wrote:

> On 12/19/18 12:54 AM, abhishpaliwal wrote:
> > create a shell script from below given code. name it testing.sh.
> >
> > #!/bin/bash
> >  logger "Started testing"
> >  while(true); do
> > while (true); do ls > /dev/null ; done
> >  done
> >
> > Steps.
> > # chmod 755 /root/testing.sh
> > modify any /etc/init.d/"service" file add "/root/testing.sh" inside the
> > start case.
> > "service" is any service file starting during start up.
> > Make sure above service is started during the startup.
>
> What is the hard limit on the number of processes for a process started in
> this environment? (The value of `ulimit -n'.)
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>


-- 




Regards
Abhishek Paliwal

Re: memory leak in bash only during boot up in Bash-3.2.48

[Chet Ramey]

On 12/19/18 12:54 AM, abhishpaliwal wrote:
> create a shell script from below given code. name it testing.sh. 
> 
> #!/bin/bash 
>  logger "Started testing" 
>  while(true); do 
> while (true); do ls > /dev/null ; done 
>  done 
> 
> Steps. 
> # chmod 755 /root/testing.sh 
> modify any /etc/init.d/"service" file add "/root/testing.sh" inside the
> start case. 
> "service" is any service file starting during start up. 
> Make sure above service is started during the startup. 

What is the hard limit on the number of processes for a process started in
this environment? (The value of `ulimit -n'.)

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

memory leak in bash only during boot up in Bash-3.2.48

[abhishpaliwal]

create a shell script from below given code. name it testing.sh. 

#!/bin/bash 
 logger "Started testing" 
 while(true); do 
while (true); do ls > /dev/null ; done 
 done 

Steps. 
# chmod 755 /root/testing.sh 
modify any /etc/init.d/"service" file add "/root/testing.sh" inside the
start case. 
"service" is any service file starting during start up. 
Make sure above service is started during the startup. 

Example : 
Add testing to audit init script 
start(){ 
echo -n "Starting testing.sh" 
/root/testing.sh & 

After boot up 

root@localhost:/root> ps fax | grep testing 
751 ? S 0:11 /bin/bash /root/testing.sh 
9684 ttyS0 S+ 0:00 \_ grep testing 

root@localhost:/root> cat /proc/751/smaps | grep -A10 heap 
01419000-016ad000 rw-p  00:00 0 [heap] 
Size: 2640 kB 
Rss: 2536 kB 
Pss: 2536 kB 
Shared_Clean: 0 kB 
Shared_Dirty: 0 kB 
Private_Clean: 0 kB 
Private_Dirty: 2536 kB 
Referenced: 2536 kB 
Swap: 0 kB 
KernelPageSize: 4 kB 

root@localhost:/root> cat /proc/751/smaps | grep -A10 heap 
01419000-016ce000 rw-p  00:00 0 [heap] 
Size: 2772 kB 
Rss: 2740 kB 
Pss: 2734 kB 
Shared_Clean: 0 kB 
Shared_Dirty: 12 kB 
Private_Clean: 0 kB 
Private_Dirty: 2728 kB 
Referenced: 2740 kB 
Swap: 0 kB 
KernelPageSize: 4 kB

But when we are testing the same on Bash-4.4.0, its not consuming the memory
and steady as well, below are the logs 

root@localhost:/root> ps fax | grep testing
  964 ?S  0:02 /bin/bash /root/testing.sh
11484 pts/0S+ 0:00  \_ grep testing
root@localhost:/root> cat /proc/964/smaps | grep -A10 heap
00b1b000-00b32000 rw-p  00:00 0 
[heap]
Size: 92 kB
Rss:  88 kB
Pss:  88 kB
Shared_Clean:  0 kB
Shared_Dirty:  0 kB
Private_Clean: 0 kB
Private_Dirty:88 kB
Referenced:   88 kB
Swap:  0 kB
KernelPageSize:4 kB
root@localhost:/root> cat /proc/964/smaps | grep -A10 heap
00b1b000-00b32000 rw-p  00:00 0 
[heap]
Size: 92 kB
Rss:  88 kB
Pss:  60 kB
Shared_Clean:  0 kB
Shared_Dirty: 56 kB
Private_Clean: 0 kB
Private_Dirty:32 kB
Referenced:   88 kB
Swap:  0 kB



--
Sent from: http://gnu-bash.2382.n7.nabble.com/

答复: memory leak in bash

[wuzongyong (A)]

Well, the recent version runs ok, I just wan't to find the commit solving this 
bug.

Thanks,
Zongyong Wu

-邮件原件-
发件人: Eduardo Bustamante [mailto:dual...@gmail.com] 
发送时间: 2017年5月11日 22:39
收件人: wuzongyong (A) <wuzongyo...@huawei.com>
抄送: bug-bash@gnu.org; Wanzongshun (Vincent) <wanzongs...@huawei.com>
主题: Re: memory leak in bash

On Thu, May 11, 2017 at 3:47 AM, wuzongyong (A) <wuzongyo...@huawei.com> wrote:
[...]
> My bash version is  version 4.2.46(1)-release, valgrind version is  3.11.0 , 
> could someone help to tell me if it is a bug please? And I wanna to know the 
> deeply level reason.

Try with a more recent version. Version 4.2.46 is a few years old now.

dualbus@debian:~$ cat script
#! /bin/bash
rc_exit ()
{
exit 0
}
rc_exit

dualbus@debian:~$ valgrind bash ./script ==14004== Memcheck, a memory error 
detector ==14004== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et 
al.
==14004== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright 
info ==14004== Command: bash ./script ==14004== ==14004== ==14004== HEAP 
SUMMARY:
==14004== in use at exit: 0 bytes in 0 blocks
==14004==   total heap usage: 482 allocs, 482 frees, 31,142 bytes allocated
==14004==
==14004== All heap blocks were freed -- no leaks are possible ==14004== 
==14004== For counts of detected and suppressed errors, rerun with: -v 
==14004== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

dualbus@debian:~$ bash --version
GNU bash, version 4.4.11(1)-release (x86_64-pc-linux-gnu) Copyright (C) 2016 
Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Re: memory leak in bash

[Eduardo Bustamante]

On Thu, May 11, 2017 at 3:47 AM, wuzongyong (A)  wrote:
[...]
> My bash version is  version 4.2.46(1)-release, valgrind version is  3.11.0 , 
> could someone help to tell me if it is a bug please? And I wanna to know the 
> deeply level reason.

Try with a more recent version. Version 4.2.46 is a few years old now.

dualbus@debian:~$ cat script
#! /bin/bash
rc_exit ()
{
exit 0
}
rc_exit

dualbus@debian:~$ valgrind bash ./script
==14004== Memcheck, a memory error detector
==14004== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==14004== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==14004== Command: bash ./script
==14004==
==14004==
==14004== HEAP SUMMARY:
==14004== in use at exit: 0 bytes in 0 blocks
==14004==   total heap usage: 482 allocs, 482 frees, 31,142 bytes allocated
==14004==
==14004== All heap blocks were freed -- no leaks are possible
==14004==
==14004== For counts of detected and suppressed errors, rerun with: -v
==14004== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

dualbus@debian:~$ bash --version
GNU bash, version 4.4.11(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

memory leak in bash

[wuzongyong (A)]

Hi,

I met a memory leak to use valgrind run shell code below:

#! /bin/bash
rc_exit ()
{
exit 0
}
rc_exit

And the valgrind log just like that:

HEAP SUMMARY:
==27459== in use at exit: 21,968 bytes in 464 blocks
==27459==   total heap usage: 568 allocs, 104 frees, 29,714 bytes allocated
==27459==
==27459== 72 (32 direct, 40 indirect) bytes in 1 blocks are definitely lost in 
loss record 267 of 288
==27459==at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27459==by 0x46A1AA: xmalloc (in /usr/bin/bash)
==27459==by 0x42B44A: make_bare_simple_command (in /usr/bin/bash)
==27459==by 0x42B525: make_simple_command (in /usr/bin/bash)
==27459==by 0x42839D: yyparse (in /usr/bin/bash)
==27459==by 0x41DEF9: parse_command (in /usr/bin/bash)
==27459==by 0x41DFBB: read_command (in /usr/bin/bash)
==27459==by 0x41E1BB: reader_loop (in /usr/bin/bash)
==27459==by 0x41C8BD: main (in /usr/bin/bash)
==27459==
==27459== LEAK SUMMARY:
==27459==definitely lost: 32 bytes in 1 blocks
==27459==indirectly lost: 40 bytes in 2 blocks
==27459==  possibly lost: 0 bytes in 0 blocks
==27459==still reachable: 21,896 bytes in 461 blocks
==27459== suppressed: 0 bytes in 0 blocks
==27459== Reachable blocks (those to which a pointer was found) are not shown.
==27459== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==27459==
==27459== For counts of detected and suppressed errors, rerun with: -v
==27459== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

My bash version is  version 4.2.46(1)-release, valgrind version is  3.11.0 , 
could someone help to tell me if it is a bug please? And I wanna to know the 
deeply level reason.


Thanks,
Zongyong Wu

Re: Memory leak in bash 4.3

[Chet Ramey]

 My hope was that the updated patch I'm about to send would be released
 as bash43-040 in bash-4.3-patches/. Does that make sense, or am I
 missing something obvious?

Here's a corrected patch, and eventually this will be released as an
official patch.

Chet

*** ../bash-4.3-patched/subst.c 2014-10-01 12:57:47.0 -0400
--- subst.c 2015-06-22 09:16:53.0 -0400
***
*** 5783,5787 
if (pflags  PF_ASSIGNRHS)
  {
!   temp = array_variable_name (name, tt, (int *)0);
if (ALL_ELEMENT_SUB (tt[0])  tt[1] == ']')
temp = array_value (name, quoted|Q_DOUBLE_QUOTES, 0, atype, ind);
--- 5783,5787 
if (pflags  PF_ASSIGNRHS)
  {
!   var = array_variable_part (name, tt, (int *)0);
if (ALL_ELEMENT_SUB (tt[0])  tt[1] == ']')
temp = array_value (name, quoted|Q_DOUBLE_QUOTES, 0, atype, ind);

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Memory leak in bash 4.3

[Jean Delvare]

Hi Chet,

On Mon, 22 Jun 2015 09:40:36 -0400, Chet Ramey wrote:
  My hope was that the updated patch I'm about to send would be released
  as bash43-040 in bash-4.3-patches/. Does that make sense, or am I
  missing something obvious?
 
 Here's a corrected patch, and eventually this will be released as an
 official patch.
 
 Chet
 
 *** ../bash-4.3-patched/subst.c   2014-10-01 12:57:47.0 -0400
 --- subst.c   2015-06-22 09:16:53.0 -0400
 ***
 *** 5783,5787 
 if (pflags  PF_ASSIGNRHS)
   {
 !   temp = array_variable_name (name, tt, (int *)0);
 if (ALL_ELEMENT_SUB (tt[0])  tt[1] == ']')
   temp = array_value (name, quoted|Q_DOUBLE_QUOTES, 0, atype, ind);
 --- 5783,5787 
 if (pflags  PF_ASSIGNRHS)
   {
 !   var = array_variable_part (name, tt, (int *)0);
 if (ALL_ELEMENT_SUB (tt[0])  tt[1] == ']')
   temp = array_value (name, quoted|Q_DOUBLE_QUOTES, 0, atype, ind);
 

Looks good, thank you!

-- 
Jean Delvare
SUSE L3 Support

Re: Memory leak in bash 4.3

[Chet Ramey]

On 6/15/15 9:09 AM, Jean Delvare wrote:

 I made
 this change back in October 2014 as part of an unrelated fix for an
 array element quoting issue.

 I'll submit an updated patch.

See above.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Memory leak in bash 4.3

[Jean Delvare]

On Mon, 15 Jun 2015 09:12:58 -0400, Chet Ramey wrote:
 On 6/15/15 9:09 AM, Jean Delvare wrote:
 
  I made
  this change back in October 2014 as part of an unrelated fix for an
  array element quoting issue.
 
  I'll submit an updated patch.
 
 See above.

I know it's fixed in the development branch. However the leak is still
present in the latest stable release of bash, 4.3.39. Pretty much every
recent distribution out there is shipping bash version 4.3.x so they
are all affected by the leak.

My hope was that the updated patch I'm about to send would be released
as bash43-040 in bash-4.3-patches/. Does that make sense, or am I
missing something obvious?

Thanks,
-- 
Jean Delvare
SUSE L3 Support

Re: Memory leak in bash 4.3

[Jean Delvare]

Hi Chet,

Thanks for your reply.

On Thu, 11 Jun 2015 10:54:31 -0400, Chet Ramey wrote:
 On 6/9/15 3:42 PM, Jean Delvare wrote:
 
  I think I have a minimal test case now:
  
  #!/bin/bash
  
  declare -a ARRAY
  
  ARRAY[0]=foo
  FOO=${ARRAY[0]} # -- leaks
  echo $FOO
  
  And a candidate fix:
  
  bash 4.3: Fix memory leak in parameter_brace_expand_word
 
 Thanks for the report.  This is the right place for the patch.  I made
 this change back in October 2014 as part of an unrelated fix for an
 array element quoting issue.
 
 (Your patch reuses `temp', but array_variable_part returns a SHELL_VAR *.)

Good point. gcc warns about it but I did not pay attention, sorry about
that.

As we do not use the result of function array_variable_part, I think
the easiest way to fix the warning is to not assign it to any variable.
I'll submit an updated patch.

Thanks again,
-- 
Jean Delvare
SUSE L3 Support

Re: Memory leak in bash 4.3

[Chet Ramey]

On 6/9/15 3:42 PM, Jean Delvare wrote:

 I think I have a minimal test case now:
 
 #!/bin/bash
 
 declare -a ARRAY
 
 ARRAY[0]=foo
 FOO=${ARRAY[0]} # -- leaks
 echo $FOO
 
 And a candidate fix:
 
 bash 4.3: Fix memory leak in parameter_brace_expand_word

Thanks for the report.  This is the right place for the patch.  I made
this change back in October 2014 as part of an unrelated fix for an
array element quoting issue.

(Your patch reuses `temp', but array_variable_part returns a SHELL_VAR *.)

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Memory leak in bash 4.3

[Jean Delvare]

Hi all,

I have a bash script that runs as a daemon so it never exits, and its
memory consumption when run under bash 4.3.x (as reported as RSS by ps)
keeps increasing over time, apparently without any bound (starts at 3
MB and I've seen it reach 40 MB and still growing.) The memory
consumption of the same daemon script run under bash 4.2.x is steady. I
also tested a recent devel branch snapshot (bash-20150529) and
there is no memory increase there either. So the leak must have been
introduced during development between the 4.2 and 4.3 branches, and
fixed during development of between the 4.3 and upcoming 4.4 branches.

I would have bisected it, however I can't make much sense of the
branches in the git repository. Apparently snapshots are imported into
the devel branch from another source, so bisecting wouldn't lead to an
individual commit. Also I couldn't figure out which devel snapshots (if
any) correspond to versions 4.2.0 and 4.3.0 respectively so I don't
know where to start bisecting.

I built bash 4.3.39 with --without-bash-malloc and then ran the daemon
under valgrind for 1 minute. The back trace for the leak is:

3,973 bytes in 430 blocks are definitely lost in loss record 1,610 of 1,613
   at 0x4C277AB: malloc (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x471A2A: xmalloc (xmalloc.c:112)
   by 0x461A84: array_variable_name (arrayfunc.c:917)
   by 0x457731: parameter_brace_expand_word (subst.c:5785)
   by 0x452DA4: param_expand (subst.c:7385)
   by 0x4550E9: expand_word_internal (subst.c:8393)
   by 0x4566DB: call_expand_word_internal.constprop.14 (subst.c:3299)
   by 0x456829: expand_string_assignment (subst.c:3387)
   by 0x4509DB: expand_string_if_necessary (subst.c:3092)
   by 0x450E3E: do_assignment_internal (subst.c:2823)
   by 0x45859E: expand_word_list_internal (subst.c:2912)
   by 0x4360B7: execute_simple_command (execute_cmd.c:4000)

I do not have a minimum test case yet, but I wanted to ask if the back
trace above provides enough hints to recognize an already know bug and
the commit that fixed it?

Thanks,
-- 
Jean Delvare
SUSE L3 Support

Re: Memory leak in bash 4.3

[Jean Delvare]

On Tue, 9 Jun 2015 18:02:31 +0200, Jean Delvare wrote:
 I built bash 4.3.39 with --without-bash-malloc and then ran the daemon
 under valgrind for 1 minute. The back trace for the leak is:
 
 3,973 bytes in 430 blocks are definitely lost in loss record 1,610 of 1,613
at 0x4C277AB: malloc (in 
 /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x471A2A: xmalloc (xmalloc.c:112)
by 0x461A84: array_variable_name (arrayfunc.c:917)
by 0x457731: parameter_brace_expand_word (subst.c:5785)
by 0x452DA4: param_expand (subst.c:7385)
by 0x4550E9: expand_word_internal (subst.c:8393)
by 0x4566DB: call_expand_word_internal.constprop.14 (subst.c:3299)
by 0x456829: expand_string_assignment (subst.c:3387)
by 0x4509DB: expand_string_if_necessary (subst.c:3092)
by 0x450E3E: do_assignment_internal (subst.c:2823)
by 0x45859E: expand_word_list_internal (subst.c:2912)
by 0x4360B7: execute_simple_command (execute_cmd.c:4000)
 
 I do not have a minimum test case yet, but I wanted to ask if the back
 trace above provides enough hints to recognize an already know bug and
 the commit that fixed it?

I think I have a minimal test case now:

#!/bin/bash

declare -a ARRAY

ARRAY[0]=foo
FOO=${ARRAY[0]} # -- leaks
echo $FOO

And a candidate fix:

bash 4.3: Fix memory leak in parameter_brace_expand_word

Fix a memory leak in function parameter_brace_expand_word. If
expanding an array reference and PF_ASSIGNRHS is set, function
array_variable_name is called but the allocated string is not
needed and is ignored. It must be freed before going on, otherwise
memory is leaked. Function array_variable_part does that for us
so just call it instead of function array_variable_name (as is
done in later versions of bash.)
---
 subst.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/subst.c
+++ b/subst.c
@@ -5782,7 +5782,7 @@ expand_arrayref:
   /* XXX - does this leak if name[@] or name[*]? */
   if (pflags  PF_ASSIGNRHS)
 {
-  temp = array_variable_name (name, tt, (int *)0);
+  temp = array_variable_part (name, tt, (int *)0);
   if (ALL_ELEMENT_SUB (tt[0])  tt[1] == ']')
temp = array_value (name, quoted|Q_DOUBLE_QUOTES, 0, atype, ind);
  else


-- 
Jean Delvare
SUSE L3 Support