Re: net/asterisk13: memory leak under 12-CURRENT?

2017-09-27 Thread O. Hartmann 
On Wed, 27 Sep 2017 12:51:18 +0200
Hans Petter Selasky  wrote:

> On 09/27/17 09:05, Guido Falsi wrote:
> > On 09/26/2017 15:41, O. Hartmann wrote:  
> >> On Tue, 26 Sep 2017 15:06:23 +0200
> >> Guido Falsi  wrote:  
> >   
> >> Since I run net/asterisk with automatic module loading (I'm new to
> >> asterisk), this is very likely and might cause the problem somehow.
> >>  
> > 
> > You can exclude single modules from autoloading via modules.conf.
> >   
> >>> Not sure, restarting the daemon should free any leaked memory the daemon
> >>> has. If a killed process leaves memory locked at the system level there
> >>> should be some other cause.  
> >>
> >> Even with no runnidng asterisk, memory level drops after the last shutdown
> >> of asterisk and keeps that low. Even for weeks! My router never shows that
> >> high memory consumption, even under load.  
> > 
> > But while asterisk is running does the memory usage increase unbounded
> > till filling all available memory or does it stabilize at some point?
> > 
> > Asterisk is relatively memory hungry, especially with all modules
> > enabled. It also caches and logs various information in RAM, even doing
> > "nothing" it will cache and log that "nothing" activity. If memory does
> > stabilize after some point it's not really a leak but it's standard
> > memory usage. To reduce it you should disable all unused modules.
> >   
> >>
> >> The question would be: how to use vmstat to give hints for those familiar
> >> with memory subsystems to indicate a real bug?
> >>
> >> I tried to find some advices, but maybe my English isn't good enough to
> >> make google help.  
> > 
> > I'm not able to give you a correct indication, but if the memory usage
> > is not increasing indefinitely but is stabilizing I'd say it's not
> > really a leak.
> >   
> 
> Did you look at the output from "vmstat -m" and "vmstat -z" ?
> 
> --HPS

I did not, but now I will ;-)

Thanks for the hint!

Kind regards,
Oliver
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

FreeBSD Quarterly Status Report - Second Quarter 2017

2017-09-27 Thread Benjamin Kaduk 
FreeBSD Quarterly Status Report - 2nd Quarter 2017

   FreeBSD continues to defy the rumors of its demise.

   Much of the development work done this quarter was not particularly
   visible, especially the effort needed to ensure the upcoming 11.1
   release has as few regressions as possible. Planning is also well under
   way for the 10.4 maintenance release which will quickly follow it.

   Further work focused on moving the arm architectures' support closer to
   tier-1 status and improving documentation. In addition, large changes
   were made to the src and ports trees.

   These projects and others are further detailed below.

   --Mark Linimon
 __

   The deadline for submissions covering the period from July to September
   2017 is October 21, 2017.
 __

FreeBSD Team Reports

 * FreeBSD Release Engineering Team
 * Ports Collection
 * The FreeBSD Core Team
 * The FreeBSD Foundation
 * The Postmaster Team

Projects

 * 64-bit Inode Numbers
 * Capability-Based Network Communication for Capsicum/CloudABI
 * Ceph on FreeBSD
 * DTS Updates

Kernel

 * Coda revival
 * FreeBSD Driver for the Annapurna Labs ENA
 * Intel 10G Driver Update
 * pNFS Server Plan B

Architectures

 * FreeBSD on Marvell Armada38x
 * FreeBSD/arm64

Userland Programs

 * DTC
 * Using LLVM's LLD Linker as FreeBSD's System Linker

Ports

 * A New USES Macro for Porting Cargo-Based Rust Applications
 * GCC (GNU Compiler Collection)
 * GNOME on FreeBSD
 * KDE on FreeBSD
 * New Port: FRRouting
 * PHP Ports: Help Improving QA
 * Rust
 * sndio Support in the FreeBSD Ports Collection
 * TensorFlow
 * Updating Port Metadata for non-x86 Architectures
 * Xfce on FreeBSD

Documentation

 * Absolute FreeBSD, 3rd Edition
 * Doc Version Strings Improved by Their Absence
 * New Xen Handbook Section

Miscellaneous

 * BSD Meetups at Rennes (France)

Third-Party Projects

 * HardenedBSD
 __

FreeBSD Team Reports

FreeBSD Release Engineering Team

   Links
   FreeBSD 11.1-RELEASE Schedule
URL: https://www.FreeBSD.org/releases/11.1R/schedule.html
   FreeBSD Development Snapshots
URL: https://download.FreeBSD.org/ftp/snapshots/ISO-IMAGES/

   Contact: FreeBSD Release Engineering Team 

   The FreeBSD Release Engineering Team is responsible for setting and
   publishing release schedules for official project releases of FreeBSD,
   announcing code freezes, and maintaining the respective branches, among
   other things.

   The FreeBSD 11.1-RELEASE cycle started on May 19, and continued as
   scheduled. FreeBSD consumers are urged to test whenever possible to
   help ensure the reliability and stability of the upcoming second
   release from the stable/11 branch.

   This project was sponsored by The FreeBSD Foundation.
 __

Ports Collection

   Links
   About FreeBSD Ports
URL: https://www.FreeBSD.org/ports/
   Contributing to Ports
URL: 
https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html
   FreeBSD Ports Monitoring
URL: http://portsmon.freebsd.org/index.html
   Ports Management Team Website
URL: https://www.freebsd.org/portmgr/index.html
   FreeBSD portmgr on Twitter (@freebsd_portmgr)
URL: https://twitter.com/freebsd_portmgr/
   FreeBSD Ports Management Team on Facebook
URL: https://www.facebook.com/portmgr
   FreeBSD Ports Management Team on Google+
URL: https://plus.google.com/communities/108335846196454338383

   Contact: René Ladan 
   Contact: FreeBSD Ports Management Team 

   This quarter, 2017Q2, broke the 30,000 ports landmark for the first
   time. The PR count is currently just under 2,500, with almost 600 of
   them unassigned. This quarter saw almost 7,400 commits from 171
   committers. More PRs got closed this quarter than last quarter, but
   also more PRs got sent in, both of which are good to see.

   Over the past three months, we welcomed four new committers: Bradley T.
   Hughes (bhughes@), Danilo G. Baio (dbaio@), Jochen Neumeister
   (joneum@), and Richard Gallamore (ultima@). kan@ re-joined us as a
   ports committer. One commit bit, that of bf@, was taken in for
   safekeeping after a long period of inactivity.

   On the management side, the Ports Management Team welcomed back bapt@,
   who is working on several new features for the Ports Tree. The Ports
   Management Team also had its annual real-life meeting during BSDCan.

   On the infrastructure side, three new USES values were introduced:
 * cargo, to ease the porting of Rust packages or binaries using the
   cargo command

Re: panic: softdep_deallocate_dependencies: dangling deps

2017-09-27 Thread Steve Kargl 
On Wed, Sep 27, 2017 at 03:13:21PM -0700, Steve Kargl wrote:
> 
> Hmmm,
> 
> %  kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.0
> Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug...done.
> ABI doesn't support a vmcore target
> 
> OK, so debugging is broken :-/
> 

It seems the devel/gdb80 port installs a kgdb symlink
to kgdb80.  So, I'm not getting the correct kgdb due to 
my path.  Ugh.

-- 
Steve
20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4
20161221 https://www.youtube.com/watch?v=IbCHE-hONow
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

panic: softdep_deallocate_dependencies: dangling deps

2017-09-27 Thread Steve Kargl 
Just got this panic on 

FreeBSD troutmask.apl.washington.edu 12.0-CURRENT FreeBSD 12.0-CURRENT
#0 r321800: Mon Jul 31 13:48:43 PDT 2017
kargl@:/data/obj/usr/src/sys/SPEW  amd64

core.txt.0 contains

panic: softdep_deallocate_dependencies: dangling deps
cpuid = 7
time = 1506549566
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe023a281710
vpanic() at vpanic+0x19c/frame 0xfe023a281790
panic() at panic+0x43/frame 0xfe023a2817f0
softdep_deallocate_dependencies() at softdep_deallocate_dependencies+0x76/frame 
0xfe023a281810
brelse() at brelse+0x149/frame 0xfe023a281870
bufwrite() at bufwrite+0x65/frame 0xfe023a2818b0
softdep_process_journal() at softdep_process_journal+0x7a8/frame 
0xfe023a281950
softdep_process_worklist() at softdep_process_worklist+0x80/frame 
0xfe023a2819b0
softdep_flush() at softdep_flush+0xff/frame 0xfe023a2819f0
fork_exit() at fork_exit+0x75/frame 0xfe023a281a30
fork_trampoline() at fork_trampoline+0xe/frame 0xfe023a281a30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

__curthread () at ./machine/pcpu.h:232
232 __asm("movq %%gs:%1,%0" : "=r" (td)
(kgdb) #0  __curthread () at ./machine/pcpu.h:232
#1  doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:318
#2  0x805879eb in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:386
#3  0x80587e66 in vpanic (fmt=, ap=0xfe023a2817d0)
at /usr/src/sys/kern/kern_shutdown.c:779
#4  0x80587c83 in panic (fmt=)
at /usr/src/sys/kern/kern_shutdown.c:710
#5  0x80787f56 in softdep_deallocate_dependencies (
bp=0xfe01f008d8b8) at /usr/src/sys/ufs/ffs/ffs_softdep.c:14304
#6  0x8061dd69 in buf_deallocate (bp=0xfe01f008d8b8)
at /usr/src/sys/sys/buf.h:429
#7  brelse (bp=0xfe01f008d8b8) at /usr/src/sys/kern/vfs_bio.c:2348
#8  0x8061b9e5 in bufwrite (bp=0xfe01f008d8b8)
at /usr/src/sys/kern/vfs_bio.c:1914
#9  0x8079bec8 in softdep_process_journal (mp=, 
needwk=0x0, flags=)
at /usr/src/sys/ufs/ffs/ffs_softdep.c:3559
#10 0x80785dc0 in softdep_process_worklist (mp=0xf80007eef000, 
full=0) at /usr/src/sys/ufs/ffs/ffs_softdep.c:1592
#11 0x807894ff in softdep_flush (addr=0xf80007eef000)
at /usr/src/sys/ufs/ffs/ffs_softdep.c:1397
#12 0x80555075 in fork_exit (
callout=0x80789400 , arg=0xf80007eef000, 
frame=0xfe023a281a40) at /usr/src/sys/kern/kern_fork.c:1038
#13 
(kgdb) 

Hmmm,

%  kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.0
GNU gdb (GDB) 8.0 [GDB v8.0 for FreeBSD]
Copyright (C) 2017 Free Software Foundation, Inc.
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug...done.
ABI doesn't support a vmcore target

OK, so debugging is broken :-/

-- 
Steve
20170425 https://www.youtube.com/watch?v=VWUpyCsUKR4
20161221 https://www.youtube.com/watch?v=IbCHE-hONow
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: r324053: kldxref: Parse error of description string U32:vendor;U32:device;P;D:human

2017-09-27 Thread David Wolfskill 
On Wed, Sep 27, 2017 at 05:08:23AM -0700, David Wolfskill wrote:
> On Wed, Sep 27, 2017 at 11:23:22AM +0200, O. Hartmann wrote:
> > The installation of a newly, prsitine build kernel fails on CURRENT with the
> > error shown below:
> > 
> > [...]
> > kldxref /boot/kernel
> > kldxref: Parse error of description string U32:vendor;U32:device;P;D:human
> > *** [afterinstall] Error code 1
> > 
> > make[3]: stopped in /usr/src/sys/modules
> > 
> 
> I also encountered this (though merely with the usual
> "WITH_META_MODE=yes"), updating from r324007 to r324054.
> 

The code in question appears to have been added in r324038, affecting
src/sys/dev/drm2/i915/i915_drv.c:1239 and
src/sys/dev/drm2/radeon/radeon_drv.c:404.

Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
http://www.cbc.ca/news/opinion/donald-trump-playbook-1.4265374

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


signature.asc
Description: PGP signature

Re: r324053: kldxref: Parse error of description string U32:vendor;U32:device;P;D:human

2017-09-27 Thread David Wolfskill 
On Wed, Sep 27, 2017 at 11:23:22AM +0200, O. Hartmann wrote:
> The installation of a newly, prsitine build kernel fails on CURRENT with the
> error shown below:
> 
> [...]
> kldxref /boot/kernel
> kldxref: Parse error of description string U32:vendor;U32:device;P;D:human
> *** [afterinstall] Error code 1
> 
> make[3]: stopped in /usr/src/sys/modules
> 

I also encountered this (though merely with the usual
"WITH_META_MODE=yes"), updating from r324007 to r324054.

-- 
David H. Wolfskill  da...@catwhisker.org
http://www.cbc.ca/news/opinion/donald-trump-playbook-1.4265374

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


signature.asc
Description: PGP signature

Re: FreeBSD, IPFW and the SIP/VoIP NAT problem

2017-09-27 Thread Damjan Jovanovic 
On Wed, Sep 27, 2017 at 1:16 PM, O. Hartmann 
wrote:

> On Tue, 26 Sep 2017 16:26:51 +0200
> Damjan Jovanovic  wrote:
>
> > On Tue, Sep 26, 2017 at 3:44 PM, O. Hartmann 
> > wrote:
> >
> > > On Tue, 26 Sep 2017 11:00:45 +0200
> > > Damjan Jovanovic  wrote:
> > >
> > > > On Tue, Sep 26, 2017 at 10:35 AM, O. Hartmann <
> ohartm...@walstatt.org>
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > trying to build a FreeBSD based router/PBX (Asterisk 13)
> appliance, I
> > > ran
> > > > > into
> > > > > several problems. My questions might have a "noobish" character,
> so my
> > > > > apology,
> > > > > my experiences with IPFW are not as thorough as they should be.
> > > > >
> > > > > Before I'll got into medias res, aquestion about Pine64/AARCH64:
> > > > >
> > > > > - port net/asterisk13 is supposed to build only on armv6, is
> aarch64
> > > about
> > > > >   coming soon also supported?
> > > > > - would a Pine64 running CURRENT (12) be sufficient as a PBX
> platform
> > > > > (assumed
> > > > >   having 2 GB of RAM)?
> > > > >
> > > > > My main concern is about IPFW (we do not use PF for some reasons, I
> > > have to
> > > > > stay with IPFW).
> > > > >
> > > > > I'm a customer of two ITSPs and my SoHo network is behind NAT and
> not
> > > yet
> > > > > IPv6.
> > > > > The FreeBSD system acting as a router is supposed to have a jail
> soon
> > > > > containing the Asterisk 13 IP PBX (at the moment running on the
> main
> > > > > system).
> > > > > To provide access to the VoIP infrastructure inside/behind the
> > > router/NAT
> > > > > system, the in-kernel NAT facility of FreeBSD is forwarding the
> > > relevant
> > > > > UPD/TCP ports for VoIP to its destination network, and here I have
> a
> > > > > problem to
> > > > > solve.
> > > > >
> > > > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other
> > > ports,
> > > > > it
> > > > > is a mess and pain in the arse to forward a whole range, say
> 11000/udp
> > > -
> > > > > 35000/udp, which is a range one of my providers is sending RTP on.
> A
> > > second
> > > > > provider uses another range for RTP, starting at 5000/udp. So, the
> > > logical
> > > > > consequence would be a union set up UDP range to forward, which
> exapnds
> > > > > then
> > > > > form 5000/udp to 45000/udp - which is much more a pain ...
> > > > >
> > > > > One of the most disturbing and well known problems is that due to
> the
> > > > > stateful
> > > > > firewall the RTP session very often is half duplex - it seems one
> > > direction
> > > > > of the RTP connection doesn't make it through IPFW/NAT. As often I
> > > search
> > > > > the
> > > > > net, I always get informed this is a typical problem and solutions
> are
> > > > > provided by so called ALGs - since SIP protocol's SDP indicates
> within
> > > the
> > > > > payload of the packets on which UDP ports both ends wish to
> establish
> > > their
> > > > > RTP session, it would be "easy" to pinhole the IPFW on exactly
> those
> > > ports
> > > > > for
> > > > > a theoretical large number of sessions, if IPFW could "divert"
> those
> > > > > packets
> > > > > to an instance inspecting SDP (or whatever is used for the RTP port
> > > > > indication, I'm new to that, sorry for the terminology) and then
> > > pinholing
> > > > > the
> > > > > NAT/IPFW for exactly this purpose without the forwarding mess. I
> came
> > > along
> > > > > netgraph() while searching for hints and hooks, but it seems a
> complete
> > > > > Linux
> > > > > domain, when it somes to appliances like VoIP/IP PBX.
> > > > >
> > > > > Either, the problem is that trivial on FreeBSD, so no further
> > > mentioning is
> > > > > necessary (which would explain the vast emptyness of explanations,
> > > hints
> > > > > and
> > > > > so on) or FreeBSD is a complete wasteland on this subject - which I
> > > also
> > > > > suspect, since pfSense and OPNsense must have come along with such
> > > problems
> > > > > and I simply do not know or recognise the software used for those
> > > purposes.
> > > > >
> > > > > So, if someone enlightened in this matter stumbles over my
> question and
> > > > > could
> > > > > delegate me onto the right way (ports, ng_XXX netgraph ficilities
> to
> > > look
> > > > > at,
> > > > > some ipfw techniques relevant to the problem apart from the stupid
> > > simple
> > > > > forwarding large ranges of ports) - I'd appreciate this and
> > > > >
> > > > > thanks in advance for patience and help,
> > > > >
> > > > > Oliver
> > > > >
> > > >
> > > >
> > > > Hi
> > > >
> > > > It might be easier if you just enable STUN on Asterisk, and build
> FreeBSD
> > > > from source with my [largely neglected :( ] patch on
> > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219918
> > > >
> > > > That way Asterisk should dynamically discover consistent external
> > > mappings
> > > > for connections, making port forwarding RTP

Re: FreeBSD, IPFW and the SIP/VoIP NAT problem

2017-09-27 Thread Damjan Jovanovic 
On Tue, Sep 26, 2017 at 11:27 AM, Guido Falsi  wrote:

> On 09/26/2017 10:35, O. Hartmann wrote:
>
> > of the RTP connection doesn't make it through IPFW/NAT. As often I
> search the
> > net, I always get informed this is a typical problem and solutions are
> > provided by so called ALGs - since SIP protocol's SDP indicates within
> the
>
> This would require coding it in IPFW, and the load on the firewall could
> be significant.
>
> It could be done in userland maybe, leveraging divert(4) and having a
> daemon listening there and doing the extra work, but this would be quite
> expensive. Depending on your call volume the load could be too much for
> your firewall.
>
>
SIP headers like Proxy-Authorization need to send a cryptographic quality
hash of data that includes the password and the SDP when qop=auth-int, and
the ALG needs to change the IP address and port in the SDP, which changes
this hash. The ALG would have to know your password to calculate the new
hash.

A SIP ALG can thus only work with the weaker qop=auth protection, which
doesn't hash the SDP and is thus less secure (MITM attacks can
capture/modify RTP in transit), and even then it would have to be careful
not to change the SIP headers which are included in the hash.

Since it is the provider that chooses the allowed qop, a general SIP ALG is
impossible unless the ALG knows the password.

Linux has a SIP ALG in iptables, and it's full of problems and best
disabled.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: FreeBSD, IPFW and the SIP/VoIP NAT problem

2017-09-27 Thread O. Hartmann 
On Tue, 26 Sep 2017 16:26:51 +0200
Damjan Jovanovic  wrote:

> On Tue, Sep 26, 2017 at 3:44 PM, O. Hartmann 
> wrote:
> 
> > On Tue, 26 Sep 2017 11:00:45 +0200
> > Damjan Jovanovic  wrote:
> >  
> > > On Tue, Sep 26, 2017 at 10:35 AM, O. Hartmann 
> > > wrote:
> > >  
> > > > Hello,
> > > >
> > > > trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I  
> > ran  
> > > > into
> > > > several problems. My questions might have a "noobish" character, so my
> > > > apology,
> > > > my experiences with IPFW are not as thorough as they should be.
> > > >
> > > > Before I'll got into medias res, aquestion about Pine64/AARCH64:
> > > >
> > > > - port net/asterisk13 is supposed to build only on armv6, is aarch64  
> > about  
> > > >   coming soon also supported?
> > > > - would a Pine64 running CURRENT (12) be sufficient as a PBX platform
> > > > (assumed
> > > >   having 2 GB of RAM)?
> > > >
> > > > My main concern is about IPFW (we do not use PF for some reasons, I  
> > have to  
> > > > stay with IPFW).
> > > >
> > > > I'm a customer of two ITSPs and my SoHo network is behind NAT and not  
> > yet  
> > > > IPv6.
> > > > The FreeBSD system acting as a router is supposed to have a jail soon
> > > > containing the Asterisk 13 IP PBX (at the moment running on the main
> > > > system).
> > > > To provide access to the VoIP infrastructure inside/behind the  
> > router/NAT  
> > > > system, the in-kernel NAT facility of FreeBSD is forwarding the  
> > relevant  
> > > > UPD/TCP ports for VoIP to its destination network, and here I have a
> > > > problem to
> > > > solve.
> > > >
> > > > While it is sumple and easy to forward 5060/udp, 5070/tcp and other  
> > ports,  
> > > > it
> > > > is a mess and pain in the arse to forward a whole range, say 11000/udp  
> > -  
> > > > 35000/udp, which is a range one of my providers is sending RTP on. A  
> > second  
> > > > provider uses another range for RTP, starting at 5000/udp. So, the  
> > logical  
> > > > consequence would be a union set up UDP range to forward, which exapnds
> > > > then
> > > > form 5000/udp to 45000/udp - which is much more a pain ...
> > > >
> > > > One of the most disturbing and well known problems is that due to the
> > > > stateful
> > > > firewall the RTP session very often is half duplex - it seems one  
> > direction  
> > > > of the RTP connection doesn't make it through IPFW/NAT. As often I  
> > search  
> > > > the
> > > > net, I always get informed this is a typical problem and solutions are
> > > > provided by so called ALGs - since SIP protocol's SDP indicates within  
> > the  
> > > > payload of the packets on which UDP ports both ends wish to establish  
> > their  
> > > > RTP session, it would be "easy" to pinhole the IPFW on exactly those  
> > ports  
> > > > for
> > > > a theoretical large number of sessions, if IPFW could "divert" those
> > > > packets
> > > > to an instance inspecting SDP (or whatever is used for the RTP port
> > > > indication, I'm new to that, sorry for the terminology) and then  
> > pinholing  
> > > > the
> > > > NAT/IPFW for exactly this purpose without the forwarding mess. I came  
> > along  
> > > > netgraph() while searching for hints and hooks, but it seems a complete
> > > > Linux
> > > > domain, when it somes to appliances like VoIP/IP PBX.
> > > >
> > > > Either, the problem is that trivial on FreeBSD, so no further  
> > mentioning is  
> > > > necessary (which would explain the vast emptyness of explanations,  
> > hints  
> > > > and
> > > > so on) or FreeBSD is a complete wasteland on this subject - which I  
> > also  
> > > > suspect, since pfSense and OPNsense must have come along with such  
> > problems  
> > > > and I simply do not know or recognise the software used for those  
> > purposes.  
> > > >
> > > > So, if someone enlightened in this matter stumbles over my question and
> > > > could
> > > > delegate me onto the right way (ports, ng_XXX netgraph ficilities to  
> > look  
> > > > at,
> > > > some ipfw techniques relevant to the problem apart from the stupid  
> > simple  
> > > > forwarding large ranges of ports) - I'd appreciate this and
> > > >
> > > > thanks in advance for patience and help,
> > > >
> > > > Oliver
> > > >  
> > >
> > >
> > > Hi
> > >
> > > It might be easier if you just enable STUN on Asterisk, and build FreeBSD
> > > from source with my [largely neglected :( ] patch on
> > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219918
> > >
> > > That way Asterisk should dynamically discover consistent external  
> > mappings  
> > > for connections, making port forwarding RTP unnecessary.
> > >
> > > Damjan  
> >
> > STUN is enabled, but my providers do not support STUN.
> >
> > I try to figure out how SIP works exactly to make my problem more precise.
> > I
> > also try to understand the aim of your patch - as far as I know, it does
> > exactly

Re: FreeBSD, IPFW and the SIP/VoIP NAT problem

2017-09-27 Thread Guido Falsi 
Sorry the top posting but Before anything else I need to state a few points.

First this post is getting quite VoIP and asterisk specific, so please
direct any further replies to me at my email. We're off subject on this
list.

Also before giving specific answers I need to clarify a few details
about SIP/SDP and RTP, since, from your message it looks like you have
some misconceptions about some details. In specific how RTP connection
endpoints are negotiated and how the connections actually happen after
that. (I'll make a general description to be sure everything is clear)

SIP accepts connections on only one port (UDP usually but TCP is also
supported), 5060 by default, but can be changed. This IP:port
combination is what is registered with people wanting to call you, be it
SIP REGISTER on a provider, a DNS SRV record, a blind connection on 5060
to an IP returned by an A record or manual registration via your
provider website. (these are all method I've seen actually used, there
may be others)

After the caller connects via SIP to the called party they negotiate the
connection. The "media channel" is usually negotiated by embedding SDP
packets in the SIP exchange (it's all plain text, you can actually sniff
it from the asterisk console, and in fact it's often necessary to be
done to understand problems).

There is absolutely no rule forcing RTP to be used as the media channel
protocol, there are also others, but for phone calls that's what is
usually negotiated. But RTP is a different protocol whose only relation
to SIP is it being commonly used to actually relay the voice
communication between endpoints.

RTP is strictly defined to use UDP transport and is a monodirectional
protocol, so for VoIP communication you need two RTP channels (or
connections if you prefer), one for inbound and one for outbound.

In the SDP negotiation each party (there no differentiation between
caller and called party) will state where he expects the other party to
send it's audio. Various information will be exchanged, including
protocol audio format but most important for us IP:port.

After negotiation each party performs the connection to the other. Let
me rephrase: Your asterisk will be telling your provider to which
IP:port is should send it's UDP connection for the audio stream, so you
can't control where you are connecting (but that's not a problem, NAT
correctly handles that) you have FULL CONTROL of the UDP port range the
provider MUST use to connect to you (that is unless they are getting out
of their way with a non compliant implementation, in such a case, look
for a better provider).

That said your situation is quite easy to solve without any special
firewall configuration except redirecting a small (50-100) number of UDP
ports of your choice.

In real world situations this is quite handy since you rarely have full
control of the router/firewall and many times all you get is a basic DSL
modem/router with very limited FW/NAT functionality.

On the other hand if you actually need thousands of UDP ports(that is
thousands of simultaneous calls), most probably you also can get a
static IP address for your PBX.

On 09/26/2017 15:29, O. Hartmann wrote:
> On Tue, 26 Sep 2017 11:27:05 +0200
> Guido Falsi  wrote:
> 
> I already tried to build net/asterisk13 on my AARCH64 jail, but since I'd like
> to have the databases/postgresql96-client aboard and this specific port fails
> to build, I gave up - it is, by the way, the only port (pgsql) as far as I
> know that fails in my poudriere cross compiling jail. I did not get further,
> but I saw that it is supposed to build only for amd64 and armv6.
>  
>>
>> In such a case would you be willing to test port changes on the hardware
>> to actually check it runs?
> 
> I'd like to if the efforts are not to much time consuming - I do not have a
> working Pine64 image anymore, but I have a Pine64 with 2GB RAM at hand. Months
> ago I started playing with cross compiling world/kernel for AARCH64, but I'm
> not familiar with crochet and preparing the SD image - but willing to do. But
> beware: my home box preparing the cross cimpilation is not the fastest!
>  

Actually, I don't have ARM64 hardware available right now and no ARM64
jail right now, maybe in a not to distant future I'll have that and be
able to perform specific tests, but things standing as they are I cannot
state anything definitive on this subject.

> For the moment, the ARM based PBX should perform SoHo tasks - three, up to ten
> lines at maximum. 

So you need at most 1 port for SIP and 20 for RTP...a 50 ports range
will have space to spare.

> 
>>
>> On the other hand if you plan doing a lot of audio transcoding or some
>> video transcoding, load can get up quite fast. Compressed codecs like
>> the "simple" G729 will make your load grow relatively fast even with
>> audio transcoding. Transcoding also lowers call quality so it should
>> anyway be avoided as much as possible.
> 
> Good to know. But

Re: net/asterisk13: memory leak under 12-CURRENT?

2017-09-27 Thread O. Hartmann 
On Wed, 27 Sep 2017 09:05:42 +0200
Guido Falsi  wrote:

> On 09/26/2017 15:41, O. Hartmann wrote:
> > On Tue, 26 Sep 2017 15:06:23 +0200
> > Guido Falsi  wrote:  
> 
> > Since I run net/asterisk with automatic module loading (I'm new to
> > asterisk), this is very likely and might cause the problem somehow.
> >   
> 
> You can exclude single modules from autoloading via modules.conf.

I tried, but I need to study first more documentation to explore what is
prerequisite and what is optional.

> 
> >> Not sure, restarting the daemon should free any leaked memory the daemon 
> >> has. If a killed process leaves memory locked at the system level there 
> >> should be some other cause.  
> > 
> > Even with no runnidng asterisk, memory level drops after the last shutdown
> > of asterisk and keeps that low. Even for weeks! My router never shows that
> > high memory consumption, even under load.  
> 
> But while asterisk is running does the memory usage increase unbounded
> till filling all available memory or does it stabilize at some point?

As far as I could observe, a three day test run of the router/firewall/asterisk
box drained around 500 MB of memory: starting at boot time with ~3700 MB,
asterisk leaves the box with ~3640 MB after bein started and after three days
the system reached ~3150 MB. Stopping asterisk gave back some memory, so ~3300
MB then was for days the final result - not recovering anything further. I use
TEMPFS, if it matters, but I checked /tmp and /var/, there were no remnant
files so far. TMPVAR is only allowed to have 256 MB.
 
> 
> Asterisk is relatively memory hungry, especially with all modules
> enabled. It also caches and logs various information in RAM, even doing
> "nothing" it will cache and log that "nothing" activity. If memory does
> stabilize after some point it's not really a leak but it's standard
> memory usage. To reduce it you should disable all unused modules.
> 
> > 
> > The question would be: how to use vmstat to give hints for those familiar
> > with memory subsystems to indicate a real bug?
> > 
> > I tried to find some advices, but maybe my English isn't good enough to make
> > google help.  
> 
> I'm not able to give you a correct indication, but if the memory usage
> is not increasing indefinitely but is stabilizing I'd say it's not
> really a leak.
> 

Can't say whether it is stabilising or not - I think the runtime is too short.
I'll check first to disable some modules in the first place and then try to
perform a test with several days of asterisk enabled.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: net/asterisk13: memory leak under 12-CURRENT?

2017-09-27 Thread Hans Petter Selasky 

On 09/27/17 09:05, Guido Falsi wrote:

On 09/26/2017 15:41, O. Hartmann wrote:

On Tue, 26 Sep 2017 15:06:23 +0200
Guido Falsi  wrote:



Since I run net/asterisk with automatic module loading (I'm new to asterisk),
this is very likely and might cause the problem somehow.



You can exclude single modules from autoloading via modules.conf.


Not sure, restarting the daemon should free any leaked memory the daemon
has. If a killed process leaves memory locked at the system level there
should be some other cause.


Even with no runnidng asterisk, memory level drops after the last shutdown of
asterisk and keeps that low. Even for weeks! My router never shows that high
memory consumption, even under load.


But while asterisk is running does the memory usage increase unbounded
till filling all available memory or does it stabilize at some point?

Asterisk is relatively memory hungry, especially with all modules
enabled. It also caches and logs various information in RAM, even doing
"nothing" it will cache and log that "nothing" activity. If memory does
stabilize after some point it's not really a leak but it's standard
memory usage. To reduce it you should disable all unused modules.



The question would be: how to use vmstat to give hints for those familiar with
memory subsystems to indicate a real bug?

I tried to find some advices, but maybe my English isn't good enough to make
google help.


I'm not able to give you a correct indication, but if the memory usage
is not increasing indefinitely but is stabilizing I'd say it's not
really a leak.



Did you look at the output from "vmstat -m" and "vmstat -z" ?

--HPS
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: net/asterisk13: memory leak under 12-CURRENT?

2017-09-27 Thread Guido Falsi 
On 09/27/2017 11:27, O. Hartmann wrote:
> On Wed, 27 Sep 2017 09:05:42 +0200
> Guido Falsi  wrote:
>> But while asterisk is running does the memory usage increase unbounded
>> till filling all available memory or does it stabilize at some point?
> 
> As far as I could observe, a three day test run of the 
> router/firewall/asterisk
> box drained around 500 MB of memory: starting at boot time with ~3700 MB,
> asterisk leaves the box with ~3640 MB after bein started and after three days
> the system reached ~3150 MB. Stopping asterisk gave back some memory, so ~3300
> MB then was for days the final result - not recovering anything further. I use
> TEMPFS, if it matters, but I checked /tmp and /var/, there were no remnant
> files so far. TMPVAR is only allowed to have 256 MB.
> 

These numbers really don't tell us anything. The system has anyway been
running for days, depending on configuration daemons like cron and ntp
are running and performing tasks, things are being cached and so on, so
that difference after three days could be perfectly normal overhead.

You need to investigate the amount of memory allocated to asterisk with
ps and top and check if that stabilizes. A few days, at most a week
would be enough. After that, if it's not stabilizing you can start
thinking on a leak, but still can't assume where the leak is happening.


> Can't say whether it is stabilising or not - I think the runtime is too short.
> I'll check first to disable some modules in the first place and then try to
> perform a test with several days of asterisk enabled.
> 

Whatever you prefer, but trying a few days uptime with all modules
enabled is zero cost and east to do. Also you started this report with
THIS configuration, changing configuration would prevent from comparing
results. Let's test one thing at a time.

-- 
Guido Falsi 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

r324053: kldxref: Parse error of description string U32:vendor;U32:device;P;D:human

2017-09-27 Thread O. Hartmann 
The installation of a newly, prsitine build kernel fails on CURRENT with the
error shown below:

[...]
kldxref /boot/kernel
kldxref: Parse error of description string U32:vendor;U32:device;P;D:human
*** [afterinstall] Error code 1

make[3]: stopped in /usr/src/sys/modules
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: net/asterisk13: memory leak under 12-CURRENT?

2017-09-27 Thread Guido Falsi 
On 09/26/2017 15:41, O. Hartmann wrote:
> On Tue, 26 Sep 2017 15:06:23 +0200
> Guido Falsi  wrote:

> Since I run net/asterisk with automatic module loading (I'm new to asterisk),
> this is very likely and might cause the problem somehow.
> 

You can exclude single modules from autoloading via modules.conf.

>> Not sure, restarting the daemon should free any leaked memory the daemon 
>> has. If a killed process leaves memory locked at the system level there 
>> should be some other cause.
> 
> Even with no runnidng asterisk, memory level drops after the last shutdown of
> asterisk and keeps that low. Even for weeks! My router never shows that high
> memory consumption, even under load.

But while asterisk is running does the memory usage increase unbounded
till filling all available memory or does it stabilize at some point?

Asterisk is relatively memory hungry, especially with all modules
enabled. It also caches and logs various information in RAM, even doing
"nothing" it will cache and log that "nothing" activity. If memory does
stabilize after some point it's not really a leak but it's standard
memory usage. To reduce it you should disable all unused modules.

> 
> The question would be: how to use vmstat to give hints for those familiar with
> memory subsystems to indicate a real bug?
> 
> I tried to find some advices, but maybe my English isn't good enough to make
> google help.

I'm not able to give you a correct indication, but if the memory usage
is not increasing indefinitely but is stabilizing I'd say it's not
really a leak.

-- 
Guido Falsi 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"