Re: TTL expired in transit to qemu virtual machine.

[Mimiko]

On 23.03.2017 14:14, Greg Wooledge wrote:

Run "uname -a" (and also "uptime") and post the results.


You a right. I didn't do a restart after last kernel update.

After a restart, TTL problem didn't occur anymore.

Also I've noted that the particular vm had INPUT default policy to DROP and no 
icmp accepted. Strange that some time I did get an answer on ping.
Accepting icmp requests gives me stable replies.

Thank you all for answering.

--
Mimiko desu.

Re: TTL expired in transit to qemu virtual machine.

[Greg Wooledge]

On Thu, Mar 23, 2017 at 08:36:38AM +0200, Mimiko wrote:
> Well, doing regulat apt-get update && apt-get upgrade, I fought that kernel 
> is also upgraded. I've seen this several times. How comes it wasn't updated 
> to 3.2.86-1.

Run "uname -a" (and also "uptime") and post the results.  After a kernel
package is upgraded, you do have to reboot to run the new kernel, and
uptime will tell us how long ago you did that.

> As for apt-cache search '^linux-image-*' :
> linux-image-3.2.0-4-amd64 - Linux 3.2 for 64-bit PCs
[...]

> Then doing apt-get install linux-image-amd64 says it already latest version.

It's important to note that the kernel package name is *not* the same
as the exact kernel version.  That's why you want to run "uname -a".
It will show both of them.

The kernel package name 3.2.0-4-amd64 refers to the ABI (module
compatibility) of the kernel.  A module built for any kernel with this
package name will work with any other version, so you don't have to
rebuild all of your third party kernel modules every time you upgrade
the kernel.

Re: TTL expired in transit to qemu virtual machine.

[Mimiko]

On 21.03.2017 02:25, david...@freevolt.org wrote:

It is not clear to me whether your question


I regularly do apt-get upgrade, but not to next Debian version. So, how this 
kernel be old for Debian 7?


is a request for information, or merely rhetorical (ie, an assertion
that your kernel is in fact current relative to debian 7).

In case you were asking, I'm still using debian 7 too,

  $ cat /etc/debian_version
  7.11

and, for what it's worth, in case it interests you

  $ uname -a
  Linux leviathan 3.2.0-4-686-pae #1 SMP Debian 3.2.86-1 i686 GNU/Linux

I notice the version you posted here (3.2.84-1) is slightly older than
mine. If like myself you plan on running the 3.2.0-4-* release for a
while, It might[1] be worth upgrading at least to version 3.2.86-1:


Well, doing regulat apt-get update && apt-get upgrade, I fought that kernel is also upgraded. I've seen this several times. How comes it wasn't 
updated to 3.2.86-1.


As for apt-cache search '^linux-image-*' :
linux-image-3.2.0-4-amd64 - Linux 3.2 for 64-bit PCs
linux-image-3.2.0-4-amd64-dbg - Debugging symbols for Linux 3.2.0-4-amd64
linux-image-3.2.0-4-rt-amd64 - Linux 3.2 for 64-bit PCs, PREEMPT_RT
linux-image-3.2.0-4-rt-amd64-dbg - Debugging symbols for Linux 3.2.0-4-rt-amd64
linux-image-2.6-amd64 - Linux for 64-bit PCs (dummy package)
linux-image-amd64 - Linux for 64-bit PCs (meta-package)
linux-image-rt-amd64 - Linux for 64-bit PCs (meta-package), PREEMPT_RT

Then doing apt-get install linux-image-amd64 says it already latest version.

Anyway, this slightly different (minor) version difference could be the problem?

--
Mimiko desu.

Re: TTL expired in transit to qemu virtual machine.

[davidson]

On Mon, 20 Mar 2017, Mimiko wrote:

On 18.03.2017 07:22, Igor Cicimov wrote:

>uname -a
Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.84-1 x86_64 GNU/Linux

That's an really old kernel, I don't start anything virtual these days 
without at least 3.13.x kernel.


I regularly do apt-get upgrade, but not to next Debian version. So, how this 
kernel be old for Debian 7?


TLDR:

 1. I have no clues to offer regarding OP's virtualisation/networking
 troubles, but I wish them the best of fortune.

 2. OP's kernel version does not look quite current to me, even for
 wheezy (debian 7) and it might be a matter of concern (though I claim
 no particular expertise).

Longer version:

It is not clear to me whether your question

I regularly do apt-get upgrade, but not to next Debian version. So, how this 
kernel be old for Debian 7?


is a request for information, or merely rhetorical (ie, an assertion
that your kernel is in fact current relative to debian 7).

In case you were asking, I'm still using debian 7 too,

  $ cat /etc/debian_version
  7.11

and, for what it's worth, in case it interests you

  $ uname -a
  Linux leviathan 3.2.0-4-686-pae #1 SMP Debian 3.2.86-1 i686 GNU/Linux

I notice the version you posted here (3.2.84-1) is slightly older than
mine. If like myself you plan on running the 3.2.0-4-* release for a
while, It might[1] be worth upgrading at least to version 3.2.86-1:

  https://security-tracker.debian.org/tracker/CVE-2016-5195

There is also a newer kernel release available, for those who choose
to install it explicitly[3], from the wheezy-backports repository[2],
(see the 3.16.0-0.bpo releases near the end):

  $ apt-cache search '^linux-image-*'
  linux-image-3.2.0-4-486 - Linux 3.2 for older PCs
  linux-image-3.2.0-4-686-pae - Linux 3.2 for modern PCs
  linux-image-3.2.0-4-686-pae-dbg - Debugging symbols for Linux 3.2.0-4-686-pae
  linux-image-3.2.0-4-amd64 - Linux 3.2 for 64-bit PCs
  linux-image-3.2.0-4-rt-686-pae - Linux 3.2 for modern PCs, PREEMPT_RT
  linux-image-3.2.0-4-rt-686-pae-dbg - Debugging symbols for Linux 
3.2.0-4-rt-686-pae
  linux-image-2.6-486 - Linux for older PCs (dummy package)
  linux-image-2.6-686 - Linux for modern PCs (dummy package)
  linux-image-2.6-686-bigmem - Linux for PCs with 4GB+ RAM (dummy package)
  linux-image-2.6-686-pae - Linux for modern PCs (dummy package)
  linux-image-2.6-amd64 - Linux for 64-bit PCs (dummy package)
  linux-image-486 - Linux for older PCs (meta-package)
  linux-image-686 - Linux for modern PCs (dummy package)
  linux-image-686-bigmem - Linux for PCs with 4GB+ RAM (dummy package)
  linux-image-686-pae - Linux for modern PCs (meta-package)
  linux-image-amd64 - Linux for 64-bit PCs (meta-package)
  linux-image-rt-686-pae - Linux for modern PCs (meta-package), PREEMPT_RT
  linux-image-3.16.0-0.bpo.4-586 - Linux 3.16 for older PCs
  linux-image-3.16.0-0.bpo.4-686-pae - Linux 3.16 for modern PCs
  linux-image-3.16.0-0.bpo.4-686-pae-dbg - Debugging symbols for Linux 
3.16.0-0.bpo.4-686-pae
  linux-image-3.16.0-0.bpo.4-amd64 - Linux 3.16 for 64-bit PCs
  linux-image-586 - Linux for older PCs (meta-package)
  linux-image-686-pae-dbg - Debugging symbols for Linux 686-pae configuration 
(meta-package)

But regarding backports, do see https://backports.debian.org/FAQ/

| Q: Is there security support for packages from backports.debian.org?
| 
| A: Unfortunately not. This is done on a best effort basis by the

| people who track the package, usually the ones who originally did
| upload the package into backports. When security related bugs are
| fixed in Debian unstable the backporter is permitted to upload the
| package from directly there instead of having to wait until the fix
| hits testing. You can see the open issues for wheezy-backports in
| the security tracker (though there may be false positives too, the
| version compare isn't perfect yet)

[1] The security-tracker page for CVE-2016-5195 lists version 78-1 as
'vulnerable', and 86-1 as 'fixed'. It is unclear to me what this
is supposed to say about OP's kernel, which is apparently at 84-1.

[2] https://backports.debian.org/Instructions/#index2h2
[3] https://backports.debian.org/Instructions/#index3h2

Re: TTL expired in transit to qemu virtual machine.

[Mimiko]

On 18.03.2017 07:22, Igor Cicimov wrote:

>uname -a
Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.84-1 x86_64 GNU/Linux

That's an really old kernel, I don't start anything virtual these days without 
at least 3.13.x kernel.


I regularly do apt-get upgrade, but not to next Debian version. So, how this 
kernel be old for Debian 7?


Hmmm, this doesn't make much sense to me, more specifically this part:

vlan-raw-device bond0
bridge_ports bond0

Whats the purpose exactly of the vlan? Usually, and that is how I do it, you 
would split the VLAN's coming from the switch trunk port over the bond
and attach them to separate bridges lets say:


Don't mind that line. I putted it as a reminder to continue extend networking 
in using vlans. For now there is no vlan used.



That's strange indeed, except if br0 is used by something else like libvrit 
network that sets up the interface for proxy-arp. What's the output of:

# brctl showmacs br0


port no mac addris local?   ageing timer
  1 bridge mac  yes0.00
  8 fe:54:00:13:76:5b   yes0.00
  5 fe:54:00:44:ac:39   yes0.00
  6 fe:54:00:a8:35:fb   yes0.00
  2 fe:54:00:c2:64:2d   yes0.00
  4 fe:54:00:d5:17:fb   yes0.00
  3 fe:54:00:de:83:2d   yes0.00



# ip route show


10.0.0.0/8 via 10.10.10.254 dev br0
10.10.0.0/16 dev br0  proto kernel  scope link  src 10.10.10.10



# arp -n


gives a long list for local and virtual ip/mac



on the host, and:

# ip link show


I can't connect using ssh, only vnc.
This shows lo and eth0 with corresponding mac and is UP


# ip route show

10.10.0.0/16 dev eth0 proto kernel scope link src 10.10.10.3
169.254.0.0/16 dev eth0 scope link metric 1002
default via 10.10.10.254 dev eth0


# arp -n


gives a long list for local and virtual ip/mac




on the problematic vm and on one of the good vm's?


Thouse 3 commands shows the same (except ip and mac) on both vm's.




To find the loop I would start by doing ping between good and bad vm (both 
directions in turns) and check the traffic on the host interface:

# tcpdump -ennqt -i br0 \( arp or icmp \)

and corresponding network devices on both vm's too.


This gives me lot of output regarding arp and icmp on LAN.


Any sysctl settings you might have changed on the host?


net.ipv4.ip_forward=1
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2


--
Mimiko desu.

Re: TTL expired in transit to qemu virtual machine.

[Mimiko]

On 17.03.2017 13:21, Dominik George wrote:

iptables -L FORWARD -nv

Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
XX ACCEPT all  --  br0br0 0.0.0.0/0
0.0.0.0/0

What is that supposed to do?

Forwarding on the IP layer, by definition, happens between different interfaces.

Although this rule is most likely a no-op, you should remove it to reduce 
confusion.


This rule allows traffic to virtuals. Without this rule virtuals does not have access to LAN. Yes, I know that bridge setup does not need iptables, 
but reality is this.



--
Mimiko desu.

Re: TTL expired in transit to qemu virtual machine.

[Igor Cicimov]

Hi Mimiko,

On Fri, Mar 17, 2017 at 9:58 PM, Mimiko  wrote:

> Hello.
>
> I've setup qemu/kvm and installed several virtual machines. Access and
> ping to some virtuals are ok, but one have a stable problem not receiving
> correctly packets. First, this is the environment:
>
> >uname -a
> Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.84-1 x86_64 GNU/Linux
>

That's an really old kernel, I don't start anything virtual these days
without at least 3.13.x kernel.


>
> >libvirtd --version
> libvirtd (libvirt) 0.9.12.3
>
> >cat /etc/network/interfaces
> auto lo
> iface lo inet loopback
>
> auto eth0
> iface eth0 inet manual
>
> auto eth1
> iface eth1 inet manual
>
> auto bond0
> iface bond0 inet manual
> bond-slaves eth0 eth1
> bond-mode balance-alb
> bond-miimon 100
> bond-downdelay 200
> bond-updelay 200
>
> auto br0
> iface br0 inet static
> address 10.10.10.10
> netmask 255.255.0.0
> vlan-raw-device bond0
> bridge_ports bond0
> bridge_stp off
> bridge_fd 0
> bridge_ageing 0
> bridge_maxwait 0
>

Hmmm, this doesn't make much sense to me, more specifically this part:

vlan-raw-device bond0
bridge_ports bond0

Whats the purpose exactly of the vlan? Usually, and that is how I do it,
you would split the VLAN's coming from the switch trunk port over the bond
and attach them to separate bridges lets say:

# VLAN51
auto br-vlan51
iface br-vlan51 inet manual
vlan-raw-device bond0
bridge_ports *bond0.51*
bridge_stp off
bridge_maxwait 0
bridge_fd 0

# VLAN52
auto br-vlan52
iface br-vlan52 inet manual
vlan-raw-device bond0
bridge_ports *bond0.52*
bridge_stp off
bridge_maxwait 0
bridge_fd 0

If the intention was to pass through the tagged traffic to the VM's then
vlan-raw-device part is not needed at all.


> Virtual machines connects to LAN using bridge:
> >virt-install  --network=bridge:br0,model=virtio 
>
> All virtuals has networking configuret like that. Also in iptables is an
> entry to allow access to virtuals:
>
> >iptables -L FORWARD -nv
> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>  pkts bytes target prot opt in out source
>  destination
>  XX ACCEPT all  --  br0br0 0.0.0.0/0
> 0.0.0.0/0
>
> Most virtuals does not have networking problems, but some times they can't
> be reached. For now only one virtual machines have this problem:
> From windows machine ping virtual machine:
>
> >ping 10.10.10.3
>
> Reply from 10.10.10.10: TTL expired in transit.
> Reply from 10.10.10.10: TTL expired in transit.
> Reply from 10.10.10.10: TTL expired in transit.
> Reply from 10.10.10.10: TTL expired in transit.
> Request timed out.
> Request timed out.
> Request timed out.
> Request timed out.
> Request timed out.
> Request timed out.
> Reply from 10.10.10.10: TTL expired in transit.
> Reply from 10.10.10.10: TTL expired in transit.
> Reply from 10.10.10.10: TTL expired in transit.
> Reply from 10.10.10.10: TTL expired in transit.
>
> >tracert -d 10.10.10.3
>
> Tracing route to 10.10.10.3 over a maximum of 30 hops
>
>   1<1 ms<1 ms<1 ms  10.10.10.10
>   2<1 ms<1 ms<1 ms  10.10.10.10
>   3<1 ms<1 ms * 10.10.10.10
>   4 1 ms<1 ms<1 ms  10.10.10.10
>   5<1 ms<1 ms * 10.10.10.10
>
> So packet goes round on interfaces of server hosting virtuals.
>

Yep typical routing loop.


>
> Virtuals are linux different flavour and one windows. This problem may
> occur on any of this virtuals.
>
> I've observed that for this particular virtual, which have problem, the
> arp table of host assigned self mac to the virtual's IP, not the mac
> configured for virtual machine.
>

That's strange indeed, except if br0 is used by something else like libvrit
network that sets up the interface for proxy-arp. What's the output of:

# brctl showmacs br0
# ip route show
# arp -n

on the host, and:

# ip link show
# ip route show
# arp -n

on the problematic vm and on one of the good vm's?

To find the loop I would start by doing ping between good and bad vm (both
directions in turns) and check the traffic on the host interface:

# tcpdump -ennqt -i br0 \( arp or icmp \)

and corresponding network devices on both vm's too.


>
> What could be the problem?
>
>
Any sysctl settings you might have changed on the host?


> --
> Mimiko desu.
>
>

Re: TTL expired in transit to qemu virtual machine.

[Dominik George]

Hi,

> >iptables -L FORWARD -nv
>Chain FORWARD (policy DROP 0 packets, 0 bytes)
>pkts bytes target prot opt in out source  
>destination
>XX ACCEPT all  --  br0br0 0.0.0.0/0   
>0.0.0.0/0

What is that supposed to do?

Forwarding on the IP layer, by definition, happens between different interfaces.

Although this rule is most likely a no-op, you should remove it to reduce 
confusion.

-nik

TTL expired in transit to qemu virtual machine.

[Mimiko]

Hello.

I've setup qemu/kvm and installed several virtual machines. Access and ping to some virtuals are ok, but one have a stable problem not receiving 
correctly packets. First, this is the environment:


>uname -a
Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.84-1 x86_64 GNU/Linux

>libvirtd --version
libvirtd (libvirt) 0.9.12.3

>cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

auto eth1
iface eth1 inet manual

auto bond0
iface bond0 inet manual
bond-slaves eth0 eth1
bond-mode balance-alb
bond-miimon 100
bond-downdelay 200
bond-updelay 200

auto br0
iface br0 inet static
address 10.10.10.10
netmask 255.255.0.0
vlan-raw-device bond0
bridge_ports bond0
bridge_stp off
bridge_fd 0
bridge_ageing 0
bridge_maxwait 0

Virtual machines connects to LAN using bridge:
>virt-install  --network=bridge:br0,model=virtio 

All virtuals has networking configuret like that. Also in iptables is an entry 
to allow access to virtuals:

>iptables -L FORWARD -nv
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target prot opt in out source   destination
 XX ACCEPT all  --  br0br0 0.0.0.0/00.0.0.0/0

Most virtuals does not have networking problems, but some times they can't be 
reached. For now only one virtual machines have this problem:
From windows machine ping virtual machine:

>ping 10.10.10.3

Reply from 10.10.10.10: TTL expired in transit.
Reply from 10.10.10.10: TTL expired in transit.
Reply from 10.10.10.10: TTL expired in transit.
Reply from 10.10.10.10: TTL expired in transit.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 10.10.10.10: TTL expired in transit.
Reply from 10.10.10.10: TTL expired in transit.
Reply from 10.10.10.10: TTL expired in transit.
Reply from 10.10.10.10: TTL expired in transit.

>tracert -d 10.10.10.3

Tracing route to 10.10.10.3 over a maximum of 30 hops

  1<1 ms<1 ms<1 ms  10.10.10.10
  2<1 ms<1 ms<1 ms  10.10.10.10
  3<1 ms<1 ms * 10.10.10.10
  4 1 ms<1 ms<1 ms  10.10.10.10
  5<1 ms<1 ms * 10.10.10.10

So packet goes round on interfaces of server hosting virtuals.

Virtuals are linux different flavour and one windows. This problem may occur on 
any of this virtuals.

I've observed that for this particular virtual, which have problem, the arp table of host assigned self mac to the virtual's IP, not the mac 
configured for virtual machine.


What could be the problem?

--
Mimiko desu.