pf reset connection for ssh user when authpf user resets ssh connection

[Pathik]

I am trying to set up pf along with authpf on my bastion host.


Requirements for bastion hosts are as below:

   1. Admin users – have access to all resources on internal network when
   they ssh using port 22
   2. Other users — have access to limited resources in internal network
   through ssh port 2222 after they have authpf themselves to port 22

To achieve above, I have pf rules set as below:

————

ext_if = "em1"

int_if = "em2"

dns_servers = “{some_ip_here}"



set skip on lo

block return log all


# Allow everyone to ssh in

pass in log quick on $ext_if inet proto tcp from any to $ext_if port ssh
modulate state


# Track all authpf users in a table

table <authpf_users> persist


# Allow ICMP for debugging

pass in quick inet proto icmp from any to any


# Let me have basic connections

pass in quick log (all) inet proto udp from any to $dns_servers port 53
keep state


pass in quick inet proto tcp from $int_if to $int_if:network port {ssh http
https 3306} modulate state

pass quick on $int_if inet proto udp


# Load authpf anchor

anchor "authpf/*”

————


Authpf has 2 files

   1. /etc/authpf/authpf.conf is empty
   2. /etc/authpf/users/$user/authpf.rules is as below

————

ext_if="em1"

int_if="em2"


table <limited_nodes> file "/etc/pf/table/limited_resources"

pass log (all) inet proto tcp from <authpf_users> to $ext_if port 2222
modulate state

pass log (all) inet proto tcp from $int_if to <limited_nodes> port 22
modulate state

————


With above configuration in place, here’s symptoms for the problem.

   1. admin user is logged in to bastion host on port 22 and has a working
   shell(ksh,bash), has a source ip of 1.2.3.4
   2. Other user logs in to port 22 and gets authpf shell has a source ip
   of 1.2.3.4
   3. Other user disconnects its port 22 connection using ctrl+c, he is
   released from authpf and his entries get cleared in pf table.
   4. Admin user who is logged in from another terminal from source ip
   1.2.3.4 also receives a disconnect with message "Write failed: Broken pipe”
   on his ssh working shell.

If I remove ‘block return log all’’ from pf.conf then admin user is not
getting the disconnect as noted in step 4. But that is not desirable as we
need default block.


Has anyone tried to implement anything like above?

Is there some caveats between pf and authpf which I do not know and is
causing this behavior?