baseline_shorewall

The cake function baseline_shorewall executes the Ansible role-shorewall, which installs and configures shorewall.

shorewall and shorewall6 generate and manage iptables rules for IPv4 and IPv6.

Simplified role-shorewall works as follows:

• Merge the variables shorewall(6)_{zones,interfaces,snat,tunnels,policies} from various sources
• Install shorewall apt packages
• Template config files below /etc/default/shorewall(6) and /etc/shorewall(6)/
• Manage the shorewall and shorewall6 systemd services

Information

Key	Value
Playbook path	plays/baseline/shorewall.yml
Role	https://git.blunix.com/ansible-roles/role-shorewall
Tags	https://git.blunix.com/ansible-roles/role-shorewall/-/tags
Defaults	https://git.blunix.com/ansible-roles/role-shorewall/-/tree/master/defaults/main

Config file	Description	Documentation
/etc/shorewall(6)/shorewall.conf	Main config file	https://shorewall.org/manpages/shorewall.conf.html
/etc/shorewall(6)/interfaces	Network interfaces	https://shorewall.org/manpages/shorewall-interfaces.html
/etc/shorewall(6)/policy	Policies (accept, reject, drop) for when no rule applies	https://shorewall.org/manpages/shorewall-policy.html
/etc/shorewall(6)/rules	Firewall rules	https://shorewall.org/manpages/shorewall-rules.html
/etc/shorewall(6)/snat	Source NAT	https://shorewall.org/manpages/shorewall-snat.html
/etc/shorewall(6)/tunnels	Tunnels	https://shorewall.org/manpages/shorewall-tunnels.html
/etc/shorewall(6)/zones	Network zones	https://shorewall.org/manpages/shorewall-zones.html

role-shorewall design

shorewall_*_default

To template the config files listed above, variables named shorewall_<config-file-name> are constructed by the role, for example shorewall_interfaces. These variables differ between servers, for example some servers have additional network interfaces.

role-shorewall provides default variables to template these config files. These default variables are named shorewall_<config-file>_default. For example all servers have a network interfaces named mesh defined in shorewall_interfaces_default. No manual configuration in inventory/ is required for that.

shorewall_*_custom

To append custom variables to these defaults, shorewall_<config-file>_custom variables can be defined in the inventory/. Example:

inventory/group_vars/cus_tool_prod_jenkins.yml:

shorewall_rules_custom:

  # Using a shorewall macro: https://shorewall.org/Macros.html
  - name: allow incoming web traffic from pub
    src: pub
    dest: local
    action: Web(ACCEPT)

  # Using a custom definition
  - name: 'allout incoming HTTP/HTTPS from wgadm via gateways'
    src: "mesh:{{ hostvars['cus-util-prod-gateway-1']['nic_wg_mesh_ip'] }},{{ hostvars['cus-util-prod-gateway-2']['nic_wg_mesh_ip'] }}"
    dest: local
    action: Web(ACCEPT)

inventory/group_vars/util_gateway.yml:

shorewall_interfaces_custom:
  - name: wgadm
    zone: wgadm
    options: tcpflags,nosmurfs,routefilter,logmartians

Will result in:

gateway-1:/etc/shorewall/interfaces:

pub     eth0    nosmurfs,routefilter=2,tcpflags,dhcp
mesh    mesh    tcpflags,nosmurfs,routefilter,logmartians
wgadm   wgadm   tcpflags,nosmurfs,routefilter,logmartians  <== additional line here

shorewall_rule_templates

Often there are many similar firewall rules that need to be defined for a larger group, but not all servers. Hence, role-shorewall provides templates for firewall rules:

• shorewall templates
• shorewall6 templates

These rules can be included like so:

inventory/group_vars/util_monitoring.yml:

shorewall_rule_templates:
  - inc_pub_ssh
  - util_monitoring

inventory/group_vars/cus_www_prod_lb.yml:

shorewall_rule_templates:
  - inc_pub_ssh
  - inc_pub_http

Most of the rule_templates are for the utility servers, but the following might be useful for custom servers as well:

• inc_pub_http: allow incoming HTTP(S) on the pub interface
• inc_mesh_gw_http: allow incoming HTTP(S) on the mesh interface from gateway-1 and -2 to make it available to employees
• inc_pub_ssh: allow incoming SSH on the pub interface
• out_pub_ssh: allow outgoing SSH via the pub interface

Example

Allow SSH from pub on all servers via IPv4 and IPv6:

inventory/group_vars/all.yml:

shorewall_rule_templates:
  - inc_pub_ssh
shorewall6_rule_templates:
  - inc_pub_ssh

On the backup server, allow SSH from pub and configure the rules for a borgbackup server:

inventory/group_vars/util_backup.yml:

shorewall_rule_templates:
  - inc_pub_ssh
  - util_backup

Enable IP forwarding, allow incoming HTTP from pub, the employee VPN and allow outgoing SSH to a hetzner storagebox:

inventory/group_vars/cus_www_prod_web.yml:

# Enable IP forwarding
shorewall_conf_ip_forwarding: "Yes"

# Custom rules for this webserver
shorewall_rules_custom:
  - name: allow wgadm:all access to https
    src: "mesh:{{ hostvars['cus-util-prod-gateway-1']['nic_wg_mesh_ip'] }},{{ hostvars['cus-util-prod-gateway-2']['nic_wg_mesh_ip'] }}"
    dest: local
    proto: tcp
    dest_ports: 443
    action: ACCEPT
  - name: allow incoming https from pub
    src: pub
    dest: local
    proto: tcp
    dest_ports: 443
    action: ACCEPT
  - name: allow outgoing SSH to hetzner storagebox
    src: local
    dest: 'pub:u123456.your-storagebox.de'
    action: SSH(ACCEPT)

Docker

For servers that run docker simply set shorewall_docker: True. The role will take care of the rest and set an ALLOW policy for the docker interfaces. Shorewall will save iptables rules created by docker-compose and alike during server reboots. shorewall clear however will destroy those rules and you have to restart the docker daemon afterwards.

Usage

Checking for changes first

Sometimes developers add new rules by hand to /etc/shorewall/rules. These rules might not be expected to be overwritten and have to be transferred to inventory/.

Because of this, cake -f baseline_shorewall --changes should be executed before running cake -f baseline_shorewall --no-check. This way you can view the changes Ansible would make to the files:

-# allow contacting my favorite third party API
-ACCEPT local pub:1.2.3.4 tcp 11746

Copied to the inventory/, this would be written:

shorewall_rules_custom:
  - name: allow contacting my favorite third party API
    src: local
    dest: "pub:1.2.3.4"
    proto: tcp
    dest_ports: 11746
    action: ACCEPT

When everything is updated, preferably double check again without -nc, then apply the changes with:

cake -f baseline_shorewall -i pub -nc

After adding new hosts to inventory/hosts

After adding new hosts to the inventory/hosts file, role-shorewall will setup the new IPs to be allowed wireguard-mesh connections from/to all other servers. This will change the /etc/shorewall/rules file on all servers. Example:

 # allow outgoing mesh connections to all servers pub IPs
-ACCEPT local pub:1.2.3.4,2.3.4.5 udp,tcp 51819
+ACCEPT local pub:1.2.3.4,2.3.4.5,5.6.7.8,3.5.7.9 udp,tcp 51819

 # allow incoming mesh connections from all servers pub IPs
-ACCEPT pub:1.2.3.4,2.3.4.5 local udp,tcp 51819
+ACCEPT pub:1.2.3.4,2.3.4.5,5.6.7.8,3.5.7.9 local udp,tcp 51819

Common commands

Manually add a temporarily (minutes, hours) rule to shorewall to allow a connection:

/etc/shorewall/rules:

# Allow SSH to my-server.example.com
SSH(ACCEPT) local pub:my-server.example.com

# Allow tcp 9876 to IP 4.5.6.7
ACCEPT local pub:4.5.6.7 tcp 9876

Then check the config files for errors:

shorewall check
shorewall6 check

And restart shorewall:

systemctl restart shorewall.service
systemctl restart shorewall6.service

To disable shorewall (ALLOW all connections)

shorewall clear
shorewall6 clear

If you use the Ansible role, shorewall will not restart if shorewall check fails.

Debugging

Logfiles

To view rejected packages on the central log server:

root@cus-util-prod-log-1 ~ # journalctl --file /var/log/journal/remote/all.journal -f _HOSTNAME=cus-util-prod-monitoring-1 | grep -i shorewall

Jul 03 12:42:13 cus-util-prod-monitoring-1 kernel: shorewall:local-pub:REJECT:IN= OUT=ens6 SRC=85.215.174.107 DST=8.8.8.8 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=54075 DF PROTO=TCP SPT=35210 DPT=333 WINDOW=64240 RES=0x00 SYN URGP=0 

To trigger a package that will be rejected:

root@cus-util-prod-monitoring-1 ~ # telnet 8.8.8.8 333
Trying 8.8.8.8...
telnet: Unable to connect to remote host: Connection refused

First install and timeout

When you run baseline_shorewall for the first time on a new server, the restart shorewall task will most likely not complete and timeout - feel free to CRTL + C after you see:

RUNNING HANDLER [blunix_role-shorewall_11.0.0 : show check shorewall syntax output] ***
[...]
    Shorewall configuration verified
[timeout]

As shorewall kills the open SSH connection during its first start this is expected.