PowerDNS: use dnsdist to send NOTIFY packets to hidden primary

Overview

This shows how to use dnsdist, which is a dns proxy from the good folks at PowerDNS

The use case here is a DNS layout which has 3 servers

• ns0 - hidden primary (formerly known as master).
• ns1 / ns2 - nameservers in use, listed on whois records.

This is using the MySQL backend with MariaDB replication, so ns0 is the only one that can write to the database. ns1/ns2 intentionally have read only MySQL passwords.

The Issue

This all would work fine if all zones you host are NATIVE powerdns format, in that you use MySQL replication to get data to sync between servers. The issue arises if you are SECONDARY (formerly known as slave) to zones, where the PRIMARY is out of your control.

Primary servers will follow RFC1996 and send NOTIFY packets to ns1/ns2 as they're the NS records on the domain. Those servers are ready only, so what can they do with said packets?

The Fix

dnsdist can be setup EDNS Client Subject (ECS) extensions to forward the packets to the hidden primary. Without ECS, the packets are not spoofed so stats on the target server will just show a lot of source packets coming from your nameservers, NOT the original source.

Thanks to everyone in the PowerDNS IRC channel to help figure this all out. (Yes IRC still exists)

Our Layout is

• ns0 - powerdns running natively on port 53. The primary DNS server must allow AXFR from this IP as we're not doing anything special to reroute this request back via ns1/ns2.
• ns1 / ns2 - powerdns running on port 5353, i've left this on the public ip but firewalled off so we can query it directly, remotely. dnsdist is running on 53.

PowerDNS Config

PowerDNS is running on a nonstandard port on the public nameservers ns1/ns2. primary=no and secondary=no are set as ns1/ns2 should not process NOTIFY requests as they have read only SQL access to their databases.

• Relevant contents of /etc/pdns/pdns.conf: on ns1/ns2

local-port=5353
edns-subnet-processing=yes
trusted-notification-proxy=10.20.108.53, 2001:db80:108:1::53, 10.30.163.53, 2001:db80:2:163::53
primary=no
secondary=no 

dnsdist Config

This dnsdist config is slightly longer than it may need to be, but I wanted to keep ipv4 and ipv6 traffic separate. By default, pools can send ipv6 clients to an ipv4 pool and vice versa. I wanted to have better tabs on what real level of ipv4/ipv6 traffic was coming in.

I find the web interface to be incredibly useful to troubleshoot real time as well, so left that part in as well.

• Entire contents of /etc/dnsdist/dnsdist.conf. This may be harder to read as we're using Lua variables which makes it easier to copy between servers. (variable string concatenation in lua is the .. ':' .. stuff)

------------------------------------------
-- BEGIN CONFIG OPTIONS
my_v4 = "10.20.163.53"
my_v6 = "[2001:0db8:163::53]"
my_port_dnsdist = "53"
my_port_powerdns = "5353"
my_port_web = "8083"
my_pass_web_user = "<redacted>"
my_pass_web_api = "<redacted>"
my_web_sources = "10.10.47.0/24, !192.0.2.1"
-- where is the real ns0 server
ns0_v4 = "10.20.18.53"
ns0_v6 = "[2001:0db8:18::53]"
-- END CONFIG OPTIONS
------------------------------------------
 
 
------------------------------------------
-- STATIC SETTINGS, KEEP SAME ON NS1/NS2

-- which IPs may query us, all types
setACL({'0.0.0.0/0', '::/0'})
 
setLocal(my_v4 .. ':' .. my_port_dnsdist)
addLocal(my_v6 .. ':' .. my_port_dnsdist)
 
-- Webserver has useful stats, comment to not use it
setWebserverConfig({password=my_pass_web_user, apiKey=my_pass_web_api, acl=my_web_sources, statsRequireAuthentication=TRUE})
webserver(my_v4 .. ':' .. my_port_web)
 
-- load balance policy to use. Doesn't do much if only a single server in pool, if commented it uses "leastOutstanding"
-- setServerPolicy(firstAvailable)
 
-- local powerdns service
newServer({address=my_v4 .. ':' .. my_port_powerdns, source=my_v4, name='self_v4', pool='self_v4', order=1})
newServer({address=my_v6 .. ':' .. my_port_powerdns, source=my_v6, name='self_v6', pool='self_v6', order=1})
 
-- create "ns0" server pool, which supports ECS. last 3 flags below enable ECS from dnsdist
newServer({address=ns0_v4 .. ':53', source=my_v4, name="ns0_v4", pool="ns0_v4", useClientSubnet=true, setECSSourcePrefixV4(32), setECSSourcePrefixV6(128) })
newServer({address=ns0_v6 .. ':53', source=my_v6, name="ns0_v6", pool="ns0_v6", useClientSubnet=true, setECSSourcePrefixV4(32), setECSSourcePrefixV6(128) })
 
 
------------------------------------------
-- BEGIN NOTIFY REQUESTS
 
-- rule to send ALL NOTIFY packets. Here as a backup since we're using the next rule
addAction(AndRule({
    makeRule('0.0.0.0/0'),
    OpcodeRule(DNSOpcode.Notify),
    }),
    PoolAction('ns0_v4')
)
addAction(AndRule({
    makeRule('::/0'),
    OpcodeRule(DNSOpcode.Notify),
    }),
    PoolAction('ns0_v6')
)
 
-- END NOTIFY REQUESTS
------------------------------------------
 
------------------------------------------
-- NORMAL TRAFFIC leave at end of file
 
addAction(makeRule('0.0.0.0/0'), PoolAction('self_v4'))
addAction(makeRule('::/0'),      PoolAction('self_v6'))
 
-- these should never get hit, but just in case
addAction(AllRule(), PoolAction('self_v4'))
addAction(AllRule(), PoolAction('self_v6'))