Well, under some circumstances, I'd expect a relative flury, or "bursts" of notify activity. E.g. I do TLS(/"SSL") certs via LetsEncrypt.org (LE), using DNS verification. Some/many of those certs will have multiple Subject Alternative Name (SAN) domains, and there may be fair number of such certs. So, adding the relevant records for verification, and then removing them after having obtained the issued cert(s) - that can be a fair burst of activity. But those generally wouldn't persist over a long period of time. E.g. every 90 seconds over many hours or days or more ... then something else is going on. So ... let me dig (pun not quite intended but apt) a bit more and peek what's currently goin' on there ...

https://www.mpaoli.net/~michael/bin/DNS_CK $ DNS_CK mpaoli.net FQDN: mpaoli.net.: authority: mpaoli.net. 172800 IN NS ns0.balug.org. mpaoli.net. 172800 IN NS ns0.mpaoli.net. mpaoli.net. 172800 IN NS ns1.linuxmafia.com. mpaoli.net. 172800 IN NS nsx.sunnyside.com. mpaoli.net. 172800 IN NS nsy.sunnysidex.com. mpaoli.net. 172800 IN SOA ns0.mpaoli.net. Michael.Paoli.cal.berkeley.edu. 1717399740 10800 3600 3600000 86400 @96.86.170.229 (ns0.balug.org.) mpaoli.net. 172800 IN SOA ns0.mpaoli.net. Michael.Paoli.cal.berkeley.edu. 1717399740 10800 3600 3600000 86400 @2001:470:1f05:19e::2 (ns0.balug.org.) mpaoli.net. 172800 IN SOA ns0.mpaoli.net. Michael.Paoli.cal.berkeley.edu. 1717399740 10800 3600 3600000 86400 @96.86.170.226 (ns0.mpaoli.net.) mpaoli.net. 172800 IN SOA ns0.mpaoli.net. Michael.Paoli.cal.berkeley.edu. 1717399740 10800 3600 3600000 86400 @2001:470:67:76f::2 (ns0.mpaoli.net.) mpaoli.net. 172800 IN SOA ns0.mpaoli.net. Michael.Paoli.cal.berkeley.edu. 1717399740 10800 3600 3600000 86400 @96.95.217.99 (ns1.linuxmafia.com.) mpaoli.net. 172800 IN SOA ns0.mpaoli.net. Michael.Paoli.cal.berkeley.edu. 1717399740 10800 3600 3600000 86400 @50.242.105.52 (nsx.sunnyside.com.) ;; communications error to 2603:3024:180d:f100:50:242:105:34#53: timed out ;; communications error to 2603:3024:180d:f100:50:242:105:34#53: timed out ;; communications error to 2603:3024:180d:f100:50:242:105:34#53: timed out ;; no servers could be reached @2603:3024:180d:f100:50:242:105:34 (nsx.sunnyside.com.) mpaoli.net. 172800 IN SOA ns0.mpaoli.net. Michael.Paoli.cal.berkeley.edu. 1717399740 10800 3600 3600000 86400 @50.18.139.240 (nsy.sunnysidex.com.) mpaoli.net. 172800 IN SOA ns0.mpaoli.net. Michael.Paoli.cal.berkeley.edu. 1717399740 10800 3600 3600000 86400 @2600:1f1c:528:c500:5e0b:8a37:6598:356c (nsy.sunnysidex.com.) $

Okay, so there's also (probably) unrelated issue with: 2603:3024:180d:f100:50:242:105:34#53 nsx.sunnyside.com.

96.95.217.99 (ns1.linuxmafia.com.) shows current SERIAL, so seems it's been updated relatively lately ... using Dynamic DNS (DDNS) with UNIX epoch based time for SOA SERIAL so ... that was likely updated roughly around ... $ date -Iseconds -d @1717399740 2024-06-03T00:29:00-07:00 $ Since I've also got automatic zone signing with DNSSEC, the SOA SERIAL - between unsigned (internal) and signed (what's actually served up) may and typically will differ a bit, and the thus the timings may not so precisely align to UNIX epoch (logs show, in many cases, both the signed and unsigned SERIAL - most notably when freshly signing or the like). Anyway, that's not all that long ago - don't think I manually changed something then, but may have been automatic or scheduled (e.g. at(1)), e.g. clearing out some temporary goop I earlier added, or may have been automatic occasional rolling of DNSSEC related data or the like. Let's see, out of curiosity, peeking at my logs ... 2024-06-03T07:28:55.951311+00:00 tigger named[385668]: zone mpaoli.net/IN (signed): reconfiguring zone keys 2024-06-03T07:28:55.953811+00:00 tigger named[385668]: zone mpaoli.net/IN (signed): next key event: 03-Jun-2024 08:28:55.946 2024-06-03T07:29:00.976632+00:00 tigger named[385668]: zone mpaoli.net/IN (signed): sending notifies (serial 1717399740) Okay, ... that much, at least in-and-of itself would seem to make sense ... But checking further, I'm seeing way too many notifies being sent, And without good clear reason as to why. { gzip -d < syslog.7.gz gzip -d < syslog.6.gz gzip -d < syslog.4.gz gzip -d < syslog.3.gz gzip -d < syslog.2.gz cat < syslog.1 cat < syslog } | grep -a ' tigger named[[0-9]*]: zone mpaoli.net/IN (signed): sending notifies (serial 1717399740)' | sed -ne '1p;$p;$=' 2024-06-03T07:29:00.976632+00:00 tigger named[385668]: zone mpaoli.net/IN (signed): sending notifies (serial 1717399740) 2024-06-04T02:06:33.456563+00:00 tigger named[423849]: zone mpaoli.net/IN (signed): sending notifies (serial 1717399740) 740 So ... 740 notifies sent within that timespan shown above. That's far too many for that zone in that range of time, notably same SOA SERIAL, so the data itself hasn't even changed. Anyway, that's issue on my end to find and fix.

So, I wouldn't expect ongoing NOTIFY every 90s ... but well, do have far too many going, on, for reason(s) to be determined (and for me to fix). I did also do major OS (Debian: buster 10 --> bullseye 11 --> bookworm 12) upgrade on one of the DNS servers (that very one) not too long ago (bit over two weeks ago: $ TZ=GMT0 stat -c '%z %Z %n' /etc/debian_version 2024-05-14 07:37:15.000000000 +0000 1715672235 /etc/debian_version $ ) ... so, maybe that caused something? Who knows. Anyway, I'll look into and fix that.

And checking any possibly relevant SOA serial mismatches on ns1.linuxmafia.com for "my" domains ... $ DNS_CK ... no SOA SERIAL discrepancies found for ns1.linuxmafia.com - all lined up as expected and current.

Jun 3 14:48:39 linuxmafia named[15622]: client 73.189.65.18#16833: received notify for zone 'mpaoli.net'

But yeah, that wouldn't be from me. 96.86.170.226 is the only IP I definitely should be sending NOTIFY from for mpaoli.net, though there might possibly be some additional (e.g. other authoritatives, IPv6, ...), but certainly not 73.189.65.18 - not my IP at all. Sounds like some drain damaged Comcast Business SecurityEdge, or other, DNS fckery - at least that'd be my first guess (and especially given their track record).

So ... let me check for bit ... everything I've got going out to 96.95.217.99:53 UDP and everything 96.95.217.99:53 UDP has coming in that looks like DNS NOTIFY ... So ... on my end of things, and point where I can snag any relevant traffic, I do see apparently (too) periodic NOTIFY: # tcpdump -i eno1 -n -s 0 -v udp and host 96.95.217.99 and port 53 tcpdump: listening on eno1, link-type EN10MB (Ethernet), snapshot length 262144 bytes 02:30:41.945105 IP (tos 0x0, ttl 64, id 64173, offset 0, flags [none], proto UDP (17), length 126) 96.86.170.226.51682 > 96.95.217.99.53: 62396 notify [b2&3=0x2400] [1a] SOA? mpaoli.net. (98) 02:30:41.975644 IP (tos 0x0, ttl 54, id 56888, offset 0, flags [none], proto UDP (17), length 56) 96.95.217.99.53 > 96.86.170.226.51682: 62396 notify*- 0/0/0 (28) 02:32:12.436952 IP (tos 0x0, ttl 64, id 1569, offset 0, flags [none], proto UDP (17), length 126) 96.86.170.226.23199 > 96.95.217.99.53: 48406 notify [b2&3=0x2400] [1a] SOA? mpaoli.net. (98) 02:32:12.468027 IP (tos 0x0, ttl 54, id 56894, offset 0, flags [none], proto UDP (17), length 56) 96.95.217.99.53 > 96.86.170.226.23199: 48406 notify*- 0/0/0 (28) ^C 4 packets captured 4 packets received by filter 0 packets dropped by kernel # Yeah, definitely too frequently - something for me to fix on my end.

And, let's see what we can see of that at/around the ns1.linuxmafia.com. end: This was done on guido, on physical interface connecting to hardwired network, There is (or should be) no NAT/SNAT involved on these IPs, and in promiscuous mode there (default), should see ns1.linuxmafia.com's traffic fine (and bit easier and more capable to see it on guido with the much more recent OS and software there): # tcpdump -i enp6s0 -n -s 0 -v udp and host 96.95.217.99 and port 53 | grep --line-buffered -B 1 -F -i notify tcpdump: listening on enp6s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 02:36:44.479125 IP (tos 0x0, ttl 54, id 25954, offset 0, flags [none], proto UDP (17), length 126) 96.86.170.226.45546 > 96.95.217.99.53: 34616 notify [b2&3=0x2400] [1a] SOA? mpaoli.net. (98) 02:36:44.480772 IP (tos 0x0, ttl 64, id 56916, offset 0, flags [none], proto UDP (17), length 56) 96.95.217.99.53 > 96.86.170.226.45546: 34616 notify*- 0/0/0 (28) -- 02:38:14.967803 IP (tos 0x0, ttl 54, id 36016, offset 0, flags [none], proto UDP (17), length 126) 96.86.170.226.63880 > 96.95.217.99.53: 31947 notify [b2&3=0x2400] [1a] SOA? mpaoli.net. (98) 02:38:14.969472 IP (tos 0x0, ttl 64, id 56922, offset 0, flags [none], proto UDP (17), length 56) 96.95.217.99.53 > 96.86.170.226.63880: 31947 notify*- 0/0/0 (28) ^C534 packets captured 537 packets received by filter 0 packets dropped by kernel

# So ... not seeing surprises with that traffic (other than notifies too frequently sent on my side).

As for 73.189.65.18 ... well, let's see ... # tcpdump -i enp6s0 -n -s 0 -v ( port 53 or port 5353 ) and host 73.189.65.18 tcpdump: listening on enp6s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel # Waited fair while and ... really not seeing anything there currently. Trying bit more generally ... # tcpdump -i enp6s0 -n -s 0 -v port 5353 ... yeah, that captured too much (irrelevant IPs and various false positives) ... # tcpdump -i enp6s0 -n -s 0 -v port 5353 and net 96.80.0.0/12 meanwhile on [ns1.]linuxmafia.com.: $ dig -p 5353 +nomultiline +notcp +ignore @96.86.170.229 +noall +answer sflug.com. SOA ;; connection timed out; no servers could be reached $ And our capture then has: # tcpdump -i enp6s0 -n -s 0 -v port 5353 and net 96.80.0.0/12 tcpdump: listening on enp6s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 03:09:16.695708 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 55) 96.95.217.99.42136 > 96.86.170.229.5353: 62654+ SOA (QM)? sflug.com. (27) 03:09:21.696282 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 55) 96.95.217.99.42136 > 96.86.170.229.5353: 62654+ SOA (QM)? sflug.com. (27) 03:09:26.696984 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 55) 96.95.217.99.42136 > 96.86.170.229.5353: 62654+ SOA (QM)? sflug.com. (27) So ... client sends, server receives and responds, client never receives response. Other Internet clients don't have this issue. And if same client does that query using port 53 or protocol TCP, then issue isn't seen.

As for 73.189.65.18, that may take more digging/persistence/investigation to find/know what's up with that.

On Mon, Jun 3, 2024 at 3:29 PM Rick Moen rick@linuxmafia.com wrote:

OK, those fsckers are _definitely_ screwing with my DNS, again.

See that "73.189.65.18" IP? Well...

$ dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net. $

I'm betting this also somehow accounts for my mameserver mysteriously getting NOTIFYs for domain mpaoli.net every 90 seconds.

----- Forwarded message from logcheck system account logcheck@linuxmafia.com -----

Date: Mon, 03 Jun 2024 15:02:02 -0700 From: logcheck system account logcheck@linuxmafia.com To: root@linuxmafia.com Subject: linuxmafia.com 2024-06-03 15:02 System Events

System Events

Jun 3 14:48:39 linuxmafia named[15622]: client 73.189.65.18#16833: received notify for zone 'mpaoli.net' Jun 3 14:48:39 linuxmafia named[15622]: zone mpaoli.net/IN: refused notify from non-master: 73.189.65.18#16833 Jun 3 14:48:44 linuxmafia named[15622]: client 73.189.65.18#16833: received notify for zone 'mpaoli.net' Jun 3 14:48:44 linuxmafia named[15622]: zone mpaoli.net/IN: refused notify from non-master: 73.189.65.18#16833

----- End forwarded message -----