----- Forwarded message from Al awbalug@sunnyside.com -----
Date: Tue, 4 Jun 2024 16:29:04 -0700 From: Al awbalug@sunnyside.com To: Rick Moen rick@linuxmafia.com Subject: Re: [BALUG-Admin] Comcast Business apparently blocking 5353 UDP Re: linuxmafia.com "retry limit exceeded"
Rick, you're at the right place - that gear icon and right side panel on business.comcast.com is just the right thing. And I think the situation as you're outlining it is right to me. So the answer to your question, broadly, is yes I think you have it right. If you end up at securityedge.comcast.com, IMHO you've gone too far. My sense is that all that stuff is disabled back at the right side panel... Once SE (security edge) is disabled I think everything is. That said, you're being smart about it - if symptoms persist, drill down and look into individual settings for various elements of SE and just make sure they're all off - in case Comcast can't quite sort out how to actually disable stuff. AFAIK however your nets (yours and Michaels) are unrestricted. My tests from here are that access to both 96.86.170.229 and 96.95.217.99 on port 53 is not blocked (and not just those /32s but the entire subnet in each case). I am looking back over email from the last few days trying to sort out where 73.189.65.18 crept into the conversation. As I mentioned I have been unable to focus sufficiently on this the last few days, and missed where that came from. I also haven't looked closely enough at the discussion to see if what I am trying to reproduce isn't exactly where you're having trouble. I'll go back over the notes and see if I can pay more attention to the details and whether I can actually add any insight to the discussion. Al
----- End forwarded message -----
To clarify, I noticed "73.189.65.18" as the source of NOTIFYs for Michael's domains, which can legitimately come _only_ from Michael's authoritative nameserver, IP 96.86.170.229.
And 73.189.65.18 is Comcast's _own_ IP, not Michael's.
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
So, something is rotten, there. I'm immediately inclined to suspect that Comcast is playing man-in-the-middle games with DNS traffic. Which, if true, suggest Comcast acting like a rogue state security agency or one operating on behalf of a totalitarian state. Not a good look.
ooooooohhhh, that's weird and yucky. something rotten in CC BUS land. I wonder if Michael looks at his modem internal page whether that 73.x.xxx addr is his modem's upstream WAN address, or maybe the upstream router's IP? IIRC Michael already look at the modem config page, so I guess that's already been checked. I know if I traceroute to Michael, I don't see 73.anything.
I just set up a better test and I can dig on 53 or 5353 just fine, EXCEPT when I do it from my Comcast Business IPv4 net. That times out. So I think that's the same problem you all are describing.
I think 5353 was picked just because it was handy, but it's also the multicast DNS port or something. I wonder if 5454 or some such would show up unscathed (see via tcpdump).
I wonder if MDNS is part of Comcast's problem. Or maybe the modem has a virus, since 5353 is widely used by exploits. For the partial but extensive list see Port 5353 (tcp/udp) :: SpeedGuide https://www.speedguide.net/port.php?port=5353
On 6/4/2024 17:14, Rick Moen wrote:
----- Forwarded message from Alawbalug@sunnyside.com -----
Date: Tue, 4 Jun 2024 16:29:04 -0700 From: Alawbalug@sunnyside.com To: Rick Moenrick@linuxmafia.com Subject: Re: [BALUG-Admin] Comcast Business apparently blocking 5353 UDP Re: linuxmafia.com "retry limit exceeded"
Rick, you're at the right place - that gear icon and right side panel on business.comcast.com is just the right thing. And I think the situation as you're outlining it is right to me. So the answer to your question, broadly, is yes I think you have it right. If you end up at securityedge.comcast.com, IMHO you've gone too far. My sense is that all that stuff is disabled back at the right side panel... Once SE (security edge) is disabled I think everything is. That said, you're being smart about it - if symptoms persist, drill down and look into individual settings for various elements of SE and just make sure they're all off
- in case Comcast can't quite sort out how to actually disable stuff.
AFAIK however your nets (yours and Michaels) are unrestricted. My tests from here are that access to both 96.86.170.229 and 96.95.217.99 on port 53 is not blocked (and not just those /32s but the entire subnet in each case). I am looking back over email from the last few days trying to sort out where 73.189.65.18 crept into the conversation. As I mentioned I have been unable to focus sufficiently on this the last few days, and missed where that came from. I also haven't looked closely enough at the discussion to see if what I am trying to reproduce isn't exactly where you're having trouble. I'll go back over the notes and see if I can pay more attention to the details and whether I can actually add any insight to the discussion. Al
----- End forwarded message -----
To clarify, I noticed "73.189.65.18" as the source of NOTIFYs for Michael's domains, which can legitimately come _only_ from Michael's authoritative nameserver, IP 96.86.170.229.
And 73.189.65.18 is Comcast's _own_ IP, not Michael's.
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
So, something is rotten, there. I'm immediately inclined to suspect that Comcast is playing man-in-the-middle games with DNS traffic. Which, if true, suggest Comcast acting like a rogue state security agency or one operating on behalf of a totalitarian state. Not a good look.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
Al, yep, I think you more-or-less found same. And ... yeah, "interesting" ... and, alas, not surprising, you find same issues from your Comcast network, but not your other network(s) (e.g. AT&T if I'm not mistaken). Likewise I saw no issue when I tried the other day from AWS. And 5353 ... shouldn't matter. In general ISP shouldn't be blocking ports without customer's knowledge and consent. They are after all an Internet Service Provider, so should be providing service to The Internet, not disservice of semi-randomly blocking or interfering with such communication flows ... yet with Comcast [Business] we have/find ... ugh, yeah, that. Anyway, some months ago, 5353 worked perfectly fine. That was originally put in place as work-around for Comcast Business's screwing around with 53 notably breaking 53 with their SecurityEdge goop. Once that had been addressed and was working again, took out 5353 ... until Comcast again broke 53, then put the 5353 back - this time intending it to be for longer term. Last I looked and Comcast's track record, I'm inclined to leave it there at least for a continuous year's time of Comcast not screwing it (at least 53) up ... hasn't been a year's time yet, so I've not decommissioned the 5353 workaround. And no, I'm not willing to reconfigure to a different alternative work-around port every time Comcast changes and breaks the current workaround port. Not going there. This sh*t should work, and Comcast ought not be fscking around with it and continuing to, and unpredictably at that, break things. So, at current they've broken 5353 ... dear knows what else they've broken that we don't even know about because we haven't explicitly tested or otherwise noticed. But ISPs shouldn't be breaking things like that. But, alas, ... Comcast ... and Comcast Business, at that.
Anyway, that shouldn't be broken. So, if you (Al) also want to open support case with Comcast, please feel free, and can also cross-reference that Comcast ticket # which I earlier noted and also passed along to Rick. If you don't have the direct # for the tech on that ticket #, let me know, and I can (off-list) pass that to you too (but I think that may also be on-list with at least one of these list postings?). Anyway, at persent, I've got that service (also) on 5353, and current plans are to continue that until at least ... $ grep -F -e 5353 ~/calendar 2024-09-24 >= 10:56AM US/Pacific PST8PDT -07:00, if Comcast Business hasn't screwed up DNS with linuxmafia.com for more than a year, probably time to remove the port 5353 work-arounds. $ And since that's not a super high availability service (e.g. no redundancy on ISP or AC power, etc.), often prudent to be sure it's up first, e.g. check on TCP or port 53 - as I did in that earlier test script, it only proceeds to 5353 UDP if the preceding tests with 5353 TCP and UDP 53 are successful. And, shown again here for convenience, if one doesn't have it handy: time dig -p 5353 +tcp +nomultiline @96.86.170.229 +noall +answer sflug.com. SOA && time dig -p 53 +notcp +ignore +nomultiline @96.86.170.229 +noall +answer sflug.com. SOA && time dig -p 5353 +notcp +ignore +nomultiline @96.86.170.229 +noall +answer sflug.com. SOA
On Tue, Jun 4, 2024 at 6:11 PM Al aw009@sunnyside.com wrote:
ooooooohhhh, that's weird and yucky. something rotten in CC BUS land. I wonder if Michael looks at his modem internal page whether that 73.x.xxx addr is his modem's upstream WAN address, or maybe the upstream router's IP? IIRC Michael already look at the modem config page, so I guess that's already been checked. I know if I traceroute to Michael, I don't see 73.anything.
I just set up a better test and I can dig on 53 or 5353 just fine, EXCEPT when I do it from my Comcast Business IPv4 net. That times out. So I think that's the same problem you all are describing.
I think 5353 was picked just because it was handy, but it's also the multicast DNS port or something. I wonder if 5454 or some such would show up unscathed (see via tcpdump).
I wonder if MDNS is part of Comcast's problem. Or maybe the modem has a virus, since 5353 is widely used by exploits. For the partial but extensive list see Port 5353 (tcp/udp) :: SpeedGuide
On 6/4/2024 17:14, Rick Moen wrote:
----- Forwarded message from Al awbalug@sunnyside.com -----
Date: Tue, 4 Jun 2024 16:29:04 -0700 From: Al awbalug@sunnyside.com To: Rick Moen rick@linuxmafia.com Subject: Re: [BALUG-Admin] Comcast Business apparently blocking 5353 UDP Re: linuxmafia.com "retry limit exceeded"
Rick, you're at the right place - that gear icon and right side panel on business.comcast.com is just the right thing. And I think the situation as you're outlining it is right to me. So the answer to your question, broadly, is yes I think you have it right. If you end up at securityedge.comcast.com, IMHO you've gone too far. My sense is that all that stuff is disabled back at the right side panel... Once SE (security edge) is disabled I think everything is. That said, you're being smart about it - if symptoms persist, drill down and look into individual settings for various elements of SE and just make sure they're all off
- in case Comcast can't quite sort out how to actually disable stuff.
AFAIK however your nets (yours and Michaels) are unrestricted. My tests from here are that access to both 96.86.170.229 and 96.95.217.99 on port 53 is not blocked (and not just those /32s but the entire subnet in each case). I am looking back over email from the last few days trying to sort out where 73.189.65.18 crept into the conversation. As I mentioned I have been unable to focus sufficiently on this the last few days, and missed where that came from. I also haven't looked closely enough at the discussion to see if what I am trying to reproduce isn't exactly where you're having trouble. I'll go back over the notes and see if I can pay more attention to the details and whether I can actually add any insight to the discussion. Al
----- End forwarded message -----
To clarify, I noticed "73.189.65.18" as the source of NOTIFYs for Michael's domains, which can legitimately come _only_ from Michael's authoritative nameserver, IP 96.86.170.229.
And 73.189.65.18 is Comcast's _own_ IP, not Michael's.
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
So, something is rotten, there. I'm immediately inclined to suspect that Comcast is playing man-in-the-middle games with DNS traffic. Which, if true, suggest Comcast acting like a rogue state security agency or one operating on behalf of a totalitarian state. Not a good look.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
I think "c-73-189-65-18.hsd1.ca.comcast.net" is their naming system for end modems, not internal infrastructure, but don't quote me. When I look up the names of routers in the traceroute I don't think I see that type of name, but it may be that's an irresponsible poorly verified comment on my part.
For example from traceroute to Michael: 186.78.151.162.in-addr.arpa domain name pointer po-1-rur101.pinole.ca.sfba.comcast.net.
ooooooooooooooh, omigosh - I just perfected a traceroute to Michael, entirely within the CC network:
traceroute to 96.86.170.229 (96.86.170.229), 30 hops max, 60 byte packets 1 50.242.105.62 2.080 ms 2.743 ms 3.439 ms 2 10.61.209.66 14.028 ms 10.61.209.67 13.729 ms 10.61.209.66 14.373 ms 3 96.216.9.141 12.757 ms 96.216.9.137 13.045 ms 96.216.9.141 13.401 ms 4 68.85.154.113 21.023 ms 68.85.154.117 21.093 ms 68.85.154.113 20.944 ms 5 96.108.99.249 26.011 ms 25.722 ms 26.272 ms 6 68.86.143.89 23.688 ms 68.86.143.93 21.156 ms 20.778 ms 7 162.151.86.58 22.077 ms 20.330 ms 162.151.87.226 20.215 ms 8 162.151.79.134 21.787 ms 162.151.78.186 17.686 ms 162.151.79.134 21.466 ms 9 68.85.103.154 19.388 ms 68.85.191.206 12.456 ms 68.85.103.154 14.255 ms 10 73.189.65.18 21.185 ms 36.832 ms 31.356 ms 11 96.86.170.229 36.366 ms 34.933 ms 34.918 ms root@routr0:/z/r/srv#
It appears that 73.thing is in fact Michael's upstream router.
Ok, did everyone else already know that? Am I late to the party?
On 6/4/2024 17:14, Rick Moen wrote:
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
I think "c-73-189-65-18.hsd1.ca.comcast.net" is their naming system for end modems, not internal infrastructure, but don't quote me. When I look up the names of routers in the traceroute I don't think I see that type of name, but it may be that's an irresponsible poorly verified comment on my part.
For example from traceroute to Michael: 186.78.151.162.in-addr.arpa domain name pointer po-1-rur101.pinole.ca.sfba.comcast.net.
ooooooooooooooh, omigosh - I just perfected a traceroute to Michael, entirely within the CC network:
traceroute to 96.86.170.229 (96.86.170.229), 30 hops max, 60 byte packets 1 50.242.105.62 2.080 ms 2.743 ms 3.439 ms 2 10.61.209.66 14.028 ms 10.61.209.67 13.729 ms 10.61.209.66 14.373 ms 3 96.216.9.141 12.757 ms 96.216.9.137 13.045 ms 96.216.9.141 13.401 ms 4 68.85.154.113 21.023 ms 68.85.154.117 21.093 ms 68.85.154.113 20.944 ms 5 96.108.99.249 26.011 ms 25.722 ms 26.272 ms 6 68.86.143.89 23.688 ms 68.86.143.93 21.156 ms 20.778 ms 7 162.151.86.58 22.077 ms 20.330 ms 162.151.87.226 20.215 ms 8 162.151.79.134 21.787 ms 162.151.78.186 17.686 ms 162.151.79.134 21.466 ms 9 68.85.103.154 19.388 ms 68.85.191.206 12.456 ms 68.85.103.154 14.255 ms 10 73.189.65.18 21.185 ms 36.832 ms 31.356 ms 11 96.86.170.229 36.366 ms 34.933 ms 34.918 ms root@routr0:/z/r/srv#
It appears that 73.thing is in fact Michael's upstream router.
Ok, did everyone else already know that? Am I late to the party?
On 6/4/2024 17:14, Rick Moen wrote:
----- Forwarded message from Al awbalug@sunnyside.com -----
Date: Tue, 4 Jun 2024 16:29:04 -0700 From: Al awbalug@sunnyside.com To: Rick Moen rick@linuxmafia.com Subject: Re: [BALUG-Admin] Comcast Business apparently blocking 5353 UDP Re: linuxmafia.com "retry limit exceeded"
Rick, you're at the right place - that gear icon and right side panel on business.comcast.com is just the right thing. And I think the situation as you're outlining it is right to me. So the answer to your question, broadly, is yes I think you have it right. If you end up at securityedge.comcast.com, IMHO you've gone too far. My sense is that all that stuff is disabled back at the right side panel... Once SE (security edge) is disabled I think everything is. That said, you're being smart about it - if symptoms persist, drill down and look into individual settings for various elements of SE and just make sure they're all off
- in case Comcast can't quite sort out how to actually disable stuff.
AFAIK however your nets (yours and Michaels) are unrestricted. My tests from here are that access to both 96.86.170.229 and 96.95.217.99 on port 53 is not blocked (and not just those /32s but the entire subnet in each case). I am looking back over email from the last few days trying to sort out where 73.189.65.18 crept into the conversation. As I mentioned I have been unable to focus sufficiently on this the last few days, and missed where that came from. I also haven't looked closely enough at the discussion to see if what I am trying to reproduce isn't exactly where you're having trouble. I'll go back over the notes and see if I can pay more attention to the details and whether I can actually add any insight to the discussion. Al
----- End forwarded message -----
To clarify, I noticed "73.189.65.18" as the source of NOTIFYs for Michael's domains, which can legitimately come _only_ from Michael's authoritative nameserver, IP 96.86.170.229.
And 73.189.65.18 is Comcast's _own_ IP, not Michael's.
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
So, something is rotten, there. I'm immediately inclined to suspect that Comcast is playing man-in-the-middle games with DNS traffic. Which, if true, suggest Comcast acting like a rogue state security agency or one operating on behalf of a totalitarian state. Not a good look.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
Actually I take it back, 73.189.65.18 must be the WAN address of Michael's modem. That won't appear on a message from Michael, unless somehow NAT got involved in the modem? Not sure if that's quite right. I think NAT would come from the last of the assigned static IPv4 addresses, but IIRC I have also seen messages from a modem's WAN address.
Most interesting.
On 6/4/2024 17:43, Al wrote:
I think "c-73-189-65-18.hsd1.ca.comcast.net" is their naming system for end modems, not internal infrastructure, but don't quote me. When I look up the names of routers in the traceroute I don't think I see that type of name, but it may be that's an irresponsible poorly verified comment on my part.
For example from traceroute to Michael: 186.78.151.162.in-addr.arpa domain name pointer po-1-rur101.pinole.ca.sfba.comcast.net.
ooooooooooooooh, omigosh - I just perfected a traceroute to Michael, entirely within the CC network:
traceroute to 96.86.170.229 (96.86.170.229), 30 hops max, 60 byte packets 1 50.242.105.62 2.080 ms 2.743 ms 3.439 ms 2 10.61.209.66 14.028 ms 10.61.209.67 13.729 ms 10.61.209.66 14.373 ms 3 96.216.9.141 12.757 ms 96.216.9.137 13.045 ms 96.216.9.141 13.401 ms 4 68.85.154.113 21.023 ms 68.85.154.117 21.093 ms 68.85.154.113 20.944 ms 5 96.108.99.249 26.011 ms 25.722 ms 26.272 ms 6 68.86.143.89 23.688 ms 68.86.143.93 21.156 ms 20.778 ms 7 162.151.86.58 22.077 ms 20.330 ms 162.151.87.226 20.215 ms 8 162.151.79.134 21.787 ms 162.151.78.186 17.686 ms 162.151.79.134 21.466 ms 9 68.85.103.154 19.388 ms 68.85.191.206 12.456 ms 68.85.103.154 14.255 ms 10 73.189.65.18 21.185 ms 36.832 ms 31.356 ms 11 96.86.170.229 36.366 ms 34.933 ms 34.918 ms root@routr0:/z/r/srv#
It appears that 73.thing is in fact Michael's upstream router.
Ok, did everyone else already know that? Am I late to the party?
On 6/4/2024 17:14, Rick Moen wrote:
----- Forwarded message from Al awbalug@sunnyside.com -----
Date: Tue, 4 Jun 2024 16:29:04 -0700 From: Al awbalug@sunnyside.com To: Rick Moen rick@linuxmafia.com Subject: Re: [BALUG-Admin] Comcast Business apparently blocking 5353 UDP Re: linuxmafia.com "retry limit exceeded"
Rick, you're at the right place - that gear icon and right side panel on business.comcast.com is just the right thing. And I think the situation as you're outlining it is right to me. So the answer to your question, broadly, is yes I think you have it right. If you end up at securityedge.comcast.com, IMHO you've gone too far. My sense is that all that stuff is disabled back at the right side panel... Once SE (security edge) is disabled I think everything is. That said, you're being smart about it - if symptoms persist, drill down and look into individual settings for various elements of SE and just make sure they're all off
- in case Comcast can't quite sort out how to actually disable stuff.
AFAIK however your nets (yours and Michaels) are unrestricted. My tests from here are that access to both 96.86.170.229 and 96.95.217.99 on port 53 is not blocked (and not just those /32s but the entire subnet in each case). I am looking back over email from the last few days trying to sort out where 73.189.65.18 crept into the conversation. As I mentioned I have been unable to focus sufficiently on this the last few days, and missed where that came from. I also haven't looked closely enough at the discussion to see if what I am trying to reproduce isn't exactly where you're having trouble. I'll go back over the notes and see if I can pay more attention to the details and whether I can actually add any insight to the discussion. Al
----- End forwarded message -----
To clarify, I noticed "73.189.65.18" as the source of NOTIFYs for Michael's domains, which can legitimately come _only_ from Michael's authoritative nameserver, IP 96.86.170.229.
And 73.189.65.18 is Comcast's _own_ IP, not Michael's.
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
So, something is rotten, there. I'm immediately inclined to suspect that Comcast is playing man-in-the-middle games with DNS traffic. Which, if true, suggest Comcast acting like a rogue state security agency or one operating on behalf of a totalitarian state. Not a good look.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
Quoting Al (awbalug@sunnyside.com):
Actually I take it back, 73.189.65.18 must be the WAN address of Michael's modem. That won't appear on a message from Michael, unless somehow NAT got involved in the modem? Not sure if that's quite right. I think NAT would come from the last of the assigned static IPv4 addresses, but IIRC I have also seen messages from a modem's WAN address.
Interest, if so -- but nonetheless completely improper. NOTIFY should only ever come from the domain master nameserver's IP.
Well, of course it's all improper. I try to figure out what flavor of improper because Comcast seems to need help thinking. I admit I have had no success in telling them anything, however. I just wonder if they are doing something with MDNS/Zero-config that might have been part of a new s/w release, and they got it all wrong. Or maybe they're changing policies and products, and somehow that leaked in to a legacy config and was of course broken. Enquiring (ala Natl Enquirer) minds wanna know.
On 6/4/2024 18:14, Rick Moen wrote:
Quoting Al (awbalug@sunnyside.com):
Actually I take it back, 73.189.65.18 must be the WAN address of Michael's modem. That won't appear on a message from Michael, unless somehow NAT got involved in the modem? Not sure if that's quite right. I think NAT would come from the last of the assigned static IPv4 addresses, but IIRC I have also seen messages from a modem's WAN address.
Interest, if so -- but nonetheless completely improper. NOTIFY should only ever come from the domain master nameserver's IP.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
Uhm, ... almost exactly correct. ;-)
One would think that ... but that's not quite how many/most DNS servers and their software behave and/or behave by default (and is often quite configurable too). Many will, by default, issue NOTIFY from all authoritative nameservers. At first that might seem odd, but, I believe the logic goes about like this: the overhead is low clients that don't care can/will (mostly) ignore authoritative (whether primary or secondary) has no way of knowing how other authoritatives downstream of it are configured, so, e.g. some authoritatives may only get their data via other secondary(/ies), and not direct from master, etc. So, DNS, etc., trying to be resilient and robust and fault tolerant and mostly continue working under even far from ideal circumstances, that's probably not a horrible design decision for default behaviors. Of course one can also make good arguments to have different or different default behavior ... and most nameservers (and including BIND9) are highly configurable as to how they'll actually behave in those regards.
But NOTIFY coming from other than actual authoritative nameservers (be they master/primary or a slave/secondary), something is quite odd or screwed up there. E.g. like some funky NAT/SNAT mapping (which shouldn't apply in our cases here, as we're talking Internet globally routable public IPs here, no NAT/SNAT or other mapping of the sort should at all apply between these), or, somebody <me giving Comcast Business the hairy eyeball look again> fscking around in ways with DNS traffic that they never ought be doing - and SecurityEdge would certainly be one of the guilty culprits found in such fscking up with port 53 traffic (at least we certainly hit lots of that earlier, and maybe we're hitting more of some same and/or similar, at least partially, in those regards).
On Tue, Jun 4, 2024 at 6:14 PM Rick Moen rick@linuxmafia.com wrote:
Quoting Al (awbalug@sunnyside.com):
Actually I take it back, 73.189.65.18 must be the WAN address of Michael's modem. That won't appear on a message from Michael, unless somehow NAT got involved in the modem? Not sure if that's quite right. I think NAT would come from the last of the assigned static IPv4 addresses, but IIRC I have also seen messages from a modem's WAN address.
Interest, if so -- but nonetheless completely improper. NOTIFY should only ever come from the domain master nameserver's IP.
Quoting Michael Paoli (michael.paoli@berkeley.edu):
Many will, by default, issue NOTIFY from all authoritative nameservers. At first that might seem odd, but, I believe the logic goes about like this: the overhead is low clients that don't care can/will (mostly) ignore authoritative (whether primary or secondary) has no way of knowing how other authoritatives downstream of it are configured, so, e.g. some authoritatives may only get their data via other secondary(/ies), and not direct from master, etc.
Yes, as I was saying, I had a faint recollection that the matter of Aaron T. Porter's ns.primate.net issuing NOTIFY for domains on which it's secondary, not primary, had come up in some of my earlier efforts to puzzle out strange nameserver behaviour. I just couldn't remember exactly how that had unfolded -- other than my obviously having decided to take no action.
Obviously it's at worst harmless, and I can/should just add another "ignore" line to the logcheck configuration so I stop being told about it. I just was taking a moment to try to figure out whether this is deliberate behaviour and why it's there. Your answer will serve splendidly. Although, I'm bothered that the usual information sources don't seem to cover this.
On the third hand, I didn't look _too_ closely, e.g,, maybe it's covered in the Zytrax's "DNS for Rocket Scientists" or the related dead-tree book _Pro DNS and BIND_.
Slightly weird, anyway.
Rick, Michael, Now that I've resolved the crazy static-IPv6-have-it-yet-don't snafu, fixed after 20 tickets and 3 weeks of pain and persistence, I'm ready to join your ticket(s) on 5353. First, of course, I want to be certain that I have it right so when I open a ticket (and link it to yours) I am describing the right thing.
I think what I'm hearing is that *outbound* 5353 UDP traffic gets NAT-ed to the modem WAN address and times out because Rick blocks unknown IPs? This seems to be an issue only over the Comcast network, because no one else is doing the crazy stuff with 5353. It only happens on destinations that are on Comcast but that's just an artifact because we have not tried other 5353 sites. Aren't any probably that we know of that are handy?
I assume that understanding may be way off, but that's my opening remark.
I do not get the impression that 5353 is being blocked as an incoming message by Comcast. The assumption I think is that this is some sort of undocumented 'feature' of something some bozo thought was helpful regarding MDNS / Zero Config. No other ISP we know of seems to have had the same brain fail over this 'helpful' behavior AFAIK.
Also need to know which of the two ticket #s I see mentioned I should join.
Once I hear from you guys whether I have this right, and verify the behavior, I'll proceed with tickets.
tnx Al
On 6/4/2024 22:35, Rick Moen wrote:
Quoting Michael Paoli (michael.paoli@berkeley.edu):
Many will, by default, issue NOTIFY from all authoritative nameservers. At first that might seem odd, but, I believe the logic goes about like this: the overhead is low clients that don't care can/will (mostly) ignore authoritative (whether primary or secondary) has no way of knowing how other authoritatives downstream of it are configured, so, e.g. some authoritatives may only get their data via other secondary(/ies), and not direct from master, etc.
Yes, as I was saying, I had a faint recollection that the matter of Aaron T. Porter's ns.primate.net issuing NOTIFY for domains on which it's secondary, not primary, had come up in some of my earlier efforts to puzzle out strange nameserver behaviour. I just couldn't remember exactly how that had unfolded -- other than my obviously having decided to take no action.
Obviously it's at worst harmless, and I can/should just add another "ignore" line to the logcheck configuration so I stop being told about it. I just was taking a moment to try to figure out whether this is deliberate behaviour and why it's there. Your answer will serve splendidly. Although, I'm bothered that the usual information sources don't seem to cover this.
On the third hand, I didn't look _too_ closely, e.g,, maybe it's covered in the Zytrax's "DNS for Rocket Scientists" or the related dead-tree book _Pro DNS and BIND_.
Slightly weird, anyway.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
Okay, adjusting Subject: to better track to ... well the subject.
In brief, from at least certain Comcast Business locations (first noticed on Rick's apparently same issue found from Al's), there's issue communicating from said clients to Internet UDP port 5353 at least with server 96.86.170.229. I also, in my earlier testing (but not when I first opened CR145359298 with Comcast business), discovered that server is receiving packets and responding apparently properly with packets. And other clients (e.g. when I tested from AWS, apparently also when you, Al, tested from AT&T) have no such issue. Also no problem if protocol is switched to TCP or port to 53. So, as I'd last reported to Comcast Business (after I'd figured out issue apparently wasn't on my (Comcast) end): ----- 2024-06-03 11:26p-12:02a US/Pacific CR145359298 CR145359298 - issue appears to be on/towards other end, different Comcast Business account, address: 1105 ALTSCHUL AVE UNIT HMOFC MENLO PARK CA 94025 issue, client IP: 96.95.217.98/29 communication fails with UDP to server: 96.86.170.229 UDP port 5353 Also observed: Server receives and responds. Response packets never make it to client (96.95.217.98). If target port is changed to 53 or protocol changed to TCP, then client successfully receives reply packets. Also able to communicate from other Internet clients to server: 96.86.170.229 UDP port 5353 ----- Anyway, that was before you, Al, confirmed you also have same issue from your Comcast Business.
As for testing, this is highly handy - it also tests to be sure the server is up and responsive over TCP and on UDP port 5353, before it even attempts 5353 UDP (in case the server is down or off-line - it isn't a particularly redundant highly available setup (email might mangle it slightly, but should be easy enough to unmangle), and this fresh test from 96.95.217.99 ([ns1.]linuxmafia.com.): $ (for po in '5353 +tcp' '53 +notcp +ignore' '5353 +notcp +ignore'; do set -- $po; time dig -p $* +nomultiline @96.86.170.229 +noall +answer sflug.com. SOA || break; done) sflug.com. 172800 IN SOA ns1.sf-lug.org. jim.well.com. 1716775812 10800 3600 3600000 86400
real 0m0.064s user 0m0.004s sys 0m0.000s sflug.com. 172800 IN SOA ns1.sf-lug.org. jim.well.com. 1716775812 10800 3600 3600000 86400
real 0m0.043s user 0m0.008s sys 0m0.000s ;; connection timed out; no servers could be reached
real 0m15.010s user 0m0.004s sys 0m0.004s $
Oh, and Al, you do still have login account on balug-sf-lug-v2.balug.org. (that's the canonical name for the VM that hosts BALUG.org, etc.), and even have some fair bits of sudo stuff there too ... if you'd like me to add some more sudo stuff so you can do some relevant tcpdump on ports 5353 and/53, just let me know and I can add that for you.
And ... yeah, we've all been guilty of it, certainly, including me, but perhaps bit better if we keep various separate unrelated (or not so related) emails in, well ... mostly separate emails ... and as relevant adjust the Subject: to track ... the (current) subject, that might make it bit easier to follow these things (and especially more easily find 'em later).
And NAT/SNAT shouldn't be relevant to this 5353 stuff and the IPs involved, nor should, e.g. Comcast Business's SecurityEdge (dis-)service be doing any mucking about interfering or breaking traffic, etc. (e.g. as it was earlier found to be doing on port 53 some months or so ago).
Anyway, also quite probable that whatever you (Al) find regarding Comcast Business and UDP port 5353 would also apply to Rick, as I believe you two are both still currently in the same situation on that (so if y'all both create tickets, or one does first and the other later, may want to cross-reference 'em to hopefully move things along more efficiently). And the one I'd opened on my end: CR145359298 so referencing that for starters may also help - but looks like the Comcast Business issue isn't on my end (at least thus far for this one).
On Thu, Jun 6, 2024 at 12:13 PM Al awbalug@sunnyside.com wrote:
Rick, Michael, Now that I've resolved the crazy static-IPv6-have-it-yet-don't snafu, fixed after 20 tickets and 3 weeks of pain and persistence, I'm ready to join your ticket(s) on 5353. First, of course, I want to be certain that I have it right so when I open a ticket (and link it to yours) I am describing the right thing.
I think what I'm hearing is that *outbound* 5353 UDP traffic gets NAT-ed to the modem WAN address and times out because Rick blocks unknown IPs? This seems to be an issue only over the Comcast network, because no one else is doing the crazy stuff with 5353. It only happens on destinations that are on Comcast but that's just an artifact because we have not tried other 5353 sites. Aren't any probably that we know of that are handy?
I assume that understanding may be way off, but that's my opening remark.
I do not get the impression that 5353 is being blocked as an incoming message by Comcast. The assumption I think is that this is some sort of undocumented 'feature' of something some bozo thought was helpful regarding MDNS / Zero Config. No other ISP we know of seems to have had the same brain fail over this 'helpful' behavior AFAIK.
Also need to know which of the two ticket #s I see mentioned I should join.
Once I hear from you guys whether I have this right, and verify the behavior, I'll proceed with tickets.
tnx Al
On 6/4/2024 22:35, Rick Moen wrote:
Quoting Michael Paoli (michael.paoli@berkeley.edu):
Many will, by default, issue NOTIFY from all authoritative nameservers. At first that might seem odd, but, I believe the logic goes about like this: the overhead is low clients that don't care can/will (mostly) ignore authoritative (whether primary or secondary) has no way of knowing how other authoritatives downstream of it are configured, so, e.g. some authoritatives may only get their data via other secondary(/ies), and not direct from master, etc.
Yes, as I was saying, I had a faint recollection that the matter of Aaron T. Porter's ns.primate.net issuing NOTIFY for domains on which it's secondary, not primary, had come up in some of my earlier efforts to puzzle out strange nameserver behaviour. I just couldn't remember exactly how that had unfolded -- other than my obviously having decided to take no action.
Obviously it's at worst harmless, and I can/should just add another "ignore" line to the logcheck configuration so I stop being told about it. I just was taking a moment to try to figure out whether this is deliberate behaviour and why it's there. Your answer will serve splendidly. Although, I'm bothered that the usual information sources don't seem to cover this.
On the third hand, I didn't look _too_ closely, e.g,, maybe it's covered in the Zytrax's "DNS for Rocket Scientists" or the related dead-tree book _Pro DNS and BIND_.
Slightly weird, anyway.
Just to clarify, the notifies sent to Rick from your .229 were NAT-ed to your modem's WAN address. (Yes, yes, I know, that makes no sense, but it did it anyway. and I merely observe.) My question is, were the notifies going to Rick's port 5353 instead of 53? Or were they coming from your port 5353? Or both? Your answer will weigh heavily on my interpretation of what's going on and my design of an experiment here. Maybe Rick captured his named log that showed the 73.xxx number and he knows the port numbers on both ends. Rick probably sent it in an email, but a lot of messages to comb back through.
On 6/6/2024 18:54, Michael Paoli wrote:
Okay, adjusting Subject: to better track to ... well the subject.
In brief, from at least certain Comcast Business locations (first noticed on Rick's apparently same issue found from Al's), there's issue communicating from said clients to Internet UDP port 5353 at least with server 96.86.170.229. I also, in my earlier testing (but not when I first opened CR145359298 with Comcast business), discovered that server is receiving packets and responding apparently properly with packets. And other clients (e.g. when I tested from AWS, apparently also when you, Al, tested from AT&T) have no such issue. Also no problem if protocol is switched to TCP or port to 53. So, as I'd last reported to Comcast Business (after I'd figured out issue apparently wasn't on my (Comcast) end):
2024-06-03 11:26p-12:02a US/Pacific CR145359298 CR145359298 - issue appears to be on/towards other end, different Comcast Business account, address: 1105 ALTSCHUL AVE UNIT HMOFC MENLO PARK CA 94025 issue, client IP: 96.95.217.98/29 communication fails with UDP to server: 96.86.170.229 UDP port 5353 Also observed: Server receives and responds. Response packets never make it to client (96.95.217.98). If target port is changed to 53 or protocol changed to TCP, then client successfully receives reply packets. Also able to communicate from other Internet clients to server: 96.86.170.229 UDP port 5353
Anyway, that was before you, Al, confirmed you also have same issue from your Comcast Business.
As for testing, this is highly handy - it also tests to be sure the server is up and responsive over TCP and on UDP port 5353, before it even attempts 5353 UDP (in case the server is down or off-line - it isn't a particularly redundant highly available setup (email might mangle it slightly, but should be easy enough to unmangle), and this fresh test from 96.95.217.99 ([ns1.]linuxmafia.com.): $ (for po in '5353 +tcp' '53 +notcp +ignore' '5353 +notcp +ignore'; do set -- $po; time dig -p $* +nomultiline @96.86.170.229 +noall +answer sflug.com. SOA || break; done) sflug.com. 172800 IN SOA ns1.sf-lug.org. jim.well.com. 1716775812 10800 3600 3600000 86400
real 0m0.064s user 0m0.004s sys 0m0.000s sflug.com. 172800 IN SOA ns1.sf-lug.org. jim.well.com. 1716775812 10800 3600 3600000 86400
real 0m0.043s user 0m0.008s sys 0m0.000s ;; connection timed out; no servers could be reached
real 0m15.010s user 0m0.004s sys 0m0.004s $
Oh, and Al, you do still have login account on balug-sf-lug-v2.balug.org. (that's the canonical name for the VM that hosts BALUG.org, etc.), and even have some fair bits of sudo stuff there too ... if you'd like me to add some more sudo stuff so you can do some relevant tcpdump on ports 5353 and/53, just let me know and I can add that for you.
And ... yeah, we've all been guilty of it, certainly, including me, but perhaps bit better if we keep various separate unrelated (or not so related) emails in, well ... mostly separate emails ... and as relevant adjust the Subject: to track ... the (current) subject, that might make it bit easier to follow these things (and especially more easily find 'em later).
And NAT/SNAT shouldn't be relevant to this 5353 stuff and the IPs involved, nor should, e.g. Comcast Business's SecurityEdge (dis-)service be doing any mucking about interfering or breaking traffic, etc. (e.g. as it was earlier found to be doing on port 53 some months or so ago).
Anyway, also quite probable that whatever you (Al) find regarding Comcast Business and UDP port 5353 would also apply to Rick, as I believe you two are both still currently in the same situation on that (so if y'all both create tickets, or one does first and the other later, may want to cross-reference 'em to hopefully move things along more efficiently). And the one I'd opened on my end: CR145359298 so referencing that for starters may also help - but looks like the Comcast Business issue isn't on my end (at least thus far for this one).
On Thu, Jun 6, 2024 at 12:13 PM Al awbalug@sunnyside.com wrote:
Rick, Michael, Now that I've resolved the crazy static-IPv6-have-it-yet-don't snafu, fixed after 20 tickets and 3 weeks of pain and persistence, I'm ready to join your ticket(s) on 5353. First, of course, I want to be certain that I have it right so when I open a ticket (and link it to yours) I am describing the right thing.
I think what I'm hearing is that *outbound* 5353 UDP traffic gets NAT-ed to the modem WAN address and times out because Rick blocks unknown IPs? This seems to be an issue only over the Comcast network, because no one else is doing the crazy stuff with 5353. It only happens on destinations that are on Comcast but that's just an artifact because we have not tried other 5353 sites. Aren't any probably that we know of that are handy?
I assume that understanding may be way off, but that's my opening remark.
I do not get the impression that 5353 is being blocked as an incoming message by Comcast. The assumption I think is that this is some sort of undocumented 'feature' of something some bozo thought was helpful regarding MDNS / Zero Config. No other ISP we know of seems to have had the same brain fail over this 'helpful' behavior AFAIK.
Also need to know which of the two ticket #s I see mentioned I should join.
Once I hear from you guys whether I have this right, and verify the behavior, I'll proceed with tickets.
tnx Al
On 6/4/2024 22:35, Rick Moen wrote:
Quoting Michael Paoli (michael.paoli@berkeley.edu):
Many will, by default, issue NOTIFY from all authoritative nameservers. At first that might seem odd, but, I believe the logic goes about like this: the overhead is low clients that don't care can/will (mostly) ignore authoritative (whether primary or secondary) has no way of knowing how other authoritatives downstream of it are configured, so, e.g. some authoritatives may only get their data via other secondary(/ies), and not direct from master, etc.
Yes, as I was saying, I had a faint recollection that the matter of Aaron T. Porter's ns.primate.net issuing NOTIFY for domains on which it's secondary, not primary, had come up in some of my earlier efforts to puzzle out strange nameserver behaviour. I just couldn't remember exactly how that had unfolded -- other than my obviously having decided to take no action.
Obviously it's at worst harmless, and I can/should just add another "ignore" line to the logcheck configuration so I stop being told about it. I just was taking a moment to try to figure out whether this is deliberate behaviour and why it's there. Your answer will serve splendidly. Although, I'm bothered that the usual information sources don't seem to cover this.
On the third hand, I didn't look _too_ closely, e.g,, maybe it's covered in the Zytrax's "DNS for Rocket Scientists" or the related dead-tree book _Pro DNS and BIND_.
Slightly weird, anyway.
Probably don't muddy the waters with other distractions. Rick apparently noticed some stuff in his logs that apparently looked like some NOTIFY traffic from unexpected IP(s). Maybe that's perhaps related or a clue, maybe it's totally different issue or even red herring.
UDP communications with port 5353, that's a solid issue. If you want to go down rabbit hole of NOTIFY from unexpected IPs, hey, have at it - but I'm not sending 'em, so you'll have to find those yourself (or get 'em from Rick).
On Thu, Jun 6, 2024 at 8:05 PM Al awbalug@sunnyside.com wrote:
Just to clarify, the notifies sent to Rick from your .229 were NAT-ed to your modem's WAN address. (Yes, yes, I know, that makes no sense, but it did it anyway. and I merely observe.) My question is, were the notifies going to Rick's port 5353 instead of 53? Or were they coming from your port 5353? Or both? Your answer will weigh heavily on my interpretation of what's going on and my design of an experiment here. Maybe Rick captured his named log that showed the 73.xxx number and he knows the port numbers on both ends. Rick probably sent it in an email, but a lot of messages to comb back through.
On 6/6/2024 18:54, Michael Paoli wrote:
Okay, adjusting Subject: to better track to ... well the subject.
In brief, from at least certain Comcast Business locations (first noticed on Rick's apparently same issue found from Al's), there's issue communicating from said clients to Internet UDP port 5353 at least with server 96.86.170.229. I also, in my earlier testing (but not when I first opened CR145359298 with Comcast business), discovered that server is receiving packets and responding apparently properly with packets. And other clients (e.g. when I tested from AWS, apparently also when you, Al, tested from AT&T) have no such issue. Also no problem if protocol is switched to TCP or port to 53. So, as I'd last reported to Comcast Business (after I'd figured out issue apparently wasn't on my (Comcast) end):
2024-06-03 11:26p-12:02a US/Pacific CR145359298 CR145359298 - issue appears to be on/towards other end, different Comcast Business account, address: 1105 ALTSCHUL AVE UNIT HMOFC MENLO PARK CA 94025 issue, client IP: 96.95.217.98/29 communication fails with UDP to server: 96.86.170.229 UDP port 5353 Also observed: Server receives and responds. Response packets never make it to client (96.95.217.98). If target port is changed to 53 or protocol changed to TCP, then client successfully receives reply packets. Also able to communicate from other Internet clients to server: 96.86.170.229 UDP port 5353
Anyway, that was before you, Al, confirmed you also have same issue from your Comcast Business.
As for testing, this is highly handy - it also tests to be sure the server is up and responsive over TCP and on UDP port 5353, before it even attempts 5353 UDP (in case the server is down or off-line - it isn't a particularly redundant highly available setup (email might mangle it slightly, but should be easy enough to unmangle), and this fresh test from 96.95.217.99 ([ns1.]linuxmafia.com.): $ (for po in '5353 +tcp' '53 +notcp +ignore' '5353 +notcp +ignore'; do set -- $po; time dig -p $* +nomultiline @96.86.170.229 +noall +answer sflug.com. SOA || break; done) sflug.com. 172800 IN SOA ns1.sf-lug.org. jim.well.com. 1716775812 10800 3600 3600000 86400
real 0m0.064s user 0m0.004s sys 0m0.000s sflug.com. 172800 IN SOA ns1.sf-lug.org. jim.well.com. 1716775812 10800 3600 3600000 86400
real 0m0.043s user 0m0.008s sys 0m0.000s ;; connection timed out; no servers could be reached
real 0m15.010s user 0m0.004s sys 0m0.004s $
Oh, and Al, you do still have login account on balug-sf-lug-v2.balug.org. (that's the canonical name for the VM that hosts BALUG.org, etc.), and even have some fair bits of sudo stuff there too ... if you'd like me to add some more sudo stuff so you can do some relevant tcpdump on ports 5353 and/53, just let me know and I can add that for you.
And ... yeah, we've all been guilty of it, certainly, including me, but perhaps bit better if we keep various separate unrelated (or not so related) emails in, well ... mostly separate emails ... and as relevant adjust the Subject: to track ... the (current) subject, that might make it bit easier to follow these things (and especially more easily find 'em later).
And NAT/SNAT shouldn't be relevant to this 5353 stuff and the IPs involved, nor should, e.g. Comcast Business's SecurityEdge (dis-)service be doing any mucking about interfering or breaking traffic, etc. (e.g. as it was earlier found to be doing on port 53 some months or so ago).
Anyway, also quite probable that whatever you (Al) find regarding Comcast Business and UDP port 5353 would also apply to Rick, as I believe you two are both still currently in the same situation on that (so if y'all both create tickets, or one does first and the other later, may want to cross-reference 'em to hopefully move things along more efficiently). And the one I'd opened on my end: CR145359298 so referencing that for starters may also help - but looks like the Comcast Business issue isn't on my end (at least thus far for this one).
On Thu, Jun 6, 2024 at 12:13 PM Al awbalug@sunnyside.com wrote:
Rick, Michael, Now that I've resolved the crazy static-IPv6-have-it-yet-don't snafu, fixed after 20 tickets and 3 weeks of pain and persistence, I'm ready to join your ticket(s) on 5353. First, of course, I want to be certain that I have it right so when I open a ticket (and link it to yours) I am describing the right thing.
I think what I'm hearing is that *outbound* 5353 UDP traffic gets NAT-ed to the modem WAN address and times out because Rick blocks unknown IPs? This seems to be an issue only over the Comcast network, because no one else is doing the crazy stuff with 5353. It only happens on destinations that are on Comcast but that's just an artifact because we have not tried other 5353 sites. Aren't any probably that we know of that are handy?
I assume that understanding may be way off, but that's my opening remark.
I do not get the impression that 5353 is being blocked as an incoming message by Comcast. The assumption I think is that this is some sort of undocumented 'feature' of something some bozo thought was helpful regarding MDNS / Zero Config. No other ISP we know of seems to have had the same brain fail over this 'helpful' behavior AFAIK.
Also need to know which of the two ticket #s I see mentioned I should join.
Once I hear from you guys whether I have this right, and verify the behavior, I'll proceed with tickets.
tnx Al
On 6/4/2024 22:35, Rick Moen wrote:
Quoting Michael Paoli (michael.paoli@berkeley.edu):
Many will, by default, issue NOTIFY from all authoritative nameservers. At first that might seem odd, but, I believe the logic goes about like this: the overhead is low clients that don't care can/will (mostly) ignore authoritative (whether primary or secondary) has no way of knowing how other authoritatives downstream of it are configured, so, e.g. some authoritatives may only get their data via other secondary(/ies), and not direct from master, etc.
Yes, as I was saying, I had a faint recollection that the matter of Aaron T. Porter's ns.primate.net issuing NOTIFY for domains on which it's secondary, not primary, had come up in some of my earlier efforts to puzzle out strange nameserver behaviour. I just couldn't remember exactly how that had unfolded -- other than my obviously having decided to take no action.
Obviously it's at worst harmless, and I can/should just add another "ignore" line to the logcheck configuration so I stop being told about it. I just was taking a moment to try to figure out whether this is deliberate behaviour and why it's there. Your answer will serve splendidly. Although, I'm bothered that the usual information sources don't seem to cover this.
On the third hand, I didn't look _too_ closely, e.g,, maybe it's covered in the Zytrax's "DNS for Rocket Scientists" or the related dead-tree book _Pro DNS and BIND_.
Slightly weird, anyway.
No NAT/SNAT for my* 96.86.170.224/29 *well, 5 of those are "mine", and 3 Comcast's: 96.86.170.224 network (Comcast's) 96.86.170.225 ("mine") 96.86.170.226 www.mpaoli.net., tigger.mpaoli.net, ... ("mine") 96.86.170.227 ("mine") 96.86.170.228 ("mine") 96.86.170.229 balug.org, etc. ("mine") 96.86.170.230 gateway/router (Comcast's) 96.86.170.231 broadcast (Comcast's) So, 73.189.65.18 is likely Comcast's WAN side. Though at moment, next to last hop I see ... 68.85.103.154 ... but I did that from Rick's, so ... probably different routing from Comcast Business to Comcast Business, vs., e.g. from AT&T to Comcast. And ... minutes later I try and see next to last of: 73.189.65.18 (to balug.org., but same target subnet) So, likely they've got load balancing / failover in their routing too.
On Tue, Jun 4, 2024 at 5:48 PM Al awbalug@sunnyside.com wrote:
Actually I take it back, 73.189.65.18 must be the WAN address of Michael's modem. That won't appear on a message from Michael, unless somehow NAT got involved in the modem? Not sure if that's quite right. I think NAT would come from the last of the assigned static IPv4 addresses, but IIRC I have also seen messages from a modem's WAN address.
Most interesting.
On 6/4/2024 17:43, Al wrote:
I think "c-73-189-65-18.hsd1.ca.comcast.net" is their naming system for end modems, not internal infrastructure, but don't quote me. When I look up the names of routers in the traceroute I don't think I see that type of name, but it may be that's an irresponsible poorly verified comment on my part.
For example from traceroute to Michael: 186.78.151.162.in-addr.arpa domain name pointer po-1-rur101.pinole.ca.sfba.comcast.net.
ooooooooooooooh, omigosh - I just perfected a traceroute to Michael, entirely within the CC network:
traceroute to 96.86.170.229 (96.86.170.229), 30 hops max, 60 byte packets 1 50.242.105.62 2.080 ms 2.743 ms 3.439 ms 2 10.61.209.66 14.028 ms 10.61.209.67 13.729 ms 10.61.209.66 14.373 ms 3 96.216.9.141 12.757 ms 96.216.9.137 13.045 ms 96.216.9.141 13.401 ms 4 68.85.154.113 21.023 ms 68.85.154.117 21.093 ms 68.85.154.113 20.944 ms 5 96.108.99.249 26.011 ms 25.722 ms 26.272 ms 6 68.86.143.89 23.688 ms 68.86.143.93 21.156 ms 20.778 ms 7 162.151.86.58 22.077 ms 20.330 ms 162.151.87.226 20.215 ms 8 162.151.79.134 21.787 ms 162.151.78.186 17.686 ms 162.151.79.134 21.466 ms 9 68.85.103.154 19.388 ms 68.85.191.206 12.456 ms 68.85.103.154 14.255 ms 10 73.189.65.18 21.185 ms 36.832 ms 31.356 ms 11 96.86.170.229 36.366 ms 34.933 ms 34.918 ms root@routr0:/z/r/srv#
It appears that 73.thing is in fact Michael's upstream router.
Ok, did everyone else already know that? Am I late to the party?
On 6/4/2024 17:14, Rick Moen wrote:
----- Forwarded message from Al awbalug@sunnyside.com -----
Date: Tue, 4 Jun 2024 16:29:04 -0700 From: Al awbalug@sunnyside.com To: Rick Moen rick@linuxmafia.com Subject: Re: [BALUG-Admin] Comcast Business apparently blocking 5353 UDP Re: linuxmafia.com "retry limit exceeded"
Rick, you're at the right place - that gear icon and right side panel on business.comcast.com is just the right thing. And I think the situation as you're outlining it is right to me. So the answer to your question, broadly, is yes I think you have it right. If you end up at securityedge.comcast.com, IMHO you've gone too far. My sense is that all that stuff is disabled back at the right side panel... Once SE (security edge) is disabled I think everything is. That said, you're being smart about it - if symptoms persist, drill down and look into individual settings for various elements of SE and just make sure they're all off
- in case Comcast can't quite sort out how to actually disable stuff.
AFAIK however your nets (yours and Michaels) are unrestricted. My tests from here are that access to both 96.86.170.229 and 96.95.217.99 on port 53 is not blocked (and not just those /32s but the entire subnet in each case). I am looking back over email from the last few days trying to sort out where 73.189.65.18 crept into the conversation. As I mentioned I have been unable to focus sufficiently on this the last few days, and missed where that came from. I also haven't looked closely enough at the discussion to see if what I am trying to reproduce isn't exactly where you're having trouble. I'll go back over the notes and see if I can pay more attention to the details and whether I can actually add any insight to the discussion. Al
----- End forwarded message -----
To clarify, I noticed "73.189.65.18" as the source of NOTIFYs for Michael's domains, which can legitimately come _only_ from Michael's authoritative nameserver, IP 96.86.170.229.
And 73.189.65.18 is Comcast's _own_ IP, not Michael's.
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
So, something is rotten, there. I'm immediately inclined to suspect that Comcast is playing man-in-the-middle games with DNS traffic. Which, if true, suggest Comcast acting like a rogue state security agency or one operating on behalf of a totalitarian state. Not a good look.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
Your mdem's web page will tell you what the modem's WAN address is. Gateway > Connection > Comcast Network should show WAN IP Address (IPv4):98.45.214.155 (Ok, the IP is on my modem, but supposedly the 73.thing will be on yours) It will certainly be interesting if it isn't the 73.whatever NAT normally uses the 96.86.170.230 address (in your case) but in some cases I believe it sometimes uses the WAN address. I don't have it right in front of me, and it's been a while, so I can't back it up, and maybe I'm wrong but I'm pretty sure I've seen that. Why 5353? is a whole other speculation exercise.
On 6/4/2024 19:59, Michael Paoli wrote:
No NAT/SNAT for my* 96.86.170.224/29 *well, 5 of those are "mine", and 3 Comcast's: 96.86.170.224 network (Comcast's) 96.86.170.225 ("mine") 96.86.170.226www.mpaoli.net., tigger.mpaoli.net, ... ("mine") 96.86.170.227 ("mine") 96.86.170.228 ("mine") 96.86.170.229 balug.org, etc. ("mine") 96.86.170.230 gateway/router (Comcast's) 96.86.170.231 broadcast (Comcast's) So, 73.189.65.18 is likely Comcast's WAN side. Though at moment, next to last hop I see ... 68.85.103.154 ... but I did that from Rick's, so ... probably different routing from Comcast Business to Comcast Business, vs., e.g. from AT&T to Comcast. And ... minutes later I try and see next to last of: 73.189.65.18 (to balug.org., but same target subnet) So, likely they've got load balancing / failover in their routing too.
On Tue, Jun 4, 2024 at 5:48 PM Alawbalug@sunnyside.com wrote:
Actually I take it back, 73.189.65.18 must be the WAN address of Michael's modem. That won't appear on a message from Michael, unless somehow NAT got involved in the modem? Not sure if that's quite right. I think NAT would come from the last of the assigned static IPv4 addresses, but IIRC I have also seen messages from a modem's WAN address.
Most interesting.
On 6/4/2024 17:43, Al wrote:
I think "c-73-189-65-18.hsd1.ca.comcast.net" is their naming system for end modems, not internal infrastructure, but don't quote me. When I look up the names of routers in the traceroute I don't think I see that type of name, but it may be that's an irresponsible poorly verified comment on my part.
For example from traceroute to Michael: 186.78.151.162.in-addr.arpa domain name pointer po-1-rur101.pinole.ca.sfba.comcast.net.
ooooooooooooooh, omigosh - I just perfected a traceroute to Michael, entirely within the CC network:
traceroute to 96.86.170.229 (96.86.170.229), 30 hops max, 60 byte packets 1 50.242.105.62 2.080 ms 2.743 ms 3.439 ms 2 10.61.209.66 14.028 ms 10.61.209.67 13.729 ms 10.61.209.66 14.373 ms 3 96.216.9.141 12.757 ms 96.216.9.137 13.045 ms 96.216.9.141 13.401 ms 4 68.85.154.113 21.023 ms 68.85.154.117 21.093 ms 68.85.154.113 20.944 ms 5 96.108.99.249 26.011 ms 25.722 ms 26.272 ms 6 68.86.143.89 23.688 ms 68.86.143.93 21.156 ms 20.778 ms 7 162.151.86.58 22.077 ms 20.330 ms 162.151.87.226 20.215 ms 8 162.151.79.134 21.787 ms 162.151.78.186 17.686 ms 162.151.79.134 21.466 ms 9 68.85.103.154 19.388 ms 68.85.191.206 12.456 ms 68.85.103.154 14.255 ms 10 73.189.65.18 21.185 ms 36.832 ms 31.356 ms 11 96.86.170.229 36.366 ms 34.933 ms 34.918 ms root@routr0:/z/r/srv#
It appears that 73.thing is in fact Michael's upstream router.
Ok, did everyone else already know that? Am I late to the party?
On 6/4/2024 17:14, Rick Moen wrote:
----- Forwarded message from Alawbalug@sunnyside.com -----
Date: Tue, 4 Jun 2024 16:29:04 -0700 From: Alawbalug@sunnyside.com To: Rick Moenrick@linuxmafia.com Subject: Re: [BALUG-Admin] Comcast Business apparently blocking 5353 UDP Re: linuxmafia.com "retry limit exceeded"
Rick, you're at the right place - that gear icon and right side panel on business.comcast.com is just the right thing. And I think the situation as you're outlining it is right to me. So the answer to your question, broadly, is yes I think you have it right. If you end up at securityedge.comcast.com, IMHO you've gone too far. My sense is that all that stuff is disabled back at the right side panel... Once SE (security edge) is disabled I think everything is. That said, you're being smart about it - if symptoms persist, drill down and look into individual settings for various elements of SE and just make sure they're all off
- in case Comcast can't quite sort out how to actually disable stuff.
AFAIK however your nets (yours and Michaels) are unrestricted. My tests from here are that access to both 96.86.170.229 and 96.95.217.99 on port 53 is not blocked (and not just those /32s but the entire subnet in each case). I am looking back over email from the last few days trying to sort out where 73.189.65.18 crept into the conversation. As I mentioned I have been unable to focus sufficiently on this the last few days, and missed where that came from. I also haven't looked closely enough at the discussion to see if what I am trying to reproduce isn't exactly where you're having trouble. I'll go back over the notes and see if I can pay more attention to the details and whether I can actually add any insight to the discussion. Al
----- End forwarded message -----
To clarify, I noticed "73.189.65.18" as the source of NOTIFYs for Michael's domains, which can legitimately come _only_ from Michael's authoritative nameserver, IP 96.86.170.229.
And 73.189.65.18 is Comcast's _own_ IP, not Michael's.
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
So, something is rotten, there. I'm immediately inclined to suspect that Comcast is playing man-in-the-middle games with DNS traffic. Which, if true, suggest Comcast acting like a rogue state security agency or one operating on behalf of a totalitarian state. Not a good look.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
Gateway > Connection > Comcast Network WAN IP Address (IPv4): 73.189.65.18 WAN Static IP Address (IPv4): 96.86.170.230 WAN Default Gateway Address (IPv4): 73.189.64.1 WAN IP Address (IPv6): 2001:558:6045:b3:9887:23a:a685:f346 WAN Default Gateway Address (IPv6): fe80::201:5cff:fea Thanks ... well, that at least does eliminate a bit of guesswork as to what/where that/those IPs are. And as for what NAT uses ... have to check from relevant client IP ... https://www.wiki.balug.org/wiki/doku.php?id=system:what_is_my_ip_address So, behind Comcast's NAT on the "modem"/router ... $ ip -4 a s | sed -ne '/inet/!d;/inet 127./!p'; curl -s https://ip4only.me/api/ inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3 IPv4,96.86.170.230,v1.1,,,See http://ip6.me/docs/ for api documentation $ Or from another such client host, and preceding with a check that doesn't traverse NAT/SNAT to see the IPv4 IP on my external interface ...: $ ssh -q myip@10.1.10.2 10.1.10.142 $ curl -s https://ip4only.me/api/ IPv4,96.86.170.230,v1.1,,,See http://ip6.me/docs/ for api documentation $ So, not too surprisingly, it uses the 96.86.170.230 IP, that's IP of router/gateway itself, static, and in all cases, relevant return traffic would need come to or via that IP anyway. So, no big surprises there. For IPv4, probably the only logical possibilities open to Comcast without consuming more IPv4 IPs (a precious resource) would be: 73.189.65.18 as that's the IP on WAN side ... but that wouldn't work as efficiently for communicating with the rest of 96.86.170.224/29 So probably simpler for them to use 96.86.170.230 ... or it might even be a limitation of their "router"/"modem" software/firmware and its NAT/SNAT capabilities, on where it can do that. And, also easier and more consistent, e.g. when I'm doing stuff on firewall(s) elsewhere and whitelisting my "home" IPv4 sources, I'll typically just whitelist 96.86.170.224/29 and that covers all the possibilities IPv4-wise that I'd be originating out to The Internet on globally routable IP addresses.
On Tue, Jun 4, 2024 at 8:09 PM Al awbalug@sunnyside.com wrote:
Your mdem's web page will tell you what the modem's WAN address is. Gateway > Connection > Comcast Network should show WAN IP Address (IPv4):98.45.214.155 (Ok, the IP is on my modem, but supposedly the 73.thing will be on yours) It will certainly be interesting if it isn't the 73.whatever NAT normally uses the 96.86.170.230 address (in your case) but in some cases I believe it sometimes uses the WAN address. I don't have it right in front of me, and it's been a while, so I can't back it up, and maybe I'm wrong but I'm pretty sure I've seen that. Why 5353? is a whole other speculation exercise.
On 6/4/2024 19:59, Michael Paoli wrote:
No NAT/SNAT for my* 96.86.170.224/29 *well, 5 of those are "mine", and 3 Comcast's: 96.86.170.224 network (Comcast's) 96.86.170.225 ("mine") 96.86.170.226 www.mpaoli.net., tigger.mpaoli.net, ... ("mine") 96.86.170.227 ("mine") 96.86.170.228 ("mine") 96.86.170.229 balug.org, etc. ("mine") 96.86.170.230 gateway/router (Comcast's) 96.86.170.231 broadcast (Comcast's) So, 73.189.65.18 is likely Comcast's WAN side. Though at moment, next to last hop I see ... 68.85.103.154 ... but I did that from Rick's, so ... probably different routing from Comcast Business to Comcast Business, vs., e.g. from AT&T to Comcast. And ... minutes later I try and see next to last of: 73.189.65.18 (to balug.org., but same target subnet) So, likely they've got load balancing / failover in their routing too.
On Tue, Jun 4, 2024 at 5:48 PM Al awbalug@sunnyside.com wrote:
Actually I take it back, 73.189.65.18 must be the WAN address of Michael's modem. That won't appear on a message from Michael, unless somehow NAT got involved in the modem? Not sure if that's quite right. I think NAT would come from the last of the assigned static IPv4 addresses, but IIRC I have also seen messages from a modem's WAN address.
Most interesting.
On 6/4/2024 17:43, Al wrote:
I think "c-73-189-65-18.hsd1.ca.comcast.net" is their naming system for end modems, not internal infrastructure, but don't quote me. When I look up the names of routers in the traceroute I don't think I see that type of name, but it may be that's an irresponsible poorly verified comment on my part.
For example from traceroute to Michael: 186.78.151.162.in-addr.arpa domain name pointer po-1-rur101.pinole.ca.sfba.comcast.net.
ooooooooooooooh, omigosh - I just perfected a traceroute to Michael, entirely within the CC network:
traceroute to 96.86.170.229 (96.86.170.229), 30 hops max, 60 byte packets 1 50.242.105.62 2.080 ms 2.743 ms 3.439 ms 2 10.61.209.66 14.028 ms 10.61.209.67 13.729 ms 10.61.209.66 14.373 ms 3 96.216.9.141 12.757 ms 96.216.9.137 13.045 ms 96.216.9.141 13.401 ms 4 68.85.154.113 21.023 ms 68.85.154.117 21.093 ms 68.85.154.113 20.944 ms 5 96.108.99.249 26.011 ms 25.722 ms 26.272 ms 6 68.86.143.89 23.688 ms 68.86.143.93 21.156 ms 20.778 ms 7 162.151.86.58 22.077 ms 20.330 ms 162.151.87.226 20.215 ms 8 162.151.79.134 21.787 ms 162.151.78.186 17.686 ms 162.151.79.134 21.466 ms 9 68.85.103.154 19.388 ms 68.85.191.206 12.456 ms 68.85.103.154 14.255 ms 10 73.189.65.18 21.185 ms 36.832 ms 31.356 ms 11 96.86.170.229 36.366 ms 34.933 ms 34.918 ms root@routr0:/z/r/srv#
It appears that 73.thing is in fact Michael's upstream router.
Ok, did everyone else already know that? Am I late to the party?
On 6/4/2024 17:14, Rick Moen wrote:
----- Forwarded message from Al awbalug@sunnyside.com -----
Date: Tue, 4 Jun 2024 16:29:04 -0700 From: Al awbalug@sunnyside.com To: Rick Moen rick@linuxmafia.com Subject: Re: [BALUG-Admin] Comcast Business apparently blocking 5353 UDP Re: linuxmafia.com "retry limit exceeded"
Rick, you're at the right place - that gear icon and right side panel on business.comcast.com is just the right thing. And I think the situation as you're outlining it is right to me. So the answer to your question, broadly, is yes I think you have it right. If you end up at securityedge.comcast.com, IMHO you've gone too far. My sense is that all that stuff is disabled back at the right side panel... Once SE (security edge) is disabled I think everything is. That said, you're being smart about it - if symptoms persist, drill down and look into individual settings for various elements of SE and just make sure they're all off
- in case Comcast can't quite sort out how to actually disable stuff.
AFAIK however your nets (yours and Michaels) are unrestricted. My tests from here are that access to both 96.86.170.229 and 96.95.217.99 on port 53 is not blocked (and not just those /32s but the entire subnet in each case). I am looking back over email from the last few days trying to sort out where 73.189.65.18 crept into the conversation. As I mentioned I have been unable to focus sufficiently on this the last few days, and missed where that came from. I also haven't looked closely enough at the discussion to see if what I am trying to reproduce isn't exactly where you're having trouble. I'll go back over the notes and see if I can pay more attention to the details and whether I can actually add any insight to the discussion. Al
----- End forwarded message -----
To clarify, I noticed "73.189.65.18" as the source of NOTIFYs for Michael's domains, which can legitimately come _only_ from Michael's authoritative nameserver, IP 96.86.170.229.
And 73.189.65.18 is Comcast's _own_ IP, not Michael's.
:r! dig -x 73.189.65.18 +short c-73-189-65-18.hsd1.ca.comcast.net.
So, something is rotten, there. I'm immediately inclined to suspect that Comcast is playing man-in-the-middle games with DNS traffic. Which, if true, suggest Comcast acting like a rogue state security agency or one operating on behalf of a totalitarian state. Not a good look.
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin
BALUG-Admin mailing list BALUG-Admin@lists.balug.org https://lists.balug.org/cgi-bin/mailman/listinfo/balug-admin