Monday, April 20, 2026
[Incident Report #033][DNS] Suspended lookups for ‘look.com’
What Happened?
NS2 has been seeing a low-volume, but constant stream of lookups for
‘look.com’ for the past few days. These lookups are for ANY and are spread
across a handful of IPv4 addresses. Seeing as most lookups only make a
handful of request before the requesting machine has the info it needs,
we are dropping these lookups for a little while because our NOC is treating
it as internet background radiation.
We were seeing the following issues:
- Steady, low rate, constant lookups for ANY against ‘look.com’
What did we do to fix this?
- Temporarily dropping lookup request for ‘look.com’
Saturday, April 18, 2026
[Incident Report #032][DNS] SSL Expiry on NS1 and NS2
What Happened?
Our SSL certs for NS1 and NS2 expired earlier today. Currently our process
for handling the updating of SSL certs is not automated and requires our
team to manually install new certs and then reload the servers one after
the other. Usually this is on our internal calendar and is handled three to
four days before EOL. That didn’t happen this time.
We were seeing the following issues:
- Loss of DNS over HTTPS support
- Loss of DNS over TLS support
What did we do to fix this?
- We pulled new certs and updated the cert store
- We reloaded both name servers to restore service
Everything should be operational and peachy again!
Saturday, March 28, 2026
Ongoing Name Server Attacks
FurrIX is seeing attacks on our name servers that have not let
up for a few hours now and as a result we have had to tighten
our rate limits and start dropping excessive traffic.
The way things are going, we will not be letting up on our rate
limits any time soon.
If you are being affected by these changes, you can send an
email off to ‘nameservers at marbledfennec dot net’ and request
a whitelisting that will bypass the limits. We will require knowing
you use case, however.
Wednesday, March 25, 2026
[Name Servers] Updated Configuration
We have been seeing a lot of DNS amplification attacks ongoing
during the past few weeks and have adjusted our name server
configuration to start rate-limiting and dropping request a fair
bit sooner.
We have also changed the graphs that we are showing for
the server status page.
Tuesday, March 24, 2026
[Incident Report #031][NET] Connectivity Issues with 2602:F992:EC::/46
Update:
Appears our static route did not get migrated to the new hardware
at the data center. This has been resolved and our services should
be back online momentarily!
What Happened?
Our upstream provider performed some upkeep on the data center on
Mar 20th and since then our routed subnet has not be working properly.
We are looking into this as it affect our IPAM, NMS, PBS and a few other
services that assist us with managing our network.
This is also affects our secondary name server, as it currently does not
have IPv6 service.
We are seeing the following issues:
- No IPv6 connectivity for NS2, affecting private and public resolution request
- No PBS backups for either MFN, FurrIX or Maows.Gay
- No NMS accounting or error tracking
- No status graphs on our website are updating
What are we doing to work on this?
- We have performed internal and external networking checks
- We have reached out to our upstream provider with our findings
At the moment, we will have to wait to hear back from the data center.