Packetfabric / Unitas
Under Investigation 07/08/2025 - Network Interruptions NL
- Thread starter KH-DanielP
- Start date
Howdy,
I understand your concern and frustrations, I get it. We're doing everything in our power to ensure services are restored as quickly as possible. You're welcome to review our SLA and Terms of Service regarding any "actions" but such talk is neither helpful nor productive at this stage.
Our one and only goal is to work to ensure services are restored as quickly as possible.
digitalboxgr
New Member
How can you claim to be doing everything in your power to restore services as quickly as possible when you don’t even have an ETA?
I don’t think you have any idea.
I don’t think you have any idea.
We've been given confirmation that all equipment, including upstream equipment within all DRT facilities is functioning properly, however we've been told this may be tied to a "Power Event" at Equinix AMS1 where apparently all of the public uplinks feed from.
I don't have any additional information beyond that at this time.
Howdy,
I'm sorry you feel that way, but since the situation is not under my direct control there are two things I can do.
1) Apply as much pressure to all appropriate escalation paths with our vendors as possible, which I am continually doing.
2) Communicate any relevant information to our customers regarding the issue as it becomes available.
We're going to ensure everyone gets back online as quickly as possible. As I've mentioned before plans were already significantly in motion to mitigate additional issues such as these, and we were weeks away from announcing those upgrades for existing customers in Netherlands.
I get it, downtime sucks, really shouldn't happen with the redundancies in place, the timing is shit and we'd all rather be going about our days dealing with the thousands of other problems that life throws at us. But here we are, and we're in this until it's sorted and everyone is back online.
Thank you very much, we do appreciate your efforts and we do not have any doubts that you are doing your best since we had a very good experience with your support team.
our problem is that some of our customers has mission critical hosting's and are pushing on us asking to get approximate ETA which you do not have (understandable since the issue not at your side).
waiting for a new redundant connection to another upstream providers if any.
once you got any ETA's please share with us ASAP.
our problem is that some of our customers has mission critical hosting's and are pushing on us asking to get approximate ETA which you do not have (understandable since the issue not at your side).
waiting for a new redundant connection to another upstream providers if any.
once you got any ETA's please share with us ASAP.
I completely understand, and I do appreciate your understanding. In normal scenarios such as a node outage, we're able to give estimated times as it's a well defined process, in this instance, I'm very hesitant to put any type of number to it.
I'm receiving regular updates as to the status of their investigation and how things are progressing, and have received one just now.
It appears that the root issue is being worked on now, or at least isolated to the core equipment located in Equinix AM1 facility. My hope is that this means we'll see service restoration soon, but I'm not privy to the extent of the problems there other than Equinix AM1 appears to have scuffed a power event.
I do 100% understand, I'd just hate to guess at a time frame only for it to be wrong. I can confirm they appear to have isolated the issue, and have passed along some information regarding it, but no ETA has been provided yet.
I am sorry, I do not mean to sound aggressive, but everyone in the company is breathing down my back for accessing the data on the server, and this situation made my life 1000 times harder right now. The issue is that this downtime happened too sudden and no backups were made in the past few days. I am glad it is only the core equipment and nothing more damaged to the power infrastructure.
You're fine, I completely understand.
All data / servers etc. are not impacted , it's purely the connectivity to them so rest assured your data is there and safe.
I take failures like this very personal, I've been in this industry now for almost three decades and I still remember every failure large and small and every action taken after each one of them to help prevent or eliminate circumstances like them in the future.
Right now we've got well over half a million in equipment either sitting at, or on the way to Iron Mountain in Amsterdam along with multiple circuits being provisioned there for in-essence a full hardware refresh for our entire Amsterdam deployment. I know that's not overly relevant to the case now, but we've had this plan in the works for the last 6~ months with it scheduled to go-live at the beginning of August.
Once that's done, not only will we have a fully upgraded Network infrastructure (like we completed in WA and GA) but a full HW refresh for everyone as well. Sadly, it's not August yet so that isn't in production and client traffic is not yet shifted to it.
Regardless I'm rambling a bit, but we absolutely take every failure to heart, and do everything in our power to mitigate against them, just sometimes we fail, or fall a bit short timeline wise, but we'll never stop trying to better ourselves.
Hi KnownHost team,
I completely understand the difficulty of the situation you’re facing and I know issues like this are sometimes beyond anyone’s immediate control. Everyone here in Europe is getting hammered by their clients because of this downtime, including myself.
However, what I don’t understand is why no email was sent out to all affected customers as soon as this problem occurred. Many of us found out only because we checked the forum or noticed our services offline, which adds to the frustration and the number of support tickets you receive.
A proactive email notification would have helped us inform our own clients faster and manage expectations better.
Could you please consider improving communication during such incidents in the future?
Thanks for your efforts and for keeping us updated here on the forum.
I completely understand the difficulty of the situation you’re facing and I know issues like this are sometimes beyond anyone’s immediate control. Everyone here in Europe is getting hammered by their clients because of this downtime, including myself.
However, what I don’t understand is why no email was sent out to all affected customers as soon as this problem occurred. Many of us found out only because we checked the forum or noticed our services offline, which adds to the frustration and the number of support tickets you receive.
A proactive email notification would have helped us inform our own clients faster and manage expectations better.
Could you please consider improving communication during such incidents in the future?
Thanks for your efforts and for keeping us updated here on the forum.
GreatMarko
Member
Howdy,
Thanks for the suggestion, we have discussed internally a different type of notice/status system for any events, perhaps even tied into the billing portal.
Historically we've recommended folks subscribe to the appropriate regional notice section here on the forums as you'd get emails from any posts relevant to your region.
digitalboxgr
New Member
Maybe. it has only t do with Knownhost because the status of Equinix Networks does not have issues, including Amsterdam AM1
status.equinixmetal.com
Equinix Metal Status
Welcome to Equinix Metal's home for real-time and historical data on system performance.
status.equinixmetal.com
To be honest, that I am not 100% certain on.
I do not know if this event specifically impacts just Packet Fabrics presence in Equinix AMS1, or if it's more wide spread.
Hi Daniel,
I’d like to mention that I personally was never aware that we should subscribe separately to a regional notice section in order to receive emails about incidents like this. I assumed that major outages affecting customers would automatically trigger an email to all impacted accounts.
GreatMarko
Member
Ooo... slightly off-topic, but does that mean that NVMe VPS availability will finally be coming to the EU data center?!
It's very possible it's isolated to the Packet Fabric presence within Equinix AMS1, anything there would be referenced by status.equinix.com which is locked behind their portal and only displayed as customer specific dependent upon your services with them.Maybe. it has only t do with Knownhost because the status of Equinix Networks does not have issues, including Amsterdam AM1
Equinix Metal Status
Welcome to Equinix Metal's home for real-time and historical data on system performance.
I have access to it, but not to any AMS1 details for Equinix.