Status mreže News & Information

19:01 Switch has been replaced but server did not get back online; DC personal dispatched to the server for further investigation.
19:25 There is a secondary issue with the network configuration which is not operating as it should be, they are working on this. 
20:42  The DC techs are still working on the issue. Once we have more information we will update it here.
20:57 Escalated as per instructions to DC management (informative to keep everyone in the loop) 
21:25 after the broken switch there is still something not functioning correctly in the network; the network technicians are looking into the issue once more. 
23:01 server was briefly online but in a wrong kernel version which caused other issues; we are still working on the issue. 

23:18 Status Update: this issue started with a network issue in the DC, that is what we know, but unfortunately this may have started a list of chain reactions ; somewhere something else did go wrong; we where able to briefly start the system with an old centos kernel; which fortunately means that we can access the data if needed, the bad thing is that we where still not able to get the server working; we are trying to resolve this but for this we need a so called rescue system; a system which boots up a Linux OS which allows us to manually mount the server and boot it up to see what is happening as we are sitting behind the console. But also this functionality is not working. We keep working on this issue. If we can not resolve this issue by morning we will be forced to install a new server and start restores; something we would like to avoid. Our team is working hard to resolve the issue the soonest possible. 

Installation of a patch to prevent the Intel meltdown bug has resulted in the server crashing & it being unable to boot up.

12:21am As SSDSB04 server will not even load past the initial boot screen we are currently restoring all accounts to a new server - SSDSB06.

Our nameserver IPs have been updated:

ns1.affordahost.net 62.138.16.101
ns2.affordahost.net 198.20.101.186

If you use your own custom nameservers you will need to update them to these IPs to restore access to your website.

We are currently investigating a problem with the Meliai server. Updates to follow.

0310: Issue has been identified & is currently being fixed - we hope to have everything back up & running in the next couple of hours. Full info will be provided soon

Full details: Meliai Server crashed after an installation of an anti DOS patch in the kernel, installing the patch rendered the server partly unreachable so the changes had to be rolled back. Normal service has resumed, we apologise for any inconvenience caused.

• We are receiving reports that services are not available.
• This has been confirmed by us and we are currently investigating the issue with our service provider.
• The service provider has now confirmed there is an issue and is investigating further.
• The service provider has confirmed the server is unreachable and is sending a technician to check the hardware.
• The server is now back online and should be reachable.

Short explaination:
On Monday (30th November, 2009) your website will be unreachable for a short period of time (aprox. 15-20 minutes) between 6:00-7:30 GMT due to scheduled emergency network maintenance.

Further details:
The following notification is to inform you of EMERGENCY MAINTENANCE on the IP Network. Due to the severity, the time frame between this notice and the maintenance window has been shortened.
In conjunction with Cisco Systems we have identified a service impacting bug in the current IOS version running on this router. A vendor recommended upgrade has been provided as a resolution and needs to be implemented as soon as possible.

Original Issue Reference:
With the introduction of 4-byte ASNs (Autonomous System Numbers), a newly devised method of injecting a specifically crafted prefix into the global route table allows for the BGP process to be reset on older IOS code.
http://www.cisco.com/warp/public/707/cisco-sa-20090729-bgp.shtml

Customer Impact:
During this maintenance, customers will notice a complete loss of connectivity to their servers on the frontend network (public network). Backend network (private network) connectivity will NOT be impacted during this maintenance. While the upgrade duration is scheduled for 1 hour, we only expect between 15-20 minutes of downtime as the routers reload and fully converge.

At approximately 11:55 AM GMT, engineers were alerted to connectivity issues within our DC network. After our initial investigation, it was determined that FCR01 had suffered a software crash and had to be reloaded. Once the router stabilised at 12:21 GMT, Engineers determined that the best course of action was to upgrade the router to the latest IOS and perform another reload at 12:41 GMT. Connectivity was fully restored at 12:47am GMT and no other service interruption is expected.

During the course of this event, there was intermitting connectivity and packet loss to servers connected behind FCR01. Clients behind other FCR routers would not have been effected. At this time, all connectivity has been restored to the effected network segments.

We sincerely apologise for any inconvenience this may have caused you, and thank you for your patience in this matter.