Network issue

Network issue

17-05-2016 16:01:15 - 17-05-2016 18:15:00
Status: resolved

There is a problem on our network. Untill further notice updates regarding this incident will be placed on http://www.bit.org/.

 


Update placed at 17/05/2016, 16:59

The following is the incident as it appeared on www.bitorg. The incident there will be closed soon.

2016-05-17 15:45h

There is a problem in our network. More info when available on this site.

=====

Update 16:00h

Large parts of the network are available again.

====

Update 16:10h

An incident in our core switching caused the network issue. Traffic coming from certain paths to our network might still be disrupted. Engineers are working on this issue.

====

Update 16:17h

The complete network is available again.


Update placed at 17/05/2016, 17:02

The incident has been resolved. Our apologies for the inconvenience. More information regarding the incident will follow on this website as soon as our network engineers finished their analysis. If you have any questions regarding this incident, please contact the Customer Care department by phone on +31 (0)318 648 688 or by email on support@bit.nl.


Update placed at 17/05/2016, 17:56

The disruption in our network on May 17th has been caused by a failing core switch. Unfortunately no logging is available that can give any clues to the cause of this failure. Because this is the second time in a relative short period that this switch fails, we will replace the switch. This replacement will be announced in a separate maintenance notice.

The problematic switch led to the unavaivalability of several paths within our network. The precautions for these type of problems did not perform as expected. The core routers saw themselves in a 'split brain' situation. As a result rerouting to other paths took longer than wished for and necessary. Our network engineers will analyse the cause of this split brain situation and will take appropiate actions to prevent this in the future.

Again we apologise for any inconvenience.


Update placed at 18/05/2016, 14:16

The RFO regarding this outage is being made. The RFO will be made available on our website as soon as this RFO is ready.


Update placed at 19/05/2016, 10:41

The RFO regarding this outage is available (in Dutch) and can be found here.