Maintenance load balancers

Maintenance load balancers

30-01-2020 00:00:00 - 30-01-2020 06:00:00

Urgency: Planned
Affected services:
- Load balanced services
- Shared hosting
Expected impact:
- Short disruption of load balanced services
Customer intervention required: No
Reference number: 166503

Summary:

On Thursday January 30th between 00:00 hrs and 06:00 hrs we will perform maintenance on our load balancers. During this maintenance we will apply a software update. We will also disable TLS version 1.0 en 1.1 and enable TLS 1.3.

Details:

During this maintenance, we will switch between the active and backup load balancer a number of times. This switchover can cause a short disruption of one or a few seconds to all load balanced services. When using a secure connection to a SSL VIP, the load balancer will offer TLS version 1.0, 1.1 or 1.2. Because TLS versions 1.0 and 1.1 are considered weak and insecure we will disable these protocols. Starting from this maintenace, only TLS version 1.2 will remain and 1.3 will be offered as a new version. This applies to all SSL VIPS and customers using our Shared Hosting platform. You can see on your invoice if you are using our Shared Hosting platform at 'linux-webruimte' or 'windows-webruimte'.
Update placed at 30-01-2020, 5.10h

Unfortunately, the software upgrade did not go as planned. After upgrading the first load balancer a problem was detected with one its network interfaces. Because there was no clear cause for the instability of this interface and this was part of a redundant set, engineers decided to temporarily disable it and continue with the upgrade of the other load balancer after switching between the master and standby. After the upgrade, the other load balancer showed identical problems with one instable network interface. At this point a high priority case was opened with our vendor and sessions have been started to gather logs and other relevant data. It turned out that the instability was caused by a bug in the kernel of the load balancer's operating system. As adviced by the vendor we decided to roll back to the previous version of the software we were running.

We want to emphasize that this bug has had no affect on the availability of our load balanced services. At all times the active load balancer was available. We did have to do an additonal number of failovers between the master and standby unit, which may have caused very short disruptions in load balanced traffic.

In addition, we want to point out that although TLS 1.3 is not available now, we did disable TLS 1.0 and 1.1, since these are considered insecure.