Emergency Maintenance shared filesystem (CephFS)

Emergency Maintenance shared filesystem (CephFS)

05-12-2019 23:00:00 - 06-12-2019 02:00:00

Urgency: Emergency

Affected services:
- Shared Linux Hosting;
- No VNC (console) access to virtual machines in BIT portal;
- Shared filesystems on CephFS.
Expected impact:
- Websites on the Shared Linux Hosting platform might be unreachable for a short period of time;
- The upload server of the Shared Linux hosting platform will be unreachable for a short period of time;
- In the BIT portal it will not be possible to obtain (VNC) console access to virtual machines for a short period of time;
- Customers who use shared filesystem CephFS will experience hindrance.
Customer intervention required: No

Summary:
In consultation with the CephFS developers we will apply a patch to our Ceph Metadata Servers (MDS). This in an attempt to mitigate or prevent interruptions on our CephFS-cluster. In this maintenance window the CephFS-cluster will not be redundant and will be unreachable for short periods of time.

Details:
This emergency maintenance is not being carried out during a normal maintenance window (00:00 - 07:00 hrs), because this overlaps with the period the MDS-server(s) are used the most intense (during backup period). We have therefore decided to execute this maintenance during lower traffic hours. During the maintenance the shared filesystem CephFS will not be available at least once, but possibly multiple times, for a short period of time. The MDS-servers will be updated with newly patched CephFS software packages. The standby server will be re-installed and provisioned with a modified version of the CephFS (MDS) software. This to hopefully prevent interruptions we experienced recently (Outage-storage-systems) (CephFS-storage-issue). During the maintenance on the standby MDS server the CephFS cluster is temporary not redundant.