Description
As a fast follow on to the work completed for the critical database outage the previous evening (9/24). Additional configuration of the database servers was required to ensure more reliable data replication moving forward.
Root Cause/Remediation
The initial configuration of the new standby database system and replication method were suboptimal and through monitoring it was identified that load on the system started to cause replication delay and needing immediate resolution. The problems were causing excessive local WAL transfer and storage and the potential for less reliable replication. The standby was brought up-to-date and the database configuration was updated to ensure more reliable replication and availability.
Posted Sep 28, 2018 - 09:09 MDT
Completed
As a fast follow on to the work completed for the critical database outage in the evening of 9/24 (MST). Additional configuration of the replication servers and services were required to ensure high availability of the data moving forward.
Posted Sep 25, 2018 - 23:13 MDT
Verifying
Verification is currently underway for the maintenance items.
Posted Sep 25, 2018 - 22:44 MDT
Update
Teem is continuing the maintenance work and will provide additional updates when available.
Posted Sep 25, 2018 - 22:04 MDT
In progress
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Sep 25, 2018 - 21:00 MDT
Scheduled
We need to shedule emergency database maintenance that will cause disruption to services at 9PM tonight (Tuesday, September 25th). We apologize for this inconvenience.
Posted Sep 25, 2018 - 20:35 MDT
This scheduled maintenance affected: Web Interface, Mobile Data, API, Google Apps Calendar, and Exchange Sync.