This incident has been resolved. Teem has been monitoring performance for several hours and will now close this incident.
At approximately 10:45 MDT internal systems began detecting an anomaly affecting the API. Triage began and the issue escalated over the next hour. Customers may have experienced intermittent access to the platform (devices, mobile and plugins) until 2:30 MDT when a temporary patch was deployed. Work continued to correct the root cause (a caching system that became unavailable cascaded to the web server pools causing them to drop offline, until the system was restored/rebuilt fully early this morning) through the monitoring period. All systems have been operational since approximately 2:30 MDT and full remediation work has been completed. Additional information on this incident will be provided at the completion of Teem's internal review process.
Posted Apr 14, 2020 - 12:18 MDT
A fix has been implemented and we are monitoring the results.
Posted Apr 14, 2020 - 03:41 MDT
We are currently investigating this issue.
Posted Apr 14, 2020 - 00:30 MDT
This incident affected: Web Interface, Mobile Data, API, EventBoard, and LobbyConnect.