29th May 2019

Web dashboard Elevated API error rates

**Update: ** log processing is now nominal. We will continue to watch performance over the next hours

--

**Update: ** we're currently processing logs but we're catching up with old work, we will continue to watch performance over the next hours

--

**Update: ** we're restarting the service

--

**Update: ** the database corruption has happened to be more severe than expected. Our high-availability plan has also failed, we're now restoring backups. We're sorry for the inconveniences caused.

--

Update: we just migrated to the new database server, logs are being processed in properly but everything's a bit slower than usual due to the extra work being handled

--

Update: the errors seem to come from a corrupted database index. We're failing over to a new database.

--

We have detected the API is returning an unusual number of errors. We're investigating the root cause.