KumpeApps LLC - BOT Server Down (should not affect most websites) – Incident details

All systems operational

BOT Server Down (should not affect most websites)

Resolved
Operational
Started 9 months agoLasted 2 days

Affected

Websites

Operational from 6:30 AM to 3:22 AM, Major outage from 3:22 AM to 3:14 PM, Operational from 3:14 PM to 3:14 PM, Major outage from 3:14 PM to 3:15 PM, Operational from 3:15 PM to 12:40 AM

Kumpe3D

Operational from 6:30 AM to 3:22 AM, Major outage from 3:22 AM to 3:14 PM, Operational from 3:14 PM to 3:14 PM, Major outage from 3:14 PM to 3:15 PM, Operational from 3:15 PM to 12:40 AM

Apps

Operational from 6:30 AM to 4:47 PM, Partial outage from 4:47 PM to 3:15 PM, Operational from 3:15 PM to 12:40 AM

Kumpe3D Kiosk

Operational from 6:30 AM to 4:47 PM, Partial outage from 4:47 PM to 3:15 PM, Operational from 3:15 PM to 12:40 AM

BOTs

Major outage from 6:30 AM to 12:40 AM

KumpeApps Credentials Manager

Major outage from 6:30 AM to 3:14 PM, Operational from 3:14 PM to 3:14 PM, Major outage from 3:14 PM to 3:15 PM, Partial outage from 3:15 PM to 12:40 AM

Updates
  • Resolved
    Resolved
    This incident has been resolved.
  • Update
    Update

    We have restored Kumpe3D systems and are in progress of restoring the other BOTS. Credentials manager is back up and running but we are still working on restoring it's data.

  • Update
    Update

    Hosting company has reinstalled our server causing 100% data loss. Backups have bene restord but credential data was evidently not in our backup. Credential data will have to be manually restored then service tokens will have to be re-issued to all BOTS. This will take some time to complete.

  • Identified
    Identified

    Our BOT server is currently down due to a failed migration by our host provider. We are working with our host provider for a swift resolution. This will only affect automated background systems and internal applications.

  • Investigating
    Investigating

    Our BOT Server is down and under investigation.