Backup Data Center (DC2) Connectivity Issues
Incident Report for Solve Networks
Resolved
Connectivity has been fully restored at Data Center 2 and geographic redundancy is active for MNO Tunnel infrastructure. All services operating normally.
Posted Mar 19, 2021 - 15:10 CDT
Update
Mobile Network Operator backup tunnels migrated to DC1, prepared for manual failover scenario if need arises, while DC2 connectivity issue fix completed.
Posted Mar 19, 2021 - 09:41 CDT
Monitoring
CCM Databases, Data Analytics, and Customer VPN Gateway are operating normally. DC2 Mobile Network Operator backup tunnels inoperational until ISP Circuit fully resolved at DC2.
Posted Mar 18, 2021 - 13:27 CDT
Update
CCM Databases and Data Analytics may be slow to respond and delayed in updating.
Posted Mar 18, 2021 - 08:30 CDT
Update
Work completed to replicate Customer VPN Gateway at Data Center 1.
Posted Mar 18, 2021 - 00:00 CDT
Identified
The issue has been identified and a fix is being implemented.
Posted Mar 17, 2021 - 23:58 CDT
Update
We are continuing to investigate this issue.
Posted Mar 17, 2021 - 21:31 CDT
Investigating
Connectivity issues affecting Data Center 2 are impacting Customer VPN Gateway as well as backup tunnels between Solve Networks infrastructure and Mobile Network Operators. Primary Tunnels to MNOs are operational at this time. Internet Access of Enhanced SIMs is also operational.
Posted Mar 17, 2021 - 20:09 CDT
This incident affected: Verizon (Verizon Enhanced Network Infrastructure), AT&T (AT&T Enhanced Network Infrastructure), and Solve Networks (Clarity Connectivity Management (ccm.solvenetworks.com), IPsec VPN Service).