TalkTalk Hosted Voice Issues

Incident
July 05, 4:31pm BST

TalkTalk Hosted Voice Issues

Status: Closed
Start: June 29, 10:00am BST
End: July 05, 4:31pm BST
Duration: 6 days 6 hours 30 minutes
Affected Components:
Update

June 29, 10:00am BST

June 29, 10:00am BST

We have become aware of an issue affecting some users of the TalkTalk Hosted Voice service.

Mitel Users are unaffected at this time, so this issue seems specific to Generic Licence holders (Generic SIP).

So far we have ascertained that these users cannot make or receive calls, so this is a complete outage for these users.

TalkTalk are aware of the issue and have engineers working on the problem.

We apologise for any inconvenience caused.

We aim to keep you updated of any movement on this issue as we know more.

Update

June 29, 12:44pm BST

June 29, 12:44pm BST

Good Afternoon,

The TalkTalk team are investigating this issue further with the provider, Mitel.

Work is still being carried out to identify the root cause of this issue and apply a resolution, however rest assured this is being investigated at high priority by the TalkTalk Team, and they have advised a further update will be provided as soon as possible.

Update

June 29, 5:09pm BST

June 29, 5:09pm BST

Good Afternoon,

The TalkTalk Team are still investigating this further with Mitel, however they have had reports that affected customers are now coming back online, and are able to make/receive calls again.

We will aim to provide further updates as soon as possible.

Monitoring

June 29, 5:59pm BST

June 29, 5:59pm BST

Good Afternoon,

We have had a further update from the TalkTalk Team and the issue should now be resolved.

Please can you retest and if you are still experiencing issues, please get in touch with the Technical Support Team to investigate further.

We will keep this case in a monitoring state before closing.

Update

June 30, 3:19pm BST

June 30, 3:19pm BST

Good Afternoon,

We have had further correspondence from the TalkTalk Team and unfortunately the issue looks to still be present.

While some customers are reporting that they are able to make and receive calls, there are others instances in which this is failing, or have gone offline later on in the day.

The TalkTalk Team are still investigating this at a high priority with Mitel, and we aim to provide a further update as soon as possible.

Update

July 01, 9:34am BST

July 01, 9:34am BST

Good Morning,

After further investigation from the TalkTalk Team, affected SBC's have been removed from service, and work has been completed to rebuild the databases on each device. 

All SBC's have now been added back into the cluster and we can now see Mitel handsets are now connecting with successful calls being made. 

We will be keeping this open for monitoring purposes, and if you still encounter issues after retesting, please get in touch with the Technical Support team to investigate further.

Resolved

July 05, 4:31pm BST

July 05, 4:31pm BST

Good Afternoon,

After further investigation from the TalkTalk Team, the issue looks to now be resolved.

The Team advised there was a corruption in the synchronisation database that keeps all of the SBC nodes in the cluster in sync. The cluster was rebuilt and a full database synchronisation completed to resolve the problem.

If you are continuing to experience issues after reconfiguring the affected users, please raise a ticket to our Technical Support Team and we will investigate these further on a case by case basis.