We apologise for any frustration we caused our customers due to our website being down last week. This problem was unfortunately the result of a series of consecutive mishaps and we acknowledge our part in these incidences.
The problem started on Wednesday afternoon when we experienced a firewall failure. In such an emergency, our provisioned response took place where we shut down our ports and transferred our service to our back-up production site. This unfortunately meant that services such as the Standard Bank homepage, Online Banking and messaging of the One Time Passwords were affected.
Between 14:40 and 15:45, customers were unable to access the site as we were working to bring up services on our back-up site. After 16:00, everything was restored on our side and all services could be used and accessed by our customers ... or so we thought.
On Wednesday afternoon a second problem occurred. We were unaware that Vodacom had made some changes to their exchange service providers. This meant that the routing to some services for Vodacom 3G users was affected and could not be properly accessed.
This problem was only brought to our attention on Thursday morning. We immediately brought together the relevant teams from Standard Bank, Vodacom and SAIX to get to the bottom of the issue. The complex problem was eventually fixed in the early afternoon.
"It's imperative for us that we not only protect the integrity of our systems and our customers’ information, but also ensure that we keep our users abreast of issues when we are aware of them. Due to the fact that the outage on Wednesday afternoon was unplanned, we couldn't be proactive in our communication, but we constantly briefed our call centre staff about the latest developments," says Itumeleng Monale, Director of Standard Bank’s Self-Service Banking.
Monale was in touch with major mainstream media throughout the morning. This included an interview live on radio with John Robbie on 702 just after 08:00. She was also interviewed by Moneyweb and ITWeb.
We will use this as a case study of how not to handle such problems should these problems arise in the future.
