Jim Delurgio says he’s bracing for the worst.
The chief information officer at Irvine-based Buyproduce once considered the threat of rolling blackouts “media hype”,until a wave hit Orange County a few weeks back.
Now he knows the story is real. And he’s taking no chances this summer, when power officials project more than a month’s worth of rolling blackout days.
“Everything here is set to default over to redundant power,” Delurgio said. “We expect to have several blackouts this summer and to keep working through them. Our expectation is not to have to send employees home.”
Buyproduce, which offers electronic business services to food growers and shippers, isn’t alone when it comes to dot-coms planning ahead. Power outages are particularly acute for Internet companies: having a site go down for even an hour means lost sales. Analysts say that’s something many struggling dot-coms can’t afford.
“They would be losing an essential, and for some, a critical business channel,” said Ekaterina Walsh of Cambridge, Mass.-based market researcher Forrester Research Inc. “If they lose that, they not only lose customer confidence but a great revenue opportunity.”
Roger Piwonka, chief information officer at Irvine-based online mortgage brokerage LoanGenie.com said blackouts could potentially cost his business “thousands of dollars a minute.” Moreover, he said, a non-operating site “sends a message of unreliability.”
“That would impact reputation more than the monetary,” he said.
Like others, LoanGenie isn’t leaving things to chance. About a year and a half ago, the 3-year-old company tapped Irvine-based focusCORE Inc. to house and run its electronic operations at a local data center. The move virtually guarantees LoanGenie’s site will be up and running, even during rolling blackouts, Piwonka said. Like other data centers, focusCORE’s facility can run for days off the power grid.
Other OC Internet companies also have outsourced operations to local data centers. The service isn’t cheap,running LoanGenie $10,000 to $20,000 a month, Piwonka said. But he said it’s still less expensive than having an onsite generator, which can cost $200,000 and up.
LoanGenie’s corporate offices don’t have the same safety net. Instead, Piwonka said they rely on about 40 minutes of back-up power before things go dark, which is what happened a few weeks back when the electricity went down for up to two hours in some parts of Irvine.
“We’re gone if the (power outage) is over an hour,” Piwonka said. “And if the phone lines go down, we can’t control that. Although we would like to try.”
At Buyproduce, company officials have invested $12,000 on back-up battery power equipment to run their internal network for up to five hours in case of a power outage.
But Delurgio still wasn’t comfortable. He feared even back-up and detailed shut-down procedures wouldn’t be enough. So he turned to Santa Clara-based Exodus Communications Inc.’s Irvine data center. So have Aliso Viejo-based buy.com Inc. and Irvine-based Autobytel.com Inc.
Delurgio moved Buyproduce’s “mission critical” servers,used to run internal operations,to Exodus’ facility. With its multiple back-up generators, the Exodus data center can keep production humming for two weeks or more during power outages.
Without giving specific numbers, Delurgio called the move an “expensive” alternative,but worth it. The data center gives him “peace of mind” and buys him more time,an extra 45 minutes,to run his office network on battery power without having to worry about the Web site.
“We’ll take that any time,” Delurgio said.
Brent Rusick, senior vice president of operations and technology at buy.com, said Exodus hosts the company’s online store, basically ensuring “99.9% up time” during outages.
“From purely a power standpoint, that is probably the least of our concerns in terms of risk and stability to the site,” Rusick said. “We’re a 24-by-7 store, so we have to make sure that the lights are always on.”
Plus, buy.com installed a diesel generator at its corporate offices when it moved into the building a few years back. The equipment can flow back-up power to the business “as long as we have fuel,” Rusick said. n
