Monday, June 13, 2005

Almost live, from Sunday Bay, Part 1

After a good two weeks of struggle (oh my gosh, has it been two weeks already?), situation in Sunny Bay finally settled down a little, and we were able to relax.

Our trusty developer from Beijing who came in just a couple days before station launch, solved a huge list of memory leak problem with the software right before the launch on 1st June. Granted that memory leak problems are more often than not caused by carelessness while coding, he did an amazing job fixing what's broken. We also added a watchdog program both PA Server and DVAS Server so that they can restart themselves when there's trouble. So everything looks nice and shinny and full of hope on the day of launch.

On the day of launch, all kinds of random requests got pushed through to our system. Some we ignore without problem, as per specifications. Others belong to a camp of tricks performed by the client sides that aren't supposed to happen, and very often bring our system to a crawl if not completely down.

We'll talk about some of the fun (translation: rediculous) things we see while we debug the system in my up-coming posts.

My apologies for this weeks of silence. I'd been on 12 hours worknight and on-call basis. My life basically revolves around going on site to debug, going to meetings to explain the problems, and writing reports. No, not fun at all.

No comments: