Why healthcare.gov Crashed */?> Why healthcare.gov Crashed

Posted by · November 13, 2013 10:49 am

healthcare-obamacareBy now, most of us have heard about the problems facing people trying to use healthcare.gov, the US government’s website created to allow people to get health insurance. It’s bad enough that while 4.7 million people visited the site on its first day, only six people were able to successfully enroll on the first day, and now the Obama administration is calling in the big guns to try and get the site fixed by the end of November.

But last month, when beavoid asked a straight question about the failings of the site, it was Ray_Paseur who posted one of the most concise and complete descriptions of the problems.

Actually, I think Avik Roy’s Forbes Article was exactly on point. The implementation team knew last winter that this rollout was going to be a disaster.

If you want to see some of the things wrong with the healthcare.gov, open a private browsing window and do “view source.”

Then look at the output of the validator.

Then try to envision assembling a staff of software development professionals who understood the risks of passing personal data around in clear text through JavaScript, who understood the APIs of 50 different state health care programs, and who understood how to build to scale.

Add to that a system that has, literally, nobody in charge. Congress wants to hold hearings to find out what went wrong and they can’t even figure out who to subpoena. The initial bids were $90MM. The as-built cost at the opening failure was $580MM. The contractors who received the money say it’s the Government’s fault. Secretary Sebelius says she didn’t tell the President it was not working — he had to find out just like everyone else. The program is totally rudderless.

Anyone who has experience in large-scale systems development has been required to read Fred Brooks seminal treatise, The Mythical Man Month. Among other things he advises that “adding manpower to a late software project makes it later” and “plan to throw one away.” This latter point is where most of the Obamacare software failures are originating. What Brooks teaches us is that even the most well-reasoned and thoughtfully designed software will not work right at its first implementation, and that designs will, of necessity, change when the software is first built. It’s not a question of whether you will need to throw the first build away, because you will have to throw it away. The only question is whether you will deliver it to the customers and make them throw it away. The site suffers from both accidental complexity and essential complexity. Oh, by the way, there was a design criterion that the site was required to keep the rates secret from the clients until it had gathered all of their personal information.

Was there ever an Alpha Release of healthcare.gov? Was there ever a private Beta? Was there ever a public Beta? Trying to release a system like this without formal testing cycles is not a path to success.

The Government has spent more money on this web site than were spent to build Twitter and Facebook together. And nobody is going to be held accountable for the failures. The contractors will get paid no matter how bad the cost overruns. The Government employees who are party to the mess will still have their jobs and pensions, and the American public will be stuck with the bill.

Executive summary: Political leadership, but no technological leadership, a palpable lack of understanding of the principles of large-scale systems development, inadequate testing, a hard deadline. All that and I’ll bet if you peel the onion, somewhere in the inner layers you will find SELECT * with no LIMIT clause!

Here is the source code from the first private viewing of healthcare.gov — what does it tell you when you find variables assignments commented out in deployed code?

Finally, we were sent this story — and we have no doubts about its veracity — from one of EE’s members:

I was in Canada last week for a leadership retreat (current president of the association is based there) and it turns out that the brother of one of the attendees is a Vice President at CGI and he joined us for one of the casual dinners. So of course all of the people at the tabled badgered him and me into a conversation.

Me: So…how’s work?
Him: <glares>
Me: You know, I probably could have rolled out a better site using WordPress, Gravity Forms, and WP Engine for large scale hosting. I did the math…would have been a lot less money and the data would still have been accessible to the state exchanges.
Him: <examines remnants of drink, flags waiter for refill>
Me: Anyway, bonne chance!

**This post was originally published in the November 6, 2013 edition of the Experts Exchange Community Newsletter**