What customers look for

I learnt something today. I’ve been involved in evaluating vendor reponses to an RFP for a small-ish bespoke system development. The functional spec and “detailed design” had already been completed for the system, so it was purely the build and deploy phase to be done. My role was evaluating the technical competence of the vendors' proposals.

So I read the proposals, I listened to their presentations, I asked probing questions. And I formed an opinion. Then it came time for the evaluation team to get together and compare notes.

For one of the vendors, I had significant concerns about their lead technical “architect”. He was clearly a cowboy, and had a sloppy approach (a small example: the proposal defined MVC as “multiple virtual storage”… wah??). He seemed to have a shallow understanding of J2EE architecture and fundamentals. When I raised this with the evaluation team, I was met with “but he responded calmly under the pressure of our questioning”.

The “business” people on the evaluation team (that is, the representatives of the system’s end users) were quite openly stating that they were less concerned with the vendor’s actual technical ability and depth than with their ability to “remain calm under pressure”. Sure, I can see that’s important, but what’s the use if the pressure grew out of their inability to actually do the work in the first place?

This is a triumph of form over function. The management of this large corporation were effectively saying “we care more that the vendor puts on a good front, and has someone to kick, than actually builds a well designed system”.

Reinforcing this was my second lesson. Part of the evaluation process involves going through a spreadsheet scoring the vendor’s response against requirements (functional and non). There are four sheets, one of which is for the technical element (the others are project management, commercial and vendor stability). The technical part was weighted at 10% of the total score! Again, management is saying “we care more that the vendor looks good and presents a good management face than whether they can actually deliver the system”.

I’m being perhaps a little unfair. As stated above, this project was purely for the build phase, the initial design being already complete. I suppose in this circumstance, project management does play a large role in determining success, on the assumption that all the big technical and architectural decisions have already been made.

But I think it’s an interesting lesson for those of us who care deeply about technical quality, right down to the level of implementation. I may care, and if you’re reading this perhaps you do too, but in the corporate world, it’s just not that important. What’s important there is making the right sounds and playing the right game.