1. Home
  2. John Miano
  3. H-1B Opiate Has Thwarted Advancement in Software Engineering

H-1B Opiate Has Thwarted Advancement in Software Engineering

By John Miano on February 16, 2017

Over the weekend I did an interview with blogger Stefan Molyneux. Like myself, Stefan worked in software development. So we got to talk shop.

After that interview, I thought that I might try to explain the why of H-1B. That requires a look at the state of the software industry.

There is a great disconnect between the perceived state of software engineering and its actual state. We talk about "high tech", but in reality software engineering is in its infancy. Software engineering, as we know it, is only about 60 years old. Compare that to civil engineering where there are thousands of years of experience.

Software engineering is as mature as the civil engineering of two thousand years ago. In fact, software engineering is probably less advanced than that.

Think about the age of the cathedral builders. They did not have the advantage of calculus and formal structural analysis. Their master masons designed cathedrals by instinct — and in most cases they were successful. But there were disasters.

Beauvais Cathedral collapsed. So did Ely Cathedral and Lincoln Cathedral. But for the most part the medieval cathedral builders were successful.

And the medieval cathedral builders were positively advanced compared to today's software engineers. Software disasters today are much more common than cathedral disasters in the Middle Ages.

Here are some reports on the rate of software engineering failure. I have tried to pick out reports that reflect the full range of such studies.

The variation in the numbers here is the result of studies measuring different things; especially different measures of success or failure.

For any software project there is a range of outcomes. A project can be an unqualified success, an unqualified failure, or something in between. That in between includes projects that run way over budget, like the Obamacare website. Software projects that are completed but do not meet expectations or provide value also fit in this category

If we cut through the hype in these reports, we find that roughly a quarter of software projects are successful, a quarter are complete disasters, and about half fall somewhere in between.

To put this into perspective, this would be equivalent to a quarter of buildings erected falling down and another half requiring extensive reconstruction to be made usable. If just 1 percent of buildings collapsed there would be enormous pressure to take corrective action, but a quarter of software projects doing the same is accepted as normal. Any way you look at these figures, the current state of software engineering is primitive.

Whenever a disaster occurs, the human instinct is to cover it up. That usually is not possible in civil engineering where a building collapse produces a pile of debris.

When a software project fails, the only visible evidence is an accounting ledger that is much easier to conceal. When a software project fails, all involved, from the executives who cooked up the idea to the consulting companies that built it, try to cover up the disaster. Because of this ability to conceal, there has not been strong effort to prevent software disasters from occurring.

We can distill all these software engineering failures into a simple business problem: Software costs too much.

If you wanted to address that cost problem in a scientific way you would:

  • Use development tools that maximize productivity;
  • Set up physical work environments to maximize productivity;
  • Ensure developers have the right equipment to do their jobs most efficiently;
  • Put in place personnel policies to reward and retain the most productive developers; and
  • Put in place software management policies to ensure systems are properly designed and to prevent scope creep.

That kind of change requires knowledge of the software development process. These are issues that your average accountant has no conceptual understanding of.

But accountants make high-level decisions in corporations.

When you put the software cost problem in the hand of accountants, they understand it as: American computer programmer cost $90 an hour and Indian computer programmers cost $80 an hour.

The obvious flaw in that accountant analysis is that in the best case it simply makes disasters cost less.

I shared with Stefan an experience I had before law school. I joined a project written in C++ that had serious performance problems. The original developers were not familiar with table joins in database queries. They wrote the system so that it used millions of nested queries where one query would suffice. I was in the process of replacing these millions of queries with single queries that obviously boosted performance tremendously.

While working on this issue, a senior accountant-type in the company (without consulting anyone familiar with the structure of the system) hired an Indian body shop to rewrite the entire system in Java to solve this problem. Their legion of H-1B programmers slavishly copied the C++ code into Java, including the parts that used millions of queries where one would suffice. The result was even slower than the original — another software disaster.

It is this fundamental problem of software cost that is driving the demand for H-1B workers. Unfortunately, cheap labor on H-1B became a distraction from fixing the underlying problems in software engineering. The H-1B opiate has thwarted the advancement of software engineering and undermines American leadership in this area.