The Killer Robot Case

Have you ever read the killer robot case? Well, just in case you don't, this is a good (and long) story about ethical scenario which occurs in a development of robot. In short, the programmer who wrote the codes for this robot, was mistakenly wrote a wrong formula which cause the robot to perform inappropriately and killed its operator. After a deep investigations held inside the company, it was found that the company also did some unethical actions which responsible direct/indirectly to the death of the operator.

Hmm, I guess you need to read it. It's a good case, believe me! You can read the story here. If you have other links for the same case, I would really thank you for sharing.

Now, the case has been assigned as a homework in one of training class that I attended. The training is about Software Quality, and one of the sub topic is about ethics related to Software Engineering. My writing below, was my answer for that homework. I shared it here, because I think it will be helpful for others who work on the same topic, not by giving them direct answer, but to better analyze the case so they can answer better than my answer. If you have worked on the same topic in the past and would like to share anything about this, I would really thank you for sharing it.

So, we were asked to identify the top 5 violation of code of ethics in this case, explain why and what should have been done instead.

And here is my answer.

Violation 1
The quality assurance division was made false claim that they said the software was successfully been tested and error free. They made a fake testing document which said that the. This violation is shown by Cindy Yardley who made fake test document as per request from Ray Johnson. Ray Johnson was also made fake claims that he used to convince Cindy Yardley, that the software is safe that no one will know the test document was a fake. This is violation for the following codes:

1. Code number 1.06 which says “Be fair and avoid deception in all statements, particularly public ones, concerning software or related documents, methods and tools”.
2. Code number 6.07 which says “Be accurate in stating the characteristics of software on which they work, avoiding not only false claims but also claims that might reasonably be supposed to be speculative, vacuous, deceptive, misleading, or doubtful”.
3. Code number 6.08 which says “Take responsibility for detecting, correcting, and reporting errors in software and associated documents on which they work”.

Both Cindy Yardley and Ray Johnson should never make deceptive claim regarding the application, since it may lead to serious problem (cause death to the operator).

Violation 2
People in Techtronics never inform the operator that the software has exceptional conditions which were clearly stated in the requirements document, as stated by Ruth Whitherspoon that there are exceptional conditions written in the document section 5.2.4.26. This is violation for code number 1.04 which says “Disclose to appropriate persons or authorities any actual or potential danger to the user, the public, or the environment, that they reasonably believe to be associated with software or related documents”. According to this code, the people from Techtronics should warn the operator prior to operate the robot, and even give them appropriate training about how to act when such exceptional conditions occur.

Violation 3
Some of people involved in the project had no appropriate experience and educational backgrounds. This is a violation for the following codes:

1. Code number 2.01 which says “Provide service in their areas of competence, being honest and forthright about any limitations of their experience and education”.
2. Code number 3.04 which says “Ensure that they are qualified for any project on which they work or propose to work by an appropriate combination of education and training, and experience”.
3. Code number 5.04 which says “Assign work only after taking into account appropriate contributions of education and experience tempered with a desire to further that education and experience”.
   

The violations were shown by the following acts:

1. Randy Samuels has no educational background in mathematics, statistics and physics. Knowledge on those areas is needed to be able to judge that the business rules, formulas, and logics are correctly implemented within the application. It is true that Randy has experience in programming, but it doesn’t make Randy the appropriate person to judge that the code that he wrote has correct logic or not.
2. Ray Johnson who leads the Robotic Division had no sufficient experience in software development, since his background is from manufacturing. Therefore his decision was inappropriate for example when he insisted to add developers to speed up the project which is based on his experience in manufacturing field
3. The decision that has been made by the management to place Sam Reynolds as the project manager was also violation of the codes above, since Sam has no experience in Robotic software. Instead, Sam was previously experienced in data processing which characteristics of the projects have differences from robotic software.

To avoid the violations, the executives or managements should place people who have expertise in area of robotic programming.

Violation 4
Ray Johnson was unethically order Cindy Yardley to create fake test document by using his power as Head of Robotics Division. This action was a violation of the code of ethic since Johnson was asking his engineer to do such action that was inconsistent with the code of ethic. This is a violation for the following codes:

1. Code number 5.11 which says “Not ask a software engineer to do anything inconsistent with this Code”.
2. Code number 8.08 which says “Not influence others to undertake any action that involves a breach of this Code”.

According to the codes above, Johnson should never use his power to influences his staffs to do any unethical actions. He should let his engineers to work professionally based on their expertise and knowledge.

Violation 5
The project was not developed based on a realistic schedule and estimations. Both Ray Johnson and Sam Reynolds realize that it was impossible to deliver the product perfectly within the time constraint provided by the executives (1 January). However, both Johnson and Reynolds were choosing to force their engineers to do the impossible project, instead of trying to convince the executives that their time constraint given to them was inappropriate. This is violation for the following codes:

1. Code number 3.01 which says “Strive for high quality, acceptable cost and a reasonable schedule, ensuring significant tradeoffs are clear to and accepted by the employer and the client, and are available for consideration by the user and the public”.
2. Code number 5.05 which says “Ensure realistic quantitative estimates of cost, scheduling, personnel, quality and outcomes on any project on which they work or propose to work, and provide an uncertainty assessment of these estimates”.