Accountability with AI

This post is a public version of my comments submitted to the US government’s request for information on artificial intelligence. As such, it’s primarily plain text and contains little formatting, as it was submitted as plain text. The examples are simplified to be as comprehensive to the layman as possible.

For some reason, the government makes the request for comments inordinately hard to find. Tarah Wheeler has an excellent post on how hard it is to find here.

Comments to the Office of Science and Technology Policy

My comment is specifically about safety measures that must be taken in the development of AI technology to ensure that people’s rights and safety are protected in an equitable manner.

If no other part of this comment is read, it’s imperative that this one thing be communicated: Computers cannot be held accountable, and therefore must not make decisions unsupervised.

Due to complicated issues involving the nature of problem-solving computer programs, computers may come to conclusions that seem reasonable to them but are unacceptable or dangerous to people. To prevent them from automatically acting upon these conclusions, it’s imperative that a person review and approve suggested actions provided by computer assistance before they are enacted.

Consider the example of a computer system assigning dorm rooms at a college. Imagine that it evaluates how well matched people are to their preferred rooms using a score, and will select the highest scoring arrangement. If one person is in a room that they prefer, the arrangement gets 1 point. We wouldn’t want someone in a wheelchair to be in an upstairs room that they couldn’t get to, so we might assign this situation -1000 points. Unfortunately, this would let the computer select a situation where 1002 people get their preferred rooms and 1 person in a wheelchair gets an inaccessible room, over a situation where 1 person gets the room they want, and everyone else gets an acceptable but not preferred room. If the computer had no oversight, it might deem this an acceptable solution, and nobody would realize until a disabled person tried to move into their second floor room on the first day of school.

Additionally, there remains the possibility of purposeful accountability dodging. Humans must be responsible for the decisions of computers they create and run. Otherwise, unscrupulous parties could outsource any unethical decision to a computer, and completely avoid liability for any damage caused. For this reason, it’s of paramount importance that usage of AI tools not diminish one’s culpability (legal or otherwise) in making poor decisions.