Whatever the outcome, though, there are more immediate challenges to consider, challenges that need to be addressed today if AI technology is going to be harnessed in an ethical, responsible way. Four sets of issues stand out here: Data biases; Accountability and trust; AI usage; and AI safety.
If these areas of concern are attended to now, then it should be possible to safeguard ourselves from a number of future hazards, while hopefully minimising the risk of a net-based super-intelligence system wiping us all out.
Data Biases
The first major issue when thinking about AI and ethics is data. The majority of data produced is “artificial”. Take, for example, stock prices and smartphone activity. This kind of data is created by arbitrary human behaviour rather than, say, from observations of natural phenomena. And this means it automatically inherits the same cognitive biases as humans. (The psychological definition of a bias is instructive: it is a “systematic deviation from a standard of rationality or good judgment”).
So, if data biases stem from human cognitive biases, the obvious solution would be to remove any data that could bias an AI engine. And yet, even if that were possible, there are two significant problems to think about:
• We do not have a universal understanding of ethics.
• Even if we did, there is no guarantee that AI would not autonomously learn the same biases.
These two points draw attention to the difficulty of embedding human ethics into machine design. Given that we cannot agree upon what constitutes “good ethics”, it is therefore hard to know which set of values AI should have. It could also be argued that ethics is fluid, that the acceptance or rejection of certain ethics changes over time. But, supposing we could agree what ethics is, there is always the chance that AI would learn the same biases we possess. Considering all this could be useful; perhaps we will end up learning about our own ethics and value systems as the development of AI progresses.
Accountability and Trust
Although it might appear to be a purely philosophical issue, data bias relates to a set of practical questions: how far should algorithms be trusted? And who is to blame if AI gets it wrong?
Imagine a doctor who uses an algorithm to diagnose a patient. In 99.99% of cases, the computer gets it right — and, of course, it never gets tired, it analyses billions of records, and it sees patterns that a human eye cannot perceive. But what, if in the remaining 0.01% of cases, the doctor’s instinct tells them something different to the machine result (and the doctor’s instinct turns out to be correct)? What if the doctor then decides to follow the advice the machine recommends and the patient dies? In terms of accountability, this is a grey area. The most straightforward solution to understanding who is liable for a certain AI tool is thinking about the following three groups:
• AI systems
• Designers (although usually AI teams consist of hundreds of people and shifting liability on to this group could discourage many from entering the field);
• Organizations responsible for running the system
There is no easy answer to the questions surrounding accountability and liability but a good starting point is provided by Sorelle Friedler and Nicholas Diakopoulos. They suggest considering accountability through the lens of five core principles:
• Responsibility: a person should be identified to deal with unexpected outcomes, not in terms of legal responsibility but rather as a single point of contact;
• Explainability: a decision process should be explainable not technically but rather in an accessible form to anyone;
• Accuracy: garbage in, garbage out is likely to be the most common reason for the lack of accuracy in a model. The data and error sources need then to be identified, logged, and benchmarked;
• Auditability: third parties should be able to probe and review the behaviour of an algorithm;
• Fairness: algorithms should be evaluated for discriminatory effects.
Aside from liability issues, it is necessary to think about how we trust algorithms. It is understandable that a doctor who has studied for 12 years might be reluctant to hand over complete control to a machine. This reluctance is known as algorithm adversion and is emerging as a real problem for algorithm-assisted tasks. Recent studies suggest that those who use algorithms still prefer to retain some degree of control over the machine. This leads to another area of contention: the alignment problem. If we only ever want AI technology to share our same goals and behaviours, then what is the point of having it at all?
AI Usage and the Control Problem
Everything we have discussed so far has been based on two assumptions we have not yet considered. Firstly, that everyone is going to benefit from AI and, secondly, that everyone will be able to use it. However, these assumptions might prove false. Many of us will indirectly benefit from AI applications (e.g., in medicine, manufacturing, etc.) but we might live in a society in which only a few big companies drive the AI supply and offer fully-functional AI services, which might not be affordable to everyone (to say nothing about their impartiality). What this comes down to is a policy choice between AI democratisation and Centralised AI. This is a serious policy issue, which needs to be sorted out today. Both choices contain their respective benefits and risks:
• AI democratisation increases the benefits and the rate of development of the technology but carries with it the risks associated with malicious usage and/or system collapse. And how should it be regulated?
• Centralised AI, on the other hand, might appear as a safer option but the chances of it remaining unbiased are slim. It poses another problem, too. If it is centralised, who will control AI?
We might then need a new impartial organization to decide how and when to use AI, but history has demonstrated we are not often well-equipped when forming impartial institutional groups—especially when the stakes are so high. With regulation in mind, the regulatory powers should be strict enough to deal with cases like AI-to-AI conflicts (for example, when two AIs made by two different companies conflict and give different outcomes) or deciding the ethical use of a certain tool—but not so strict to prevent research and development or full access to everyone.
AI Safety
As soon as AI becomes a commodity, it will be used maliciously. No matter the risk we face, it appears that AI will be dominated by some sort of exponential chaotic underlying structure and getting the minor things wrong could in turn produce catastrophic consequences. This is why it is of primary importance to understand every minor nuance and to solve them without underestimating any potential risk.
Amodei et al. (2016) draft a set of five core problems in AI safety, which is worth citing:
• Avoiding negative side effects;
• Avoiding reward hacking;
• Scalable oversight (respecting aspects of the objective that are too expensive to be frequently evaluated during training);
• Safe exploration (learning new strategies in a non-risky way);
• Robustness to distributional shift (can the machine adapt itself to different environments?).
This is a good categorization of AI risks but I’d like to add one more: the interaction risk or the way in which we interact with the machines. Our relationship with AI promises to be extremely fruitful but it comes with several risks. For instance, the so-called dependence threat, which would see humans relying too heavily on AI.
A final thought. Most of us advocate full transparency of methods, data and algorithms used in the decision-making process. But full transparency carries with it the risk of manipulation. This is not necessarily in reference to cyber-attacks or other forms of criminal activity, but more generally to the idea that once the rules of the game are clear and the processes reproducible, it is easier for anyone to hack the game itself.
