source('code/mlc_vs_opm.R')4 Connection with machine learning and AI
4.1 Inferences with machine-learning algorithms
Some works in machine learning focus on “guessing the correct answer”, and this focus is reflected in the way their machine-learning algorithms – especially classifiers – are trained and used.
In § 2.3 we emphasized that “guessing successfully” can be a misleading goal, however, because it can lead us away from guessing optimally. We shall now see two simple but concrete examples of this.
A “max-success” classifier vs an optimal classifier
We shall compare the results obtained in some numerical simulations by using
- a Machine-Learning Classifier trained to do most successful guesses
- a prototype “Optimal Predictor Machine” trained to make the optimal decision
For the moment we treat both as “black boxes”, that is, we don’t study yet how they’re calculating their outputs (although you may already have a good guess at how the Optimal Predictor Machine works).
Their operation is implemented in this R script that we now load:
This script simply defines the function hitsvsgain():
hitsvsgain(
ntrials,
chooseAtrueA,
chooseAtrueB,
chooseBtrueB,
chooseBtrueA,
probsA
)having six arguments:
ntrials: how many simulations of guesses to makechooseAtrueA: utility gained by guessingAwhen the successful guess is indeedAchooseAtrueB: utility gained by guessingAwhen the successful guess isBinsteadchooseBtrueB: utility gained by guessingBwhen the successful guess is indeedBchooseBtrueA: utility gained by guessingBwhen the successful guess isAinsteadprobsA: a tuple of probabilities (between0and1) to be used in the simulations (recycling it if necessary), for the successful guess beingA; the corresponding probabilities forBare therefore1-probsA. If this argument is omitted it defaults to0.5(not very interesting)
Example 1: electronic component
Let’s apply our two classifiers to the Accept or discard? problem of § 1. We call A the alternative in which the element won’t fail before one year, and should therefore be accepted if this alternative were known at the time of the decision. We call B the alternative in which the element will fail within a year, and should therefore be discarded if this alternative were known at the time of the decision. Remember that the crucial point here is that the classifiers don’t have this information at the moment of making the decision.
We simulate this decision for 100 000 components (“trials”), assuming that the probabilities of failure can be 0.05, 0.20, 0.80, 0.95. The values of the arguments should be clear:
hitsvsgain(
ntrials = 100000,
chooseAtrueA = +1,
chooseAtrueB = -11,
chooseBtrueB = 0,
chooseBtrueA = 0,
probsA = c(0.05, 0.20, 0.80, 0.95)
)
Trials: 100000
Machine-Learning Classifier: successes 87649 ( 87.6 %) | total gain -23353
Optimal Predictor Machine: successes 72604 ( 72.6 %) | total gain 11162Note how the machine-learning classifier is the one that makes most successful guesses (around 88%), and yet it leads to a net loss! If the utility were in kroner, this classifier would cause the company producing the components a net loss of more than 20 000 kr.
The optimal predictor machine, on the other hand, makes fewer successful guesses overall (around 72%), and yet it leads to a net gain! It would earn the company a net gain of around 10 000 kr.
Example 2: find Aladdin! (image recognition)
A typical use of machine-learning classifiers is for image recognition: for instance, the classifier guesses whether a particular subject is present in the image or not.
Intuitively one may think that “guessing successfully” should be the best goal here. But exceptions to this may be more common than one thinks. Consider the following scenario:
Bianca has a computer folder with 10 000 photos. Some of these include her beloved cat Aladdin, who sadly passed away recently. She would like to select all photos that include Aladdin and save them in a separate “Aladdin” folder. Doing this by hand would take too long, if at all possible; so Bianca wants to employ a machine-learning classifier.
For Bianca it’s important that no photo with Aladdin goes missing, so she would be very sad if any photo with him weren’t correctly recognized; on the other hand she doesn’t mind if some photos without him end up in the “Aladdin” folder – she can delete them herself afterwards.
Let’s apply and compare our two classifiers to this image-recognition problem, using again the hitsvsgain() function. We call A the case where Aladdin is present in a photo, and B where he isn’t. To reflect Bianca’s preferences, let’s use these “emotional utilities”:
chooseAisA = +2: Aladdin is correctly recognizedchooseBisA = -2: Aladdin is not recognized and photo goes missingchooseBisB = +1: absence of Aladding is correctly recognizedchooseAisB = -1: photo without Aladding end up in “Aladding” folder
and let’s say that the photos may have probabilities 0.3, 0.4, 0.6, 0.7 of including Aladding:
hitsvsgain(
ntrials = 10000,
chooseAtrueA = +2,
chooseAtrueB = -1,
chooseBtrueB = 1,
chooseBtrueA = -2,
probsA = c(0.3, 0.4, 0.6, 0.7)
)
Trials: 10000
Machine-Learning Classifier: successes 6487 ( 64.9 %) | total gain 4462
Optimal Predictor Machine: successes 5930 ( 59.3 %) | total gain 5304Again we see that the machine-learning classifier makes more successful guesses than the optimal predictor machine, but the latter yields a higher “emotional utility”.
You may sensibly object that this result could depend on the peculiar utilities or probabilities chosen for this example. The next exercise helps answering your objection.
4.2 What is “Artificial Intelligence”?
“AI” as opposed to what?
The field of Artificial Intelligence is vast, and its boundaries are not clear-cut. Different books give slightly different definitions of AI. In everyday parlance the term “AI” is moreover used in ways that are not technically correct – a bit like it happens with physics terms such as “energy” or “force”. In this course we want to use AI in a technically more correct way.
The discussion of the possible definitions of AI could take several chapters. Let’s try a shorter approach, by examining why the two words “artificial” and “intelligence” are used specifically.
Artificial as opposed to what? As opposed to natural for example. So it denotes something human-made, as opposed to something directly found in nature; say in an orangutan or in a dolphin.
Intelligence as opposed to what? As opposed to stupidity. The definition of “intelligence” itself, even natural intelligence, is still quite open. Generally we mean something that is logical or rational. Thus an agent that breaks some logical procedure, or that does not follow a procedure that it claims to follow, is not “intelligent”.
Of course neither term is fully dichotomous: we can distinguish different degrees of artificiality and of intelligence.
“Intelligence” is not “human-likeness”
We can distinguish two distinct endeavours in the field of Artificial Intelligence, considered in its most general extension:
- achieving human-like behaviour;
- achieving intelligent reasoning, or we could say logical or rational reasoning.
It’s important to recognize immediately that these two endeavours may not be mutually compatible. We often associate human behaviour with error-making and irrationality. We may say that a person is very irrational, yet we don’t say that because of this the person is inhuman.
Given the incompatible character of the two endeavours above, we must be very clear and conscious about which goal we’re trying to achieve; otherwise we won’t achieve any goal at all. And in technical discussions we must be careful to adopt the correct terminology. In particular we should avoid the term “intelligent” when we instead mean “human-like”, and vice versa.
An example of such confusion is with present-day large language models (LLMs), and in particular those with a Generative Pre-training Transformer (GPT) architecture. In many media they are referred to as “AI systems”; yet what they achieve is not intelligence, but rather human-like language processing – including non-intelligent processing.
If you have access to a large language model, you have surely witnessed examples of stupid output1. You can try a variation of the following experiment:
1 often euphemistically called “hallucination” because this term may increase sales, whereas “stupid” would risk decreasing sales.
- Ask the LLM to write down a short list of some set, for instance of all Norwegian counties.
- Ask the LLM to select from the list only those item that have one or more letter “r” in their name. See the result.
- Ask the LLM to give you a step-by-step procedure to achieve the selection required in the previous step.
Typically a LLM fails at task 2., even if it can give a completely sound procedure in task 3. Clearly it isn’t internally following the logical procedure.
This is the reason why in this course we do not categorize LLMs as “artificial intelligence”, but rather as human-mimicking machines. But we shall consider possible ways in which a true intelligence framework could be built into these machines.