34
Example application: “small language model”
ADA511
0.4
2025-10-28
licence
Dear student
and aspiring data- & AI-engineer
Preface
An invitation
1
Accept or discard?
2
Framework
3
Basic decision problems
4
Connection with machine learning and AI
Inference I
Working with R, I
5
What is an inference?
6
Sentences
7
Truth inference
8
Probability inference
9
Shortcut rules
10
Monty Hall and related inference problems
11
Second connection with machine learning
Data I
12
Quantities and data types
13
Joint quantities and complex data types
Inference II
14
Probability distributions
Working with R, II
15
Joint probability distributions
Working with R, III
16
Marginal probability distributions
17
Conditional probability and learning
Learning and conditional probability: a summary
18
Information, relevance, independence, association
19
Third connection with machine learning
Data II
20
Populations and variates
21
Statistics
22
Subpopulations and conditional frequencies
23
Infinite populations and samples
Inference III
24
A categorization of inferences
25
Exchangeable beliefs
26
Inferences from frequencies
27
Inference about frequencies
28
Example of belief over frequencies: the Dirichlet-mixture distribution
29
Final inference formulae for exchangeable beliefs
A prototype Optimal Predictor Machine
30
A look behind
31
Implementing an OPM
32
Prototype code and workflow
33
Example application: adult-income task
34
Example application: “small language model”
Decision-making
35
Utilities
36
Making decisions
37
The prototype Optimal Predictor Machine makes decisions
Approximations: machine learning
38
Introduction to machine learning
39
Decision trees
40
Neural networks
41
Large Language Models
42
Decisions: limitations of present-day machine-learning algorithms
43
Evaluation practices and utilities
Conclusion
What next?
Further reading
Thanks
References
34
Example application: “small language model”
Published
2025-10-28
\(\DeclarePairedDelimiter{\set}{\{}{\}}\)
33
Example application: adult-income task
35
Utilities