Understand how AI works with a tiny model

Every programmer’s first program is hello world. It is the smallest thing you can run that proves the whole setup works. So think of this as the hello world of AI. A model so tiny you can read the whole thing on one screen, that still answers a question.

I asked it a simple question, who is the president of france, and it answered macron. Then I changed one number in the code and the same question answered modi. That one number is the thing everyone keeps calling a weight.

No training here, and no library. I set the weights by hand so you can see them. Run on .NET 8.

dotnet new console

The model is three small things

First the question, already split into words. Then the answers the model is allowed to give. Then the weights, which say how much each word votes for each answer.

string[] question = { "who", "is", "the", "president", "of", "france" };
string[] answers = { "macron", "modi", "trump" };

var weights = new Dictionary<string, Dictionary<string, double>>
{
    ["france"]    = new() { ["macron"] = 5.0, ["modi"] = 0.0, ["trump"] = 0.0 },
    ["india"]     = new() { ["macron"] = 0.0, ["modi"] = 5.0, ["trump"] = 0.0 },
    ["america"]   = new() { ["macron"] = 0.0, ["modi"] = 0.0, ["trump"] = 5.0 },
    ["president"] = new() { ["macron"] = 1.0, ["modi"] = 1.0, ["trump"] = 1.0 },
};

Read the weights like a voting sheet. The word france gives 5 votes to macron and nothing to the others. The word president gives a small 1 vote to everyone, because it does not point to any one country. A big number is a strong opinion.

How it picks the answer

For each answer we walk through the words in the question and add up that word’s votes. The answer with the highest total wins. That is the whole model.

string best = answers[0];
double bestScore = double.MinValue;
foreach (string answer in answers)
{
    double score = 0.0;
    foreach (string word in question)
        if (weights.ContainsKey(word) && weights[word].ContainsKey(answer))
            score += weights[word];
    Console.WriteLine($"  {answer,-8} score = {score}");
    if (score > bestScore) { bestScore = score; best = answer; }
}
Console.WriteLine($"answer  : {best}");

Run it

dotnet run -c Release

question: who is the president of france

scores (add up each word's weight for that answer):
  macron   score = 6
  modi     score = 1
  trump    score = 1

answer  : macron

Notice the score 6 is nowhere in the code. It is not written down. The program builds it while running. You can follow it word by word.

Question word	Weight for macron	Running score
who	not in the weights	0
is	not in the weights	0
the	not in the weights	0
president	1.0	1
of	not in the weights	1
france	5.0	6

So the score starts at 0, president adds 1, france adds 5, and it ends at 6. The other two answers only got the small 1 vote from president, so they sat at 1. Macron had the highest total.

Change one weight and the answer moves

Make the word france vote for modi instead. Change its line to this.

["france"] = new() { ["macron"] = 0.0, ["modi"] = 5.0, ["trump"] = 0.0 },

Run it again.

question: who is the president of france

scores (add up each word's weight for that answer):
  macron   score = 1
  modi     score = 6
  trump    score = 1

answer  : modi

Same question, one number changed, different answer. The 5 votes moved from macron to modi, so now modi ends at 6.

Change the question instead

Put france back to macron, and change the question to ask about america.

string[] question = { "who", "is", "the", "president", "of", "america" };

question: who is the president of america

scores (add up each word's weight for that answer):
  macron   score = 1
  modi     score = 1
  trump    score = 6

answer  : trump

Now the word america is in the question, and america gives its 5 votes to trump. The words france and india are still in the weights, but they are not in this question, so their votes never got counted.

What this is really showing you

In a real AI model these weights are not typed by hand. They are learned during training, when the model reads a huge pile of web pages and books and slowly sets millions of these numbers until the right answers score highest. There are billions of them, and after training they are frozen. But the machine underneath is the same as what you just ran. Words go in, each one adds its weight, the highest total is the answer.

Change a few numbers in this file and you are doing by hand the exact thing training does on its own.