My Notes: Makemore - Character Level Language Model

It looks like you're working through a tutorial or exercise related to natural language processing (NLP) using PyTorch. Specifically, you are dealing with the process of generating text based on probabilities derived from character-level data.

Here's a summary and some additional insights for each step:

1-3. Preparing Data

You've already prepared your dataset by converting words into lists of characters and then into numerical indices using itos (index-to-string) mapping.

4. Generating Counts Matrix

The counts matrix (N) is constructed to store the frequency of transitions between characters in the text corpus. The shape (C+1, C) indicates that you have an additional row for start-of-word tokens and columns representing each character.

5-6. Handling Start Tokens

You've correctly handled special start tokens by adding a row and column to your counts matrix N. This ensures that transitions from the start token to any other character are captured.

7. Converting Counts to Probabilities

By normalizing the counts in each row, you convert them into probabilities. For example, the first row of p gives the probability distribution for which character is likely to be the starting character of a word

Read the full article at DEV Community

---

Want to create content about this topic? Use Nemati AI tools to generate articles, social posts, and more.