Sunday, October 13, 2024

The sheer stupidity of using the r’s in strawberry question as evidence of LLM incompetence

The meme of asking an LLM how many R's are in the word strawberry went viral because of public ignorance about how LLMs work. An LLM doesn't even see the individual letters in a word. It takes its input in the form of word-sized chunks. As such, questions relating to individual letters, like asking it to count the number of letters or asking which words start with a specific letter, are uniquely difficult and completely unrelated to performance on other tasks in general.

If you think about it, the only way it can even infer strawberry has any r's at all is purely based on context clues from somewhere deep in its training set where someone may have spelled out the word "b-e-r-r-y" with dashes in some random blog or forum post. Therefore, the fact they even sometimes get these types of questions right should be considered nothing short of amazing.

3 comments:

  1. NewsVersal provides verified updates, essential stories, and breaking headlines across politics and technology, offering readers clarity, insight, and understanding while staying informed with dependable and timely news

    ReplyDelete
  2. NewsIfe provides verified updates, key headlines, and breaking stories across technology, business, and politics, ensuring readers experience a sense of calm, knowledge, and trust while staying fully informed daily.

    ReplyDelete
  3. NewsWeen delivers verified headlines, key updates, and breaking stories across technology, politics, and business, giving readers insight, confidence, and understanding while staying informed with accurate and dependable reporting

    ReplyDelete