Welcome back, future AI architect! In our journey with any-llm, we’ve explored how to interact with various Large Language Models (LLMs) to generate text and understand their reasoning capabilities. Today, we’re taking a step back to dive into a fundamental concept that underpins many advanced AI applications: embeddings.

This chapter will demystify embeddings, explaining what they are, why they’re incredibly useful, and how any-llm provides a unified, straightforward way to generate them from different providers. We’ll move from theoretical understanding to practical application, showing you how to generate embeddings and use them for powerful tasks like semantic similarity. Get ready to transform text into numerical representations that unlock new dimensions of understanding!

Before we start, make sure you’re comfortable with basic any-llm setup and making simple LLM calls, as covered in previous chapters. We’ll build on that foundation to explore this exciting new capability.

Core Concepts: Understanding Embeddings

Imagine trying to explain the meaning of words or sentences to a computer. It’s tough, right? Computers are great with numbers, but human language is rich, nuanced, and full of context. This is where embeddings come to the rescue!

What are Embeddings?

At their core, embeddings are numerical representations of text. Think of them as high-dimensional vectors (lists of numbers) where each number captures a different semantic aspect of the original text. When words or phrases have similar meanings, their corresponding embedding vectors will be “close” to each other in this high-dimensional space. Conversely, unrelated texts will have vectors that are “far apart.”

For example, the word “king” and “queen” might have vectors that are close, while “king” and “table” would be much further apart. Even more powerfully, the relationship between “king” and “queen” might be similar to the relationship between “man” and “woman” in this vector space. That’s pretty cool, isn’t it?

Why are Embeddings Important?

Embeddings are the secret sauce behind many intelligent features you interact with daily. Here’s why they’re so crucial:

  1. Semantic Search: Instead of keyword matching, you can search for concepts. If you query “recipes for healthy breakfast,” an embedding-powered search could find documents about “nutritious morning meals” even if the exact keywords aren’t present.
  2. Information Retrieval (RAG): In Retrieval Augmented Generation (RAG) systems, embeddings help LLMs “look up” relevant information from a vast knowledge base before generating a response, drastically improving accuracy and reducing hallucinations.
  3. Clustering and Classification: Grouping similar documents or categorizing text based on its content becomes much easier when you can compare their numerical embeddings.
  4. Recommendation Systems: Suggesting similar items or content based on user interactions or item descriptions.

Essentially, embeddings allow computers to understand the meaning of text, not just its characters. This opens up a world of possibilities for building smarter, more context-aware AI applications.

How LLMs Generate Embeddings

Embedding models, often specialized neural networks, are trained on massive amounts of text data. During training, they learn to map words, sentences, or even entire documents into these dense vector representations. The magic is that the model learns to place semantically similar texts close together in the vector space.

Different embedding models (e.g., from OpenAI, Mistral, or local models like those available via Ollama) will generate embeddings with different dimensions (lengths of the vector) and in slightly different “spaces.” However, any-llm helps you switch between these providers with ease, maintaining a consistent API for your application logic.

Let’s visualize this conceptual flow:

flowchart LR A["Your Text Input"] --> B{"Embedding Model"} B --> C["Numerical Vector (Embedding)"] C --> D["Vector Database / Application Logic"]

In this simple flowchart, your raw text goes into an Embedding Model, which transforms it into a numerical vector. This vector can then be stored in a vector database or used directly by your application for various tasks.

any-llm and Embeddings: A Unified Approach

Just like any-llm provides a single interface for LLM completions, it does the same for generating embeddings. This means you can switch between different embedding providers (like OpenAI, Mistral, or local Ollama models) without changing your core code. This flexibility is a game-changer for experimenting with different models or deploying to various environments.

The any_llm.embedding module is your gateway to this functionality. Let’s see it in action!

Step-by-Step Implementation: Generating Your First Embedding

First, ensure you have any-llm-sdk installed with the necessary provider extras. For this example, we’ll assume you’ve installed it with support for a cloud provider like OpenAI or a local provider like Ollama.

# As of 2025-12-30, ensure you have the latest stable version.
# For example, to install with OpenAI and Ollama support:
pip install 'any-llm-sdk[openai,ollama]' --upgrade

Prerequisites: If you’re using a cloud provider like OpenAI or Mistral, make sure your API key is set as an environment variable (e.g., OPENAI_API_KEY or MISTRAL_API_KEY). For Ollama, ensure it’s running locally and you’ve pulled a model like nomic-embed-text.

# Example for OpenAI
export OPENAI_API_KEY="your_openai_api_key_here"

# Example for Ollama (if not already running)
ollama pull nomic-embed-text

Now, let’s write some Python code to generate an embedding.

1. Importing the embedding module

Open your Python editor and start by importing the embedding module from any_llm.

# embeddings_example.py
import os
from any_llm import embedding
from any_llm.schemas import EmbeddingProvider # We'll use this to specify the provider

Here, we import os to check environment variables, embedding for the core functionality, and EmbeddingProvider to explicitly select our provider.

2. Configuring Your Embedding Provider and Model

Before generating an embedding, we need to tell any-llm which provider and model to use.

# embeddings_example.py (continued)
# ...
# Configure your provider.
# You can choose from EmbeddingProvider.OPENAI, EmbeddingProvider.MISTRAL, EmbeddingProvider.OLLAMA, etc.
# For this example, let's use OpenAI. If you prefer Ollama, change the provider and model.
chosen_provider = EmbeddingProvider.OPENAI

# Select an embedding model.
# For OpenAI, 'text-embedding-3-small' is a good, cost-effective choice as of late 2025.
# For Ollama, 'nomic-embed-text' is a popular option.
chosen_model = "text-embedding-3-small" # Or "nomic-embed-text" for Ollama

# Ensure the API key is set if using a cloud provider
if chosen_provider == EmbeddingProvider.OPENAI and not os.environ.get("OPENAI_API_KEY"):
    print("Error: OPENAI_API_KEY environment variable not set.")
    exit()

print(f"Using embedding provider: {chosen_provider.value} with model: {chosen_model}")

We define chosen_provider and chosen_model. It’s good practice to add a check for the API key if you’re using a cloud provider, preventing runtime errors.

3. Generating a Single Embedding

Now for the exciting part: generating an embedding for a simple piece of text.

# embeddings_example.py (continued)
# ...

text_to_embed = "The quick brown fox jumps over the lazy dog."

print(f"\nGenerating embedding for: '{text_to_embed}'")

try:
    # The create function takes the text, provider, and model
    embedding_response = embedding.create(
        text=text_to_embed,
        provider=chosen_provider,
        model=chosen_model
    )

    # The result is a list of embeddings. For a single text, it will be a list with one item.
    single_embedding = embedding_response.embeddings[0]

    print(f"Embedding generated successfully!")
    print(f"Embedding dimensions (length): {len(single_embedding.embedding)}")
    print(f"First 5 values of embedding: {single_embedding.embedding[:5]}...")
    # print(f"Full embedding: {single_embedding.embedding}") # Uncomment to see the full vector

except Exception as e:
    print(f"An error occurred during embedding generation: {e}")

The embedding.create() function is your workhorse. It takes the text, provider, and model as arguments. The response object contains a list of Embedding objects, each holding the actual numerical vector. We print its length and the first few values to get a sense of it.

Run this script! You should see output similar to this (values will differ):

Using embedding provider: openai with model: text-embedding-3-small

Generating embedding for: 'The quick brown fox jumps over the lazy dog.'
Embedding generated successfully!
Embedding dimensions (length): 1536
First 5 values of embedding: [-0.00761234, 0.00398765, -0.01234567, 0.00112233, -0.00556677]...

Congratulations, you’ve just converted human language into a machine-understandable vector!

4. Generating Embeddings for Multiple Texts (Batch)

For efficiency, any-llm also allows you to generate embeddings for a list of texts in a single call. This is often faster and more cost-effective for cloud providers.

# embeddings_example.py (continued)
# ...

texts_to_embed_batch = [
    "Artificial intelligence is transforming industries.",
    "Machine learning is a subset of AI.",
    "The future of technology is exciting and full of possibilities."
]

print(f"\nGenerating embeddings for multiple texts:")
for text in texts_to_embed_batch:
    print(f"- '{text}'")

try:
    batch_embedding_response = embedding.create(
        text=texts_to_embed_batch, # Pass a list of strings
        provider=chosen_provider,
        model=chosen_model
    )

    print(f"\nBatch embeddings generated successfully!")
    for i, emb_obj in enumerate(batch_embedding_response.embeddings):
        print(f"Text {i+1} embedding dimensions: {len(emb_obj.embedding)}")
        print(f"  First 3 values: {emb_obj.embedding[:3]}...")

except Exception as e:
    print(f"An error occurred during batch embedding generation: {e}")

Notice how text now accepts a list of strings. The embedding_response.embeddings will then contain an Embedding object for each input string, in the same order.

This batch processing capability is incredibly useful when you’re indexing a large number of documents for a semantic search engine or RAG system.

Practical Application: Semantic Similarity

Now that we can generate embeddings, let’s put them to work! A common and powerful use case is calculating semantic similarity. This allows us to find how “alike” two pieces of text are, based on their meaning rather than just shared words.

The most common way to measure similarity between two embedding vectors is using cosine similarity. It measures the cosine of the angle between two vectors. A cosine similarity of 1 means the vectors point in the exact same direction (perfect similarity), 0 means they are orthogonal (no similarity), and -1 means they point in opposite directions (perfect dissimilarity).

Let’s add a helper function for cosine similarity and then use it to find the most similar document to a query.

# embeddings_example.py (continued)
# ...
import numpy as np # We'll need NumPy for vector operations

# Helper function to calculate cosine similarity
def cosine_similarity(vec1, vec2):
    """Calculates the cosine similarity between two vectors."""
    dot_product = np.dot(vec1, vec2)
    norm_vec1 = np.linalg.norm(vec1)
    norm_vec2 = np.linalg.norm(vec2)
    if norm_vec1 == 0 or norm_vec2 == 0:
        return 0.0 # Handle cases where a vector might be all zeros
    return dot_product / (norm_vec1 * norm_vec2)

print("\n--- Semantic Similarity Example ---")

# Our query
query_text = "What is the capital of France?"

# Some candidate documents
documents = [
    "Paris is the capital and most populous city of France.",
    "The Eiffel Tower is a famous landmark in Paris.",
    "London is the capital of the United Kingdom.",
    "Artificial intelligence is a rapidly developing field."
]

print(f"\nQuery: '{query_text}'")
print("Candidate Documents:")
for i, doc in enumerate(documents):
    print(f"  [{i+1}] {doc}")

try:
    # 1. Generate embedding for the query
    query_embedding_response = embedding.create(
        text=query_text,
        provider=chosen_provider,
        model=chosen_model
    )
    query_vector = np.array(query_embedding_response.embeddings[0].embedding)

    # 2. Generate embeddings for all documents
    document_embeddings_response = embedding.create(
        text=documents,
        provider=chosen_provider,
        model=chosen_model
    )
    document_vectors = [np.array(emb.embedding) for emb in document_embeddings_response.embeddings]

    # 3. Calculate similarity and find the best match
    similarities = []
    for i, doc_vector in enumerate(document_vectors):
        sim = cosine_similarity(query_vector, doc_vector)
        similarities.append((sim, documents[i]))
        print(f"  Similarity with Doc {i+1}: {sim:.4f}")

    # Sort to find the most similar document
    most_similar_doc = max(similarities, key=lambda item: item[0])

    print(f"\nMost semantically similar document:")
    print(f"  '{most_similar_doc[1]}' (Similarity: {most_similar_doc[0]:.4f})")

except Exception as e:
    print(f"An error occurred during semantic similarity calculation: {e}")

Here, we added numpy for efficient vector operations. We define cosine_similarity function, then generate embeddings for a query_text and a list of documents. Finally, we calculate the similarity between the query and each document, identifying the best match.

Run this updated script. You should observe that the document “Paris is the capital and most populous city of France.” has the highest similarity score with the query “What is the capital of France?”. This demonstrates the power of embeddings in understanding context and meaning!

Mini-Challenge

You’ve successfully found the single most similar document. Now, let’s level up!

Challenge: Modify the semantic similarity example to find the top 2 most similar documents to the query_text. Print them out in descending order of similarity.

Hint: After calculating all similarities, you can sort the similarities list (which contains (similarity_score, document_text) tuples) and then slice it to get the top N results.

What to observe/learn: This exercise reinforces your understanding of working with lists of data and applying sorting logic, a common task in information retrieval.

Common Pitfalls & Troubleshooting

Working with embeddings, especially across different providers, can sometimes lead to small hurdles. Here are a few common issues and how to troubleshoot them:

  1. API Key / Environment Variable Not Set:

    • Symptom: AuthenticationError, KeyError, or NoneType object errors from cloud providers.
    • Fix: Double-check that OPENAI_API_KEY, MISTRAL_API_KEY, etc., are correctly set in your environment before running your Python script. Remember to restart your terminal or IDE if you’ve just set them.
    • Best Practice: Never hardcode API keys directly in your code. Always use environment variables or a secure configuration management system.

  2. Incorrect Model Name:

    • Symptom: InvalidModelError, NotFoundError, or similar messages indicating the model doesn’t exist or isn’t available for your chosen provider.
    • Fix: Verify the exact model name with the provider’s official documentation. For example, OpenAI’s embedding models have specific names like text-embedding-3-small. For Ollama, ensure you’ve pulled the model (ollama pull nomic-embed-text) and that its name matches what you’re using in your code.

  3. Ollama Server Not Running / Model Not Pulled:

    • Symptom: Connection errors (ConnectionRefusedError), ModelNotFound from any-llm when using EmbeddingProvider.OLLAMA.
    • Fix: Ensure the Ollama server is actively running on your machine. Also, confirm you’ve pulled the specific embedding model you’re trying to use (e.g., ollama pull nomic-embed-text). You can check available models with ollama list.

  4. Dimensionality Mismatch (Less Common with any-llm, but good to know):

    • Symptom: Errors in cosine_similarity or other vector operations if you try to compare embeddings generated by different models that produce different vector lengths (e.g., comparing a 1536-dimension vector with a 768-dimension vector).
    • Fix: Ensure that all embeddings you intend to compare or use together are generated by the same embedding model. any-llm helps manage this by making it easy to stick to one model for a given task.

Summary

Phew! You’ve just taken a significant leap into understanding one of the most powerful concepts in modern AI: embeddings.

Here’s what we covered:

  • What are Embeddings? Numerical vector representations of text that capture semantic meaning.
  • Why They Matter: Essential for semantic search, RAG, clustering, and classification, allowing computers to understand the meaning of language.
  • any-llm’s Unified Approach: How any-llm simplifies generating embeddings across various providers (OpenAI, Mistral, Ollama) with a consistent API using any_llm.embedding.create().
  • Practical Implementation: Step-by-step code examples for generating single and batch embeddings.
  • Semantic Similarity: Using cosine similarity to measure the conceptual closeness between texts, demonstrating a core application of embeddings.
  • Troubleshooting: Common issues like API key problems, incorrect model names, and Ollama setup.

You now have the tools to convert raw text into a rich numerical format, opening doors to building incredibly smart and context-aware applications. In the next chapter, we’ll explore how to handle asynchronous operations with any-llm, a crucial skill for building responsive and scalable AI systems.

References

This page is AI-assisted and reviewed. It references official documentation and recognized resources where relevant.