Structured Response

You can enforce a particular response format from an LLM by providing a JSON schema to the .respond() method. This guarantees that the model's output conforms to the schema you provide.

The JSON schema can either be provided directly, or by providing an object that implements the lmstudio.ModelSchema protocol, such as pydantic.BaseModel or lmstudio.BaseModel.

The lmstudio.ModelSchema protocol is defined as follows:

@runtime_checkable
class ModelSchema(Protocol):
    """Protocol for classes that provide a JSON schema for their model."""

    @classmethod
    def model_json_schema(cls) -> DictSchema:
        """Return a JSON schema dict describing this model."""
        ...

When a schema is provided, the prediction result's parsed field will contain a string-keyed dictionary that conforms to the given schema (for unstructured results, this field is a string field containing the same value as content).

Enforce Using a Class Based Schema Definition

If you wish the model to generate JSON that satisfies a given schema, it is recommended to provide a class based schema definition using a library such as pydantic or msgspec.

Pydantic models natively implement the lmstudio.ModelSchema protocol, while lmstudio.BaseModel is a msgspec.Struct subclass that implements .model_json_schema() appropriately.

Define a Class Based Schema

from pydantic import BaseModel

# A class based schema for a book
class BookSchema(BaseModel):
    title: str
    author: str
    year: int

Generate a Structured Response

result = model.respond("Tell me about The Hobbit", response_format=BookSchema)
book = result.parsed

print(book)
#           ^
# Note that `book` is correctly typed as { title: string, author: string, year: number }

Enforce Using a JSON Schema

You can also enforce a structured response using a JSON schema.

Define a JSON Schema

# A JSON schema for a book
schema = {
  "type": "object",
  "properties": {
    "title": { "type": "string" },
    "author": { "type": "string" },
    "year": { "type": "integer" },
  },
  "required": ["title", "author", "year"],
}

Generate a Structured Response

result = model.respond("Tell me about The Hobbit", response_format=schema)
book = result.parsed

print(book)
#     ^
# Note that `book` is correctly typed as { title: string, author: string, year: number }