AI Task

The AI Task integrationIntegrations connect and integrate Home Assistant with your devices, services, and more. [Learn more] lets your automations, scripts, and the Home Assistant interface use AI for a single job. For example, to describe what a camera sees, to generate an image, or to decide whether a delivery driver is at the door.

Note

Building block integration

This ai task is a building block integration that cannot be added to your Home Assistant directly but is used and provided by other integrations.

A building block integration differs from the typical integration that connects to a device or service. Instead, other integrations that do integrate a device or service into Home Assistant use this ai task building block to provide entities, services, and other functionality that you can use in your automations or dashboards.

If one of your integrations features this building block, this page documents the functionality the ai task building block offers.

The AI task entitiesAn entity represents a sensor, actor, or function in Home Assistant. Entities are used to monitor physical properties or to control other entities. An entity is usually part of a device or a service. [Learn more] come from AI integrations. For example, OpenAI, Google Gemini, and Ollama add them. For the full list of AI integrations, refer to AI integrations.

Ways to use AI tasks

You can use AI tasks in the following ways:

  • In automations and scripts, with the AI task actions. With them, you can generate text or structured data, generate images, and evaluate questions. For all actions, refer to List of actions.
  • In the Home Assistant interface, with the Suggest button. For example, when you save an automation, it suggests a name, a description, a category, and labels.

Not every AI task supports every action. For example, some AI tasks can generate data, but not images.

AI tasks, AI tools, and conversation agents

AI tasks and AI toolsA function that conversation agents and AI apps can use to read information, control entities, or manage Home Assistant. AI tools are grouped into sets. Each set provides different functionality. Some sets only work for administrators. [Learn more] work in opposite directions:

  • With AI tools, a conversation agentSoftware that Home Assistant uses to understand a request and decide what to do. Home Assistant has a built-in conversation agent. Integrations can add others, including ones that use AI. Conversation agents that use AI can also use AI tools. Each assistant uses a conversation agent. [Learn more] or an AI appThird-party software that connects to Home Assistant through the Model Context Protocol integration. An AI app can use the AI tools that you allow. [Learn more] uses Home Assistant. During a conversation, it decides which AI tools to call. For example, it calls an AI tool to turn off a light.
  • With an AI task, Home Assistant uses the AI model of an AI integration. Your automation, your script, or the Suggest button sends a single job to an AI task, and gets the result back. For example, when someone rings the doorbell, your automation asks an AI task whether the camera image shows a delivery driver. There’s no conversation, and an AI task doesn’t use AI tools.

So an AI task can’t control your devices. To control your devices with AI, use a conversation agent with the Device control set of AI tools. For details, refer to Sets of AI tools.

Choosing the preferred AI task entities

When an action doesn’t name an AI task entity, Home Assistant uses the preferred one. The Suggest button also uses the preferred AI task entity for data generation. You can choose a different one for each kind of task. For example, a fast model to generate text, and another model that can generate images.

Prerequisites:

  • An administrator account in Home Assistant.
  • An AI integration that adds AI tasks. For example, OpenAI.

To choose the preferred AI task entities:

  1. Go to Settings > System > AI.
  2. On the AI suggestions card, under Data generation tasks, select an AI task entity.
    • The list only shows AI tasks that can generate data.
  3. Optional: under Image generation tasks, select an AI task entity that can generate images.
  4. Optional: under Decisions, select an AI task entity that can evaluate questions.
  5. Select Save.
    • Result: The actions and the Suggest button use these AI task entities when you don’t select one yourself.

AI task states

The stateThe state holds the information of interest of an entity, for example, if a light is on or off. Each entity has exactly one state and the state only holds one value at a time. However, entities can store attributes related to that state such as brightness, color, or a unit of measurement. [Learn more] of an AI task entityAn entity represents a sensor, actor, or function in Home Assistant. Entities are used to monitor physical properties or to control other entities. An entity is usually part of a device or a service. [Learn more] is a timestamp showing the date and time when the AI task was last used. Home Assistant stores the timestamp in UTC, for example, 2026-01-01T12:00:00.123456+00:00. The Home Assistant interface shows it in your local date and time format.

In addition, the entity can have the following states. Each item shows the interface label, followed by the stored state:

  • Unavailable (unavailable): The entity is currently unavailable.
  • Unknown (unknown): The state is not yet known.

List of actions

The AI Task integrationIntegrations connect and integrate Home Assistant with your devices, services, and more. [Learn more] provides the following actions. Each link below opens a dedicated page with examples, parameters, and a step-by-step UI walkthrough.

  • Clear images (ai_task.clear_images) Deletes images generated by AI tasks.

  • Evaluate (ai_task.evaluate) Uses AI to evaluate questions about supplied text, structured data, or attachments.

  • Generate data (ai_task.generate_data) Uses AI to run a task that generates data, such as text or structured output.

  • Generate image (ai_task.generate_image) Uses AI to generate an image from a set of instructions.

For an overview of every action across all integrations, see the actions reference.

Examples

Automation: thank a delivery driver at the front door

When someone rings the doorbell, use Evaluate to check a camera snapshot for a delivery driver. If the estimated probability is at least 90%, use Speak to play a thank-you message through the doorbell speaker.

Prerequisites:

  • A doorbell with a camera and a doorbell event entity that supports the Doorbell rang trigger.
  • A doorbell speaker exposed as a media player that supports text-to-speech playback, such as a supported UniFi Protect doorbell. Not every doorbell supports this.
  • A configured text-to-speech entity and a preferred AI task evaluation entity that supports image attachments.

In the automation editor, use this sequence:

  1. Add a Doorbell rang trigger and select your front door doorbell.
  2. Add an Evaluate action with a question named delivery_driver: “Is the person at the front door a delivery driver delivering a package?” Use the noul question type, attach the front door camera, and store the response in delivery.
  3. Add an If-then action with a template condition that checks whether delivery.answers.delivery_driver.noul is at least 0.9.
  4. In the Then branch, add Speak, select your text-to-speech entity and doorbell media player, and enter “Thank you for delivering the package!”.

The example below includes the question format and condition. Replace the event, camera, media player, and text-to-speech entity IDs with your own. It uses your preferred evaluation entity; you can also set entity_id in the Evaluate action’s data.

YAML example for thanking a delivery driver

Paste this into the YAML editor of a new automation:

AutomationAutomations in Home Assistant allow you to automatically respond to things that happen in and around your home. [Learn more]
alias: "Thank a delivery driver at the front door"
triggers:
  - trigger: doorbell.rang
    target:
      entity_id: event.front_door_doorbell
actions:
  - action: ai_task.evaluate
    data:
      task_name: "Check for a delivery driver"
      questions:
        delivery_driver:
          type: noul
          instructions: >
            Is the person at the front door a delivery driver delivering
            a package? Use visible evidence, such as a package being
            carried or placed by the door.
      attachments:
        - media_content_id: media-source://camera/camera.front_door
          media_content_type: image/jpeg
    response_variable: delivery
  - if: "{{ delivery.answers.delivery_driver.noul >= 0.9 }}"
    then:
      - action: tts.speak
        target:
          entity_id: tts.home_assistant_cloud
        data:
          media_player_entity_id: media_player.front_door_doorbell
          message: "Thank you for delivering the package!"

The noul value is the model’s estimated probability, not a guarantee. Adjust the 0.9 threshold to suit your camera view. Below that threshold, the automation does not play a message.

Template entity counting items on a camera

template:
  - triggers:
      - trigger: homeassistant
        event: start
      - trigger: time_pattern
        minutes: "/5"  # update every 5 minutes
    actions:
      - action: ai_task.generate_data
        data:
          task_name: "{{ this.entity_id }}"
          instructions: >-
            This is the inside of my goose coop. How many birds (chickens, geese, and
            ducks) are inside the coop?
          structure:
            birds:
              selector:
                number:
          attachments:
            media_content_id: media-source://camera/camera.chicken_coop
            media_content_type: image/jpeg
        response_variable: result
    sensor:
      - name: "Chickens"
        state: "{{ result.data.birds }}"
        state_class: total

Alternative ideas: detect number of parking spots available, count people in a room, or detect if a door is open.

Structured output example

# Example: Generate weather and indoor comfort report
script:
- alias: "Weather and comfort report"
  sequence:
    - action: ai_task.generate_data
      data:
        task_name: "weather comfort report"
        instructions: |
          Based on the current conditions:
          - Outdoor temperature: {{ states('sensor.outdoor_temperature') }}°C
          - Weather condition: {{ states('weather.home') }}
          - Indoor temperature: {{ states('sensor.living_room_temperature') }}°C
          - Indoor humidity: {{ states('sensor.living_room_humidity') }}%

          Generate a funny weather description and assess indoor comfort level.
        structure:
          weather_description:
            description: "A humorous description of the current weather outside"
            required: true
            selector:
              text:
          indoor_comfort:
            description: "Assessment of how comfortable it is inside compared to outside"
            required: true
            selector:
              text:
      response_variable: comfort_report
    - action: notify.persistent_notification
      data:
        title: "🏠 Home climate report"
        message: |
          🌤️ **Weather outside:**
          {{ comfort_report.data.weather_description }}

          🛋️ **Indoor comfort:**
          {{ comfort_report.data.indoor_comfort }}

Simple text generation example

# Example: Generate a notification when garage door is left open
automation:
- alias: "Garage door notification"
  triggers:
    - trigger: state
      entity_id: cover.garage_door
      to: 'on'
      for:
        minutes: 10
  actions:
    - action: ai_task.generate_data
      data:
        task_name: "garage door left open comment"
        instructions: "Generate a funny notification that garage door was left open"
      response_variable: generated_text
    - action: notify.persistent_notification
      data:
        message: "{{ generated_text.data }}"

Weather visualization example

# Example: Up-to date weather image
automation:
  - alias: "Update image when weather changes"
    triggers:
      - trigger: state
        entity_id: weather.home
    actions:
      - alias: "Generate an image with AI Task"
        action: ai_task.generate_image
        response_variable: generated_image
        data:
          task_name: weather visualization
          instructions: >-
            New York when the weather is {{ states("weather.home") }}

      - alias: "Send out a manual event to update the image entity"
        event: new_weather_image
        event_data:
          url: '{{ generated_image.url }}'

template:
  - trigger:
      - alias: "Update image when a new weather image is generated"
        trigger: event
        event_type: new_weather_image
    image:
      - name: "AI generated image of New York"
        url: "http://localhost:8123{{ trigger.event.data.url }}"