2025.3.4

2024.6: Dipping our toes in the world of AI using LLMs 🤖

Franck Nijhof Franck Nijhof Comments

Home Assistant Core 2024.6! 🎉

Welcome to another loaded release of Home Assistant! Packed with many exciting new features, this month is most definitely one to remember.

Let’s talk about AI. AI might just as well be the talk of the century. These so-called LLMs most certainly open up a whole new world of possibilities. When we first introduced the OpenAI and Google AI conversation agents for Assist, one of the most requested features was the ability to control your home with it.

Now for the big news:

Well, as of today, you can control your home with an AI! 🤖

But there is a lot more to this release. Super simple media player commands, more data table improvements, tag entities, collapsible blueprint sections, and so much more!

I’m personally most excited about the new features in the dashboarding area. Being able to conditionally show sections and cards is definitely a game-changer for my dashboards.

Enjoy the release!

../Frenck

Don’t forget to join our release party live stream on YouTube 5 June 2024, at 20:00 GMT / 12:00 PST / 21:00 CEST!

Voice & Assist

This month, we are continuing our work on improving Assist’s out-of-the-box capabilities: Not only are we adding features that you would expect from a voice assistant, such as media control, but we are leap-frogging ahead with some new exciting developments and experimenting with AI! 🤖

We will have more to announce during a soon-to-be-announced Voice - Chapter 7 livestream on June the 26th! Keep an eye out for that! 🎙️

Dipping our toes in the world of AI using LLMs

Our voice assistant’s brain is called a conversation agent. It is responsible for understanding the intention behind the spoken command, performing an action, and generating a response.

Since the beginning of our voice journey, we allowed you to switch your conversation agent for an LLM-based one. This led to some very funny experiments, such as our infamous interview with Mario.

Until now, these two worlds (our own Home Assistant conversation agent controlling your home and LLM-based conversation agents) did not overlap; it was impossible to control your home from an LLM-based conversation agent.

Today, we’re happy to announce that we are closing this gap!

When setting up an LLM-based conversation agent, you can decide to let your conversation agent control your home with a flick of a switch.

Screenshot of the configuration screen of the OpenAI integration, showing how to enable Home Assistant control.

With this new setting, LLM-based conversation agents can tap into our intent意图是与语音助手一起使用的术语。意图是 Home Assistant 从您的语音或文本表达中提取命令时认为您想要它执行的操作。 [Learn more] system, which powers Assist. They also get access to every entity实体表示 Home Assistant 中的传感器、执行器或功能。实体用于监控物理属性或控制其他实体。实体通常是设备或服务的一部分。 [Learn more] exposed to Assist. That way, you control what your agents have access to.

Tapping into our intent system is great because it works out of the box. LLM-based conversation agents can do everything that Assist can do. The added benefit is that they are capable of reasoning beyond words, which Assist was incapable of doing.

For example, if you have a light called “Webcam light” exposed in your “office” area, you can give direct commands such as:

Turn on the office webcam light.

That was also working with Assist previously, but you can also give more complex commands, such as:

I’m going to a meeting, can you please make sure people see my face?

The agent will figure out the intention behind the words and call the correct intent on the matching exposed entities.

What about custom intents?

Our intent system has been built from the start to be extensible. Custom integrations and even users can register their own intents. This allows you to guide the agent beyond what’s capable out of the box.

Here is an example of a custom intent that explains to the agent what to do when I leave my home.

conversation:
  intents:
    LeaveHome:
      - "Leave home"

intent_script:
  LeaveHome:
    description: "Launch the leave home script. To be used when I am about to leave my home."
    action:
      - service: script.leave_home
        data: {}
    speech:
      text: "Done"

With this YAML snippet added to my configuration.yamlconfiguration.yaml 文件是 Home Assistant 的主要配置文件。它列出了要加载的集成及其特定配置。在某些情况下，需要直接在 configuration.yaml 文件中手动编辑配置。大多数集成可以在 UI 中配置。 [Learn more], if the agent understands that I am leaving home, it will not turn off what it thinks I want, it will run my leave_home script, which is precisely what I want it to do.

This release makes it available for our OpenAI and Google AI integrations集成将 Home Assistant 与您的设备、服务等连接和集成。 [Learn more]. To make it easier to get started with LLMs, we have updated them with recommended model settings that strike the right balance between accuracy, speed, and cost. If you had them set up previously, we recommend enabling the recommended settings and emptying the instructions.

Our recommended model settings perform equally well for voice assistant tasks. Google is 14x cheaper than OpenAI, but OpenAI is better at answering non-smart home questions. We are preparing a blog post with a deep dive into the research that went into this feature, coming soon™!

Local LLMs have been supported via the Ollama integration since Home Assistant 2024.4. Ollama and the major open source LLM models are not tuned for tool calling, so this has to be built from scratch and was not done in time for this release. We’re collaborating with NVIDIA to get this working – they showed a prototype last week.

Thanks to everyone who contributed to this feature! @shulyaka, @tronikos, @allenporter, @synesthesiam, @jlpouffier, and @balloob!

Improved media player commands

We have introduced new media player intents意图是与语音助手一起使用的术语。意图是 Home Assistant 从您的语音或文本表达中提取命令时认为您想要它执行的操作。 [Learn more] that allow you to do more while saying less. You can say the following voice commands to control media players in the same area as your Assist device:

  • “Pause”
  • “Resume”
  • “Next”
  • “Set the volume to 50%”

Up until now, we have only proposed sentences targeting a specific entity by its name. Due to this limitation, these intents were not the most user-friendly, as the sentences were too long to say: “Skip to the next song on the TV”.

We added two features to get to the smallest sentences possible. Context awareness makes our voice assistant aware of the devices in the same area as the satellite. We have also created a smart matching strategy that finds the right media player to target.

For example, if you say “pause”, the voice assistant will automatically target the media player that is playing.

Just ensure your voice assistant device is assigned to an area with an exposed media player, and you are good to go.

Dashboards

Work has continued, making the dashboard customization and organization easier and more intuitive. This month, we are adding a new feature for our new sections and customization for backgrounds, which everyone in your home will enjoy.

Show sections conditionally

We added a visibility option to the new sections dashboard feature. This amazing new feature allows you to hide or show a section based on certain conditions.

For example, you may want to display only a section on mobile when you’re at home. Or, only show the switch to turn off kitchen lights when the kitchen lights are actually on. Maybe you have that one section only relevant to you or your partner, and you want to hide it from the kids.

Screenshot showing how to conditional make a section visible on the new sections dashboard.

It uses the same conditions as condition cards, meaning you have a lot of flexibility in what you can do.

Control visibility of cards

Wait, we didn’t stop there! This same visibility feature is now also available for cards! Oh yes! 🚀

Screenshot showing a tile card that is only visible to a specific person.

You can now hide or show a card based on certain conditions. This allows you to create more dynamic dashboards that adapt to your needs. The big difference is that you don’t need a condition card to use this feature. It is available directly in the card configuration in the Visibility tab.

Setting a dashboard background

@Nezz contributed a great new feature to our dashboarding UI: the ability to set a background image for your dashboard!

Previously, this was only possible by editing YAML, but now you can do it directly from the UI. Even better, you can upload an image directly from your computer or provide a URL to one!

Screenshot of the configuration screen of the OpenAI integration, showing how to enable Home Assistant control.

Photo: Body of Water Between Green Leaf Trees by Ian Turnell.

The same ability (to upload images directly from your computer) was added to the picture card, too! Thanks, @karwosts, for that one!

Data tables improvements

Like the last release, this release keeps improving our data tables.

When we introduced data groupings in the tables, you asked for a way to collapse and expand those groupings, so we did in the last release! But then we quickly realized that it was not enough, and you wanted to be able to collapse and expand all groups at once. So, we did that for this release!

Screenshot showing the new collapse and expand all feature of our data tables.

We’ve also expanded the voice assistant expose data table to allow grouping by area and domain.

Additionally, filters you’ve set in the data tables are now saved in your browser session. Meaning, if you navigate away from the page and come back, your filters will still be there. But! Each browser tab or window has its own session, meaning you can have different filters in different tabs or windows that are remembered for that specific tab or window.

Integrations

Thanks to our community for keeping pace with the new integrations集成将 Home Assistant 与您的设备、服务等连接和集成。 [Learn more] and improvements to existing ones! You’re all awesome 🥰

New integrations

We welcome the following new integrations in this release:

  • Airgradient, added by @joostlek
    Provides air quality data from your local Airgradient device.
  • APsystems, added by @mawoka-myblock
    Monitor your APsystems EZ1 microinverters.
  • Azure data explorer, added by @kaareseras
    Forward events from Home Assistant to the Azure Data Explorer for analysis.
  • IMGW-PIB, added by @bieniu
    Hydrological data from the Institute of Meteorology and Water Management—National Research Institute provides information about rivers and water reservoirs in Poland.
  • Intelligent Storage Acceleration, added by @bdraco
    Intelligent Storage Acceleration Library (ISAL) is used to accelerate our Home Assistant frontend. It is automatically enabled/activated.
  • Monzo, added by @JakeMartin-ICL
    Connect your Monzo bank account to Home Assistant and get insights into your account balance.

Noteworthy improvements to existing integrations

It is not just new integrations集成将 Home Assistant 与您的设备、服务等连接和集成。 [Learn more] that are added; existing integrations are also constantly improved. Here are some of the noteworthy changes to existing integrations:

  • Do you need a wireless device that can bridge a larger distance, like kilometers? A LoRa device might be a good option. Thanks to @angelnu we now have a working integration for The Things Network.
  • @starkillerOG added support for PIR and battery sensors to the Reolink integrations. Nice!
  • The Teslemetry integration has been expanded tremendously, adding new platforms and features. Awesome job @Bre77!
  • Limited templates can now be used to disable triggers触发器是定义的一组平台值或条件，用于触发自动化运行。 [Learn more], conditions条件是自动化的可选部分，如果不满足条件，将阻止动作触发。 [Learn more], and actions动作在 Home Assistant 的多个地方使用。作为脚本或自动化的一部分，动作定义了触发器被激活后将发生的事情。在脚本中，动作被称为序列。 [Learn more]. @farmio thought this might be helpful for blueprint蓝图是一个脚本、自动化或模板实体配置，其中某些部分标记为可配置。这允许用户基于相同的蓝图创建多个脚本、自动化或模板实体，每个都有自己的配置特定设置。 [Learn more] creators.
  • @Anrijs added support for the Aranet Radiation devices to the Aranet integration. Nice!
  • You can now set the cutting height of your Husqvarna Automower and there are switches to make it stay out of certain zones. Thanks @Thomas55555!
  • @thomaskistler added flow and rain sensor support to the Hydrawise integration. Great!
  • The SwitchBot Cloud integration now supports the SwitchBot Meter, MeterPlus, and Outdoor Meter. Thanks @laurence-presland!

Integrations reaching platinum quality level

The following integrations集成将 Home Assistant 与您的设备、服务等连接和集成。 [Learn more] reached the platinum 🏆 level on our integration quality scale. These integrations are the best of the best and give you an excellent out-of-the-box experience.

Congratulations to all the developers who worked on these integrations!

Now available to set up from the UI

While most integrations集成将 Home Assistant 与您的设备、服务等连接和集成。 [Learn more] can be set up directly from the Home Assistant user interface, some were only available using YAML configuration. We keep moving more integrations to the UI, making them more accessible for everyone to set up and use.

The following integrations are now available via the Home Assistant UI:

Farewell to the following

The following integrations集成将 Home Assistant 与您的设备、服务等连接和集成。 [Learn more] are also no longer available as of this release:

  • Ambiclimate has been removed by @gjohansson-ST. Their services have been terminated, effective March 31, 2024.

Matter turns 1.3

As one of the first in the industry, we proudly announce that Home Assistant is already based on the latest MatterMatter 是一个开源标准，定义了如何在 Wi-Fi 或 Thread 网络上控制智能家居设备。 [Learn more] specification: Matter 1.3! 😎 This updated version of the Matter SDK improves the reliability and compatibility of Matter devices.

This release also improves and expands device support. Matter-based air purifiers and room air conditioner devices are now supported, thermostats and other climate devices had important fixes around set points, and updates to our discovery ensure that we can represent more data for most devices.

A couple of Matter devices out there have so-called “custom clusters”. Custom clusters are a standardized way for manufacturers to include non-standardized data. We want to support custom clusters, so we need to add support for each one individually.

This release allows Home Assistant to be notified when the data of a custom cluster changes (instead of polling for it on an interval). This will reduce the network traffic and improve the performance of your Matter network, especially on bigger ThreadThread 是一种专为智能家居应用设计的低功耗网状网络标准。它是定义设备如何通信的协议。 [Learn more] networks.

Other noteworthy changes

There are many more improvements in this release; here are some of the other noteworthy changes this release:

  • A default code can now be set in the entity settings for every alarm control panel entities实体表示 Home Assistant 中的传感器、执行器或功能。实体用于监控物理属性或控制其他实体。实体通常是设备或服务的一部分。 [Learn more]. Nice work @gjohansson-ST!
  • @Troon added a filter to our templates模板是一种自动化定义，可以包括动作或触发值数据的变量。这允许自动化生成动态动作。 [Learn more]: add. This filter allows you to add a value similar to the existing multiply. Great!
  • A new action building block has been added to our automationsHome Assistant 中的自动化允许您自动响应在家中和周围发生的事情。 [Learn more] and scripts脚本是允许用户指定 Home Assistant 打开时要执行的动作序列的组件。 [Learn more] by @surfingbytes and @frenck: Run sequence of actions. This allows you to group multiple actions动作在 Home Assistant 的多个地方使用。作为脚本或自动化的一部分，动作定义了触发器被激活后将发生的事情。在脚本中，动作被称为序列。 [Learn more] together or run multiple sequences of actions in parallel.
  • Sensors can be grouped to calculate, for example, their sum or mean value. Thanks to @CoRfr, it is now possible to calculate the standard deviation of a group of sensors. Cool!

Disabling expiration of login tokens

A few releases ago, we introduced the automatic expiration of logins. If a login session wasn’t used for 90 days, it would expire and be automatically cleaned up.

Some of you have expressed that you would like to disable this feature for various reasons. For example, because of a secondary holiday home one doesn’t regularly visit, or because of relatives who can only come over a few times a year. We added a new option to the user profile settings to disable the expiration of specific login tokens.

Screenshot showing the option to disable the expiration of tokens in the profile page.

You can find these token in your profile settings, under the Security tab.

Tag entities!

Home Assistant has had support for tags for a long time. The tags feature allowed NFC tags or QR codes to be used with Home Assistant, for example, to trigger automationsHome Assistant 中的自动化允许您自动响应在家中和周围发生的事情。 [Learn more].

However, tags were not real entities and, thus, not always as logical to use. With this release, @gjohansson-ST changed this!

Tags are now normal entities实体表示 Home Assistant 中的传感器、执行器或功能。实体用于监控物理属性或控制其他实体。实体通常是设备或服务的一部分。 [Learn more] and can be used in automationsHome Assistant 中的自动化允许您自动响应在家中和周围发生的事情。 [Learn more], scripts脚本是允许用户指定 Home Assistant 打开时要执行的动作序列的组件。 [Learn more], templates模板是一种自动化定义，可以包括动作或触发值数据的变量。这允许自动化生成动态动作。 [Learn more], and even be added to your your dashboards!

Collapsible sections for blueprints!

@karwosts makes an appearance again this release with another one of his QoL (Quality of Live) improvements. He added the ability to add collapsible sections to blueprints蓝图是一个脚本、自动化或模板实体配置，其中某些部分标记为可配置。这允许用户基于相同的蓝图创建多个脚本、自动化或模板实体，每个都有自己的配置特定设置。 [Learn more]. This allows blueprint creators to group configuration options together or hide advanced or less common options by default.

Screenrecording showing sections of Blueprints inputs being collapsed/uncollapsed. Each section can be collapsed or uncollapsed by clicking on the section header.

Thanks @karwosts!

Read more about blueprint input sections in our documentation.

Looking for an interesting blueprint to get started? Check out the Blueprints Exchange on our community forum! There are great automation blueprints available.

Patch releases

We will also release patch releases for Home Assistant 2024.6 in June. These patch releases only contain bug fixes. Our goal is to release a patch release every Friday.

2024.6.1 - June 7

2024.6.2 - June 11

2024.6.3 - June 15

2024.6.4 - June 21

Need help? Join the community!

Home Assistant has a great community of users who are all more than willing to help each other out. So, join us!

Our very active Discord chat server is an excellent place to be at, and don’t forget to join our amazing forums.

Found a bug or issue? Please report it in our issue tracker, to get it fixed! Or, check our help page for guidance for more places you can go.

Are you more into email? Sign-up for our Building the Open Home Newsletter to get the latest news about features, things happening in our community and other news about building an Open Home; straight into your inbox.

Backward-incompatible changes

We do our best to avoid making changes to existing functionality that might unexpectedly impact your Home Assistant installation. Unfortunately, sometimes, it is inevitable.

We always make sure to document these changes to make the transition as easy as possible for you. This release has the following backward-incompatible changes:

Anova

The State and Mode entities for Anova devices have been changed to match the new protocol messaging. Any automations based on these entities will need to be updated.

(@Lash-L - #109508) (documentation)

AVM FRITZ!Box Tools

The previously deprecated services reboot, reconnect, and cleanup have been removed. Please use the corresponding button entities instead.

(@mib1185 - #118108) (documentation)

File

The notify services for the file integration now require that the file path is an allowed path. You should check the accessed file is in the allowlist_external_dirs to ensure your automations keep working.

(@jbouwh - #116861) (documentation)

MQTT

MQTT Lock now supports the lock open and opening states. When a lock is used in optimistic mode, its state will become open instead of unlocked when it is opened.

(@jbouwh - #117110) (documentation)

The behavior on processing empty and "None" state payloads for MQTT entities has been made more predictable for alarm_control_panel, climate, cover, device_tracker, lock, select, valve, and water_heater items:

  • A “None” (string) for a state payload generally means the state becomes unknown.
  • An empty payload is ignored.

Previously, a “None” state could be considered invalid or ignored.

(@jbouwh - #117813) (documentation)

OpenWeatherMap

Migration to the new library with support of OneCall API (3.0).

OWM API v2.5 will be closed in June 2024. All OpenWeatherMap integrations must be migrated to mode (API) v3.0 to avoid service interruption.

Before the migration, you must have an active subscription (be aware that subscription activation takes up to 2h). After the subscription is activated, users can migrate integrations to mode v3.0 via repair notification or manually via the integration configuration menu.

The subscription has a free tier with 1000 calls/day. Consider setting the limit on the OpenWeatherMap website to stay under the threshold where API usage incurs a cost.

OpenWeatherMap official statement

(@freekode - #116870) (documentation)

Synology DSM

The option to define a custom timeout has been removed. The timeout is set static to a sane value of 30 seconds.

(@mib1185 - #116815) (documentation)

Webmin

Duplicate entities may appear due to unique ID changes. The obsolete entities can be removed.

(@autinerd - #113084) (documentation)

World Air Quality Index (WAQI)

The state attributes of the WAQI air quality sensor have been removed. The attributes have been replaced by individual sensors; use these instead.

(@joostlek - #116595) (documentation)

Z-Wave

With this release, you will need to update your zwave-js-server instance.

You must use zwave-js-server 1.35.0 or greater (schema 35).

  • If you use the Z-Wave JS add-on, you need at least version 0.5.0.
  • If you use the Z-Wave JS UI add-on, you need at least version 3.4.2.
  • If you use the Z-Wave JS UI Docker container, you need at least version 9.10.0.
  • If you run your own Docker container or some other installation method, you will need to update your zwave-js-server instance to at least 1.35.0.

(@raman325 - #117288) (documentation)

If you are a custom integration developer and want to learn about changes and new features available for your integration: Be sure to follow our developer blog. The following are the most notable for this release:

All changes

Of course, there is a lot more in this release. You can find a list of all changes made here: Full changelog for Home Assistant Core 2024.6