Scrape

The scrape sensor integration集成将 Home Assistant 与您的设备、服务等连接和集成。 [Learn more] scrapes information from websites. The sensor loads an HTML page, and allows you to search and extract specific values. As this is not a fully featured web scraper like scrapy, it will work with simple web pages and it can be time-consuming to get the right section.

If you are not using Home Assistant Container or Home Assistant Operating System, this integration requires libxml2 to be installed. On Debian based installs, run:

sudo apt install libxml2

Both UI and YAML setup is supported while YAML provides additional configuration possibilities.

配置

要将 Scrape integration 添加到您的 Home Assistant 实例中,请使用此 My 按钮:

手动配置步骤

如果上述 My 按钮不起作用,您也可以手动执行以下步骤:

To enable this integration集成将 Home Assistant 与您的设备、服务等连接和集成。 [Learn more] using YAML, add the following lines to your configuration.yamlconfiguration.yaml 文件是 Home Assistant 的主要配置文件。它列出了要加载的集成及其特定配置。在某些情况下,需要直接在 configuration.yaml 文件中手动编辑配置。大多数集成可以在 UI 中配置。 [Learn more] file. 在更改了configuration.yamlconfiguration.yaml 文件是 Home Assistant 的主要配置文件。它列出了要加载的集成及其特定配置。在某些情况下,需要直接在 configuration.yaml 文件中手动编辑配置。大多数集成可以在 UI 中配置。 [Learn more] 文件后,重启 Home Assistant 以应用更改。 该集成现在显示在集成页面的 设置 > 设备与服务 下。其实体在集成卡片上以及实体标签上列出。

# Example configuration.yaml entry
scrape:
  - resource: https://www.home-assistant.io
    sensor:
      - name: "Current version"
        select: ".release-date"

Configuration Variables

resource string Required

The resource or endpoint that contains the value.

resource_template template Required

The resource or endpoint that contains the value with template support.

method string (Optional, default: GET)

The method of the request. Either POST or GET.

payload string (Optional)

The payload to send with a POST request. Depends on the service, but usually formed as JSON.

verify_ssl boolean (Optional, default: true)

Verify the SSL certificate of the endpoint.

timeout integer (Optional, default: 10)

Defines max time to wait data from the endpoint.

authentication string (Optional)

Type of the HTTP authentication. basic or digest.

username string (Optional)

The username for accessing the REST endpoint.

password string (Optional)

The password for accessing the REST endpoint.

headers list | template (Optional)

The headers for the requests.

params list | template (Optional)

The query params for the requests.

scan_interval integer (Optional, default: 600)

Define the refrequency to call the REST endpoint in seconds.

encoding string (Optional, default: UTF-8)

The character encoding to use if none provided in the header of the shared data.

sensor map Required

A list of sensors to create from the shared data. All configuration settings that are supported by RESTful Sensor not listed above can be used here.

name template (Optional)

Defines a template to get the name of the entity.

select string Required

Defines the HTML tag to search for. Check Beautifulsoup’s CSS selectors for details.

attribute string (Optional)

Get value of an attribute on the selected tag.

index integer (Optional, default: 0)

Defines which of the elements returned by the CSS selector to use.

value_template template (Optional)

Defines a template to get the state of the sensor.

unique_id string (Optional)

An ID that uniquely identifies this entity. Will be combined with the unique ID of the configuration block if available. This allows changing the name, icon and entity_id from the web interface.

icon template (Optional)

Defines a template for the icon of the entity.

availability template (Optional, default: true)

Defines a template to get the available state of the entity. If the template either fails to render or returns True, "1", "true", "yes", "on", "enable", or a non-zero number, the entity will be available. If the template returns any other value, the entity will be unavailable. If not configured, the entity will always be available. Note that the string comparison not case sensitive; "TrUe" and "yEs" are allowed.

unit_of_measurement string (Optional, default: None)

Defines the units of measurement of the sensor, if any. This will also display the value based on the user profile Number Format setting and influence the graphical presentation in the history visualization as a continuous value.

state_class string (Optional, default: None)

The state_class of the sensor. This will also display the value based on the user profile Number Format setting and influence the graphical presentation in the history visualization as a continuous value.

picture template (Optional)

Defines a template for the entity picture of the sensor.

device_class device_class (Optional, default: None)

Sets the class of the device, changing the device state and icon that is displayed on the UI (see below). It does not set the unit_of_measurement.

使用模板

对于传入数据,值模板将传入的 JSON 或原始数据转换为有效的有效负载。 传入的有效负载使用可能的 JSON 值进行渲染,因此在渲染时,可以使用 value_json 访问基于 JSON 的有效负载中的属性,否则可以使用 value 变量来处理非 JSON 基础的数据。

此外,this 可以作为模板中的变量使用。this 属性指的是实体的当前 实体状态。 关于 this 变量的更多信息可以在 模板文档 中找到。

Note

带 json 的示例值模板:

给定有效负载:

{ "state": "ON", "temperature": 21.902 }

模板 {{ value_json.temperature | round(1) }} 渲染为 21.9

Examples

In this section you find some real-life examples of how to use this sensor. There is also a Jupyter notebook available for this example to give you a bit more insight.

Home Assistant

The current release Home Assistant is published on homepage

scrape:
# Example configuration.yaml entry
  - resource: https://www.home-assistant.io
    sensor:
      - name: Release
        select: ".release-date"

Available implementations

Get the counter for all our implementations from the integrations page under Settings > Devices & services.

# Example configuration.yaml entry
scrape:
  - resource: https://www.home-assistant.io/integrations/
    sensor:
      - name: Home Assistant impl.
        select: 'a[href="#all"]'
        value_template: '{{ value.split("(")[1].split(")")[0] }}'

Get a value out of a tag

The German Federal Office for Radiation protection (Bundesamt für Strahlenschutz) is publishing various details about optical radiation including an UV index. This example is getting the index for a region in Germany.

# Example configuration.yaml entry
scrape:
  - resource: https://www.bfs.de/DE/themen/opt/uv/uv-index/prognose/prognose_node.html
    sensor:
      - name: Coast Ostsee
        select: "p"
        index: 19
        unit_of_measurement: "UV Index"

IFTTT status

If you make heavy use of the IFTTT web service for your automations and are curious about the status of IFTTT then you can display the current state of IFTTT in your frontend.

# Example configuration.yaml entry
scrape:
  - resource: https://status.ifttt.com/
    sensor:
      - name: IFTTT status
        select: ".component-status"

Get the latest podcast episode file URL

If you want to get the file URL for the latest episode of your favorite podcast, so you can pass it on to a compatible media player.

# Example configuration.yaml entry
scrape:
  - resource: https://hasspodcast.io/feed/podcast
    sensor:
      - name: Home Assistant Podcast
        select: "enclosure"
        index: 1
        attribute: url

Energy price

This example tries to retrieve the price for electricity.

# Example configuration.yaml entry
scrape:
  - resource: https://elen.nu/dagens-spotpris/se3-stockholm/
    sensor:
      - name: Electricity price
        select: ".text-lg.font-bold"
        index: 1
        value_template: '{{ value | replace (",", ".") | float }}'
        unit_of_measurement: "öre/kWh"

Container cleaning by CleanProfs in The Netherlands

This example gets the container type and container cleaning date for the next two cleanings.

# Example configuration.yaml entry. Change postal code and house number to your own address.
scrape:
  - resource: https://crm.cleanprofs.nl/search/planning
    method: POST
    payload: zipcode=5624JW&street_number=17
    headers:
      Content-Type: application/x-www-form-urlencoded
    sensor:
      - name: "Type container 1"
        select: "div.nk-tb-item:nth-child(2) > div:nth-child(1) > span:nth-child(1)"
      - name: "Date container 1"
        select: "div.nk-tb-item:nth-child(2) > div:nth-child(3) > span:nth-child(1) > span:nth-child(1)"
      - name: "Type container 2"
        select: "div.nk-tb-item:nth-child(3) > div:nth-child(1) > span:nth-child(1)"
      - name: "Date container 2"
        select: "div.nk-tb-item:nth-child(3) > div:nth-child(3) > span:nth-child(1) > span:nth-child(1)"