Add pvpc electricity prices integration (#32092)

* Add new integration: pvpc_hourly_pricing

to add a sensor with the current hourly price of electricity in Spain.
Configuration is done by selecting one of the 3 reference tariffs, with
1, 2, or 3 billing periods.

* Features config flow, entity registry, RestoreEntity, options flow
  to change tariff, manual yaml config as integration or sensor platform
* Cloud polling sensor with minimal API calls (3/hour at random times)
  and smart retry; fully async
* Only 1 state change / hour (only when the price changes)
* At evening, try to download published tomorrow prices, to always store
  prices info for a window of [3, 27] hours in the future.
* Include useful state attributes to program automations to be run
  at best electric prices.
* Add spanish and english translations.

* Requires `xmltodict` to parse official xml file with hourly prices
for each day.

* Update requirements and add to codeowners

* Avoid passing in hass as a parameter to the entity

Instead, create time change listeners in async_added_to_hass and
call async_generate_entity_id before async_add_entities

* Fix lint issues

* Add tests for config & options flow

* Add tests for manual yaml config

with entity definition as integration and also as a sensor platform

* Fix placement of PLATFORM_SCHEMA and update generated config_flows

* Store prices internally linked to UTC timestamps

- to deal with days with DST changes
- and work with different local timezones

* Add availability to sensor

to 'expire' the sensor if there is no connection available
 and current hour is not in the stored prices.

Also, turn off logging and retrying if prices can't be downloaded
repeatedly, by flagging `data_source_available` as False, so there is no
log-flood mess.

* Add more tests

- to cover behavior in DST changes and complete coverage of sensor logic
- to cover abort config flow

* fix linter

* Better handling of sensor availability and minor enhancements

- Emmit 1 error if data source is marked as unavailable
  (after some retries), and be silent until cloud access is recovered,
  then emmit 1 warning.
- Follow standard of camel_case keys in attributes

* Mock aiosession to not access real API, store fixture data

- Store a set of daily xml files to test sensor logic for all situations
- Mock time and session to run tests with stored API responses
- Add availability test to simulate a lost + recovery of cloud access,
  checking that logging is reasonable: 1 error to flag the continued
  disconnection + 1 warning in recovery.

* Change API endpoint to retrieve JSON data

and remove xmltodict from reqs.

It seems that this endpoint is more reliable than the XML.

* Adapt tests to new API endpoint

* Translate tariff labels to plain English

and sync the default timeout value for all ways of configuration.

* Relax logging levels to meet silver requirements

- 1 warning when becoming unavailable, another warning when recovered.
- Warnings for unexpected TimeoutError or ClientError
- Move the rest to debug level, leaving info for HA internals

Also reduce number of API calls from 3 to 2 calls/hour.

* Fix requirements

* Mod tests to work with timezone Atlantic/Canary

and fix state attributes for timezones != reference, by using 3 price
prefixes: 'price_last_day_XXh', 'price_next_day_XXh' and 'price_XXh',
all generated with local time (backend timezone)

* Try to fix CI tests

* Externalize pvpc data and simplify sensor.py

* add new `aiopvpc` to requirements
* Remove data parsing and price logic from here
* Replace some constant properties with class variables

* Simplify tests for pvpc_hourly_pricing

* Fix updater for options flow

* Updater always reloads
* `tariff` value comes 1st from entry.options, 2nd from entry.data

* Fix lint

* Bump aiopvpc

* Remove options flow and platform setup

- Remove PLATFORM_SCHEMA and async_setup_platform
- Generate config_entry.unique_id with tariff instead of entity_id, in flow step.
- Remove TariffSelectorConfigFlow
- Adapt tests to maintain full coverage

* Fix docstring on test

and rename SENSOR_SCHEMA to SINGLE_SENSOR_SCHEMA to avoid confusion

* Remove timeout manual config, fix entry.options usage, simplify unique_id

* Simplify tests

- No need for a test_setup now, as platform setup is removed and integration
  setup is already used in `test_availability`
- Simplified `_process_time_step`: only one async_fire(EVENT_TIME_CHANGED)/hour

* Fix possible duplicated update

when source is not available.

* Do not access State last_changed for log messages

* Do not update until entity is added to hass

and call to async_update after 1st download or when recovering access, so
async_write_ha_state is not called twice on those.

* minor changes

* Rename method to select current price and make it a callback
This commit is contained in:
Eugenio Panadero authored and GitHub committed 2020-03-22 14:25:31 -05:00
1 parent 52ac7285a7
commit 8d2e72cdf6
19 files changed
+3047

No files matched your search

@@ -0,0 +1 @@
"""Tests for the pvpc_hourly_pricing integration."""
@@ -0,0 +1,56 @@
"""Tests for the pvpc_hourly_pricing integration."""
import pytest
from homeassistant.components.pvpc_hourly_pricing import ATTR_TARIFF, DOMAIN
from homeassistant.const import ATTR_UNIT_OF_MEASUREMENT
from tests.common import load_fixture
from tests.test_util.aiohttp import AiohttpClientMocker
FIXTURE_JSON_DATA_2019_10_26 = "PVPC_CURV_DD_2019_10_26.json"
FIXTURE_JSON_DATA_2019_10_27 = "PVPC_CURV_DD_2019_10_27.json"
FIXTURE_JSON_DATA_2019_10_29 = "PVPC_CURV_DD_2019_10_29.json"
def check_valid_state(state, tariff: str, value=None, key_attr=None):
"""Ensure that sensor has a valid state and attributes."""
assert state
assert state.attributes[ATTR_UNIT_OF_MEASUREMENT] == "€/kWh"
try:
_ = float(state.state)
# safety margins for current electricity price (it shouldn't be out of [0, 0.2])
assert -0.1 < float(state.state) < 0.3
assert state.attributes[ATTR_TARIFF] == tariff
except ValueError:
pass
if value is not None and isinstance(value, str):
assert state.state == value
elif value is not None:
assert abs(float(state.state) - value) < 1e-6
if key_attr is not None:
assert abs(float(state.state) - state.attributes[key_attr]) < 1e-6
@pytest.fixture
def pvpc_aioclient_mock(aioclient_mock: AiohttpClientMocker):
"""Create a mock config entry."""
aioclient_mock.get(
"https://api.esios.ree.es/archives/70/download_json?locale=es&date=2019-10-26",
text=load_fixture(f"{DOMAIN}/{FIXTURE_JSON_DATA_2019_10_26}"),
)
aioclient_mock.get(
"https://api.esios.ree.es/archives/70/download_json?locale=es&date=2019-10-27",
text=load_fixture(f"{DOMAIN}/{FIXTURE_JSON_DATA_2019_10_27}"),
)
# missing day
aioclient_mock.get(
"https://api.esios.ree.es/archives/70/download_json?locale=es&date=2019-10-28",
text='{"message":"No values for specified archive"}',
)
aioclient_mock.get(
"https://api.esios.ree.es/archives/70/download_json?locale=es&date=2019-10-29",
text=load_fixture(f"{DOMAIN}/{FIXTURE_JSON_DATA_2019_10_29}"),
)
return aioclient_mock
@@ -0,0 +1,79 @@
"""Tests for the pvpc_hourly_pricing config_flow."""
from datetime import datetime
from unittest.mock import patch
from pytz import timezone
from homeassistant import data_entry_flow
from homeassistant.components.pvpc_hourly_pricing import ATTR_TARIFF, DOMAIN
from homeassistant.const import CONF_NAME
from homeassistant.helpers import entity_registry
from .conftest import check_valid_state
from tests.common import date_util
from tests.test_util.aiohttp import AiohttpClientMocker
async def test_config_flow(hass, pvpc_aioclient_mock: AiohttpClientMocker):
"""
Test config flow for pvpc_hourly_pricing.
- Create a new entry with tariff "normal"
- Check state and attributes
- Check abort when trying to config another with same tariff
- Check removal and add again to check state restoration
"""
hass.config.time_zone = timezone("Europe/Madrid")
mock_data = {"return_time": datetime(2019, 10, 26, 14, 0, tzinfo=date_util.UTC)}
def mock_now():
return mock_data["return_time"]
with patch("homeassistant.util.dt.utcnow", new=mock_now):
result = await hass.config_entries.flow.async_init(
DOMAIN, context={"source": "user"}
)
assert result["type"] == data_entry_flow.RESULT_TYPE_FORM
result = await hass.config_entries.flow.async_configure(
result["flow_id"], {CONF_NAME: "test", ATTR_TARIFF: "normal"}
)
assert result["type"] == data_entry_flow.RESULT_TYPE_CREATE_ENTRY
await hass.async_block_till_done()
state = hass.states.get("sensor.test")
check_valid_state(state, tariff="normal")
assert pvpc_aioclient_mock.call_count == 1
# Check abort when configuring another with same tariff
result = await hass.config_entries.flow.async_init(
DOMAIN, context={"source": "user"}
)
assert result["type"] == data_entry_flow.RESULT_TYPE_FORM
result = await hass.config_entries.flow.async_configure(
result["flow_id"], {CONF_NAME: "test", ATTR_TARIFF: "normal"}
)
assert result["type"] == data_entry_flow.RESULT_TYPE_ABORT
assert pvpc_aioclient_mock.call_count == 1
# Check removal
registry = await entity_registry.async_get_registry(hass)
registry_entity = registry.async_get("sensor.test")
assert await hass.config_entries.async_remove(registry_entity.config_entry_id)
# and add it again with UI
result = await hass.config_entries.flow.async_init(
DOMAIN, context={"source": "user"}
)
assert result["type"] == data_entry_flow.RESULT_TYPE_FORM
result = await hass.config_entries.flow.async_configure(
result["flow_id"], {CONF_NAME: "test", ATTR_TARIFF: "normal"}
)
assert result["type"] == data_entry_flow.RESULT_TYPE_CREATE_ENTRY
await hass.async_block_till_done()
state = hass.states.get("sensor.test")
check_valid_state(state, tariff="normal")
assert pvpc_aioclient_mock.call_count == 2
@@ -0,0 +1,85 @@
"""Tests for the pvpc_hourly_pricing component."""
from datetime import datetime, timedelta
import logging
from unittest.mock import patch
from pytz import timezone
from homeassistant.components.pvpc_hourly_pricing import ATTR_TARIFF, DOMAIN
from homeassistant.const import CONF_NAME
from homeassistant.core import ATTR_NOW, EVENT_TIME_CHANGED
from .conftest import check_valid_state
from tests.common import async_setup_component, date_util
from tests.test_util.aiohttp import AiohttpClientMocker
async def _process_time_step(
hass, mock_data, key_state=None, value=None, tariff="discrimination", delta_min=60
):
state = hass.states.get("sensor.test_dst")
check_valid_state(state, tariff=tariff, value=value, key_attr=key_state)
mock_data["return_time"] += timedelta(minutes=delta_min)
hass.bus.async_fire(EVENT_TIME_CHANGED, {ATTR_NOW: mock_data["return_time"]})
await hass.async_block_till_done()
return state
async def test_availability(hass, caplog, pvpc_aioclient_mock: AiohttpClientMocker):
"""Test sensor availability and handling of cloud access."""
hass.config.time_zone = timezone("Europe/Madrid")
config = {DOMAIN: [{CONF_NAME: "test_dst", ATTR_TARIFF: "discrimination"}]}
mock_data = {"return_time": datetime(2019, 10, 27, 20, 0, 0, tzinfo=date_util.UTC)}
def mock_now():
return mock_data["return_time"]
with patch("homeassistant.util.dt.utcnow", new=mock_now):
assert await async_setup_component(hass, DOMAIN, config)
await hass.async_block_till_done()
caplog.clear()
assert pvpc_aioclient_mock.call_count == 2
await _process_time_step(hass, mock_data, "price_21h", 0.13896)
await _process_time_step(hass, mock_data, "price_22h", 0.06893)
assert pvpc_aioclient_mock.call_count == 4
await _process_time_step(hass, mock_data, "price_23h", 0.06935)
assert pvpc_aioclient_mock.call_count == 5
# sensor has no more prices, state is "unavailable" from now on
await _process_time_step(hass, mock_data, value="unavailable")
await _process_time_step(hass, mock_data, value="unavailable")
num_errors = sum(
1 for x in caplog.get_records("call") if x.levelno == logging.ERROR
)
num_warnings = sum(
1 for x in caplog.get_records("call") if x.levelno == logging.WARNING
)
assert num_warnings == 1
assert num_errors == 0
assert pvpc_aioclient_mock.call_count == 7
# check that it is silent until it becomes available again
caplog.clear()
with caplog.at_level(logging.WARNING):
# silent mode
for _ in range(21):
await _process_time_step(hass, mock_data, value="unavailable")
assert pvpc_aioclient_mock.call_count == 28
assert len(caplog.messages) == 0
# warning about data access recovered
await _process_time_step(hass, mock_data, value="unavailable")
assert pvpc_aioclient_mock.call_count == 29
assert len(caplog.messages) == 1
assert caplog.records[0].levelno == logging.WARNING
# working ok again
await _process_time_step(hass, mock_data, "price_00h", value=0.06821)
assert pvpc_aioclient_mock.call_count == 30
await _process_time_step(hass, mock_data, "price_01h", value=0.06627)
assert pvpc_aioclient_mock.call_count == 31
assert len(caplog.messages) == 1
assert caplog.records[0].levelno == logging.WARNING