Dec 29, 2020

Upgrading to Openhab 3

For my home automation system I use OpenHab.
I started using it in 2017, when setting up the automation for my new apartment.
I tried Home Assistant, which I slightly preferred, being in python, but the use of yaml for expressing rules put me off. I want to be able to express rules with “imperative” code, not with a “declarative” approach.

A couple of days ago, I migrated to the new OH version, 3.0. It is a major release, dropping compatibility with older integrations and deprecating the current set of rules, but still maintaining a “scripting” approach; also, it introduces the new concept of “semantic model”, which requires however changes in the way things and items are managed.
During the last months, I prepared my setup, moving away from older bindings (e.g. Telegram action) to use the new ones; still I am leaving the old approach for modeling, since I want to have everything as a text file in git.

In such situation, the upgrade was supposed to be quite a simple operation.
Obviously, things did not go as smooth as expected.

The upgrade

I used the list of breaking changes to determine the changes to my script; mostly, it was a matter of changes in datetime conversions.
Also, I took an extra backup, as additional security.

For the upgrade, I used the functionality from openhabian-config; other times I was using just apt, but lately I wanted to delegate more and more the administration to this tool.
During migration, I had some error messages. Unfortunately, openhabian-config was hiding the errors, but they usually went away at the second try.

File permissions

After migration, there were a lot of issues due to wrong file permissions and I had to reset them, every time restarting openhab to check if anything was yet to fix:

sudo chown -R openhab:openhab /var/log/openhab
sudo chown -R openhab:openhab /var/lib/openhab/cache
sudo chown -R openhab:openhab /var/lib/openhab/tmp

Frontail

After migration, frontail was not working any longer; as a very first measure I tried to reinstall/upgrade it to the latest version, but it took quite a long time and it was not necessary, because it was a matter of configuration: the frontail configuration had to be updated, because it was still pointing to the old openhab2 locations:

sudo sed 's/openhab2/openhab/g' /etc/systemd/system/frontail.service
sudo systemctl daemon-reload
sudo systemctl restart frontail.service

Rules update

As expected, few rules had to be updated, because of changes in date time. The biggest change was in rules creating dates from epoch as it is was no longer available; however the rules improved quite significantly so it was a welcome change.
Updating from > to isAfter() required some more effort, because in some cases it was not that self-explanatory, but again, I feel now those rules are actually better.
I had to make some smaller changes during the next couple of days, for rules that I failed to detect, or changed in the wrong way; but this was a minor thing.

Main changes:

  • getHourOfDay -> getHour

  • Do not use millis any longer.
    Very often, I was taking the milliseconds part of now, in order to compare with a date time. This is no longer working, and has been replaced by a different approach:

    var Number now_ms = now.millis
var Number wifi_LastSeen_Millis = (wifi_LastSeen_Seconds.state as Number) * 1000
if ( (now_ms - wifi_LastSeen_Millis) < 4*60*1000 ) { }

    becomes

    val wifi_LastSeen_DateTime = (wifi_LastSeen.state as DateTimeType).getZonedDateTime()
if (wifi_LastSeen_DateTime.isAfter(now.minusMinutes(4))) { }

    which is definitely better as regards readability.

  • Some items were stored as milliseconds; but now they are always stored as date time. Groups containing those items, are now date time, and aggregation becomes LATEST instead of MAX:

    Group:Number:MAX Roberto_Wifi_LastSeen "Roberto last seen [%d]" (WifiStatus)

    becomes:

    Group:DateTime:LATEST Roberto_Wifi_LastSeen "Roberto last seen [%1$td.%1$tm.%1$tY %1$tH:%1$tM]" (WifiStatus)

  • Warnings for global variables. Global (module) variables, might give a strange warning on startup. I think it is not important; nevertheless, I tried to remove all globals where possible.

  • The way to update DateTimeItem to the current time changed:

    val lastCloseTime = new DateTimeType(now.toString)
DoorLastClosedTime.sendCommand(lastCloseTime)

    becomes:

    DoorLastClosedTime.sendCommand(now)

    I am not sure why, but it works, while the older approach is not working any longer :)
    Note also that I am pretty sure there is a way to do the same from configuration, using profiles (and in particular timestamp-update) but this code is older than profiles, and I have never updated/investigated.
    Also, at the best of my knowledge profiles are managed through UI, and I am not really willing to move too much into that direction.

Issues with bindings

After update, on the karaf log started to appear warnings about “XStream not initalized”; after some analysis they turned out to be related to the Z-Wave addon, but it does not appear to be a major issue. There is already a GitHub issue; somehow, also from a cursory look at all the commits in the last few months, the binding looks a bit abandoned, but this is a different topic.

Z-Wave binding starts now quite late, and it does some heavy polling of all devices, leading to disrupted operations for about 20-30 minutes because of the excessive traffic. But this is only after restart, after that it is fine.
Note also that one side effect of such delay is that for few minutes all z-wave items will have NULL state; this might lead to some errors during the first few minutes after startup, unless script will take that case into consideration.

The Broadcomm IR blaster binding is currently not working with OH 3.0, but lately I rarely use it, so it is OK if for the moment it’s not working.
Since this is not the first time that I am in a similar situation, I might think about a small separate service to send those commands, communicating through MQTT.

Non existing bindings

I had quite a lot of warnings for non-existing bindings, like http1 or paper.
The reasons is that they were still listed in /var/lib/openhab/config/org/openhab/addons.config and they had to be removed manually from that file.

InfluxDB

There were also some issues with persistence.
The reason is that OH2.5 persists switch data type as float, while OH3.0 uses int.
This requires some migration of data in influxdb, applying something like

SELECT value::integer INTO temp FROM xxx;
DROP MEASUREMENT xxx;
SELECT value::integer INTO xxx FROM temp;
DROP MEASUREMENT temp;

for each broken measurement (in my case, only one).

Logging

The log configuration now has changed, and is managed through standard log4j configuration, in /var/lib/openhab/etc/log4j2.xml. I added the following changes:

  • Since z-wave is usually a bit problematic, I want ot have it on debug log, on a different file.
  • Disabled xtext.ide as it would dump errors while editing files in VSCode.
<!-- Zwave file appender -->
<RollingFile fileName="${sys:openhab.logdir}/zwave.log" filePattern="${sys:openhab.logdir}/zwave.log.%i" name="ZWAVE">
    <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5.5p] [%-36.36c] - %m%n"/>
    <Policies>
        <OnStartupTriggeringPolicy/>
        <SizeBasedTriggeringPolicy size="16 MB"/>
    </Policies>
</RollingFile>


<Logger additivity="false" level="DEBUG" name="org.openhab.binding.zwave">
    <AppenderRef ref="ZWAVE"/>
</Logger>

<Logger level="OFF" name="org.eclipse.xtext.ide.server.concurrent.ReadRequest"/>
<Logger level="OFF" name="org.eclipse.xtext.ide.server.concurrent.WriteRequest"/>
<Logger level="OFF" name="org.eclipse.xtext.ide.server.WorkspaceManager"/>

Openhabian

This is just a cosmetic issue; but the welcome message for openhabian should be updated as well, and this is accomplished by changing .bash_profile:

OHVERSION=$(dpkg-query --showformat='${Version}' --show openhab) OHBUILD=$(sed -n 's/build-no\s*: //p' /var/lib/openhab/etc/version.properties)

Afterthought

After few days running, overall, I think OH3 is a good release.
Logs are clean, and the entire automation feels… snappier. I do not know how to better describe; but lately I had quite a few problems related to running my automation, with things working erratically, forcing to periodic system restarts with no guarantee of improvements.
Now, since the migration, everything is working fine.
Maybe I am too optimistic: most of my automation uses Z-Wave, and there the code is essentially the same; maybe in few days it will become slower, and erratic, and I will have to restart the system every now and then.
Still, even in Z-Wave there are small differences (e.g. the synchronization at startup), so that might be the reason why now it’s better. Also, there are a ton of other changes, and that could have worked to improve stability as well.

Anyhow, I want to be optimistic :)