Home Assistant Remote Access networking

2019-12-13 Jan Olbrecht 0 Comment 12:28

Categories:

journal

I’ve written this guide for the documentation pages of the Home Assistant iOS companion app. As I’m rather proud of them I’m republishing the guide here. Jan

Companion app networking

Having your Home available anywhere and everywhere you go is important, whether you forgot to turn off the stove or you want to check the camera views because of an alert.

Because we want your smart home to be private and secure on the web, many parts of the puzzle need to align just right so everything works as you expect. This guide aims to help you understand the requirements, some of the complexities and our recommended typical solutions to setting up network access to your Home Assistant instance:

The basics: How the app talks to your Home Assistant

In order for the app to talk to HA, it needs to know its address. Just within your home network you might know that your Home Assistant is on an IP like 192.168.1.4 and listening on port 8123. If you use Hass.io and haven’t changed any of the defaults, Home Assistant will also be reachable at http://hassio.local:8123.
This is all fine and will work perfectly well as long as you never take your phone or tablet outside your home, but what if you do?
The easiest way is to subscribe to Nabu Casa Cloud for the small monthly fee of $5 US, which will solve all this for you and you can (almost) stop reading here, as well as supporting further development of Home Assistant. Nabu Casa Cloud acts as a “smart” proxy on the internet, tunnelling your frontend in an encrypted manner from your home to your phone, regardless where you are and without requiring opening your home network to inbound traffic from the internet.
If you don’t want to use Nabu Casa Cloud (which is fine, but you should still subscribe and enjoy the warm feeling of supporting Home Assistant), you need HA to be accessible from the internet. This requires opening a port on your router and getting a name for your Home Assistant on the internet. While it is possible to have your HA use Port 8123 internally and have your router do a port-forwarding from say the default https port of 443 to 8123, we recommend you NOT do this for reasons of simplicity which we will explain later. You also need a name for your Home Assistant as hassio.local is a private domain suffix that does not exist on the internet.

Dynamic DNS

Most non-business internet connections have at least one of two drawbacks: Your internet service provider typically does not give you a static IP (meaning the public IP address your modem/router is assigned will change every once in a while or even every 24 hours) and some ISPs don’t even give you a “real” IP address as they do not have enough addresses to give out. This last scenario is very common on cable providers and especially in Asia/Pacific. If your ISP says they use Carrier-grade NAT (CG-NAT) or something like Dual Stack lite (DS-lite) you likely will have this problem. If you’re impacted please see the CG-NAT and IPv6 addendum.
For dynamic, public IP addresses the solution is simple: Typically users choose a dynamic dns service such as duckdns.org which will create a unique name (e.g. my-home.duckdns.org) that is supported to be updated via your router to always point to your public address. If you have created the port-forward of TCP 8123 on your router’s public interface to TCP 8123 on your internal Home Assistant IP (say 192.168.1.4), your Home Assistant is now available on the web. You could declare victory at this point and stop but don’t – because everything at this point is unencrypted and we want you to enjoy HA in a private, secure manner.

Hairpin NAT

At this point of setting up we need to check one capability of your router: Hairpin NAT (otherwise known as NAT reflection or NAT loopback). What this means is the ability of your router to mirror a request from its inside (LAN) interface to its outside (WAN) address back to an internal IP address (in this case, your Home Assistant), thus reflecting or hairpinning the traffic. It’s easy to check if this works: Just open a browser on your phone or PC while connected to your home network and opening http://my-home.duckdns.org:8123 – if it works, you have hairpin NAT working and can go on to the next section. Most current routers will support NAT hairpinning out of the box, there are however some routers (especially if you got your router from your ISP) that do not have this ability or have it disabled. If this is the case, you need to check if you can enable it on your router or, if you can’t, you will need to set up Split Brain DNS.

Securing the connection

We’ll stay with our DuckDNS example. Using http://my-home.duckdns.org:8123 works, but anyone could be reading your traffic. Let’s change that! The DuckDNS Hass.io add-on will create a free, trusted and valid LetsEncrypt SSL certificate to use on your Home Assistant. Just follow the installation instructions here and here and you will have secure, public access to your Home Assistant. What’s great about using the DuckDNS add-on is that it uses the LetsEncrypt DNS challenge, whereby during requesting the certificate it proves “ownership” of the domain by creating a temporary DNS record. If you use a different DNS provider other than DuckDNS, you can use the LetsEncrypt add-on for Hass.io which supports proving ownership of the name either via the DNS or the http challenge. The latter requires port-forwarding TCP Port 80 on your router to your internal Home Assistant IP on TCP Port 80.

With Hairpin NAT working and SSL on your DNS domain you can now access Home Assistant securely both on the internet and at home and you should add base_url: my-home.duckdns.org:8123 to the http: section of your configuration.yaml. This is not strictly necessary but will help with auto-detection during onboarding of the iOS app, as the app will know where and how to reach your Home Assistant.

Split Brain DNS

So what’s this split brain DNS (also known as split horizon DNS, split-DNS) thing and why would I need it? If your router doesn’t do hairpin NAT, you still need to access your Home Assistant via the public DNS name, e.g. my-home.duckdns.org. Why is that? Because valid encryption via https and SSL certificates only works for public DNS names. What this means is that the certificate name on your server needs to match the DNS name you enter in your browser or app. This is fine with hairpin NAT available but becomes an issue when it’s not. In this case you need to “split” the answer your browser/app gets when it looks up the IP address behind my-home.duckdns.org – you need one answer for devices on your home network that points to the internal IP address of your Home Assistant (e.g. 192.168.1.4) and another answer for when you’re out and about [e.g. 104.25.25.31.
The easiest solution to this is to use the Hass.io add-on AdGuard Home. This can also be set up on some routers (e.g. pfSense or UniFi Security Gateways) but we’ll continue on using our example guide with the tools provided via Hass.io: So you’ve now installed the AdGuard Home add-on and changed the DNS server on your router DHCP settings to the address of your Home Assistant. You should now go to the AdGuard Home setting page in your Hass.io panel and there go to Settings -> DNS settings, then scroll down to the bottom where you have a box titled: DNS rewrites
Here you click Add DNS rewrite and enter your my-home.duckdns.org and the internal IP 192.168.1.4 of your Home Assistant, followed by clicking on save. What happens now is that all DNS queries for the address my-home.duckdns.org from inside your home network will be answered by AdGuard via its own rewrite table, thus pointing toward the internal address of your Home Assistant instead of asking public DNS servers on the web which will all answer with the public IP of your router.
Even if you don’t need split brain DNS, you may also want to set this up as it will enable you to access Home Assistant via it’s public name even when your internet connection is down and hairpin NAT won’t work. One less dependency on the Cloud!

Setting up the iOS app

If you’ve followed all our advice, your app should find your Home Assistant instance automatically during onboarding when connected to your home wifi network. You can also go through onboarding anywhere you’re connected to the internet by manually entering https://my-home.duckdns.org:8123 and the setup will finish with that address in the External URL field in the app connection settings. There should be no need to enter an internal URL as the same address will work regardless of where your phone is connected.
If you want to (or have to) use Nabu Casa Cloud, there are two more steps required:
– In iOS settings, set location access permissoin for Home Assistant to Always. This is required because starting with iOS 13, Apple will only let apps with such permission have access to the wifi SSID which is used by the app to determine whether to use internal or external URLs.
– Once permission is given, add your Home Assistant address to internal URL (if you come from the top of this article, this could be http://hassio.local:8123
– If you’ve set up Nabu Casa Cloud in your Home Assistant the checkbox to “Connect via Cloud” should now become available. Once you activate the checkbox, external URL will become deactivated.

Addendum: CG-NAT

If your ISP doesn’t give you a public IPv4 address you’re down to basically only two solutions: You can call your ISP and ask if they can give you a real address or if there is an upgrade for your connection available (oddly enough, asking nicely will work with many ISPs out there) or use Nabu Casa Cloud.

Addendum: IPv6

Since IPv6 has been rolling out for the last 20 years, chances are that along with an IPv4 address your home network will also have been provided with IPv6 addresses from your ISP. So your Home Assistant host may have it’s IPv4 address of 192.168.1.4 as well as an IPv6 address of e897:5571:5f66:21dc:51c1:28d8:3bdc:6724. Here’s where our advice for not changing the TCP port you forward to Home Assistant comes in:
– Home Assistant will listen for traffic on 192.168.1.4:8123 and [e897:5571:5f66:21dc:51c1:28d8:3bdc:6724]:8123
– If you really want to future proof your setup, you will have two DNS records for my-home.duckdns.org: An A-record pointing to your routers public IPv4 address which will be port-forwarded to your HA hosts internal address and an AAAA-record, which points directly to the IPv6 address of your HA host. Now when you access your HA remotely either protocol could be used, since all you’re entering will be https://my-home.duckdns.org:8123. If you had changed the Port on your Router to the https default 443, the connection would now fail if you suddenly ended up with a working IPv6 setup as nothing is listening on [e897:5571:5f66:21dc:51c1:28d8:3bdc:6724]:443.

Addendum: Reverse Proxy via NGINX

There are cases when having Home Assistant serve https is impossible or incompatible with some of your devices. This can be especially true with ESP-based low power IoT hardware that communicates via RestAPI and just doesn’t have the horsepower to do the SSL encryption. One example is the konnected.io Integration which requires Home Assistant to be reachable via http.
So to accomodate this and still have encryption for external access, we use a reverse proxy like NGINX. What a reverse proxy does is to act as an intermediate for your clients (Browser or App). The client talks to the reverse proxy securely via https and the proxy passes through this traffic to Home Assistant over an unencrypted http connection. Staying with our Hass.io example, we’ll assume you have already set up DuckDNS and LetsEncrypt. You should now install the Hass.io add-on NGINX Home Assistant SSL proxy and configure it according to the docs.

In your configuration.yaml file the following changes are needed:

http:
  use_x_forwarded_for: true     # To ensure HA understands that client requests come via reverse proxy
  trusted_proxies:
    - 172.30.32.0/23            # In Hass.io we need to add the Docker subnet
    - 127.0.0.1                 # Add the localhost IPv4 address
    - ::1                       # Add the localhost IPv6 address
  base_url: my-home.duckdns.org # Note we no longer have a :8123 Port here
  # Uncomment or remove the SSL certificate lines:
  # ssl_certificate: /ssl/fullchaim.pem
  # ssl_key: /ssl/privkey.pem

Once that’s done your router’s port-forwarding should be TCP 443 to your Home Assistant internal IP 192.168.1.4 Port 443. Do NOT create a forward to 192.168.1.4 Port 8123 as that is now unencrypted http and should only be accessible from your local network.
You can now access your Home Assistant via https://my-home.duckdns.org both internally and externally while having http://192.168.1.4:8123 available to be used as unencrypted endpoint for things like konnected.io.
Note: If you don’t use the NGINX Hass.io add-on but instead roll your own, please ensure that websockets support is enabled.

Maintenance and Non-OEM sparesMaintenance and Non-OEM spares

So I need to replace the battery on my trusty vacuum bot iRobot Roomba 780. Judging from my purchase history, I have had the robot for approx. 18 months when I replaced the old original battery with a knock-off spare claiming 50% more capacity (4500mAh vs 3000mAh on the original pack).

By now, 25 more months have passed and I need to get another replacement. Just looking at the timeline, there is precious little reason to even think about getting the original part when the knockoff lasted longer than the original one.

So in case anyone is wondering, here’s what I’m getting:

-Jan

Netgear ReadyNAS PRONetgear ReadyNAS PRO

For the last couple of months, I have been part of the betatest for the Netgear ReadyNAS PRO Business Edition. As the product has been released, I can now tell my story.

First of all, some history:

I’ve been a user of the PRO’s predecessor, the ReadyNAS NV+ for almost 3 years now. Its primary use has been that of a file and media streaming server. I have also had another NV+ in use as part of my employers server infrastructure, where it also works as a file server for archive and storage.
There were a couple of things that made me choose the ReadyNAS NV+ over its competition, but mostly I was convinced by reviews such as this, along with user feedback in forums and comments. For the most part, I can truthfully say I have been very happy with the NV+, as it has been rock-solid. There were no crashes or problems and configuration – at least for my usage scenario – has been dead simple.
There was only one minor gripe: Once you have a couple TBs of data, or your job includes moving hundreds of gigs of data on a regular basis, network throughput really comes to the foreground. With the NV+, under most if not all real-world applications, throughput maxed out around 30MB/s. This might not sound all that bad, considering most people still use 100Mbit networking, which tops out at around 12MB/s – but Gigabit Ethernet can move 120+MB/s and I’m an impatient fellow.

Along comes the ReadyNAS PRO, which is in most user regards an evolutionary step over the NV+:

6 hot-plug drive bays instead of 4
two Gigabit Ethernet ports (with support for teaming and failover) instead of one
x86 architecture instead of SPARC
support for RAID6
support for X-RAID2 (more on that later)

Everything else, from a user perspective is pretty much the same:

The ReadyNAS Pro uses the same interface (FrontView)
it supports the same file protocols (SMB/CIFS, AFP, NFS, FTP(S), HTTP(S))
it supports the same streaming media solutions (UPnP AV, iTunes, SqueezeCenter, Home Media Streaming Server)
DHCP, WINS, Printer Sharing
Active Directory support
UPS support

So why choose the PRO over its much cheaper predecessor, the NV+? One word: Performance!

The ReadyNAS PRO is by far the fastest networking device I’ve seen yet. To be honest, I have no idea how fast it really is, and from my experience during the beta phase, no-one else really knows either.
Why is that, you ask? Well, it turns out that no-one within the group of testers had a client available that could serve data at faster than gigabit speeds. I was in fact planning on using IEEE 802.3ad to hook up a much faster server as well as the PRO to my network – but incompatibilities with my network hardware kept me from testing it. I’m still trying to track down some Cisco hardware to have another go at this but for the time being, I will refer you to the official testing data from Netgear, which I can personally tell you to be true.

In use, the limiting factor for me has been my client computer (a ThinkPad T61 with a 200GB 7.200rpm Hitachi HDD), where I’m regularly seeing transfer rates between 50-60MB/s – a huge improvement over the NV+ which gave me 30MB/s only on a good day (I had some issues with data fragmentation as the NV+ was nearly constantly filled to max capacity…). On a side note: If you use Vista, *don’t* have any kind of media files or players open because that just murders network throughput (more on this issue here).

So what else is there to say:
The one factor that lead me to the ReadyNAS line of products was its proprietary X-RAID technology. Let me explain about this for a short while:
RAID stands for Redundant Array of Independent Disks. This means that a disk failure will not lead to you losing data. There are basically three types of RAID terminology you need to know about:

RAID 1: Mirroring – best for two disks, you loose 50% of your gross capacity
RAID 5: Striping – starting with three disks, parity information is spread so that any one disk failure will not lead to data loss
RAID 6: Striping with “hot spare” – starting with four disks, parity information is spread so that and *two* disk failure will not lead to data loss

There are also other kinds of RAID you can use like RAID 10, 50 or 60 but I won’t go into those as they don’t really apply for a NAS box. Back to the topic though: Typically, once you’ve decided on one version of RAID for your setup, you cannot go back. You cannot upgrade to larger disks, you cannot change much of anything whatsoever. That is, unless you have X-RAID:
The ReadyNAS has this trick up its sleeve which allows you to go from one disk to two (when it sets up a mirrored RAID 1 volume), then three (converting the RAID 0 to a RAID 5), up to 4 (with the PRO 6) disks. During the latter stages, all the NAS does is to expand the volume and restripe the parity information so it is evenly spread among disks. Netgear calls this horizontal expansion.
Once you hit the maximum amount of disks you could put into your box, there was only one way to upgrade to even higher capacity: Swap out each disk one-by-one for a larger capacity model, wait for resync, and once all the disks have been swapped, wait for the volume expansion to take place. With the ReadyNAS PRO, X-RAID2 is adding a new trick: vertical expansion. The moment your drive swaping allows for a redundant setup of two larger capacity disks, the NAS will expand the volume to include whatever new diskspace is available in a redundant setup. What’s that mean? Here’s an example:

Say you have 4 500GB hdds already in place and you need even more space, adding two 1TB drives would usually have lead to an additional 1TB of space – because in a RAID5 volume, all disks can only as much to the volume as the smallest disk in the array. With X-RAID2, the ReadyNAS PRO detects that there are two drives with an unused capacity of 500GB each, which it then uses to extend the volume by another 500GB, which are in a mirrored setting. Neat.

Then again, who cares? In reality, most people will probably populate the NAS with all the disks they plan to put in it from the start and never think about it again until something breaks or they run out of space. Only then does this feature become important. What’s it good for then? Peace of mind!

I have pretty much summed up most I had to say, except for one thing: It’s quiet. Really quiet. Netgear did a great job designing the hardware for this thing as all my disks have been at below 40Â°C and I’ve had this thing on my desk without the fans feeling annoying.

To sum it up: If you’re looking for a fast, stable and userfriendly device to store your data, the ReadyNAS PRO will be the best choice currently on the market. It may be more expensive than its competition – but ease of use and maturity do count for something, most certainly in a business environment. Finally, a five year warranty along with the stellar support from the ReadyNAS team over at www.readynas.com/forum should put your mind at ease… it’s certainly convinced me.

-Jan Olbrecht

My acknowledgements and thanks go to the ReadyNAS PRO beta tester group and the Jedi Council over at the ReadyNAS Forum.

Fixing Windows Admin Center ‘Can’t verify whether “cluster_name” is online’Fixing Windows Admin Center ‘Can’t verify whether “cluster_name” is online’

So you’re trying to add your Hyper-Converged Cluster to Windows Admin Center and it’s giving you the “Can’t verify whether “cluster_name” is online” treatment. You’ve checked DNS, upgraded WAC/Honolulu and tested installing it on multiple servers and workstations. Nothing helped. I have good news! for you:

I ran into this immediately after Project Honolulu became public and have been banging my head continuously. Here’s what to do:

Check the Event Viewer\Applications and Services Logs\Microsoft-ServerMangementExperience for the following entry:

400 - CimException: The xsi:type attribute (p1:MSCluster_Property_Node_PrivateProperties) does not identify an existing class.

This indicates your that WAC is connecting fine to your cluster but is running into an issue where it’s missing some cluster property.

I’ll have to give props to Robert Hochmayr here as he pointed me to the solution:

There are two private properties that are set on the cluster and its nodes which through some combination of events (like adding nodes to the cluster at a later point in time) are missing from nodes. You can find out by running the following PowerShell command on one of your S2D cluster nodes:

get-clusternode | Get-ClusterParameter

The output will look something like this:

Object Name Value Type
------ ---- ----- ----
S2D-01 S2DCacheBehavior 88 UInt64
S2D-01 S2DCacheDesiredState 2 UInt32
S2D-03 S2DCacheDesiredState 2 UInt32
S2D-03 S2DCacheBehavior 88 UInt64

Note that this was a four node cluster.. Nodes S2D-02 and S2D-04 are missing!

Off to the registry to fix it:

At HKLM\Cluster\Nodes\x\Parameters there should be two entries for the above cluster parameters. On my systems, the full registry key Parameters was missing from nodes 1 and 4 (go figure…). I added them *on each host* by running the following command lines:

REG ADD HKEY_LOCAL_MACHINE\Cluster\Nodes\1\Parameters /f /vÂ  "S2DCacheBehavior" /t REG_QWORD /d "88"
REG ADD HKEY_LOCAL_MACHINE\Cluster\Nodes\1\Parameters /f /v "S2DCacheDesiredState" /t REG_DWORD /d "2"
REG ADD HKEY_LOCAL_MACHINE\Cluster\Nodes\4\Parameters /f /vÂ  "S2DCacheBehavior" /t REG_QWORD /d "88"
REG ADD HKEY_LOCAL_MACHINE\Cluster\Nodes\4\Parameters /f /v "S2DCacheDesiredState" /t REG_DWORD /d "2"

Checking I now get the correct PowerShell output:

get-clusternode | Get-ClusterParameter

Object Name Value Type
------ ---- ----- ----
S2D-01 S2DCacheBehavior 88 UInt64
S2D-01 S2DCacheDesiredState 2 UInt32
S2D-02 S2DCacheBehavior 88 UInt64
S2D-02 S2DCacheDesiredState 2 UInt32
S2D-03 S2DCacheBehavior 88 UInt64
S2D-03 S2DCacheDesiredState 2 UInt32
S2D-04 S2DCacheBehavior 88 UInt64
S2D-04 S2DCacheDesiredState 2 UInt32

Once this was added I was immediately able to add the cluster to Windows Admin Center. No reboots or service restarts were needed.