Home Automation Server
Home automation is a concept that can be confused with smart-home. They are not the same thing, as home automation is something that has been going on for quite a while, but smart-home refers to a more modern concept.
Home Automation is a more general concept, which refers to make your home capable of performing action by itself depending on triggers, events or timing. This can be more or less smart.
The smart-home concept instead is more tightly coupled to the IoT (Internet of Things) and in general, COTS based environments. This is a more recent concept, but i don't really like it because it always match up with proprietary environments, closed-in technology and, in the best case, stuff that relies on the cloud to perform it's actions (and, in the process, ties you in an ecosystem and probably resell your privacy too).
I strongly suggest that you use a dedicated Home Automation Server. While you could use the home server to host the automation server as well, i prefer to have two discrete systems so that general services will not affect home automation and vice-versa.
Considering that home automation is something critical, which might be controlling your heating / cooling, your blinds and maybe also your intrusion detection, motion cameras and alarm system, i believe that is much better to have it independently located on it's own hardware.
From a software point of view, i choose and love Home Assistant, so i cnanot but recoment to purchase a Green to support it's development. Short of a dedicated purchase, you can also repurpose some older hardware.