For CPU, setting the Linux scheduler to the Nice level for a VM process can limit effect of CPU DoS. Using cgroups, virtual machines can also be restricted to CPU limits which supports the prevention of denial of service.

For memory, reducing or disabling swap puts limits on memory DoS.

Even without any changes to Host DoS can be effectively mitigated:
Limiting the number of logical CPUs for a VM in its settings limits the amount of Host cores it can use.
Linux's OOM-killer will identify and kill the offending VM process if causing the host to run out of memory.

virsh can set disk activity limits per vm:

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Tuning_and_Optimization_Guide/sect-Virtualization_Tuning_Optimization_Guide-BlockIO-Techniques.html

In T567#10647, @HulaHoop wrote:

blkiotune and iotune can restrict io (KVM only)
https://libvirt.org/formatdomain.html#elementsBlockTuning

There's a problem with setting this. SSD vs HDD io throughput is very different. What is reasonable for one will be excessive or too low for the other.

HulaHoop (HulaHoop):

HulaHoop added a comment.

There's a problem with setting this. SSD vs HDD io throughput is very different. What is reasonable for one will be excessive or too low for the other.

Set the limit to protect against excessive use of SSD then. SSD users
are better off then and HDD users not any worse. (HDD users would have
to manually adjust that value.) (Or auto detect somehow?)

Though I agree with anonym's argument that resource exhaustion goes against the purpose of advanced malware that wants to hide - I still looked at io limits in case you still think its valuable to set.

Turns out blkiotune can partition resources using "weighting" aka percentage besides setting arbitrary limits.

I think by "host block device" they refer to the physical device /dev/sda rather than getting into partition numbers like sda2. The latter would have made this option very messy.

https://www.debian.org/releases/stable/i386/apcs04.html.en

https://libvirt.org/formatdomain.html#elementsBlockTuning

HulaHoop (HulaHoop):

HulaHoop added a comment.
Though I agree with anonym's argument that resource exhaustion goes
against the purpose of advanced malware that wants to hide

Maybe. However, I wouldn't assume that no type of malware ever tries
resource exhaustion.

I still looked at io limits in case you still think its valuable to set.

Yes, very much worthwhile. Should have always been a standard feature of
any virtualizer. It's what one can reasonably expect, that no VM (or
application) is capable to exhaust resources so everything else suffers.
Even if not malware, but just a bug, it still is a huge usability issue
to run into cases where one thing can look up the whole computer.

Should limits be enforced for GW too?

Yes.

Done. Added io limit commits to open pull requests. Each vm can only use a maximum of 25% of the host io resources.

cgroups were mentioned by @madaidan

Cgroups may be a better option for that. Systemd uses cgroups and user sessions are run under a systemd slice by default AFAIK so we can just edit that https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html

We should be able to create a drop-in file at /lib/systemd/system/user-.slice.d/ and add something such as

[Slice]
MemoryAccounting=yes
MemoryMax=4G

which would prevent all users from using more than 4gb of memory.

There's more documentation on user slices at https://www.freedesktop.org/software/systemd/man/user@.service.html