My banking app refuses to work with the VPN on.

Some banks block traffic they cannot place geographically. Two fixes: connect to a server in your own country so the address matches where the account is, or use split tunneling to leave that one app outside the tunnel while everything else stays protected.