Thank you very much Penzoiders, every answer is super clear.
tl;dr; according to what said, and considering that it is impossibile to add the cluster directly to a vpn or at least protect it with nginx client certificate verification, the only feasibile solution is to add another piece of redundant hardware, right?
It was a good news overall, until i've read your last statement:
we are in a server farm under a floating ip dual network gateway. if it was possibile to use a different subnet i would be able to use the gw as canary, but with the notes you added the only possibility i see i adding another piece of hardware in the network, same subnet as my LAN network, to avoid exposing the cluster ip.
i think that all considered you should really add this to the requirements of the cluster:
exposing the cluster ip to the internet is the worst idea possible, as you said, so the only feasible(and best) solution implies the creation of a vpn. but if you want to create the vpn inside the cluster(as we would) you actually cant stop here, because the canary MUST be outside the cluster(for the reasons you stated, which are exactly what i was thinking about writing the first post, also considering the management ip of the switches) but in the same network of the cluster ip, so the lan, so it needs another piece of hw, which must be HA to respect dogmas of NW, which in my opinion adds complexity to the infrastructure.
I hoped the solution would be having the canary ip in a different subnetwork(pointing to the gateway, in my case), with also the addition that it would be very awesome if it could be possible to tag separately the canary ip(and network conf) and the cluster network configuration, having different network setup(which should include vlan tag) for the canary and the cluster network.
I also think that the https tunnel you are suggesting, if coming from a vm inside the cluster, would again make the cluster unreachable in case of the vm not booting for reasons(missing chunk or first boot of cluster), so also for this solution i need an external appliance.
I would really appreciate the possibility to avoid adding extra complexity to the network, any solution would be super-appreciated!
thank you in advance again!
alberto