When should I use ipsec transport mode?
I basically understand how tunnel mode and transport mode works, but I don't know when I should use one instead of another.
Among the two parties who want to communicate, if one computer B doesn't understand IPsec, I think they have to use tunnel mode, which puts original IP and payload into ESP and delivers the packet to a device near B who knows IPsec, and that device decrypts the packet and sends the decrypted packet to computer B. But what if the two computers both know IPsec, can I use transport mode? Various articles mention that if two computers are in an intranet, use transport; if they are in different networks, use a tunnel. Why? If two computers are in different networks and transport mode is used, what problem will happen?
In IPSec transport mode, only the IP payload is encrypted, and the original IP headers are left intact. It also allows devices on the public network to see the final source and destination of the packet. With this capability, you can enable special processing in the intermediate network based on the information in the IP header. However, the Layer 4 header will be encrypted, limiting the examination of the packet. Unfortunately, by passing the IP header in the clear, transport mode allows an attacker to perform some traffic analysis. Having quoted that, this is almost never the case - the underlying payload is most often a tunnelling protocol (e.g. L2TP in case of Windows or mobile clients, could be GRE), so in modern networking terms there are "mitigating" factors against traffic analysis in transport mode.