We would expect this to be a simple transport level security scenario.Think again. I had this question answered from Shy on this at techready.
The point is that IPSec removes any trace of security and so by looking at the packet WCF cannot say if it is secured or not. Hence if you had a operation contract that required some kind of protection level and you expected this requirement to be satisfied by IPSec then think again. If you plan to use IPSec as a deployment option and then you will have to remove this attribute decoration on the operation cause the underlying transport cannot claim that its secure.
Another work around would be to use a custom transport that claims it is secure and you can turn on IPSec on that. This would enable the usage of ProtectionLevel on an operation but WCF cannot gurantee the security as it takes the custom transport claim unless ofcourse you can somehow query and figure out if IPSec is enabled. This is left as an excercise.