(4.2.5) PvP alerts triggering despite being off

Bug Reports
8

It looks like this should be fixed for 4.2.5 but we need to test and confirm (at a later point).