When you configure a different time server (NTP server) under OS X, the System Settings panel doesn't actually tell you that the address is valid or not. To check this, open a terminal window and use the ntpq command as follows:
$ ntpq -p ntp1.example.com
The time server should list its sources and the command should display something roughly as follows:
remote refid st t when poll reach delay offset jitter ============================================================================== +kl10abav.xx.xxx 193.79.237.14 2 u 1030 1024 377 1.904 4.659 0.150 -kl10ab9n.xx.xxx 193.79.237.14 2 u 924 1024 377 1.805 4.721 0.100 *ntp1qvi.xxxxxxx .GPS. 1 u 921 1024 377 24.874 -1.048 0.330 +ntp1tls.xxxxxxx .GPS. 1 u 705 1024 377 28.317 -1.072 0.211
If there isn't actually a time server running, then ntpq will report:
nodename nor servname provided, or not known