Another you can do is just firewall the regular port and whitelist your static IPs, that way there is no way for someone to get in. The only downside is that if you are on vacation or something and need to SSH in you will need to login to interworx first to add your current IP to the whitelist or disable the block.
I’m late on replying to this post, however I’ll recommend another option in case someone needs it.
Instead of allowing your shell users to access their accounts using passwords consider using keys or Passwordless logins. I would also consider disabling the ability to login as root. I could go into detail, however there are a number of good guides on the net.
Consider using denyhosts as well in combination with the above method. It stops brute force attempts by banning the ip in the /etc/hosts.deny