Take a look at your log file (/var/log/secure for Fedora Core 4) and you will discover numerous automated ssh hacking attempts using dictionary attack. So now you have identified the offending addresses. How do you stop them?
Here comes the magic mantra which uses iptables (packet) firewall:
iptables -A INPUT -s a.b.c.d -j DROP
Replace a.b.c.d with the offending IP address. Repeat this for each of the offending IP addresses.