You should find out what jobs atd is running that are consuming so much time. As atd could be used to run all sorts of things, disabling it might disable all kinds of services you may need (like periodic backups or something).
Your problem may be that something is running more often than it should, or requeuing itself.
Run the command atq
as superuser to see what kind of stuff is being queued.