“We must remember not to judge any public servant by any one act, and especially should we beware of attacking the men who are merely the occasions and not the cause of disaster.”