Who haunted Clinton?

Despite the fact that Bill Clinton turned out to be one of the most successful two term presidents in recent history, his administration was dogged with controversy right from the beginning. Mostly it was a campaign to discredit him and his presidency. So where did it really come from and who were these people who were so determined to bring Clinton down?