You have physical reasons to have a slow sex drive with your epididymitis and apparently orchitis (swelling of the testicle). Physical problems often lead to emotional issues, but getting rid of the physical problems can also get rid of the emotional problems. Try searching this site:
http://www4.ncbi.nlm.nih.gov/PubMed/
and report back to this board (if you have time) if you can find a statistic on frequency of epididymitis after vasectomy.