Facebook contacts not linking correctly on htc sense...need help?

I recently got an HTC sense and I think its great how you can link your contacts to their facebooks, however, I do not want all of my Facebook friends on my contacts because 3/4 I don't even talk to. I could only get one of my contacts to link and then for some reason, it stopped working. The link option is still present, but when I try to link the contacts, it doesn't work. It shows the unlink grey symbol, instead of the green link symbol. And when I switch from unlink and click done, it doesn't do what it was supposed to. Answers and help would be great, I'm confused! I will give best answer quickly. Thank you.