Can he visit his son @ my house if I am on Housing istance?

My fiance and I were not together for the end of June, We live in HUD Subsidized housing..we have a almost 2 yr old little boy. I Started the process of taking him off and everything in the beginning of July. Now when you are on HUD you have to wait 6 months to put back on the lease after being taken off. So I am trying to do so now, but they are telling me that he cant put back on until March 1st, And he was taken off in September. And there saying he cant come over or I am not allowed any visitiors. We have a 2 yr old son together and A baby on the way. Is it a real rule he cant even visit his son?