Am I being paranoid or logical?

OK I have a mild mental disorder and live in an apartment complex with several of my peers as well as other people from all walks of life. All of my close neighbors have families with children. The complex is offering a summer lunch program for all children aged 1-17 and all the residents that are considered disabled such as myself. I do not want to go and eat because I think things such as "what makes me better than my neighbor" and "I don't want to eat with these kids my neighbors will think I'm a child stalker" Is this line of reasoning logical or paranoid?