Depending on how you invited them you can remove someone as a manager if you are the profile owner and remove them from your trusted list. Check the help file for more information. You should also review the privacy tab on your profile for greater detail.
Your privacy tab will keep non trusted users from viewing data about people that are alive.
I would start on your privacy tab as a first step.