This has never happened to me. However, I know some people are sick or family emergencies come about and at that point, that is the PCAs focal point.
FIRST, be certain everything you are reporting against the caregivers and your brother is factual. Gather all the information you need. Make sure you have someone legal on your side. First I would report to APS and get legal advice.
The Care.com Community is an advice-sharing network for parents and caregivers.