Frank Tully's father and brothers were cruel to him because they were Calydons and he was not. His father killed Mary, the woman he loved, and his brothers had to dispatch their father because he was so dangerous. They blamed him for their loss. Frank later killed his brothers in retaliation for their treatment of him.
Frank also murdered Grace and Ana Matthews' parents. He was close to the entire family until Mr. Matthews discovered Frank's plans to liberate Ezekiel. They fought over it and Frank slaughtered husband and wife.