Southern Charm star Patricia Altschul reflected on her relationship with son Whitney Sudler-Smith’s father in her new memoir.