Phishing is often used to coerce a person into sharing their sensitive information with the caller by demonstrating that they already know other sensitive information about you.
The phisher may have gathered that information about you in any number of ways, including but not limited to dumpster-diving for sensitive documents, hacking a retail merchant's system and more.
Phishers reach out to consumers pretending to be a representative of their financial institution or other organization so they can convince you to provide more information that will give them access to your accounts. The call may even be spoofed to display the phone number of the organization the scammer claims to represent.