Tax Collection Scams The IRS will not call you directly. Here are the tipoffs that you’re dealing with a scammer.