Accepting an Invitation for Scheduling Privileges

Someone sent me an invitation to have scheduling privileges for their account.  However, when I try to accept, it says: "No pending invitation, or you have already accepted it before."  But when I look in settings it says that I have scheduling privileges for no one.