to/by a person or peopleの人 means “person/people” modified by the preceding word, while に marks the person as the recipient, target, or—in passive sentences—the one who performs the action. The exact English translation depends on the context.
JA: 困ったときは、近所の人に相談してください。
EN: When you’re in trouble, please ask a neighbor for advice.