Tell each other their names when they meet
This could be done by literally saying ones name instead of "hello" or by creating a unique sign (like in sign language) and signing ones own name instead of shaking hands.
In some cultures (like parts of India) it's custom to tell your name as greeting, but in most parts of the world, this would be very strange and suspicious.
The signing, on the other hand, can be written off as dusting off your clothes or some coincidental gesture. Most "normal" humans would extend their hand to shake or bow (like in Japan), so Face Swappers could recognize each other without standing out like a sore thumb.
Wear something unique
They could wear a necklace, a set of differently shaped earings or braid their hair in a unique way and create a visual clue to their name like a nametag. A tatoo on the neck or hand can also serve as name tag.
Wearing the same jewelry is not suspicious for most people, but never changing your overly complicated hairstyle might be hard to keep up over many years, especially if your youthfull Mohican hairstyle doesn't fit your 60 years old body anymore.
Smell and other senses
Many animals have a better nose than humans and recognize their family members by smell.
In some cultures people hug or do air kisses as greetings. There you could smell the other party without being too suspicious. It gets more complicated in buisness meetings or in cultures like Japan where physical contact is avoided as much as possible.