Well, I kind of compares it to dollar billionaires, world leaders, and A-list celebrities in the real world. While people might not have heard about all of them, they most likely have heard about some of them (which even was true when silent movies was the new hot thing). The odds of meeting them, unless moving in certain circles, is very slim.

Solar and Lunars tend to lump themselves together in small groups, making it less likely to encounter them, but when doing so it is possible they're not alone.

However, it is likely the thing catching the player characters attention...