For me, I'm not that young (55), but not too old yet either. And I work in the office all day with all kinds of noise from people talking on the phone, then I eat in the cafeteria every day where it's really loud and noisy as well. So being immersive in noisy places all day long helps my brain adapt to the OPN much faster. I also wear the OPN exclusively 100% all day long and I don't use my old HAs as a fall back as well, so that also helps get me adapt much faster, too.
If your lifestyle is mostly in quiet places most of the day everyday, then I think even normal hearing people who are used to a quiet lifestyle will be bothered a lot every time they're exposed to noisy environment themselves, let alone us hearing challenged people.