Show / Hide a subsite based off of group permissions.

Hi all,

I am building an intranet with some subsites for each department underneath it.  Is anyone aware of how to show / hide a link to a subsite based off group permissions?  I was looking into enabling the audience feature but after reading, I was under the impression that it’s used within the subsite for list, pages or apps.  I broke inheritance and gave the group that I want to have access, permissions, but others are still able to see and click on the link. 

Thanks so much.