Every time you like a song, it gets added to a playlist called "Inbox", which holds all the songs you want to eventually add to your other playlists.
You should clear your inbox on a regular cadence. This involves removing songs from the inbox and adding them to your other playlists (e.g. "Party Bangers").