Is it wrong to flirt with my friend's brother?

Our families have spend a lot of time together in the past, and now just my friend and i hang out together. I basically grew up in her family. Her brother and I for the last few years have this weird thing where we'd always flirt even though he's like... 6 years older than me. We fell asleep in a bed together once and his mom 'had a talk' with him after, and lately it's just been every summer we see each other we just subtly flirt. Is that wrong?