How does faction loss works?

[Goregasmic]

I'm factionned in chardok and been buffing players/groups around entrance/CE and noticed when I buff them while in combat and their mob dies sometimes I lose faction and sometimes I don't. Like sometimes I drop a group c2 on them mid fight and I don't lose faction but sometimes I'll haste a melee and I'll lose faction. I checked the wiki and googled a bit but couldn't find anything.

Is it because if you buff and they don't take direct action toward the mob post buff you won't get hit? Like if mob is dotted and pet is on it and you buff shaman that only watches it die you're fine but if shaman reroots mob you're not fine anymore?

I thought you'd get a hit the second you interacted with a player getting a hit but apparently it is more nuanced. I'd run tests but my faction is getting low right now and I'd like to stretch what's left.