When I say my brother and his wife are heterosexual, that doesn't mean I'm talking about their sex lives. Likewise, when we say someone is gay, we're talking about sexual orientation, not their sexual activity.