How to tell if a male co-worker likes you, and flirt without crossing the line

Recently a new guy has joined my team at work – he is an account manager from another company that works with us, and we are technically his company’s client. He’s been up to our office a few times and I feel like him and I instantly hit it off. We have a great time talking and joking around however he is friendly to everyone so it’s hard to gauge whether or not he is treating me differently. I have noticed that he sneaks glances at me a lot when we are in a group situation and someone else is talking, though. He has also stayed late at our office a few times to keep talking to me however there have been other co-workers there (all male) so I can’t be 100% certain it’s to talk to me, just speculation. It’s usually me, him and one other guy who end up staying later and hanging out. He also always hugs me goodbye.

He’s only at our office once a week for a few hours, so I feel like every week is a bit like starting over from scratch, in terms of forming any deeper connection. That plus it’s a work environment which makes flirting tricky.

Bottom line – how to best know if a guy at work is interested? And how to best flirt with him to let him know I’m interested, without being too inappropriate for the office?