I just don't know how to tell someone off or caution them. What's the best way of handling these situations?
Yes, managing people is pretty easy when it's all going well. The difficulty comes when people are not performing at the right level, or repeatedly making the same mistake. Most British managers share your concern that if you reprimand someone or even take them to task, you will cease to be liked and will lose their goodwill.
The trouble about avoiding giving negative feedback and failing to make your expectations clear is that the under-performer continues in blissful ignorance (or, if they are good at picking up the vibes, fearful ignorance) about what they are doing wrong and what they could be doing differently.
There are several consequences to this. For a start, the individual misses out on an 'improvement opportunity' and, worse than this, other people get the message that poor performance and incompetence are tolerated, even condoned. The demotivating effect of seeing under-achievers get away with it cannot be underestimated.
Successful performance management requires skill in using two key levers: challenge and support. Not enough support and constant challenge - the 'Spanish Inquisition' scenario - leads to demoralised staff, resentful at the constant demands made of them, with no understanding of their position or progress to enable them to develop their skills. The opposite situation - which might apply to you - is the 'Warm Bath' scenario: there's lots of support and empathy but no challenge, resulting in mediocrity.
I suspect you're pretty strong on the support side, having developed your ability to empathise by actively listening to gain an understanding of how others feel. Now you need to work on challenge, for which the core skills are setting clear objectives, understood and agreed; giving feedback - positive and negative - and assertive communication.
The expectations element is simple enough. When you ask someone to do something, especially when you suspect it will be a stretch for them, make sure you're in a quiet place where you can both give your full attention.
Look the person in the eye and tell them what you want them to do, clearly and calmly. Check their understanding of the brief by having them tell you how they will tackle the task and asking if they have any questions.
Agree a date for completion, and put a date in both your diaries to review it. If the requirement is less a task and more a change of behaviour or attitude, set markers by which both of you can check progress.
Reviewing the task allows feedback. Again, make an appointment for a time and place where the two of you will be undisturbed. Ask your employee to review how things went and whether there is anything they would do differently another time. If they have done well, express your pleasure.
If they have failed, express your disappointment or, if appropriate, your anger.
You'll need to be specific about how you felt and the behaviour that made you feel that way. For example: 'I felt let down when you failed to produce your report on time because I was relying on it for my presentation to the board.' By making explicit the emotion their specific behaviour has aroused in you, you provide assertive feedback that will help them develop. You are also letting them know it's their behaviour you are disappointed with, not them as a person - behaviours are much easier to change than personalities.
Be economical with words and pause afterwards to allow them to respond Chances are they'll apologise, and even if they don't, they will have made a connection between their behaviour and its results. Avoid generalisations: sentences that begin 'You always...' or 'You never...' only invite denial.
Comments limited to a specific incident and the feelings it provoked are less easy to shrug off. Practice at home, with your partner, your child or your cat. After a while, what seems a little artificial will become quite natural to you.
What I'm suggesting here is the development of adult-to-adult communication, rather than the parent-to-child confrontation you perhaps envisage, which you would feel uncomfortable with and your staff member would resent; and, anyway, it wouldn't be effective in generating the performance improvement you are after. If you can reframe your role from critic, judge and jury to performance developer - in other words, coach - you may well start to feel far more positive about the whole area.