The Toxic singer is part of the new line-up on the US version of the singing talent show, and doesn't give budding singers an easy time if their renditions aren't up to scratch.
A source told People.com: "She's got a really good instinct, and when she loves something, she looks like she wants to dance on the desk. When she doesn't like someone, she is incredibly tough."
Britney's harder side was revealed at the show's auditions in Kansas at the weekend, where she starred on the panel alongside Demi Lovato, LA Reid and Louis Walsh, who had stepped in from the UK version of the show to replace Simon Cowell, who was taken ill with bronchitis.
'You have to be great'
The source added Britney told the contestant: "You have to be great to be on this show, so I vote no."
Britney, 30, is not afraid to stick up for contestants she likes, either, and the show is said to be a good boost for her own confidence.
The source said: "She's really massively grown in confidence. She loves interesting and fun auditions and is prepared to stick her neck out and support someone no matter if some of the other judges disagree."
The next round of auditions for The X Factor take place in San Francisco, USA on 16 June.