Show boss Simon Cowell is said to be keen to "take the show in a different direction", and has only guaranteed record producer LA Reid a place on the panel next year.
A source told E! News: "At this point, the only ones staying are Simon and LA Reid. Simon wants to take the show in a different direction. The plan is to part ways amicably.''
While a Fox spokesperson confirmed the departure of both Steve - whose presenting style on the show was slammed by one critic as "utterly irrelevant" - and Nicole, he could not comment on Paula's rumoured departure from the series.
However, the Welsh host took to twitter to express his disappointment at being given the chop from the singing contest after just one series.
Steve tweeted: "I wont be hosting next seasons X Factor which is a shame but I cant complain as I've had a great time. Good luck to everyone on the show. (sic)"
Music career
Meanwhile, Nicole is reportedly keen to leave the series so she can dedicate more time to her music career.
The insider added: "She wants to focus on her music career. She's spoken to Simon and he's given her his blessing.
"X Factor took a lot more of her time than she ever expected. She didn't have any time really to work on her music."
This isn't the first time Simon Cowell has made sweeping changes to the US X Factor panel.
Last year, he axed Cheryl Cole from the show after just four rounds of auditions, claiming the British singer seemed "bewildered" in America.
He said at the time: "She looked a bit bewildered. You come to a country like America, it's a bit deal, takes a while to adjust. And I took the view, along with a few other people, that she would have been more comfortable back in the UK."