The 52-year-old music mogul decided to poach Alesha Dixon from BBC show Strictly Come Dancing to get channel chiefs back for scheduling their new show at the same time as BGT, but he thinks the ITV1 series is "in a good place" to beat The Voice in the upcoming ratings battle.
He told BANG Showbiz: "The Voice was 100% intended to damage Britain's Got Talent. I thought, 'Great, we'll nick Alesha then'.
"It affects the viewers when you go head-to-head and that, to me, is silly rivalry. Of course I love competition but if you ask the BBC whether we need another singing talent show, I would query whether we do.
"They have every right to compete with us and I have every right to arm ourselves so that we're in a good place to beat them."
'Sense of humour back'
The X Factor boss also claimed BBC bosses need to "get their sense of humour back", and slammed suggestions he had offered Sir Tom Jones - a coach on The Voice - a job on one of his shows.
Speaking at the launch of Britain's Got Talent on Thursday, Simon said: "I think the BBC need to get their sense of humour back.
"It was intimated that I'd offered Tom Jones a role on one of my shows. I've genuinely met the guy once in Vegas, I think the offer he was thinking about was Opportunity Knocks. It wasn't one of my shows so he must have got the dates confused.
"They're winding this up to try and make it a big battle. The shows are going out at the same time and may the better one win. In terms of credibility, it's down to being successful."