Gary Oldman Explains Why Harry Potter and Dark Knight Trilogy 'Saved Me' After His Divorce

Gary Oldman is reflecting on how his roles in the Harry Potter movies and The Dark Knight trilogy helped him during a trying personal time.

Oldman, 65, is well known for portraying the character Sirius Black in the Potter films and he costarred with Christian Bale as Gotham City police officer (and later commissioner) Jim Gordon in Christopher Nolan's acclaimed take on Batman.

During his recent appearance on The Drew Barrymore Show, Oldman said he began turning down acting opportunities in the early 2000s in order to spend more time with his kids— Alfie, 35, Charlie John, 24, and Gulliver Flynn, 26 — after he became a single father in 2000 following his separation from third wife Donya Fiorentino.

“Thank God for Harry Potter," he said. "I tell you, the two — Batman and Harry Potter — really, they saved me, because it meant that I could do the least amount of work for the most amount of money and then be home with the kids."

Oldman has been married five times. He first wed English actress Lesley Manville in 1987, but they split in 1989 not long after the birth of their son, Alfie. He married Uma Thurman in 1990, and they also split two years later. He shares sons Charlie and Gulliber with Fiorentino, whom he was married to from 1997 to 2001.

“At 42 years old, I woke up divorced and I had custody of these boys and so that, in itself, was... That was hard because there was a shift in the industry where a lot of productions were being [filmed in] — it was Hungary, Budapest, Prague, Australia, you know, all of these places," he recalled on The Drew Barrymore Show.

During his appearance on the daytime talk show, Oldman said he took as many as 27 round trips between London, where the Dark Knight trilogy filmed, and his home in Los Angeles in order to spend time with his children. The first film in that trilogy, Batman Begins, released in 2006; the third and final film, The Dark Knight Rises, released in 2013.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

Oldman married his fourth wife, Alexandra Edenborough, on New Year’s Eve in 2008. Edenborough filed for divorce in January 2015 and cited “irreconcilable differences" at the time. He married his current wife, Gisele Schmidt, in August 2017, as he confirmed to the Associated Press later that year.

Though Oldman has expressed gratitude for playing Sirius Black, he recently shared on Josh Horowitz's Happy Sad Confused podcast that he believes his "work is mediocre" in the Potter movies.

“Maybe if I had read the books like [late costar Alan Rickman], if I had got ahead of the curve, if I had known what's coming, I honestly think I would have played it differently," Oldman said on the podcast.