The Apollo Theater is a living piece of black history. Located in the heart of Harlem on West 125th Street, the theater has operated as a refuge for black audiences and performers from its opening in 1934. Artists from James Brown and Aretha Franklin to Stevie Wonder and Lauryn Hill have graced the Apollo’s stage. In 2002, Michael Jackson gave his final onstage performance at the Apollo.

“The theatre’s story is really the story of how black America lifted itself out of oppression through music and art,” said Roger Ross Williams, the director of HBO’s new documentary The Apollo. “The artists who performed at the Apollo paved the way for everything we have now.”

Roger Ross Williams in London, 2016. Photograph: Andy Hall/The Observer

The film explores how the space has and continues to foster black talent and it offers powerful glimpses at the realities of being a black performer in a racist America. During the 40s and 50s, for example, the white owner of the Apollo would meticulously keep notecards on the personalities and professionalism of each artist. The overly harsh and petty notes were often tinged with discrimination, featuring comments like “overrated” and “talks back too much”.

We talked to Williams about archiving black history, the magic of the Apollo’s Amateur Night and being the first black director to ever win an Oscar.

What inspired you to make this film?
I was really shocked to learn that the story of the Apollo has never been the subject of a documentary. It has 85 years of history behind it, involving some of the greatest talent of all time. I grew up watching Showtime at the Apollo. When I was a student at NYU, I would go to Amateur Night. I was always blown away by the interactions between the audience and the artists. There’s no other place that has that call-and-response feel. Being at the Apollo is like church. It’s a temple.

James Brown’s gold casket arrives at the Apollo Theater in New York on 28 December 2006.

James Brown’s gold casket arrives at the Apollo Theater in New York on 28 December 2006. Photograph: Eric Thayer/Reuters

Why is the Apollo important to black history?

The Apollo is the holy grail of black music and art in America. Some of the greatest artists of our time have not only performed there but also been discovered there. That started with the Apollo opening its doors in 1934 and Ella Fitzgerald winning Amateur Night. Ella was a young 17-year-old girl who, originally, was going to dance for Amateur Night, but ended up changing her mind and singing. Other icons have performed at Amateur Night too: Billie Holiday, Lauryn Hill, Dave Chappelle – it’s endless. In fact, the Apollo was James Brown’s home so much so that he had his funeral there.

It impressive the long list of celebrities that took part in this documentary. Pharrell, Patti LaBelle, Smokey Robinson …

The Apollo is such a temple to black music that everyone jumps at the chance to talk about it. For example, Patti LaBelle got her start at the Apollo and honed her craft there. There were all these different acts – Motown acts, Patti LaBelle, Gladys Knight – doing five shows a day. They would watch each other’s acts and learn from each other. Ray Charles even helped Smokey Robinson write his music at the Apollo once.

James Brown performs with the Famous Flames at the Apollo Theater in 1964.

James Brown performs with the Famous Flames at the Apollo Theater in 1964. Photograph: Michael Ochs Archives/Getty Images

Was there a specific interview that left you starstruck?

I loved talking with Patti LaBelle. What really got to me is when she talked about how difficult it was for a black girl like her to be on the road in segregationist America. She was, as she said, “singing ten times as better as these white girls, but not getting half the respect”. But she got that love and respect when she came home to the Apollo.

I read that in 2009 you were the first African American director to win an Oscar.

Yes! Can you believe that? There’s all these great directors that have come before me and yet the Academy has yet to recognize an African American director. That’s a shame. Now, being on the board of the Academy and representing the documentary branch, it’s a great honor and responsibility and I don’t take that lightly. I’m the head of the diversity committee for the documentary branch and I am working hard to find opportunities for women, people of color – people who don’t normally have a seat at the table.

The Velvelettes perform at the Apollo in 1964.

The Velvelettes perform at the Apollo in 1964. Photograph: Michael Ochs Archives/Getty Images

Was it hard to find archival material related to the Apollo’s history? The struggles around archiving black history are well noted. Typically, there are holes and gaps in our history.

Oh, the challenge was massive. Lisa Cortez, who was a producer on this film and a Harlem resident, she would search through boxes in people’s basements. She would find moldy tapes and photos. It was really a treasure hunt. Especially because the Apollo had previously fallen into bankruptcy and disrepair and its basement in the 70s was filled with sewage. We don’t document black history as well as we should. That’s why I’m glad this film serves as its own document.

What do you think the future of the Apollo is?

I would hope it continues to be a place that finds and nurtures talent in the black community. I don’t think the Apollo wants to become a museum. They want to keep charting the way towards the future.