The Young Frankenstein director imagined filmgoers rioting in cinemas and ripping down posters and screens as they watched the now-cult comedy about a racist western town's black sheriff, played by Cleavon Little.
But instead, the film became an instant hit.
Celebrating the 40th anniversary release of the film on Blu-Ray, Brooks tells Ew.com, "I envisioned a race riot. I thought everybody would come after me and kill me for what I said about the Chinese, and the blacks, and the Jews.
"I thought if this was shown in Waco, Texas, the whites would storm the screen and cut it to ribbons. Because we were kind of hoisting the black sheriff up on our shoulders and made him a hero. But Texas liked it as much as New York."
But Brooks admits he did struggle with the use of the N-word in the film and turned to co-writer Richard Pryor to make sure he wasn't going too far: "Every time I said to Richard, 'Can I use the N-word here?' he said, 'Yes'. I said, 'Richard, it's a little dangerous here'. He said, 'Yes'."
The director insists the film could not be made today, adding, "It could hardly be made then. Certainly not 10 years before then. And now it's suddenly, it's 40 years later, it cannot be made today. That's weird. The prejudices or whatever, the restrictions, should have thoroughly diluted by now, and here we are - it's amazing. We're playing it safe.
"I don't think the individual person is playing it safe, but I think the organisations - let's call them television networks or studios - they're playing it safe. They don't want to get sued. They don't want to lose the Latino endorsement or the black endorsement or the Jewish endorsement."