I know you are going to hate this, but for me I feel pretty comfortable with keeping defense as a 90% subjective analysis. I feel like I'm keyed in enough to defense as something I really value that I've been pretty accurate in the past with which prospects are going to be good and bad defenders in the NBA. Yes there are many differences between the NCAA and the NBA when it comes to defense, but in general guys who care, have BBIQ, and the physical tools make good defenders in the NBA. Usually that stuff comes through in the film.
Personally, I think those things show through numbers more than they do through film. Maybe my opinion would change if I had infinite access and time to watch film, but I feel like I watch a decent amount of film and I'd rather start with numbers first and use film to confirm/reconsider/contextualize the numbers.
It's kind of hard to decouple the different forms of analysis though. We all have some amount film, numbers, and outside opinion baked into our opinion without even trying.