I think "push the envelope" means to take the expected, the norm, further than you've seen it go before. Once you've got a pretty good grasp on a film's subject matter (as in- how it's been handled in films previously) / theme, you won't find the phrase half as open to interpretation. Except that what you define as pushing the envelope comes down to what you've seen before.
Suicide Circle. If you've seen it, you know why.
Also, shockingly, before he fucked it up with Hostel: Part II, I say Eli Roth was making some very daring films. He's become a punchline to use when anyone hears a homophobic, racist, or sexist slur used in a movie (which is hilarious- didn't Eternal Sunshine of the Spotless Mind do the same thing just as casually?) but he actually had a point for doing it. Granted, in Cabin Fever- I still can't figure out what the fuck it is. Except that I thought the twist of the old man saying the gun in his store was for "niggers" was very clever. Other people rolled their eyes, I thought it was set up and paid off very well... let's just hope they didn't go out and buy (Lemonade?) from the kids outside because then it would be completely pointless. "Word." That kind of thing is what made me think he deserved the benefit of the doubt on the parade of "gay"s throughout. I mean, he kind of didn't tie down the series of events after a certain point. Which I liked. But that's also what made the movie lose its' considerably freaky undertone. From "Pancakes!" on, the movie becomes a David Lynch film- totally void of sensation. Before that, the movie really gave you a feeling of strange gloom.
But I consider Hostel downright groundbreaking. Not in relation to shit like The Descent so much as youth culture in general (and the stuff that was so popular at the time and years prior like Laguna Beach and The Hills among others). When you show obnoxious young hot people partying and then you start carving them up, you inevitably make a potentially unwanted moral connection between their partying like stupid, selfish robots and punishing them for it. Especially when they also happen to be deeply homophobic (rather than casually, such as the Cabin Fever knuckleheads), patronizing, hypocritically sexist, and with a fairly low opinion of other people in general. The movie did kind of imply this is what you get for being a shitty person. And, I've heard some people say the chase ending killed the movie's momentum... this certainly isn't true of a couple of short moments thrown in for very good measure (the kid gang and the reunion of Alexei and Natalya). The rewatch potential is stunted but, it was released in an insane sea of survival flicks all taking themselves very seriously. Compared to just about anything else of its' time (especially the ultra-sexualized French "horror"), it walks away looking a great deal more intelligent and friggin' relevant.
Emotionally, I thought Wendigo took some pretty big risks. To make another comparison, it had the same basic idea as Blair Witch. Except before a character goes missing and you have no clue what happened to them- the film didn't make you hate them, you meet the hostile locals BWP theorists alluded to (this also happened in Wolf Creek... why was that again?), and the filmmakers actually utilized things like writing. And talent. For some reason, I found that the film had a duty to have the husband live in the end. You can't end with Patricia Clarkson in a new millennium horror movie actually reacting to a death like the life itself meant something. These were the Bush years, people are just pawns in a chess game. Surely this movie wasn't about to go there... It Went There. A believable human reaction to a realistic tragedy. I saw it and to say the least, I was truly surprised. That's not really the direction horror was headed at that time. People were becoming a dime a dozen in The Descent, Frontier(s), The Strangers, even stuff as early back as Final Destination. (Which I think worked for that movie. I wouldn't add that to the PTE list because it was kind of a horror version of Jumanji. The deaths were brutal and shocking but the movie didn't want to take the characters' lives very seriously.)
When I think back on the "horror" movies I hate the most from the last 10 years, My Bloody Valentine keeps getting a slight pass. Being told a thousand times ahead-of-time that it's just a dumb splatterfest isn't nearly enough to numb the pain I felt but the motel scene is a good example of what the movie could have been. And could have spared us the posery of Aja's Piranha.