Ha I like this part...the rest of your post I'm not so sure about.
It would have lead to some potential for great inter group conflict. Glenn, Maggie and Merl in the same group? The woodbury and new people forming a clique against the old people? The new dad and his son wanting to take over from Rick?
Then I'd have the governor whip the woodbury general pop into a frenzy and attack the group. People killing people and zombies coming in behind them. As it appears that the woodbury people have Rick's expanded group trapped and are thinning their numbers the black guy fromt he first episode shows up and provides them an escape. Darryl and Merl are last to escape and as they are doing so something happens to Darryl and Merl sacrifices him self to save his brother and block off the escape so they can get away. Perhaps the governor grabs Darryl but Merl jumps him and they fight. The governor kills Merl but Merl takes his other eye as he dies and condems the governor to death as the zombies grab him.
That would be my season finale.