Matthias Noback has posted the next two parts of his "framework independent controllers" series (it started here) looking at avoiding annotations and tying up some loose ends.

From part two about annotations:

In the previous part of this series we decreased coupling of a Symfony controller to the Symfony2 framework by removing its dependency on the standard Controller class from the FrameworkBundle. Now we take a look at annotations. They were initially introduced for rapid development (no need to create/modify some configuration file, just solve the issues inline!) [...] This might not seem such a big problem at all, but the SensioFrameworkExtraBundle is a bundle, which means it only works in the context of a Symfony2 application. We don't want our controller to be coupled like this to the framework (at least, that is the point of this series!), so we need to remove the dependency.

He shows how to decouple this functionality through a proper routing configuration, fetching the needed data yourself for the request and generating the request object yourself. In part three he covers some of the comments already made about the series and how to take the final steps to abstracting out the controllers: removing bundle names from templates, removing the HttpFoundation dependency and letting go of "action methods".