Actually, on the TF, you can already set the aux switches to control play/stop/rec when using the looper and tap/hold/bank-down when using other effects. In this regard, the aux switches already do what you want them to do. How to set this up is on page 35 of the manual.

Are you using a single aux switch, or a double or triple? The Factors work with any of those combinations. With a triple switch, you can have hold and tap and bank-down, and play/stop/rec when using the looper.

Again, I'm not trying to be argumentative… just playing devil's advocate. It's entirely possible that I'm the one who's missing something here, and if so, finding out what that is might give me some new ideas.

And yeah, if you haven't tried an exp pedal with any of the Factor pedals, if you have a chance to, I'd highly recommend it. It triples the effective number of presets (saved setting, pedal down, and pedal up – for each preset) and all the effects are just so much more expressive, especially the PF & MF. Adding the exp pedal was what really got me addicted to the Factors, it's just so much better.