4" would be fine, you won't gain anything manifold back other than bragging rights, as long as your downpipe is still in good condition and you have no exhaust leaks, just leave it on.
Aftermarket downpipe is another part that will not do anything for you, except for lighting your wallet, a lot of labor and again bragging rights. .
You might consider stainless, Diamond Eye has the most reasonable prices on stainless steel exhaust that I've found.
2007.5 GMC Sierra 3500 SRW L.B."BLACK" C.C. 4X4 LMM Z71
DIY stock airbox mod, aFe ProGuard 7, OEM drop-in
EFILive ECM ATP - Heavy Tow Tune with Turbo Brake.
EFILive TCM Duramax Tuner
DHD Cold Side High Flow Intake Pipe, DHD PCV Re-Route, AirDog 4G100
Castrol Hypuron/Vecton 15W40 synthetic blend oil
Donaldson P550832 2 qt oil filter
2007 Yukon Denail "BLACK" 6.2 gasser
DIY stock AirBox Mod, K&N, Airaid intake, EFILive TCM only tuning