I never swap guns unless it’s a rare gun or one I don’t have. Usually they are worn guns and will more often than not be worse than yours.
Also, I noticed when I picked up a rare shotgun and swapped for my pump action it was back on my horse when I returned so I don’t think you’ll lose the gun if you do swap.