Wow, CB1100 vs a Fatboy?
About the only thing in common is the number of wheels.
But, having parted with a V-twin ,Road Star 1600 Silverado

, to get the CB I can certainly understand the attraction to the V-twin.
Heck, I'm already trying to figure out how to swing another R* to put next to the CB! But if you can only afford, or the wife will only tolerate

, one then just think about your riding style, what you want to do most and do what makes YOU happy.
Several brothers in the ALR are Harley die hards and swear by them. All have the newer Evos. Our Sgt at Arms has over 87K on a late model HD and he just did WV to MI to TX and back in one month. Two up with gear.
I've had four HDs in my time with the last a brand new '76 special edition that left a REALLY bad taste in my mouth. And I'm irrational when it comes to holding a grudge.
The best long distance machine I've ever owned was a '76 Gold Wing, one of the models before they morphed into Buicks. The two CB 750s I owned were great and several people have mocked how in love I am with the CB1100.
Ride the CB and see if a HD dealer will let you test a Fatboy (if I ever tried on again it'd be the Heritage Classic) and see which brings out the biggest, longest lasting smile. Good luck with the choice and ride safe & happy.