Re-gearing - beating a dead horse?

[4 Paws_imp]

Sorry to re-visit a subject that apparently has been beat to death, but I am.

Interested in a bit taller gearing for my 2013 5 speed, would like to drop the rpm's a bit while cruising along.
I've done the forum searches and read as many existing threads and postings as I could find multiple times and am a touch confused as to one item - if I increase the countershaft sprocket by one tooth do I need to modify the sprocket cover to alleviate clearance issues - grind away to make more room?

Does changing (reducing) just the rear sprocket by two or possibly one teeth make any appreciable difference or is it pretty much just spending money for the pleasure of spending money?