I had a Diamond Custom on my 88 GL1500 for over 180,000 miles. When riding, if it starts to rain, I stop and put a rain cover on it IF I stop and put rain gear on. If it isn't raining hard enough to make me want the rain gear, it probably isn't raining hard enough to get the seat very wet.
As for sun fading, I had my seat re-covered by Diamond Custom once because I was too stupid to put the rain cover on it when I parked in the sun. After the re-cover, I ALWAYS put the cover on and never had an issue with fade again.
And wear, yes, but after YEARS and YEARS and MILES and MILES.
My Tour DCT has a Hartco velour. Covered when parked in the sun.
I've had two of Tom Simmons seats both have/had Mercury Mark IV fabric on them and I don't even own a rain cover. I lightly spray it every 6 months with the spray you can buy at Walmart in the camping section with an orange lid and ride it. If it rains I just brush it off with my hand and go. My jeans get a bit wet but it's dry in a few minutes. Never had a problem in the 8 years I've ridden them and around 150K miles give or take.
I've had two Russell Day-Long seats and both Velour. Love both seats. Russell has covers that fit over the seat and passenger backrest to protect them from the rain and worked very well. My rear never sat in water after removing either. Takes about five seconds to install the covers. Fading was NEVER an issue either. Would do it again tomorrow if I needed a new seat.