Memorability of characters is really important so your readers don't forget about the uniqueness of your character (point in case being "Pants" from Twilight). However, making a character too flamboyant can hinder their abilities to perform a lot of duties. If you're a well-known assassin and you wear distinctive clothing that people know you by, you will not live very long. Also, it makes no sense that a peasant would be wearing something with a terribly large amount of colors. On top of this, people change their outfits. It isn't a cartoon.