Just my 2 cents.
Was the buff too much? Possibly. I only have the Chaos Cowl atm. (Mind you I can get nearly max sword/gun stats without this item anyway) The +2 ctr and +2 dmg to every weapon you have per piece gives a person free max ctr (+6) on everything that is lv 10 for using the whole Chaos Set while giving Very High (+4) Damage Attack Boost to every weapon you have. So in return your weapons only need an ASI UV allowing a person to easy max everything with the addition of trinkets and gear UVs.. well nearly max everything. I think you could make 2 damage types. Example being swords and guns but still only have VH dmg in bombs.
Anyway drawbacks I've noticed from just using one piece of Chaos Set. Status conditions. That negative status result causes a lot more trouble than I would of thought. I guess people will be aiming for max status resistance UVs when using these. I typically run with only 3 weapons since that is enough for an effective build anywhere in this game. Using one piece of Chaos Set I've noticed that all 3 of my weapons get cursed and all vial slots get cursed. Probably a problem for a person playing UFSC.
I didn't bother to type everything. Though the Chaos Set did need a buff. Maybe the buff was too much. imo I would still like to see MSI med included on it just for the lulz. :p I said for the lulz before anyone has a tantrum over that last line.
The main issue is that with things like fire and shock, it isn't hard to get OHKO'd by most attacks. Stun, poison and everything else aren't as bad, but still aren't going to be good for you. Freeze is a sore spot, but still isn't as bad as fire and shock due to the way that wears off on the 2nd hit if you're shield is decent.
If you use any other item to cover up chaos's weaknesses, then you'll be well off.