@canozo if not here is why:
The charge time reduction on a sword goes from low(from lvl 1) to med which is lvl 10. (with out any uv/trinkets or ctr sets)
You have 2 trinkets for CTR med on your sword with a lvl 10 sword I presume?

Low -> Medium -> High -> Very High ->Ultra -> Maximum!
take these "bonuses" in a numerical form,then you would get
Low = 1
Medium = 2
High = 3
Very High = 4
Ultra = 5
Maximum = 6

You said you have 2 trinkets with med ctr,that would mean you have 2 + 2 =4 so a very high ctr.add a lvl 10 sword with that so you get 4 + 2 witch adds up to a 6,no? (you understand simple math no? ) all and all you get a 6 and 6 is maximum,you can't get more then maximum! so when you put a chaos set witch has ctr med on helm and ctr med on armor you have a very high ctr( 4 in numerical bonus) just from that set.Here is the thing,you can't have more then 6(like I said already) so all you are doing is wasting your trinkets slots,or possibly armor do to all of the negative statues it gives.

Also the whole set gives out a very high ctr and very high dmg bonus.