Because the specialist armours are not really intended to be worn as a full set, but for mixing with other sets for effect.

I almsot never mix-match sets, except with armor that only has one piece like seerus. If I'm going into a dungeon with a specific set of monsters, it makes a lot more sense to wear armor that makes me fully resistant to their damage and status type, instead of wearing mixed armor and hope that I fight every monster type that I am now half resistant to before I reach the next armor station.

Pretty much, I see mixing only as a viable option for players that do not have a lot of sets, and therefore need something that would would against everything to fill any pot holes in defense they may have. As an item collector myself with over 10 5* sets and counting, there really isn't any need to mix (I don't even use my Divine set that much, since the mix defense and no normal usually makes me more vulnerable than a grey feather or skelly suit).