Instead of one item that benefits swords and guns, have 3 items- one for swords/guns, one for guns/bombs and one for swords/bombs

Each would have the same bonuses-

Guns = Damage high
Bombs = CTR high
Swords = ASI high

This way, you'd get-

Sword/Gun variant = Gun damage high + Sword ASI high

I chose to do it this way so that with chaos, the bomber sets are pointless and with black kat the gunner sets are pointless, and with either set swords come out ahead, since swords can actually use the buff. Meanwhile bombers and gunners need their buffs early game when they lack them.