I agree enough about random boss drops being annoying, but SK had a better comprimise; they're available to find (without needing to traverse some difficult dungeon or specific boss), but random enough that it takes some time to get. It worked. The SK devs do have a habit of taking small problems and overkilling fixes (think back to the Retrode/Wolver/Zombie nerfs. Instead of just nerfing one small aspect, they nerfed like five things, making them a joke) which is what the HoH is; overkill. There are plenty of other alternative suggestions here that I agree with, such as limiting recipes (the basic/mid-level ones, and keeping exotic/powerful ones to Basil), increasing the price of recipes in the HoH, and allowing random Basil to be a cheaper alternative..etc
If you can't agree to remove them from the HoH completely (and believe me, although I disagree, I do understand what you're saying about it) then there should be at least one alternative you could get down with, because you must surely see that giving players access to nigh every recipe in the game so easily is just not an inherently good thing.
The problem with "just running the CW" is that when costs are involved, sometimes you can't justify it, especially if you're not on an EP. Why should you run a CW level that won't pay very well, won't be that hard and that you've played before fifty times anyways? Might break the monotony, but there's literally no other reason or benefit making the trip worthwhile. With Basil, there was at least something exciting halfway through that you could look forward to and potentially even profit from. T'was just that one thing that made regular CW runs feel more exciting and fun.
Yes, other MMOs use random boss drops...and that mechanic is awful no matter where it's used. I for one greatly appreciate the fact that SK doesn't resort to it. Tying game progression to a random number generator does nothing but frustrate the hell out of those unlucky enough to repeatedly be denied the one thing they're looking for. (As I said before, several months' worth of T2 runs didn't net me a single Twisted Snarble Barb sighting.) I don't really understand how having the recipes available like that really speeds up progression that drastically...you still have to get the crowns needed to purchase them, plus the CE to craft them (which is no easy task today), plus the materials, which can get a bit rare for certain 5-star items. Sure, you can go P2P and buy a ton of CE, but if you're doing that, you could have bought as many recipes as you wanted from the Auction House back in the day anyway, so it doesn't change much. For anyone running the F2P route, it actually takes substantially longer now to progress, simply because of how ridiculous CE is.
As for why to still do Clockworks runs? You said it yourself: to break monotony. I don't even particularly like the Firestorm Citadel in general, and you couldn't pay me enough to repeatedly run KoA. (Hell, I still haven't gathered enough tokens for my first item, and yet I still don't run it much.) Yet I'm almost always up for a gate run, especially if there's a fun theme. As you also said, there are still boss recipes, which albeit frustrating, at least give a practical reason to go down. As I stated earlier, I don't have a fundamental problem with maybe taking some of the alternate recipe lines and reverting them back to Basil, but removing the HoH entirely is just insanity. At the end of the day, recipes aren't nearly as much of a reason for Arcade abandonment as the infinitely-repeatable boss missions are.