So I know this is a thread dig but had a couple of points that no one seems to consider when tweaking the Bose system.
Firstly, a considerable amount of vocal range comes from the door mid/tweeter. Yet no one puts a damping pad behind it on the reflective metal behind it... It's a small speaker with an open frame meaning plently of sound is being bounced off that flat steel and back through the paper cone. Only has a slight effect, but seemed to clear it up a bit.
Secondly everyone attacks the equalisation curve from the bose amp as to being some form of failure. Bose have not equalised the speaker as such, but the whole speaker AND car etc. Take the rear 6x9 (which is a WOOFER ONLY), the odd dip around 250hz will be to reduce boomyness from the fact that it is essentially mounted in a corner firing straight at a piece of glass. Changing this speaker will mean significant issues as not only will the new speaker have much more high end, but also wider dispersion from the tweeters.
Third fail most seem to make is turning up bass and treble and complaining that there's no bass. Try putting a song with deep bass lines in, and have bass set to maybe 1 at most. I have it set to 1 with treble at 1 and on songs with a dropping bass line at normal levels, it drops plenty low enough. Stop expecting bass like a set of dual 12's with 1000w of power from OEM door speakers and you will be much happier.
Another failure is that people don't realise how LIGHT the whole system is. DO you think Mazda wasn't telling bose to keep the weight down after fitting the RX8 with a carbon drive shaft and aluminium panels? The door woofers and amp weigh almost nothing and perform fairly well considering. And just to add from the OP's comments, the speaker doesn't have a passive radiator on the back. It's basically a normal speaker but has the small neo magnet mounted in the front of the cone with the suspension at the back just like normal. Just looks odd without a magnet being in the way. The frame is also on the other side of the cone. Same stuff as a normal woofer just rearranged to keep is shallow. Some slimline car subs are very similar like MB Quart's RSH slimline subs. They just make theirs pretty.
Just remember, Bose aren't amazing overall but have done a fairly respectable job for a system designed in the early 2000's in such an odd shaped car cabin. Don't think of it as an amp and speakers, think of it as an entire system tuned together. There is some clever engineering to get a reasonable sound from a well integrated package for its day.
Best bet is smart damping of the door woofers, a soft absorbing material behind the door tweeters and sane use of the bass/treble and realisation it's a light and low powered system not designed for 130db or impressing girls from half a mile away with Dubstep
Am going to try some other full range drivers for Bose tweeter replacements too, like Visaton's FRWS 5 and FRS 5X which are very similar in size when corners are trimmed.