As far as M cars go, I've had -
2005 E46 M3, manual
2008 E92 M3, manual
2008 E92 M3, manual
2018 F80 M3 ZCP, manual (current)
However, I most recently came from a 2016 F30 340i xDrive with MPPSK, Track Handling Package (which includes adaptive suspension), M-Sport Package, and Dinan lowering springs w bump stops.... actually, the car was loaded.... 19" 442M, ZTP, ZLP, ZCW, ZDA, ZDB, etc...
From a "handling" perspective, I only have 1,500 miles on the F80 but I can absolutely say that despite me driving the 340i with all the goodies it was nowhere close to "handling" like any of my M cars. Yes, I know, mine was xDrive, but the lowering springs did noticeably improve the understeer of the AWD...
There is a very big difference, IMO, between M cars and non-M cars... M package cars don't really count...
Seems like your question is if there's a noted difference between a maxed out 340i M-Sport and an M3... your answer is yes and very much worth it