The only reason why I got the Note5 was because I was tired of two phones and wanted to go dual Sim, I could have gotten a Note4 dual but just decided to get the Note5 duos instead. (Maybe should have just waited for the Note6 duos)
The Note5 has essentially nothing over the Note4, only thing I can honestly say is the upgraded finger print. If I am to be picky I would even say the Note 4 is somewhat snappier than the Note5, the 5 is by no means laggy but the 4 was quicker to me, I eagerly await Android 6.0 as the 5 should get snappier.
The dual Sim aspect works flawlessly, no complaints there.. I've had my Note4 for a year or so so I've Gotten so use to it in hand to the point where the 5 does not feel as comfortable, the 5 is definitely heavier so maybe that's why. Note4 speaker louder, flashlight is much brighter and a few other things have taken a turn for the worse in the 5.
All and all if you have a Note4 just wait for the Note 6, because honestly the Note5 does not warrant the upgrade...