I think it is time to make some measurements on the display interface: For example, you can make a full screen red color gradient and then check all bits from R7 to R0 by using an oscilloscope.
Additionally, you can clarify with the display manufacturer if the display provides only an 8bit per color component interface and if it is really able to show 8 bit per color component.