Hi!
According to some performance measurements on a STM32F407 there is no big performance drawback as long as the scratch-pad is at least 1/6...1/4 of the entire screen.
This means, you can reduce the effective memory for frame-buffer (= scratch-pad buffer) from 300k to 50k :-)
In principle, you can reduce the scratch-pad to one line - but then it requires 480 separate updates, which will take too long time to get acceptable frame-rates... The smaller the scratch-pad, the higher the number of partial screen updates.