Also, remove the use of ScratchBuffer when upmixing, as other channels may not be initialized with zeroed out data.