Recent advances in neural codec-based text-to-speech (TTS) systems have achieved remarkable audio quality; however, their reliance on large model size and heavy computational requirements limits CPU-based on-device deployment. In this work, we present a Mamba-based streaming acoustic model with two novel depthwise decoding strategies for Residual Vector Quantization: a Masked Language Model (MLM) approach and an Implicit Neural Representation (INR) approach. The MLM strategy iteratively refines tokens along the code depth axis to enhance audio quality, whereas the INR approach predicts all quantization levels in parallel to reduce computational cost. We further incorporate a zero-shot speaker embedding conditioning mechanism, enabling robust performance on unseen speakers. Experimental results demonstrate comparable or even superior improvements in both objective and subjective metrics compared to other larger TTS baseline models.