Efficient Decoder-Only Transformer with Domain-Specific Byte Pair Encoding for Python Code Generation

Efficient code generation requires a balance between model quality and computational cost, particularly when experiments are performed on resource-constrained hardware. This work investigates the effect of domain-specific Byte Pair Encoding (BPE) on a compact decoder-only Transformer for Python code generation. An 8,000-token Python-specific BPE tokenizer is compared with the 50,258-token GPT-2 tokenizer using the same compact architecture, consisting of two Transformer layers, four attention heads, a model dimension of 128, a feed-forward dimension of 512, and a maximum sequence length of 256 tokens. Experiments use 15,000 training functions, 2,000 validation functions, and 3,000 independent test functions selected with repository-level separation and cross-split duplicate-code verification. On the independent test set, the Python-specific tokenizer reduces average tokenization length by 41.94% and training blocks by 33.89%. The resulting model contains 81.37% fewer parameters, requires 93.79% less training time, and uses 72.40% less peak GPU memory during test evaluation than the GPT-2-tokenizer configuration. However, these efficiency gains are accompanied by lower language-model and code-generation performance. The Custom-BPE model obtains a test perplexity of 125.14, compared with 15.68 for the GPT-2 configuration. These perplexity values are measured over their respective token distributions and are therefore not directly comparable across vocabularies. Under controlled deterministic generation, the Custom-BPE and GPT-2 models produce syntactically valid and compilable outputs in 0% and 26% of cases, respectively. The results therefore characterize an explicit efficiency-versus-generation-quality trade-off rather than a universal quality advantage for domain-specific tokenization.

Python code generation, Byte Pair Encoding, domain-specific tokenization, decoder-only Transformer, code language modeling, resource-efficient AI, functional code-generation evaluation

1. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017. [Google Scholar] [Crossref]

2. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” OpenAI, Tech. Rep., 2019. [Google Scholar] [Crossref]

3. E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong, “CodeGen: An open large language model for code with multi-turn program synthesis,” in Proc. Int. Conf. Learn. Representations (ICLR), 2023. [Google Scholar] [Crossref]

4. H. Husain, H.-H. Wu, T. Gazit, M. Allamanis, and M. Brockschmidt, “CodeSearchNet challenge: Evaluating the state of semantic code search,” arXiv:1909.09436, 2019. [Google Scholar] [Crossref]

5. Y. Wang, W. Wang, S. Joty, and S. C. H. Hoi, “CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,” in Proc. EMNLP, 2021, pp. 8696–8708. [Google Scholar] [Crossref]

6. Q. Zheng, X. Xia, X. Zou, Y. Dong, S. Wang, Y. Xue, Z. Wang, L. Shen, A. Wang, Y. Li, T. Su, Z. Yang, and J. Tang, “CodeGeeX: A pre-trained model for code generation with multilingual benchmarking on HumanEval-X,” in Proc. 29th ACM SIGKDD Conf. Knowledge Discovery and Data Mining, 2023, pp. 5673–5684. [Google Scholar] [Crossref]

7. S. Tipirneni, M. Zhu, and C. K. Reddy, “StructCoder: Structure-aware Transformer for code generation,” ACM Trans. Knowl. Discovery Data, vol. 18, no. 3, Art. no. 70, 2024. [Google Scholar] [Crossref]

8. R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in Proc. 54th Annu. Meeting Assoc. Comput. Linguistics, 2016, pp. 1715–1725. [Google Scholar] [Crossref]

9. E. J. Hu, Y. Shen, P. Wallis, Z. A. Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” arXiv:2106.09685, 2021. [Google Scholar] [Crossref]

10. B. Rozière et al., “Code Llama: Open foundation models for code,” arXiv:2308.12950, 2023. [Google Scholar] [Crossref]

11. H. Touvron et al., “LLaMA: Open and efficient foundation language models,” arXiv:2302.13971, 2023. [Google Scholar] [Crossref]

12. R. Li et al., “StarCoder: May the source be with you!,” Trans. Mach. Learn. Res., 2023. [Google Scholar] [Crossref]

13. H. Touvron et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv:2307.09288, 2023. [Google Scholar] [Crossref]

14. M. Chen et al., “Evaluating large language models trained on code,” arXiv:2107.03374, 2021. [Google Scholar] [Crossref]

15. J. Austin et al., “Program synthesis with large language models,” arXiv:2108.07732, 2021. [Google Scholar] [Crossref]

16. I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in Proc. Int. Conf. Learn. Representations (ICLR), 2019. [Google Scholar] [Crossref]