<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN"
  "https://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink"
         xmlns:mml="http://www.w3.org/1998/Math/MathML"
         article-type="research-article"
         dtd-version="1.2">

  <!-- ============================================================ FRONT -->
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">IJLTEMAS</journal-id>
      <journal-title-group>
        <journal-title>International Journal of Latest Technology in Engineering, Management &amp; Applied Science (IJLTEMAS)</journal-title>
        <abbrev-journal-title abbrev-type="publisher">IJLTEMAS</abbrev-journal-title>
      </journal-title-group>
      <issn pub-type="epub">2278-2540</issn>
      <publisher>
        <publisher-name>IJLTEMAS</publisher-name>
      </publisher>
    </journal-meta>

    <article-meta>
      <!-- IDs -->
      <article-id pub-id-type="publisher-id">617</article-id>
            <article-id pub-id-type="doi">10.51583/IJLTEMAS.2026.151000096</article-id>
      
      <!-- Categories -->
            <article-categories>
        <subj-group subj-group-type="heading">
          <subject>Natural Language Processing</subject>
        </subj-group>
      </article-categories>
      
      <!-- Title -->
      <title-group>
        <article-title>Efficient Decoder-Only Transformer with Domain-Specific Byte Pair Encoding for Python Code Generation</article-title>
      </title-group>

      <!-- Authors -->
      <contrib-group>
                <contrib contrib-type="author">
                    <name>
            <surname>P. Venkateshwarlu</surname>
            <given-names>Dr.</given-names>
          </name>
                              <aff>
            Assistant Professor, Department of Computer Applications, Vaageswari College of Engineering (Autonomous), Karimnagar, Telangana, India                        <country>India</country>
                      </aff>
                    
        </contrib>
                <contrib contrib-type="author">
                    <name>
            <surname>M. Srinivas</surname>
            <given-names>Dr.</given-names>
          </name>
                              <aff>
            Assistant Professor, Department of CSE (AI&amp;ML), Vaageswari College of Engineering (Autonomous), Karimnagar, Telangana, India                        <country>India</country>
                      </aff>
                    
        </contrib>
                <contrib contrib-type="author">
                    <name>
            <surname>Ravikumar Thallapalli</surname>
            <given-names>Dr.</given-names>
          </name>
                              <aff>
            Assistant Professor, Department of Computer Science and Engineering, Vaageswari College of Engineering (Autonomous), Karimnagar, Telangana, India                        <country>India</country>
                      </aff>
                    
        </contrib>
              </contrib-group>

      <!-- Volume / Issue / Pages -->
            <volume>15</volume>
                  <issue>10</issue>
                        <fpage>1184</fpage>
            <lpage>1194</lpage>
            
      <!-- Dates -->
      <history>
                <date date-type="received">
          <day>28</day>
          <month>10</month>
          <year>2026</year>
        </date>
                        <date date-type="accepted">
          <day>02</day>
          <month>11</month>
          <year>2026</year>
        </date>
              </history>

            <pub-date pub-type="epub">
        <day>10</day>
        <month>11</month>
        <year>2026</year>
      </pub-date>
      
      <!-- DOI Self-URI -->
            <self-uri xlink:href="https://doi.org/10.51583/IJLTEMAS.2026.151000096"/>
      
      <!-- Keywords -->
            <kwd-group kwd-group-type="author">
                <kwd>Python code generation</kwd>
                <kwd>Byte Pair Encoding</kwd>
                <kwd>domain-specific tokenization</kwd>
                <kwd>decoder-only Transformer</kwd>
                <kwd>code language modeling</kwd>
                <kwd>resource-efficient AI</kwd>
                <kwd>functional code-generation evaluation</kwd>
              </kwd-group>
      
    </article-meta>
  </front>

  <!-- ============================================================ BODY (Abstract) -->
  <body>
        <sec>
      <title>Abstract</title>
      <p>Efficient code generation requires a balance between model quality and computational cost, particularly when experiments are performed on resource-constrained hardware. This work investigates the effect of domain-specific Byte Pair Encoding (BPE) on a compact decoder-only Transformer for Python code generation. An 8,000-token Python-specific BPE tokenizer is compared with the 50,258-token GPT-2 tokenizer using the same compact architecture, consisting of two Transformer layers, four attention heads, a model dimension of 128, a feed-forward dimension of 512, and a maximum sequence length of 256 tokens. Experiments use 15,000 training functions, 2,000 validation functions, and 3,000 independent test functions selected with repository-level separation and cross-split duplicate-code verification. On the independent test set, the Python-specific tokenizer reduces average tokenization length by 41.94% and training blocks by 33.89%. The resulting model contains 81.37% fewer parameters, requires 93.79% less training time, and uses 72.40% less peak GPU memory during test evaluation than the GPT-2-tokenizer configuration. However, these efficiency gains are accompanied by lower language-model and code-generation performance. The Custom-BPE model obtains a test perplexity of 125.14, compared with 15.68 for the GPT-2 configuration. These perplexity values are measured over their respective token distributions and are therefore not directly comparable across vocabularies. Under controlled deterministic generation, the Custom-BPE and GPT-2 models produce syntactically valid and compilable outputs in 0% and 26% of cases, respectively. The results therefore characterize an explicit efficiency-versus-generation-quality trade-off rather than a universal quality advantage for domain-specific tokenization.</p>
    </sec>
      </body>

  <!-- ============================================================ BACK (References) -->
    <back>
    <ref-list>
      <title>References</title>
            <ref id="ref1">
        <label>1</label>
        <mixed-citation>A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017.</mixed-citation>
      </ref>
            <ref id="ref2">
        <label>2</label>
        <mixed-citation>A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” OpenAI, Tech. Rep., 2019.</mixed-citation>
      </ref>
            <ref id="ref3">
        <label>3</label>
        <mixed-citation>E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong, “CodeGen: An open large language model for code with multi-turn program synthesis,” in Proc. Int. Conf. Learn. Representations (ICLR), 2023.</mixed-citation>
      </ref>
            <ref id="ref4">
        <label>4</label>
        <mixed-citation>H. Husain, H.-H. Wu, T. Gazit, M. Allamanis, and M. Brockschmidt, “CodeSearchNet challenge: Evaluating the state of semantic code search,” arXiv:1909.09436, 2019.</mixed-citation>
      </ref>
            <ref id="ref5">
        <label>5</label>
        <mixed-citation>Y. Wang, W. Wang, S. Joty, and S. C. H. Hoi, “CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,” in Proc. EMNLP, 2021, pp. 8696–8708.</mixed-citation>
      </ref>
            <ref id="ref6">
        <label>6</label>
        <mixed-citation>Q. Zheng, X. Xia, X. Zou, Y. Dong, S. Wang, Y. Xue, Z. Wang, L. Shen, A. Wang, Y. Li, T. Su, Z. Yang, and J. Tang, “CodeGeeX: A pre-trained model for code generation with multilingual benchmarking on HumanEval-X,” in Proc. 29th ACM SIGKDD Conf. Knowledge Discovery and Data Mining, 2023, pp. 5673–5684.</mixed-citation>
      </ref>
            <ref id="ref7">
        <label>7</label>
        <mixed-citation>S. Tipirneni, M. Zhu, and C. K. Reddy, “StructCoder: Structure-aware Transformer for code generation,” ACM Trans. Knowl. Discovery Data, vol. 18, no. 3, Art. no. 70, 2024.</mixed-citation>
      </ref>
            <ref id="ref8">
        <label>8</label>
        <mixed-citation>R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in Proc. 54th Annu. Meeting Assoc. Comput. Linguistics, 2016, pp. 1715–1725.</mixed-citation>
      </ref>
            <ref id="ref9">
        <label>9</label>
        <mixed-citation>E. J. Hu, Y. Shen, P. Wallis, Z. A. Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” arXiv:2106.09685, 2021.</mixed-citation>
      </ref>
            <ref id="ref10">
        <label>10</label>
        <mixed-citation>B. Rozière et al., “Code Llama: Open foundation models for code,” arXiv:2308.12950, 2023.</mixed-citation>
      </ref>
            <ref id="ref11">
        <label>11</label>
        <mixed-citation>H. Touvron et al., “LLaMA: Open and efficient foundation language models,” arXiv:2302.13971, 2023.</mixed-citation>
      </ref>
            <ref id="ref12">
        <label>12</label>
        <mixed-citation>R. Li et al., “StarCoder: May the source be with you!,” Trans. Mach. Learn. Res., 2023.</mixed-citation>
      </ref>
            <ref id="ref13">
        <label>13</label>
        <mixed-citation>H. Touvron et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv:2307.09288, 2023.</mixed-citation>
      </ref>
            <ref id="ref14">
        <label>14</label>
        <mixed-citation>M. Chen et al., “Evaluating large language models trained on code,” arXiv:2107.03374, 2021.</mixed-citation>
      </ref>
            <ref id="ref15">
        <label>15</label>
        <mixed-citation>J. Austin et al., “Program synthesis with large language models,” arXiv:2108.07732, 2021.</mixed-citation>
      </ref>
            <ref id="ref16">
        <label>16</label>
        <mixed-citation>I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in Proc. Int. Conf. Learn. Representations (ICLR), 2019.</mixed-citation>
      </ref>
          </ref-list>
  </back>
  
</article>
