Replication Data and Code for: Scale-Free Attention Dynamics in Transformer-Based Language Models
| dc.creator | Bhandari, Dhruvin | |
| dc.date | 2026-06-27 | |
| dc.date.accessioned | 2026-11-02T10:39:19Z | |
| dc.description | <p class="">This repository contains the complete experimental framework and raw data results required to replicate the findings in the manuscript "Scale-Free Attention Dynamics in Transformer-Based Language Models."</p> <p class=""><strong>Contents:</strong></p> <ul> <li><strong>Code:</strong> Jupyter notebooks for extracting attention weight matrices from multiple transformer architectures (Llama-3, GPT-2, BERT, T5, Pythia) and conducting rigorous Clauset-Shalizi-Newman (CSN) power-law fitting. It includes control experiments (Zipfian vs. Uniform training) and ablation studies (attention sink masking).</li> <li><strong>Results:</strong> Raw CSV outputs containing the calculated power-law scaling exponents (alpha), Kolmogorov-Smirnov statistics, and Vuong likelihood ratio test verdicts across all six experiments.</li> </ul> | |
| dc.identifier | https://doi.org/10.5281/zenodo.20964276 | |
| dc.identifier | oai:zenodo.org:20964276 | |
| dc.identifier.uri | https://library.dare.co.zw/handle/123456789/2879 | |
| dc.language | eng | |
| dc.publisher | Zenodo | |
| dc.relation | https://doi.org/10.5281/zenodo.20964275 | |
| dc.rights | info:eu-repo/semantics/openAccess | |
| dc.rights | Creative Commons Attribution 4.0 International | |
| dc.rights | https://creativecommons.org/licenses/by/4.0/legalcode | |
| dc.title | Replication Data and Code for: Scale-Free Attention Dynamics in Transformer-Based Language Models | |
| dc.type | info:eu-repo/semantics/other |