<link rel="stylesheet" href="styles.f3b1fba60ec7970c.css">

Replication Data and Code for: Scale-Free Attention Dynamics in Transformer-Based Language Models

dc.creatorBhandari, Dhruvin
dc.date2026-06-27
dc.date.accessioned2026-11-02T10:39:19Z
dc.description&lt;p class=""&gt;This repository contains the complete experimental framework and raw data results required to replicate the findings in the manuscript "Scale-Free Attention Dynamics in Transformer-Based Language Models."&lt;/p&gt; &lt;p class=""&gt;&lt;strong&gt;Contents:&lt;/strong&gt;&lt;/p&gt; &lt;ul&gt; &lt;li&gt;&lt;strong&gt;Code:&lt;/strong&gt;&nbsp;Jupyter notebooks for extracting attention weight matrices from multiple transformer architectures (Llama-3, GPT-2, BERT, T5, Pythia) and conducting rigorous Clauset-Shalizi-Newman (CSN) power-law fitting. It includes control experiments (Zipfian vs. Uniform training) and ablation studies (attention sink masking).&lt;/li&gt; &lt;li&gt;&lt;strong&gt;Results:&lt;/strong&gt; Raw CSV outputs containing the calculated power-law scaling exponents (alpha), Kolmogorov-Smirnov statistics, and Vuong likelihood ratio test verdicts across all six experiments.&lt;/li&gt; &lt;/ul&gt;
dc.identifierhttps://doi.org/10.5281/zenodo.20964276
dc.identifieroai:zenodo.org:20964276
dc.identifier.urihttps://library.dare.co.zw/handle/123456789/2879
dc.languageeng
dc.publisherZenodo
dc.relationhttps://doi.org/10.5281/zenodo.20964275
dc.rightsinfo:eu-repo/semantics/openAccess
dc.rightsCreative Commons Attribution 4.0 International
dc.rightshttps://creativecommons.org/licenses/by/4.0/legalcode
dc.titleReplication Data and Code for: Scale-Free Attention Dynamics in Transformer-Based Language Models
dc.typeinfo:eu-repo/semantics/other

Files

Collections