<link rel="stylesheet" href="styles.f3b1fba60ec7970c.css">

tamanaganti/abtp: Reproducibility fixes and Kendall τ verification

dc.creatortamanaganti
dc.date2026-04-03
dc.date.accessioned2026-11-02T10:44:28Z
dc.description&lt;h2&gt;Changes from v1.0&lt;/h2&gt; &lt;h3&gt;Fixed&lt;/h3&gt; &lt;ul&gt; &lt;li&gt;The §6 verification snippet in &lt;code&gt;REPRODUCE.md&lt;/code&gt; now takes &lt;strong&gt;&lt;code&gt;bri_imagenet_c&lt;/code&gt; from &lt;code&gt;generalization_results.json&lt;/code&gt;&lt;/strong&gt;, the same artifact that holds all 15 models' shift metrics. Previously it merged BRI from &lt;code&gt;imagenet_c_summary.json&lt;/code&gt; while reading outcomes from the generalization file, which is easy to get out of sync and is unnecessary for a headline check. &lt;strong&gt;Published correlation values are unchanged&lt;/strong&gt;—only the verification script was corrected.&lt;/li&gt; &lt;/ul&gt; &lt;h3&gt;Added&lt;/h3&gt; &lt;ul&gt; &lt;li&gt;&lt;strong&gt;Kendall τ&lt;/strong&gt; alongside Spearman in the correlation snippet, with expected values aligned to &lt;code&gt;scipy.stats.kendalltau&lt;/code&gt; on the bundled JSON (e.g. ImageNet-R τ = −0.390, p = 0.046; ImageNet-A τ = +0.448, p = 0.021; NINCO τ = +0.619, p ≈ 0.001). Signs and significance match Spearman.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;Pre-flight&lt;/strong&gt; Python snippet to confirm &lt;code&gt;bri_imagenet_c&lt;/code&gt; is non-null for every model record before correlating.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;Instructions&lt;/strong&gt; for re-running &lt;code&gt;10_generalization_eval_all15.py&lt;/code&gt; without stale &lt;code&gt;*_generalization.json&lt;/code&gt; cache issues (run Script 04 first; clear caches when regenerating).&lt;/li&gt; &lt;/ul&gt; &lt;h3&gt;Unchanged&lt;/h3&gt; &lt;ul&gt; &lt;li&gt;Application code under &lt;code&gt;src/&lt;/code&gt; and &lt;code&gt;scripts/&lt;/code&gt;&lt;/li&gt; &lt;li&gt;Committed result files and figures&lt;/li&gt; &lt;li&gt;Reported statistics in the paper&lt;/li&gt; &lt;/ul&gt;
dc.identifierhttps://doi.org/10.5281/zenodo.19396270
dc.identifieroai:zenodo.org:19396270
dc.identifier.urihttps://library.dare.co.zw/handle/123456789/3493
dc.publisherZenodo
dc.relationhttps://github.com/tamanaganti/abtp/tree/v1.1
dc.relationhttps://doi.org/10.5281/zenodo.19394106
dc.rightsinfo:eu-repo/semantics/openAccess
dc.rightsMIT License
dc.rightshttps://opensource.org/licenses/MIT
dc.titletamanaganti/abtp: Reproducibility fixes and Kendall τ verification
dc.typeinfo:eu-repo/semantics/other

Files

Collections