hkalbertkim/KORA: KORA v0.1.1-alpha
| dc.creator | Albert "0xARBV" Kim | |
| dc.date | 2026-05-03 | |
| dc.date.accessioned | 2026-11-02T10:32:24Z | |
| dc.description | <h1>KORA v0.1.1-alpha</h1> <h2>Summary</h2> <p>KORA v0.1.1-alpha adds CI-backed validation and the first controlled benchmark skeleton. In a deterministic-heavy benchmark skeleton, KORA-controlled execution avoided 16 of 20 simulated model invocations versus a naive direct baseline.</p> <p>This is an alpha maintenance / benchmark skeleton release. It keeps the v0.1.0-alpha terminal-first surface intact while adding a reproducible CI and benchmark path for future technical preview work.</p> <h2>What Changed Since v0.1.0-alpha</h2> <ul> <li>Added GitHub Actions CI for editable install, release smoke, and pytest validation.</li> <li>Added the <code>experiments/</code> benchmark experiment directory.</li> <li>Added the deterministic-heavy workload draft at <code>experiments/workloads/deterministic_heavy_v0.json</code>.</li> <li>Added dry-run benchmark mode for workload validation and task/category counting.</li> <li>Added direct-baseline benchmark mode to simulate one model invocation per task.</li> <li>Added KORA-controlled benchmark mode to simulate deterministic-first execution from workload metadata.</li> <li>Added the 2026-05-04 progress report.</li> <li>Added the v0.1.1-alpha release note.</li> </ul> <h2>CI-Backed Validation</h2> <p>GitHub Actions now validates changes to <code>main</code> with:</p> <pre><code class="language-bash">python3 -m pip install -e ".[dev]" ./scripts/release_smoke.sh python3 -m pytest -q </code></pre> <p>The release candidate was also verified locally with release smoke, benchmark modes, JSON validation, and pytest.</p> <h2>Benchmark Skeleton</h2> <p>This release introduces the initial benchmark structure:</p> <pre><code class="language-text">experiments/ experiments/README.md experiments/run_benchmark.py experiments/workloads/deterministic_heavy_v0.json experiments/results/.gitkeep </code></pre> <p>The benchmark runner currently supports:</p> <ul> <li><code>dry-run</code></li> <li><code>direct-baseline</code></li> <li><code>kora-controlled</code></li> </ul> <p>These modes are simulated and metadata-based. They do not call real models or external APIs.</p> <h2>Current Controlled Benchmark Result</h2> <p>| Metric | Value | |---|---:| | Total tasks | 20 | | Direct-baseline simulated model invocations | 20 | | KORA-controlled simulated model invocations | 4 | | Deterministic resolutions | 16 | | Fallback candidates | 4 | | Avoided simulated model invocations | 16 | | Avoided invocation rate | 80% |</p> <h2>How To Verify Locally</h2> <pre><code class="language-bash">./scripts/release_smoke.sh python3 -m pytest -q python3 experiments/run_benchmark.py --mode dry-run --workload experiments/workloads/deterministic_heavy_v0.json --output /tmp/kora_deterministic_heavy_v0.dry_run.json python3 experiments/run_benchmark.py --mode direct-baseline --workload experiments/workloads/deterministic_heavy_v0.json --output /tmp/kora_deterministic_heavy_v0.direct_baseline.json python3 experiments/run_benchmark.py --mode kora-controlled --workload experiments/workloads/deterministic_heavy_v0.json --output /tmp/kora_deterministic_heavy_v0.kora_controlled.json </code></pre> <p>Optional JSON checks:</p> <pre><code class="language-bash">python3 -m json.tool /tmp/kora_deterministic_heavy_v0.dry_run.json > /tmp/kora_dry_run_check.json python3 -m json.tool /tmp/kora_deterministic_heavy_v0.direct_baseline.json > /tmp/kora_direct_baseline_check.json python3 -m json.tool /tmp/kora_deterministic_heavy_v0.kora_controlled.json > /tmp/kora_kora_controlled_check.json </code></pre> <h2>Limitations</h2> <ul> <li>The benchmark is simulated.</li> <li>No real model calls are made.</li> <li>No external API calls are made.</li> <li>No real API-cost measurement is included.</li> <li>No production cost reduction claim is supported.</li> <li>No full KORA runtime integration is implemented yet.</li> <li>The workload is intentionally small and deterministic-heavy.</li> </ul> <h2>Next Planned Work</h2> <ul> <li>Decide whether benchmark result artifacts should remain generated-only or be committed under a tracked path.</li> <li>Add a concise benchmark result summary markdown if the result should be reviewed publicly.</li> <li>Extend the benchmark runner toward real KORA runtime integration in a later alpha.</li> <li>Continue expanding deterministic-heavy workloads beyond the initial 20-task draft.</li> </ul> | |
| dc.identifier | https://doi.org/10.5281/zenodo.20013234 | |
| dc.identifier | oai:zenodo.org:20013234 | |
| dc.identifier.uri | https://library.dare.co.zw/handle/123456789/2052 | |
| dc.publisher | Zenodo | |
| dc.relation | https://github.com/hkalbertkim/KORA/tree/v0.1.1-alpha | |
| dc.relation | https://doi.org/10.5281/zenodo.20013233 | |
| dc.rights | info:eu-repo/semantics/openAccess | |
| dc.rights | Apache License 2.0 | |
| dc.rights | http://www.apache.org/licenses/LICENSE-2.0 | |
| dc.title | hkalbertkim/KORA: KORA v0.1.1-alpha | |
| dc.type | info:eu-repo/semantics/other |