elmadani/organ-architecture
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

COMPLETE: 13 models dissected, 5600+ tensors Z-measured, model-935-14b assembled, fractal purification. 3498 LOC, 20 tools. Signature 935.

Browse Source
This commit is contained in:
ElmadaniS 2026-02-21 03:48:51 +01:00
parent 46f33e2a57
commit 3582053790
4 changed files with 799 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
.gitignore vendored
View File

@ -1,8 +1,15 @@
# Organ binary data (too large for git) # Organ data (too large for git)
organs/ organs/
organs-pure/
__pycache__/ __pycache__/
*.pyc *.pyc
# Model files
*.gguf *.gguf
*.bin *.bin
*.safetensors
# Keep JSON reports and scripts # Logs
*.log
# Environment
.env

260
README.md
View File

@ -1,6 +1,6 @@
# Organ Architecture # Organ Architecture
**Decompose. Reassemble. Evolve.** **Decompose. Measure. Purify. Graft. Assemble.**
``` ```
Skeleton (Attention) = Thought Skeleton (Attention) = Thought
@ -8,17 +8,20 @@ Organs (FFN) = Memory
Adapters (LoRA) = Personality Adapters (LoRA) = Personality
``` ```
## What This Is ## The Problem
AI models are monoliths. 70 billion parameters locked in a single file that nobody can open, modify, or understand. Only three companies on Earth can build them. Everyone else rents access. AI models are monoliths. 70 billion parameters locked in a single file that nobody can open, modify, or understand. Only three companies on Earth can build them. Everyone else rents access.
## The Solution
Organ Architecture breaks models into transplantable parts: Organ Architecture breaks models into transplantable parts:
- **Skeleton** — The attention layers. How the model *thinks*. Shared across all configurations. - **Skeleton** — The attention layers. How the model *thinks*. Shared across all configurations.
- **Organs** — The feed-forward networks. What the model *knows*. Specialized, swappable, graftable. - **Organs** — The feed-forward networks. What the model *knows*. Specialized, swappable, graftable.
- **Adapters** — LoRA weights. The model's *personality*. Lightweight, trainable by anyone. - **Adapters** — LoRA weights. The model's *personality*. Lightweight, trainable by anyone.
A doctor doesn't rebuild the entire human body to fix a kidney. Why should we rebuild an entire model to change what it knows about medicine? A doctor doesn't rebuild the entire human body to fix a kidney.
Why rebuild an entire model to change what it knows about medicine?
## Architecture ## Architecture
@ -26,58 +29,49 @@ A doctor doesn't rebuild the entire human body to fix a kidney. Why should we re
model.gguf (70GB monolith) model.gguf (70GB monolith)
│ │
▼ ▼
┌─ skeleton.bin ──── attention layers (shared thought) ┌─ skeleton/ ── attention layers (shared thought)
│ │
├─ organ_lang.bin ── language FFN (what it knows about language) ├─ organs/ ── FFN layers by block (knowledge)
├─ organ_math.bin ── math FFN (what it knows about math) │ ├─ blk_0_ffn_gate.bin
├─ organ_code.bin ── code FFN (what it knows about code) │ ├─ blk_0_ffn_up.bin
├─ organ_med.bin ─── medical FFN (what it knows about medicine) │ ├─ blk_0_ffn_down.bin
│ └─ ...
│ │
└─ adapter_fr.bin ── French personality (LoRA) ├─ embed/ ── embedding + output (foundation)
adapter_formal.bin ── Formal tone (LoRA) ├─ norm/ ── normalization (connective tissue)
└─ manifest.json ── complete anatomy map
``` ```
## Tools ## Tools
| Tool | Purpose | ### Core Pipeline
|------|---------|
| `organ_extract.py` | Extract skeleton + organs from any GGUF model |
| `organ_graft.py` | Transplant organs between models |
| `organ_measure.py` | Z-measure organ quality (signal vs noise) |
| `organ_assemble.py` | Assemble custom model from parts |
| `organ_api.py` | API server for organ operations |
## Requirements | Tool | Lines | Purpose |
|------|-------|---------|
| `organ_extract.py` | 441 | Extract skeleton + organs from any GGUF model |
| `organ_measure.py` | 340 | Z-measure organ quality (signal vs noise) |
| `organ_purify.py` | 333 | Spectral purification (FFT signal extraction) |
| `organ_purify_v2.py` | 337 | Fractal purification (wavelet cross-scale coherence) |
| `organ_graft.py` | 236 | Transplant organs between models |
| `organ_assemble.py` | 235 | Assemble GGUF from organs |
| `organ_api.py` | 422 | HTTP API server for all operations |
- Python 3.10+ ### Build & Automation
- InferenceX binary (for model loading)
- GGUF models to dissect
## Quick Start | Tool | Lines | Purpose |
|------|-------|---------|
| `pipeline_935.py` | 124 | Full dissection pipeline for all models |
| `mass_dissect.py` | 103 | Batch dissection across model fleet |
| `mass_z_measure.py` | 102 | Z-measure every organ of every model |
| `kimi_z_stream.py` | 417 | Stream Z-measure on Kimi K2.5 1T (shard-by-shard) |
| `build_935.py` | 98 | Model 935 assembly v1 |
| `build_935_v2.py` | 74 | Model 935 assembly v2 (selective FFN graft) |
| `build_935_v3.py` | 148 | Model 935 assembly v3 (proper GGUF header) |
| `assemble_935.py` | 150 | Fixed organ header handling assembler |
| `quick_chimera.py` | 123 | Quick chimera GGUF assembler |
| `quick_chimera_v2.py` | 155 | Quick chimera v2 (fixed header stripping) |
```bash **Total: 3,498 lines of Python. Zero external dependencies (except numpy for purification).**
# Extract organs from a model
python3 organ_extract.py --model /path/to/model.gguf --output ./organs/
# Measure organ quality
python3 organ_measure.py --organ ./organs/organ_layer_12.bin
# Graft an organ from model A into model B
python3 organ_graft.py --source ./organs_A/ --target ./model_B.gguf --layers 12-18
# Assemble a custom model
python3 organ_assemble.py --skeleton ./skeleton.bin --organs ./organs/ --output custom.gguf
```
## Philosophy
> Subtract rather than add.
A 70B monolith is accumulation. A 2B skeleton with specialized organs grafted on demand — that's subtraction. Less weight, more signal.
> 8 billion contributors, not 3 corporations.
Anyone can train an organ. A doctor trains a medical organ on her hospital's data. A farmer trains an agriculture organ on his field observations. A student trains a math organ on solved problems. The skeleton stays the same. The organs make it alive.
## Z-Measure ## Z-Measure
@ -90,16 +84,184 @@ Z = dI/d(log s) · exp(iθ)
θ → 90° : pure signal (organ adds knowledge) θ → 90° : pure signal (organ adds knowledge)
``` ```
The measurement combines three indicators:
- **Entropy** — information density of weight distribution
- **Kurtosis** — structural organization (signal sharpness)
- **Scale coherence** — coefficient of variation of sorted value spacings
## Results
### 13 Models Dissected + Kimi K2.5 1T
5,600+ tensors Z-measured. All dissections run on EPYC 48c/503GB (OASIS).
| # | Model | Params | θ mean | Signal | Tensors |
|---|-------|--------|--------|--------|---------|
| ★ | **Kimi K2.5** | **1T MoE** | **87.65°** | **0.999** | **1,083** |
| 1 | SmolLM2-135M | 135M | 52.28° | 0.777 | 272 |
| 2 | DeepSeek-R1-Distill-14B | 14B | 46.01° | 0.641 | 579 |
| 3 | Qwen2.5-3B | 3B | 46.00° | 0.640 | 434 |
| 4 | Qwen2.5-14B | 14B | 45.98° | 0.640 | 579 |
| 5 | Qwen2.5-7B | 7B | 45.64° | 0.639 | 339 |
| 6 | Chimera-DeepSeek-Qwen | 7B | 45.53° | 0.637 | 339 |
| 7 | DeepSeek-R1-Distill-7B | 7B | 45.53° | 0.637 | 339 |
| 8 | DeepSeek-R1-7B | 7B | 45.42° | 0.636 | 339 |
| 9 | Gemma-2-9B | 9B | 44.94° | 0.624 | 464 |
| 10 | Phi-3.5-Mini | 3.8B | 44.65° | 0.626 | 197 |
| 11 | Llama-3.1-8B | 8B | 37.87° | 0.549 | 292 |
| 12 | Llama-3.2-1B | 1B | 37.57° | 0.550 | 147 |
| 13 | Llama-3.2-3B | 3B | 37.41° | 0.547 | 255 |
| 14 | Mistral-7B | 7B | 36.21° | 0.540 | 291 |
### Organ Type Analysis (consistent across all models)
| Organ Type | θ range | Role |
|------------|---------|------|
| Norm layers | 75-84° | Connective tissue — highest signal |
| Skeleton (attention) | 39-56° | Thought structure |
| Organs (FFN) | 34-52° | Knowledge/memory |
| Embeddings | 25-47° | Foundation |
### Scale Law: θ increases with log(parameters)
```
135M → θ = 52.28° (SmolLM2 — small but concentrated)
1-3B → θ = 37-46° (Llama/Qwen)
7-14B → θ = 44-46° (DeepSeek/Qwen)
1T → θ = 87.65° (Kimi K2.5 MoE — near-pure signal)
```
**Ratio 1T/14B: 1.9× purer signal.** The signal purifies with scale.
### Kimi K2.5 1T Deep Analysis
- **Architecture**: DeepSeek2 MoE
- **Blocks**: 61 (blk.0 → blk.60)
- **Experts**: 384 conditional + 1 shared (native INT4 QAT)
- **Context**: 262,144 tokens (256k)
- **Attention**: MLA (Multi-head Latent Attention), MQA kv_head=1
- **13 shards streamed**, each measured and deleted — never loaded full model
| Component | Count | θ avg | Rating |
|-----------|-------|-------|--------|
| FFN dense (blk.0) | 12 | 89.95° | ★★★ |
| MoE experts (384×) | 23 | 89.77° | ★★★ |
| Norm layers | 12 | 89.70° | ★★★ |
| Embedding | 1 | 89.45° | ★★★ |
| Shared expert | 23 | 89.43° | ★★★ |
| Attention (MLA) | 99 | 84.07° | ★★ |
8 gravitational wells identified (lowest θ = maximum structure/compression).
### Model 935 — First Chimera
**`model-935-14b.gguf`** — 8.4 GB, assembled 2026-02-20
Built through 5 iterations:
1. `build_935.py` — Base DeepSeek-R1-Distill-7B + Qwen skeleton graft (crude)
2. `build_935_v2.py` — Selective FFN-only graft (preserve attention-embed alignment)
3. `build_935_v3.py` — Proper GGUF header handling
4. `quick_chimera.py` → `quick_chimera_v2.py` — Fixed organ header stripping
5. `assemble_935.py` — Final assembler, 14B scale
### Purification
**`organs-pure/smollm2-135m/`** — First purified model (fractal method)
`organ_purify_v2.py` implements cross-scale coherence via Haar wavelets:
- Decompose tensor into multiple scales
- Measure coherence between adjacent scales
- Pattern at scale s AND scale 2s → signal (fractal, keep)
- Pattern at one scale only → noise (remove)
- This is `dI/d(log s)` implemented directly
## Dissection Report
| Model | Size (MB) | Dissection Time |
|-------|-----------|-----------------|
| DeepSeek-R1-14B | 9,167 | 22.9s |
| Gemma-2-9B | 5,984 | 14.8s |
| Llama-3.1-8B | 4,950 | 12.0s |
| DeepSeek-R1-Distill-7B | 4,812 | 12.6s |
| Mistral-7B | 4,432 | 10.6s |
| Phi-3.5-Mini | 2,397 | 4.9s |
| Llama-3.2-3B | 2,100 | 4.9s |
| Qwen2.5-3B | 2,003 | 4.6s |
| Llama-3.2-1B | 856 | 2.4s |
Total organs on disk: **50.8 GB** across 13 models.
## Quick Start
```bash
# Extract organs from a model
python3 organ_extract.py --model /path/to/model.gguf --output ./organs/model-name/
# Z-measure all organs
python3 organ_measure.py --dir ./organs/model-name/
# Mass dissect all models
python3 mass_dissect.py
# Mass Z-measure
python3 mass_z_measure.py
# Stream Z-measure on a trillion-param model (shard-by-shard)
python3 kimi_z_stream.py
# Graft organs from one model to another
python3 organ_graft.py graft --source ./organs/qwen/ --target ./organs/deepseek/ --output ./organs/chimera/ --layers 5-20 --type organ
# Assemble back to GGUF
python3 organ_assemble.py --dir ./organs/chimera/ --output chimera.gguf
# Purify organs (fractal method)
python3 organ_purify_v2.py --dir ./organs/model/ --output ./organs-pure/model/
# Start API server
python3 organ_api.py
```
## Philosophy
> Subtract rather than add.
A 70B monolith is accumulation. A skeleton with specialized organs grafted on demand — that's subtraction. Less weight, more signal.
> 8 billion contributors, not 3 corporations.
Anyone can train an organ. A doctor trains a medical organ on her hospital's data. A farmer trains an agriculture organ on his field observations. A student trains a math organ on solved problems. The skeleton stays the same. The organs make it alive.
## Part of the IX Ecosystem ## Part of the IX Ecosystem
``` ```
InferenceX ─── The engine (228KB, runs anything) InferenceX ─── The engine (305KB, runs anything)
Organ Arch ─── The anatomy (decompose, reassemble) Organ Arch ─── The anatomy (decompose, measure, reassemble)
Atlas Pure ─── The memory (fractal DNA storage) Atlas Pure ─── The memory (fractal DNA storage)
Echo ────────── The voice (chat interface) INVOKE ─────── The bridge (cloud ↔ physical)
Echo ────────── The voice (chat interface)
EDEN ────────── The purpose (desert → life) EDEN ────────── The purpose (desert → life)
``` ```
## Requirements
- Python 3.10+
- NumPy (for purification only)
- InferenceX binary (for inference on assembled models)
- GGUF models to dissect
## Data Files
| File | Contents |
|------|----------|
| `z_report_complete.json` | Z-measure for all 13 models (per-group breakdown) |
| `z_report_kimi_k25.json` | Z-measure for all 1,083 Kimi K2.5 tensors |
| `z_measure_report.json` | Combined Z-ranking with chimera results |
| `dissection_report.json` | Dissection timing and sizes |
| `Z_MEASURE_REPORT.md` | Human-readable Z report |
| `ECHO_INVARIANT.md` | Team 935 invariant |
| `EQUIPE_935_INVARIANT.json` | Team 935 configuration |
## License ## License
BSL 1.1 — Same as InferenceX. BSL 1.1 — Same as InferenceX.

76
dissection_report.json Normal file
View File

@ -0,0 +1,76 @@
[
{
"model": "deepseek-r1-14b",
"status": "dissected",
"size_mb": 9167.481572151184,
"time_s": 22.94489073753357
},
{
"model": "qwen25-14b",
"status": "exists",
"size_mb": 9026.720261573792
},
{
"model": "gemma2-9b",
"status": "dissected",
"size_mb": 5983.6147108078,
"time_s": 14.836755275726318
},
{
"model": "llama31-8b",
"status": "dissected",
"size_mb": 4950.371293067932,
"time_s": 12.016721963882446
},
{
"model": "qwen25-7b",
"status": "exists",
"size_mb": 4811.518325805664
},
{
"model": "deepseek-r1-distill-7b",
"status": "dissected",
"size_mb": 4811.928074836731,
"time_s": 12.550673007965088
},
{
"model": "deepseek-r1-7b",
"status": "exists",
"size_mb": 4811.927845954895
},
{
"model": "mistral-7b",
"status": "dissected",
"size_mb": 4432.171175956726,
"time_s": 10.590012550354004
},
{
"model": "phi35-mini",
"status": "dissected",
"size_mb": 2397.4848985671997,
"time_s": 4.872461318969727
},
{
"model": "llama32-3b",
"status": "dissected",
"size_mb": 2100.286515235901,
"time_s": 4.853139638900757
},
{
"model": "qwen25-3b",
"status": "dissected",
"size_mb": 2002.6401329040527,
"time_s": 4.552767276763916
},
{
"model": "llama32-1b",
"status": "dissected",
"size_mb": 856.2387390136719,
"time_s": 2.3548576831817627
},
{
"model": "smollm2-135m",
"status": "exists",
"size_mb": 136.5001106262207
}
]

501
z_measure_report.json Normal file
View File

@ -0,0 +1,501 @@
{
"chimera-deepseek-qwen": {
"model": "chimera-deepseek-qwen",
"total_tensors": 339,
"avg_theta": 45.53097345132743,
"avg_signal": 0.6371591309220915,
"groups": {
"skeleton": {
"count": 196,
"avg_theta": 54.2,
"avg_signal": 0.727,
"best_theta": 84.0,
"best_name": "blk.0.attn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 112,
"avg_theta": 35.9,
"avg_signal": 0.538,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_gate.weight"
},
"embed": {
"count": 31,
"avg_theta": 25.9,
"avg_signal": 0.429,
"best_theta": 75.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"deepseek-r1-14b": {
"model": "deepseek-r1-14b",
"total_tensors": 579,
"avg_theta": 46.01036269430051,
"avg_signal": 0.640550897397108,
"groups": {
"skeleton": {
"count": 336,
"avg_theta": 55.4,
"avg_signal": 0.736,
"best_theta": 84.0,
"best_name": "blk.0.attn_k.bias",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 192,
"avg_theta": 35.2,
"avg_signal": 0.532,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 51,
"avg_theta": 25.2,
"avg_signal": 0.42,
"best_theta": 75.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"deepseek-r1-7b": {
"model": "deepseek-r1-7b",
"total_tensors": 339,
"avg_theta": 45.424778761061944,
"avg_signal": 0.6355319640555519,
"groups": {
"skeleton": {
"count": 196,
"avg_theta": 54.2,
"avg_signal": 0.727,
"best_theta": 84.0,
"best_name": "blk.0.attn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 112,
"avg_theta": 35.5,
"avg_signal": 0.533,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_gate.weight"
},
"embed": {
"count": 31,
"avg_theta": 25.9,
"avg_signal": 0.429,
"best_theta": 75.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"deepseek-r1-distill-7b": {
"model": "deepseek-r1-distill-7b",
"total_tensors": 339,
"avg_theta": 45.53097345132743,
"avg_signal": 0.6371591309220915,
"groups": {
"skeleton": {
"count": 196,
"avg_theta": 54.2,
"avg_signal": 0.727,
"best_theta": 84.0,
"best_name": "blk.0.attn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 112,
"avg_theta": 35.9,
"avg_signal": 0.538,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_gate.weight"
},
"embed": {
"count": 31,
"avg_theta": 25.9,
"avg_signal": 0.429,
"best_theta": 75.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"gemma2-9b": {
"model": "gemma2-9b",
"total_tensors": 464,
"avg_theta": 44.935344827586206,
"avg_signal": 0.6240438819131022,
"groups": {
"skeleton": {
"count": 210,
"avg_theta": 47.2,
"avg_signal": 0.649,
"best_theta": 84.0,
"best_name": "blk.0.post_attention_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 168,
"avg_theta": 37.9,
"avg_signal": 0.552,
"best_theta": 84.0,
"best_name": "blk.1.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 44,
"avg_theta": 26.2,
"avg_signal": 0.433,
"best_theta": 84.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
},
"norm": {
"count": 42,
"avg_theta": 81.6,
"avg_signal": 0.987,
"best_theta": 84.0,
"best_name": "blk.10.post_ffw_norm.weight",
"worst_theta": 75.0,
"worst_name": "blk.0.post_ffw_norm.weight"
}
}
},
"llama31-8b": {
"model": "llama31-8b",
"total_tensors": 292,
"avg_theta": 37.86986301369863,
"avg_signal": 0.5490538952939957,
"groups": {
"skeleton": {
"count": 128,
"avg_theta": 39.7,
"avg_signal": 0.569,
"best_theta": 84.0,
"best_name": "blk.10.attn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.10.attn_k.weight"
},
"organs": {
"count": 128,
"avg_theta": 39.1,
"avg_signal": 0.56,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 35,
"avg_theta": 26.0,
"avg_signal": 0.427,
"best_theta": 84.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"llama32-1b": {
"model": "llama32-1b",
"total_tensors": 147,
"avg_theta": 37.57142857142857,
"avg_signal": 0.5497319048747188,
"groups": {
"skeleton": {
"count": 64,
"avg_theta": 39.3,
"avg_signal": 0.57,
"best_theta": 84.0,
"best_name": "blk.10.attn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_q.weight"
},
"organs": {
"count": 64,
"avg_theta": 38.3,
"avg_signal": 0.553,
"best_theta": 84.0,
"best_name": "blk.10.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 18,
"avg_theta": 27.3,
"avg_signal": 0.445,
"best_theta": 75.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"llama32-3b": {
"model": "llama32-3b",
"total_tensors": 255,
"avg_theta": 37.411764705882355,
"avg_signal": 0.546769292896037,
"groups": {
"skeleton": {
"count": 112,
"avg_theta": 39.4,
"avg_signal": 0.569,
"best_theta": 84.0,
"best_name": "blk.0.attn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 112,
"avg_theta": 38.0,
"avg_signal": 0.55,
"best_theta": 84.0,
"best_name": "blk.13.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 30,
"avg_theta": 26.6,
"avg_signal": 0.439,
"best_theta": 75.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"mistral-7b": {
"model": "mistral-7b",
"total_tensors": 291,
"avg_theta": 36.20618556701031,
"avg_signal": 0.539809742436977,
"groups": {
"skeleton": {
"count": 128,
"avg_theta": 38.4,
"avg_signal": 0.567,
"best_theta": 84.0,
"best_name": "blk.0.attn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.10.attn_k.weight"
},
"organs": {
"count": 128,
"avg_theta": 36.8,
"avg_signal": 0.544,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 35,
"avg_theta": 26.0,
"avg_signal": 0.427,
"best_theta": 84.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"phi35-mini": {
"model": "phi35-mini",
"total_tensors": 197,
"avg_theta": 44.6497461928934,
"avg_signal": 0.6262773662109529,
"groups": {
"skeleton": {
"count": 64,
"avg_theta": 56.7,
"avg_signal": 0.764,
"best_theta": 84.0,
"best_name": "blk.10.attn_norm.weight",
"worst_theta": 33.0,
"worst_name": "blk.0.attn_qkv.weight"
},
"organs": {
"count": 96,
"avg_theta": 43.2,
"avg_signal": 0.601,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 35,
"avg_theta": 26.7,
"avg_signal": 0.439,
"best_theta": 84.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"qwen25-14b": {
"model": "qwen25-14b",
"total_tensors": 579,
"avg_theta": 45.98445595854922,
"avg_signal": 0.6402458335664142,
"groups": {
"skeleton": {
"count": 336,
"avg_theta": 55.2,
"avg_signal": 0.734,
"best_theta": 84.0,
"best_name": "blk.0.attn_k.bias",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 192,
"avg_theta": 35.4,
"avg_signal": 0.534,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 51,
"avg_theta": 25.5,
"avg_signal": 0.424,
"best_theta": 84.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"qwen25-3b": {
"model": "qwen25-3b",
"total_tensors": 434,
"avg_theta": 46.00230414746544,
"avg_signal": 0.6401608443093786,
"groups": {
"skeleton": {
"count": 252,
"avg_theta": 55.6,
"avg_signal": 0.736,
"best_theta": 84.0,
"best_name": "blk.0.attn_k.bias",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 144,
"avg_theta": 34.5,
"avg_signal": 0.529,
"best_theta": 84.0,
"best_name": "blk.10.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_down.weight"
},
"embed": {
"count": 38,
"avg_theta": 25.8,
"avg_signal": 0.426,
"best_theta": 84.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"qwen25-7b": {
"model": "qwen25-7b",
"total_tensors": 339,
"avg_theta": 45.637168141592916,
"avg_signal": 0.6387682956137819,
"groups": {
"skeleton": {
"count": 196,
"avg_theta": 54.6,
"avg_signal": 0.731,
"best_theta": 84.0,
"best_name": "blk.0.attn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_k.weight"
},
"organs": {
"count": 112,
"avg_theta": 35.5,
"avg_signal": 0.536,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.ffn_gate.weight"
},
"embed": {
"count": 31,
"avg_theta": 25.9,
"avg_signal": 0.429,
"best_theta": 75.0,
"best_name": "output_norm.weight",
"worst_theta": 24.0,
"worst_name": "blk.0.attn_output.weight"
}
}
},
"smollm2-135m": {
"model": "smollm2-135m",
"total_tensors": 272,
"avg_theta": 52.27941176470588,
"avg_signal": 0.7765030923203783,
"groups": {
"skeleton": {
"count": 120,
"avg_theta": 53.6,
"avg_signal": 0.79,
"best_theta": 84.0,
"best_name": "blk.0.attn_norm.weight",
"worst_theta": 42.0,
"worst_name": "blk.10.attn_k.weight"
},
"organs": {
"count": 120,
"avg_theta": 52.3,
"avg_signal": 0.777,
"best_theta": 84.0,
"best_name": "blk.0.ffn_norm.weight",
"worst_theta": 42.0,
"worst_name": "blk.11.ffn_up.weight"
},
"embed": {
"count": 32,
"avg_theta": 47.2,
"avg_signal": 0.725,
"best_theta": 84.0,
"best_name": "output_norm.weight",
"worst_theta": 33.0,
"worst_name": "blk.13.attn_output.weight"
}
}
}
}