# Knowledge (OKF)

Search OKF bundles with BM25F, read concepts with provenance, and follow their links.

The [Open Knowledge Format](https://github.com/GoogleCloudPlatform/open-knowledge-format)
(OKF v0.2) is a vendor-neutral way to give agents curated context: a folder of
markdown files with YAML frontmatter. Every non-reserved `.md` file is a
**concept** carrying at least a `type`. Two names are reserved: `index.md` is a
directory listing and `log.md` records change history. Ordinary markdown links
between concepts form the graph, so nothing declares it.

Context fills **lazily**. Only the root index goes into the system prompt, and
the model pulls concepts in with three read-only tools. Where
[skills](../skills/) carry instructions, knowledge bundles carry facts with
provenance.

## Authoring a concept

`knowledge/metrics/active-users.md`:

~~~markdown
---
type: Metric
title: Active Users
description: Daily and monthly active users, counted from qualifying product events.
tags: [metric, engagement]
status: stable
stale_after: 2027-09-27T00:00:00Z
verified:
  - by: human:analytics-lead
    at: 2026-09-27T00:00:00Z
sources:
  - id: dau-model
    resource: /tables/events.md
---

A user is active on a day when they emit one qualifying event.[^dau-model]
Computed from the [events table](/tables/events.md).
~~~

`/tables/events.md` is bundle-root-relative, `./events.md` is relative to the
file, and a directory link such as `tables/` resolves to `tables/index.md`.
Inline and reference-style links become graph edges. Links inside code
examples, images, HTML comments, and external URLs do not. Broken links,
unknown `type` values, and missing optional fields are legal per the spec, and
unknown frontmatter keys stay on `concept.frontmatter`.

## Loading knowledge into an agent

~~~python
from harnessx import Agent, AgentConfig

agent = Agent(
    config=AgentConfig(
        system_prompt="Answer from the knowledge bundle and cite concept paths and sources.",
    ),
    knowledge=[
        "./knowledge",                                                          # local folder
        "https://github.com/GoogleCloudPlatform/open-knowledge-format/tree/main/bundles/ga4",
    ],
)
~~~

Loading happens when the agent is constructed, like skill loading. The
`knowledge=` argument also accepts a `KnowledgeManager` or `GitSource` entries:

~~~python
from harnessx import GitSource, KnowledgeManager, OKFBundle

manager = KnowledgeManager.from_paths(
    ["./knowledge", GitSource("git@github.com:acme/kb.git", ref="v2", subdir="bundles/sales")],
    refresh=True,             # re-fetch cached clones
    max_body_chars=12_000,    # read_concept body budget (default 8,000)
    index_char_budget=4_000,  # root index characters per bundle in the prompt
)
agent = Agent(config=AgentConfig(system_prompt="..."), knowledge=manager)

bundle = OKFBundle.load("./knowledge")
concept = bundle.get("/metrics/active-users.md")
print(concept.type, concept.trust_tier, concept.is_stale(), bundle.warnings)
print(bundle.search("active users", limit=3, type="Metric", tags=["engagement"]))
print(bundle.neighbors("/metrics/active-users.md")["backlinks"])
~~~

A concept's **trust tier** records how it was verified, from `unverified`
through `machine-confirmed` to `human-reviewed`.

A bundle is named after its folder, its git subdirectory, or its repository.
Two bundles with the same name raise `ValueError`. Pass
`OKFBundle.load(root, name=...)` when they collide.

### Loading from git

Git URLs are shallow-cloned into `.agent_knowledge/` on first use and reused
until `refresh=True`. GitHub and GitLab `.../tree/<ref>/<subdir>` links select a
branch or tag and a subdirectory.

The `git` binary must be on the path. A missing binary or a failed clone raises
`RuntimeError` saying why, and a failed clone is cleaned up so the next load
can retry.

Subdirectories must resolve inside the repository. A symlink that escapes it
raises `ValueError`, and a missing subdirectory raises `FileNotFoundError`.
`OKFBundle.from_git(..., timeout=30)` bounds the wait and each git command.
Archives and other remote formats are not fetched; extract them and pass the
folder.

## What the model sees

The system prompt gains a `<knowledge>` block with the workflow and each
bundle's root `index.md`, truncated to `index_char_budget` characters. The
tools, all `ALLOW` by default:

| Tool | Arguments | Returns |
| --- | --- | --- |
| `search_concepts` | `query`, `limit`, `type`, `tags` | Ranked hits: path, type, title, description, status, score |
| `read_concept` | `path`, `max_chars` | Metadata header (type, status, trust tier, staleness, tags, sources) and the body; also reads indexes and logs |
| `get_neighbors` | `path` | Outbound links, backlinks, and unresolved links, with titles |

With several bundles loaded the tools take a `bundle` argument. Paths work
with or without `.md`, and a directory path reads its index. Lookups use an
in-memory table, never the disk, so `/../etc/passwd` returns "Concept not
found" rather than a file. A truncated body ends with a marker saying how much
was cut and how to ask for more.

## How search ranks

`search_concepts` ranks with BM25F, a keyword scoring function that weights
each field separately.

| Field | Weight |
| --- | ---: |
| Title | 3 |
| Tags | 2.5 |
| Description | 2 |
| Body | 0.5 |

Draft concepts are multiplied by 0.8 and
deprecated ones by 0.5; trust and staleness are shown when a concept is read
and do not change the score. Tokens are Unicode-normalized and case-folded. Common English singular and
plural forms match in both directions, with a 0.95 multiplier for the expanded
form. There is no general stemming, synonym
expansion, or embedding. `type` and `tags` filters are case-insensitive, all
requested tags must match, and `limit` is clamped to 1 to 50.

Scores are relative to the corpus searched. `manager.search()` uses one
combined index across every bundle; the tool searches one bundle, using that
bundle's own statistics. Override the synchronous `KnowledgeManager.search` to
plug in embeddings or hybrid retrieval. It receives the query, bundle, limit,
type, and tags, and must return hits with the same `path`, `type`, `title`,
`description`, `status`, `score`, and `bundle` keys.

## Trust, freshness, and computation contracts

Trust tiers follow the spec and come from valid `verified[].by` actors:

| Verification | Trust tier |
| --- | --- |
| No valid actor | `unverified` |
| Only `process:<id>` or `<producer>/<version>` actors | `machine-confirmed` |
| Any `human:<id>` actor | `human-reviewed` |

An actor needs a non-empty id with no whitespace. Malformed entries such as
`{}` or `{by: "human:"}` produce warnings and are left out of the normalized
metadata, while the raw values stay on `concept.frontmatter`. Trust is an
author-supplied claim; the harness does not authenticate identities.

A concept is stale on or after `stale_after`. `status` defaults to `stable`;
`draft` and `deprecated` are the other values. `read_concept` reports both in
its header, and the prompt asks the model to prefer stable, human-reviewed,
current concepts and to say when it relies on something else.

For `Attested Computation` concepts, `read_concept` also exposes the contract:
`runtime`, `parameters`, `computation`, the `executor` and its receipt fields,
the `attester`, and source credibility metadata. The harness retrieves the
contract; running the computation and attesting the result are the
application's job.

## Reacting to knowledge access

`KNOWLEDGE_ACCESSED` fires on every one of the three tool calls. Every payload
carries `tool`, `bundle` and `found`, plus:

| Tool | Also carries |
| --- | --- |
| `search_concepts` | `query`, `hits` |
| `read_concept` | `path`, `body_chars` |
| `get_neighbors` | `path`, `hits` (outbound plus backlinks) |

A missing bundle or concept still fires, with `found=False`.

~~~python
from harnessx import HookContext, HookEvent

async def on_knowledge(ctx: HookContext):
    print(ctx.data["tool"], ctx.data.get("query") or ctx.data.get("path"), ctx.data.get("hits"))

agent.hooks.on(HookEvent.KNOWLEDGE_ACCESSED, on_knowledge)
~~~

## Run the demo

Full demo: `examples/04-context/knowledge_bundles_okf.py`, with a sample bundle under
`examples/knowledge/`. Set `AGENT_KNOWLEDGE` to a folder or git URL to point
the demo at another bundle, or launch it from [harness-web](../web-demo/). See
[Examples](../examples/).