vortex_torch.cache.context

Classes

Context()

Mutable, single-instance context; populate later via .create(...).

class vortex_torch.cache.context.Context

Bases: ContextBase

Mutable, single-instance context; populate later via .create(…).

create(parent, model_runner, *, overwrite=False)

Populate this instance once (no locking). Set overwrite=True to allow re-init. NOTE: Without locking, concurrent callers may race; call from a single thread.

Parameters:

• parent (Any)

• model_runner (Any)

• overwrite (bool)

Return type:

Context

name: str

Human-readable context name.

mode: Literal['profile', 'execute']

Current operating mode.

max_new_tokens_per_batch

page_size

total_num_pages

head_dim

head_num