# Token view
> Live token view: typed text splits into tinted spans and a strip of pixels where new tokens drop in, using a rough tokenizer.
- Element: `<px-tokenize>`
- React: `import { PxTokenize } from "@/components/lumesec/px-tokenize"`
- Collection: Pixel Lab (https://elements.lumesec.ai/components/pixel-lab)
- Registry item: https://elements.lumesec.ai/r/px-tokenize.json
- Page: https://elements.lumesec.ai/components/pixel-lab/tokenize



Live preview: https://elements.lumesec.ai/view/px-tokenize

```html
<script type="module" src="https://elements.lumesec.ai/cdn/px-tokenize.js"></script>

<px-tokenize></px-tokenize>
```

## Installation

```bash
npx shadcn@latest add @lumesec/px-tokenize
```

First time with the @lumesec registry? Register it once, or install by URL:

```bash
npx shadcn@latest registry add @lumesec=https://elements.lumesec.ai/r/{name}.json
npx shadcn@latest add https://elements.lumesec.ai/r/px-tokenize.json
```

## Usage

React:

```tsx
import { PxTokenize } from "@/components/lumesec/px-tokenize";

export function Example() {
  return (
    <PxTokenize />
  );
}
```

HTML:

```html
<script type="module" src="https://elements.lumesec.ai/cdn/px-tokenize.js"></script>

<px-tokenize></px-tokenize>
```

## Behaviour

Type and the text splits into tokens, shown as tinted spans and as a strip of pixels where new tokens drop in. The tokenizer is a rough approximation.

## API reference

### Events

Events bubble and cross the shadow boundary unless the description says otherwise.

| Event    | React prop | Detail               | Description                                                                                 |
| -------- | ---------- | -------------------- | ------------------------------------------------------------------------------------------- |
| `change` | `onChange` | `{ tokens: number }` | Fires on every edit with the token count. It does not fire when the element first connects. |

## Accessibility

* The textarea has `aria-label="Text to tokenize"`.
* The tinted token spans are `aria-hidden`; the pixel strip has `role="img"` with an `aria-label` giving the token count.
* The token, character and ratio summary is an `aria-live="polite"` region. Token highlighting on hover is pointer-only.
* Reduced motion: new tokens appear on the pixel strip in place instead of dropping in, and the summary text swaps without rolling.

## Theming

The element reads your shadcn theme tokens through its shadow root, so light and dark follow your theme. The accent comes from `--lumesec`. See [Theming](/docs/theming).

This component reads `--border`, `--card`, `--foreground`, `--lumesec` and `--muted-foreground`.

To restyle only LumeSec elements, set the matching `--ui-*` overrides: `--ui-accent`, `--ui-border`, `--ui-fg`, `--ui-muted` and `--ui-surface`.

## Notes

* The tokenizer is a regular-expression approximation for illustration: words, groups of up to three digits and single symbols, with words longer than six letters split into chunks and common suffixes. Counts will not match a real model tokenizer.
* Text comes only from the built-in textarea, which starts with a sample sentence. The strip's height grows with the number of tokens.