# Tokenizer

A field that collects several short values as Tokens: email addresses, tags, labels. Typing and pressing Enter or a comma adds a token; each token has a remove button, and Backspace in the empty field removes the last one. It has Input's box and grows a line at a time as the tokens wrap.

## Usage

### When to use

- Collect several short values of the same kind in one field: email addresses, tags.
- Values the user types freely, each shown as a removable chip.

### When not to use

- One value. Use Input.
- Picking from a known list. Use Combobox, or Checkbox for a short list.
- Showing values the user can't change. Use Token.

## Variants

### size

Least height of the box from size-element-*: sm 28px, md 32px (the default), lg 36px, xl 44px with a mouse; on touch md is 36px, lg 44px and xl 48px. It grows as tokens wrap. Match the Input or Button next to it: md in forms, sm only in dense mouse-first UI, lg or xl for the main field of a screen.

Values: `sm`, `md`, `lg`, `xl`. Default: `md`.

## Examples

### Invite by email

```tsx
<Field className="w-full max-w-md">
  <FieldLabel>Invite people</FieldLabel>
  <FieldControl>
    <Tokenizer defaultValue={["ada@example.com", "grace@example.com"]} placeholder="Add an email" />
  </FieldControl>
  <FieldDescription>Press Enter or a comma after each address.</FieldDescription>
</Field>
```

### With a cap

max stops adding at three tags.

```tsx
<Field className="w-full max-w-md">
  <FieldLabel>Tags</FieldLabel>
  <FieldControl>
    <Tokenizer size="sm" max={3} defaultValue={["design", "tokens"]} placeholder="Add a tag" />
  </FieldControl>
</Field>
```

## Do and don't

### Do

- Use it for several short values of the same kind: emails, tags.
- Say in the placeholder or a description which key adds a value.
- Check each value in onValueChange and explain a rejected one in a FieldError.

### Don't

- Don't use it to pick from a known list; use Combobox or Checkbox.
- Don't use it for one value; use Input.
- Don't put long text in a token.

## Props

| Name | Type | Values | Default | Description |
| --- | --- | --- | --- | --- |
| `defaultValue` | `string` |  |  | The values on first render, as an array: ["ada@example.com"]. |
| `value` | `string` |  |  | The values as an array, when you control them. Pair it with onValueChange. |
| `onValueChange` | `function` |  |  | Called with every value after one is added or removed. |
| `placeholder` | `string` |  |  | What to type, shown while there are no tokens: Add an email. Not a label. |
| `size` | `enum` | `sm`, `md`, `lg`, `xl` | `md` | Least height of the box from size-element-*: sm 28px, md 32px (the default), lg 36px, xl 44px with a mouse; on touch md is 36px, lg 44px and xl 48px. It grows as tokens wrap. Match the Input or Button next to it: md in forms, sm only in dense mouse-first UI, lg or xl for the main field of a screen. |
| `max` | `number` |  |  | The most values it takes. At the cap the field takes no more text, and Backspace still removes. |
| `disabled` | `boolean` |  | `false` | Can't be typed in, and the tokens lose their remove buttons. |
| `name` | `string` |  |  | Form name. Each value is submitted under it. |
| `removeLabel` | `function` |  |  | Names each remove button from its value: (value) => `Remove ${value}` by default. |

## Anatomy

| Part | Description |
| --- | --- |
| `root` | Tokenizer: the field box and what it holds. |
| `group` | The field box: Input's sunken fill, control border and corners, at least the size-element height, gap-xs between tokens, py-xs. It shows the text field's focus ring, disabled look and danger border. |
| `token` | A Token per value, with a remove button named with the value. Disabled tokens have no button. |
| `placeholder` | Placeholder text in the text field, only while there are no tokens. |

## Accessibility

Role: `textbox`

| Keys | Action |
| --- | --- |
| `Enter, comma` | Turns the typed text into a token. Enter with no text submits the form as usual. |
| `Backspace` | In the empty field, removes the last token. |
| `Tab` | Moves through the tokens' remove buttons and the field. |

- Name the field with a Field label, or aria-label: the tokens are read before it, in order.
- Each remove button is named Remove and the value; removeLabel changes the words. After a remove, focus goes back to the field.
- The box shows the focus ring only while the text field has focus; a remove button draws its own.
- A value already in the list, or one past max, is not added.
- With name, each value is submitted under it in a hidden input.

## Install

```bash
npx shadcn add https://werewolf-ui.vercel.app/r/tokenizer.json
```

This also adds `styles/werewolf.css`, the tokens every component uses, and any other component it needs. Import that file once in your root CSS, after `@import "tailwindcss"`, with a path relative to that CSS file (from `app/globals.css` it is `@import "../styles/werewolf.css"`).