Documentation for Tabalyst 0.6.2 (beta, JSON format 0.2.0).

# Tabalyst documentation

> Documentation of Tabalyst 0.6.2, the open-source toolkit for reports, scans, samples and synthetic CSV generation.

Source: https://docs.tabalyst.com/ (Tabalyst 0.6.2)

Tabalyst is an open-source, local-first toolkit for understanding unfamiliar
structured data. Install it once and use its tools from the command line or
from Python:

- **[Tabalyst Report](https://docs.tabalyst.com/report.md)** turns a CSV, JSON, JSONL or Excel file into an interactive
  HTML report and a JSON profile.
- **[Tabalyst Scan](https://docs.tabalyst.com/scan.md)** describes every field of a file in one JSON document.
- **[Tabalyst Inspect](https://docs.tabalyst.com/inspect.md)** finds how to read a JSON, JSONL or Excel file.
- **[Tabalyst Sample](https://docs.tabalyst.com/sample.md)** samples CSV files to create smaller datasets.
- **[Tabalyst Generate](https://docs.tabalyst.com/generate.md)** creates synthetic CSV datasets.

```console
pip install --upgrade tabalyst
tabalyst report customers.csv
```

This creates `customers.html` (the report), `customers.json` (the profile) and
`executions.json` (the run history) beside the CSV file. Everything runs on your
computer; no data is sent anywhere.

Tabalyst is in beta and changes often: run the first command again before each
new test. Commands and the JSON formats may still change between versions. This
documentation describes the latest release published on PyPI.

## What do you want to do?

| Goal | Page |
| --- | --- |
| Install or update Tabalyst | [Install Tabalyst](https://docs.tabalyst.com/get-started/install.md) |
| Try it step by step | [Create your first report](https://docs.tabalyst.com/get-started/first-report.md) |
| See a finished report | [Examples and demos](https://docs.tabalyst.com/get-started/examples.md) |
| Report one or many CSV files | [Report CSV files](https://docs.tabalyst.com/report/csv.md) |
| Report a JSON or JSONL file | [Report JSON and JSONL files](https://docs.tabalyst.com/report/json.md) |
| Report an Excel workbook | [Report Excel workbooks](https://docs.tabalyst.com/report/excel.md) |
| Choose which array of a JSON file is analyzed | [Inspect JSON and JSONL files](https://docs.tabalyst.com/inspect/json.md) |
| Choose which sheet or table of a workbook is analyzed | [Inspect Excel workbooks](https://docs.tabalyst.com/inspect/excel.md) |
| Describe every field of a file as JSON | [Scan CSV and JSON files](https://docs.tabalyst.com/scan/files.md) |
| Cut a large CSV down | [Sample CSV files](https://docs.tabalyst.com/sample.md) |
| Create synthetic CSV data | [Tabalyst Generate](https://docs.tabalyst.com/generate.md) |
| Change delimiter, encoding or detection | [Configuration](https://docs.tabalyst.com/reference/configuration.md) |
| Find a command or an option | [Command line (CLI)](https://docs.tabalyst.com/reference/cli.md) |
| Use Tabalyst from Python | [Python API](https://docs.tabalyst.com/reference/python-api.md) |
| Read a generated JSON file in a script | [JSON profile](https://docs.tabalyst.com/report/json-profile.md), [scan format](https://docs.tabalyst.com/scan/format.md), [Inspect format](https://docs.tabalyst.com/inspect/format.md) |
| Know what is not supported yet | [Known limitations](https://docs.tabalyst.com/about/limitations.md) |

## Sections

- **Overview**: this page.
- **[Get started](https://docs.tabalyst.com/get-started.md)**: [install](https://docs.tabalyst.com/get-started/install.md),
  [first report](https://docs.tabalyst.com/get-started/first-report.md) and
  [examples and demos](https://docs.tabalyst.com/get-started/examples.md).
- **Tabalyst Report**: [overview](https://docs.tabalyst.com/report.md), [CSV](https://docs.tabalyst.com/report/csv.md),
  [JSON](https://docs.tabalyst.com/report/json.md) and [Excel](https://docs.tabalyst.com/report/excel.md) guides, the [JSON profile](https://docs.tabalyst.com/report/json-profile.md) and
  its [changelog](https://docs.tabalyst.com/report/profile-changelog.md).
- **Tabalyst Inspect**: [overview](https://docs.tabalyst.com/inspect.md), the
  [JSON](https://docs.tabalyst.com/inspect/json.md) and [Excel](https://docs.tabalyst.com/inspect/excel.md) guides, the [Inspect format](https://docs.tabalyst.com/inspect/format.md) and its
  [changelog](https://docs.tabalyst.com/inspect/format-changelog.md).
- **Tabalyst Scan**: [overview](https://docs.tabalyst.com/scan.md), the
  [guide](https://docs.tabalyst.com/scan/files.md), the [scan format](https://docs.tabalyst.com/scan/format.md), its
  [changelog](https://docs.tabalyst.com/scan/format-changelog.md) and
  [query caches](https://docs.tabalyst.com/scan/cache.md).
- **Tabalyst Sample**: [Sample CSV files](https://docs.tabalyst.com/sample.md).
- **Tabalyst Generate**: [overview and examples](https://docs.tabalyst.com/generate.md) and
  [CLI options](https://docs.tabalyst.com/reference/cli/generate.md).
- **[Reference](https://docs.tabalyst.com/reference.md)**: [command line](https://docs.tabalyst.com/reference/cli.md),
  [Python API](https://docs.tabalyst.com/reference/python-api.md),
  [configuration](https://docs.tabalyst.com/reference/configuration.md),
  [execution history](https://docs.tabalyst.com/reference/history.md) and the
  [glossary](https://docs.tabalyst.com/reference/glossary.md).
- **[About Tabalyst](https://docs.tabalyst.com/about.md)**: [concepts](https://docs.tabalyst.com/about/concepts.md),
  [known limitations](https://docs.tabalyst.com/about/limitations.md),
  [release notes](https://docs.tabalyst.com/about/releases.md) and
  [contributing and issues](https://docs.tabalyst.com/about/contributing.md).

## For AI assistants

To get help from an AI assistant, paste this prompt into a chat that can read
web pages:

```text
Read the complete Tabalyst documentation at
https://docs.tabalyst.com/llms-full.txt, then help me use Tabalyst well.
Ask me what my data looks like and what I want to learn from it, then suggest
the right commands and options. Base your answers only on that documentation.
```

If the assistant cannot open links, paste the content of the file instead.

This site publishes plain-text versions of its pages:

- [llms.txt](https://docs.tabalyst.com/llms.txt) lists every page with a
  one-line description.
- [llms-full.txt](https://docs.tabalyst.com/llms-full.txt) holds the whole
  documentation in one file.
- Every page is also available as Markdown by adding `.md` to its URL, for
  example [report.md](https://docs.tabalyst.com/report.md).

---

# Get started with Tabalyst

> Install Tabalyst, create a first report from a file of your own and browse finished examples, in three short steps.

Source: https://docs.tabalyst.com/get-started/ (Tabalyst 0.6.2)

Tabalyst turns a CSV, JSON, JSON Lines or Excel file into an interactive HTML
report and a JSON profile, on your computer. Three steps take you from nothing
to a report:

1. [Install Tabalyst](https://docs.tabalyst.com/get-started/install.md), on Windows, macOS or Linux, and
   [keep it up to date](https://docs.tabalyst.com/get-started/install.md#keep-tabalyst-up-to-date).
2. [Create your first report](https://docs.tabalyst.com/get-started/first-report.md) with a file of your own, step by
   step.
3. [Browse the examples and demos](https://docs.tabalyst.com/get-started/examples.md): finished reports for CSV,
   JSON, JSONL and Excel files, with the commands that made them.

Then pick the tool you need: [Tabalyst Report](https://docs.tabalyst.com/report.md) for reports,
[Tabalyst Inspect](https://docs.tabalyst.com/inspect.md) to choose what is analyzed in a JSON or
Excel file, [Tabalyst Scan](https://docs.tabalyst.com/scan.md) for the complete JSON description
of a file and [Tabalyst Sample](https://docs.tabalyst.com/sample.md) to cut a large CSV file
down. The [command line](https://docs.tabalyst.com/reference/cli.md) page lists every command.

---

# Install Tabalyst

> Install or update Tabalyst from PyPI on Windows, macOS or Linux, and keep it up to date before each test.

Source: https://docs.tabalyst.com/get-started/install/ (Tabalyst 0.6.2)

Tabalyst needs Python 3.11, 3.12, 3.13 or 3.14. The same command installs it
from PyPI, or updates it when it is already installed:

```console
pip install --upgrade tabalyst
tabalyst --version
```

The second command prints the installed version, for example `0.6.2`.

> **Update before each test.** Tabalyst is in beta and new versions are
> released often, with fixes and new options. Run
> `pip install --upgrade tabalyst` before testing, before following a tutorial
> and before reporting a problem. See
> [Keep Tabalyst up to date](#keep-tabalyst-up-to-date).

The sections below give the recommended steps for each system.

## Windows

1. Install Python 3.11 or later from [python.org](https://www.python.org/downloads/)
   or the Microsoft Store. Then open a new PowerShell window and check the
   version:

   ```powershell
   py --version
   ```

2. Create a virtual environment in your working folder. It keeps Tabalyst
   separate from other Python projects:

   ```powershell
   py -m venv .venv
   ```

3. Install Tabalyst into this environment. The same command updates it later:

   ```powershell
   .venv\Scripts\python.exe -m pip install --upgrade tabalyst
   ```

4. Check the installation:

   ```powershell
   .venv\Scripts\python.exe -m tabalyst --version
   ```

To type `tabalyst` directly, activate the environment first in each new
PowerShell window:

```powershell
.venv\Scripts\Activate.ps1
tabalyst --version
```

If PowerShell refuses to run `Activate.ps1` because script execution is
disabled, keep using `.venv\Scripts\python.exe -m tabalyst` instead: it accepts
exactly the same commands.

## macOS and Linux

```console
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade tabalyst
tabalyst --version
```

To install the `tabalyst` command once for your user account instead, use
[pipx](https://pipx.pypa.io/):

```console
pipx install tabalyst
```

With pipx, update with `pipx upgrade tabalyst`.

## Keep Tabalyst up to date

Tabalyst changes often during its beta. Update it before each test session,
before following this documentation and before reporting a problem: the
documentation describes the latest release, and older versions may lack
commands or options.

```console
python -m pip install --upgrade tabalyst
tabalyst --version
```

On Windows without an activated environment:

```powershell
.venv\Scripts\python.exe -m pip install --upgrade tabalyst
.venv\Scripts\python.exe -m tabalyst --version
```

Compare the printed version with the latest release on
[PyPI](https://pypi.org/project/tabalyst/). Updating keeps your reports and
configuration files: it only replaces the installed package.

## Troubleshooting

- **`tabalyst` is not recognized**: the environment is not activated, or the
  Python scripts folder is not on your `PATH`. Use `python -m tabalyst` (or
  `.venv\Scripts\python.exe -m tabalyst` on Windows): `python -m tabalyst`
  accepts the same commands as `tabalyst`.
- **`py` is not recognized on Windows**: Python is not installed, or was
  installed without the launcher. Reinstall it from python.org.
- **A command or option from this documentation is missing**: your version is
  older than the documented one. Update Tabalyst, then check
  `tabalyst --version`.
- **Unsupported Python version**: `pip` refuses to install Tabalyst on Python
  3.10 or earlier. Install a supported version and recreate the environment.

## Next step

[Create your first report](https://docs.tabalyst.com/get-started/first-report.md).

---

# Create your first report

> Tutorial that profiles a five-row CSV file with tabalyst report, opens the HTML report and reads the problems Tabalyst found.

Source: https://docs.tabalyst.com/get-started/first-report/ (Tabalyst 0.6.2)

In this tutorial you create a small CSV file, run `tabalyst report` on it, open
the HTML report and read the problems Tabalyst detects. It takes about five
minutes. You need Tabalyst installed and up to date: see
[Install Tabalyst](https://docs.tabalyst.com/get-started/install.md). If it is already installed, update it
first, since the output shown here comes from the latest release:

```console
pip install --upgrade tabalyst
```

## 1. Create a CSV file

In your working folder, create a file named `orders.csv` with this content:

```text
id,name,amount,joined,active,notes
001,Alice,12.50,2026-01-01,true,First order
002,Bob,,2026-02-01,false,
003,Charlie,25.00,2026-03-01,true,"Two items, one order"
003,Charlie,25.00,2026-03-01,true,"Two items, one order"
004,Dana,not available,2026-04-01,true,Pending review
```

This file contains deliberate problems: a repeated row, empty cells and a text
value in a numeric column.

## 2. Run the report

```console
tabalyst report orders.csv
```

Tabalyst prints a summary and the location of the report:

```text
Analyzed 5 rows and 6 columns.
Report: C:\work\orders.html
```

Three files now sit beside `orders.csv`:

```text
orders.csv
orders.html        the interactive report
orders.json        the JSON profile, for scripts and other tools
executions.json    the history of the runs in this folder
```

## 3. Open the report

Open `orders.html` in your web browser, by double-clicking it or from the
terminal:

```console
start orders.html      # Windows
open orders.html       # macOS
xdg-open orders.html   # Linux
```

The report works offline. Add `--details` to the command to create an
independent HTML page per column under `orders/`. The **Columns** table then
links to those pages. In each column page, **Overview** shows the stored values
with counts and all detected formats directly. The files contain values from
your data: share them accordingly.

## 4. Read what Tabalyst found

The report is organized in sections: **Dataset overview**, **Columns**,
**Transformations**, **Numeric analysis**, **Date analysis**,
**String analysis**, **Detectors and formats**, **Data sample** and
**Analysis settings**. A **Limits and diagnostics** section appears when the
scan stopped a measure at a limit or recorded a diagnostic.

In **Dataset overview**, the **Quality observations** card lists the problems in
`orders.csv`. Warnings come first; the two whitespace observations are
informational and are always listed, here with a count of 0.

| Problem | Where | Why |
| --- | --- | --- |
| 1 duplicate row | Record 4 | It repeats record 3 exactly |
| 2 missing cells | `amount` and `notes`, record 2 | The cells are empty |
| 1 column with mixed types | `amount` | Three numbers and the text `not available` |

Look at the columns too:

- `id` is detected as **text**, not integer: values such as `001` have leading
  zeros, so Tabalyst keeps them as text.
- `joined` is detected as a **date** in the `YYYY-MM-DD` format.
- `active` is detected as a **boolean** (`true` / `false`).

Types are descriptive hints: Tabalyst never modifies your CSV file.

## 5. Run it again

Run the same command a second time:

```console
tabalyst report orders.csv
```

Tabalyst refuses to replace the existing report:

```text
Error: Report output already exists. Use --force to replace it:
  orders.html
  orders.json
```

Add `--force` when replacing the report is intentional:

```console
tabalyst report orders.csv --force
```

`executions.json` is not replaced: each run adds an entry to it.

## What you learned

- `tabalyst report FILE.csv` creates an HTML report and a JSON profile beside
  the source file.
- The report shows detected types, missing values, duplicates and other
  problems, without changing your data.
- Existing reports are protected unless you pass `--force`.

## Next steps

- Choose the output name with `-o`, or process several files at once with
  `tabalyst report *.csv -d reports/`. Run `tabalyst report --help` for all
  options.
- Use the [JSON profile](https://docs.tabalyst.com/report/json-profile.md) in your own scripts.
- Adapt the analysis with a [configuration file](https://docs.tabalyst.com/reference/configuration.md).

---

# Examples and demos

> Synthetic example datasets and their generated reports, JSON profiles and column pages, to try Tabalyst before using your own data.

Source: https://docs.tabalyst.com/get-started/examples/ (Tabalyst 0.6.2)

The Tabalyst repository includes small, synthetic datasets with their generated
outputs. None of them contains real personal data. Use them to see what
Tabalyst produces before running it on your own files.

## Live demo

- [Example report](https://tabalyst.com/examples/report.html): the interactive
  HTML report of the insurance customers dataset.
- [Example JSON profile](https://tabalyst.com/examples/report.json): the
  profile that the report is rendered from.

## Datasets

| Input | Content | What it shows |
| --- | --- | --- |
| `basic.csv` | Five rows | A smoke test of the whole workflow |
| `insurance-customers.csv` | 3,000 fictional customers and contracts, 34 columns | A realistic CSV report, with `--details` column pages |
| `orders.json` | 60 customers with nested addresses, tags and orders | A JSON report on the one collection `$.customers[]` that Inspect finds |
| `web-events.jsonl` | 300 events with nested `user` and `device` objects, plus three bad lines | A partial JSONL report: the bad lines are excluded and counted |
| `sales.xlsx` | A workbook of 120 fictional orders under two title lines, a five-row `Regions` sheet and a note | An Excel report on the sheet `$.Orders` that Inspect selects over `$.Regions` |

Email addresses use reserved `.example` domains, and all people and identifiers
are fictional.

## Regenerate the outputs

Clone the repository at
[github.com/loribel-labs/tabalyst](https://github.com/loribel-labs/tabalyst),
install Tabalyst and run, from the repository root:

```console
tabalyst report examples/input/basic.csv -o examples/output/basic/report.html --config examples/config.json --force
tabalyst report examples/input/insurance-customers.csv -o examples/output/insurance-customers/report.html --config examples/config.json --details --force
tabalyst report examples/input/orders.json -o examples/output/orders/report.html --config examples/config.json --force
tabalyst report examples/input/web-events.jsonl -o examples/output/web-events/report.html --config examples/config.json --force
tabalyst report examples/input/sales.xlsx -o examples/output/sales/report.html --config examples/config.json --force
```

Each output folder holds an HTML report, a JSON profile and a cumulative
`executions.json`. The insurance example also has one HTML page per column under
`report/`.

## Next steps

- [Create your first report](https://docs.tabalyst.com/get-started/first-report.md) with a file of your own.
- [Report JSON and JSONL files](https://docs.tabalyst.com/report/json.md) to understand the
  `orders.json` and `web-events.jsonl` examples, and
  [Report Excel workbooks](https://docs.tabalyst.com/report/excel.md) for `sales.xlsx`.

---

# Tabalyst Report

> Overview of Tabalyst Report, the tool that turns a CSV, JSON, JSONL or Excel file into an interactive HTML report and a JSON profile, with commands, outputs and options.

Source: https://docs.tabalyst.com/report/ (Tabalyst 0.6.2)

**Tabalyst Report** analyzes a CSV, JSON, JSONL or Excel file and produces an
interactive, self-contained HTML report and a JSON profile of the same
analysis. It reads the file once, as a stream, on your computer.

```console
tabalyst report customers.csv
```

This creates `customers.html` (the report), `customers.json` (the profile) and
`executions.json` (the run history) beside the source. Your data is never
modified or sent anywhere.

## Commands at a glance

| Use case | Command | Destination |
| --- | --- | --- |
| One file, automatic name | `tabalyst report data.csv` | `data.html` beside the source |
| One file, custom name | `tabalyst report data.csv -o report.html` | The file `report.html` |
| Several files, automatic names | `tabalyst report *.csv` | Beside each source |
| Several files, one directory | `tabalyst report *.csv -d reports/` | The directory `reports/` |
| One JSON file | `tabalyst report data.json` | `data.report.html` beside the source |
| One JSONL file | `tabalyst report events.jsonl` | `events.report.html` beside the source |
| One Excel workbook | `tabalyst report sales.xlsx` | `sales.report.html` beside the source |
| One page per column | `tabalyst report data.csv --details` | The folder `data/` beside the report |
| From a scan document | `tabalyst report --scan data.scan.json` | `data.html` beside the scan |

## What the report shows

- Dataset dimensions, missing cells, duplicates and quality observations.
- Physical and semantic types, with confidence and error rates.
- Numeric, date and string-length analysis, normalization and value
  distributions.
- Distinct values, representative examples, date formats and detected meanings
  such as enumerations, email addresses, phone numbers and postal codes.
- CSV record widths, quoting, encoding and delimiter configuration.
- A bounded raw-data preview, while every record is analyzed.

The report is built on [Tabalyst Scan](https://docs.tabalyst.com/scan/files.md). Values of
sensitive fields, such as email addresses, are masked by default. Ambiguous
dates stay ambiguous: the report shows the evidence without applying it. The
HTML works without a network connection or a CDN.

## Pages

- [Read the report](https://docs.tabalyst.com/report/read-report.md): interpret every HTML section, its
  tables, indicators and limits.
- [Types and patterns](https://docs.tabalyst.com/reference/types-and-patterns.md): shared
  types and detectors, with a dedicated [pattern catalog](https://docs.tabalyst.com/reference/types-and-patterns/patterns.md).
- [Create your first report](https://docs.tabalyst.com/get-started/first-report.md): a five-minute
  tutorial on a small CSV file.
- [Report CSV files](https://docs.tabalyst.com/report/csv.md): names, batches, safe behavior and
  options.
- [Report JSON and JSONL files](https://docs.tabalyst.com/report/json.md): JSON and JSONL sources, one
  collection of records per report.
- [Report Excel workbooks](https://docs.tabalyst.com/report/excel.md): `.xlsx` and `.xlsm` sources, one table
  per report.
- [JSON profile](https://docs.tabalyst.com/report/json-profile.md): the structure of the generated `.json` file,
  and its [changelog](https://docs.tabalyst.com/report/profile-changelog.md).
- [Execution history](https://docs.tabalyst.com/reference/history.md): the `executions.json`
  file.
- [Configuration](https://docs.tabalyst.com/reference/configuration.md): adapt the analysis and the
  presentation.

---

# Read a Tabalyst Report

> Read each table and indicator in a Tabalyst Report, including its purpose, columns and limits.

Source: https://docs.tabalyst.com/report/read-report/ (Tabalyst 0.6.2)

**Tabalyst Report** analyzes one dataset and creates an interactive HTML report
and a [JSON profile](https://docs.tabalyst.com/report/json-profile.md). This guide follows the HTML sections in
display order. Use [Types and patterns](https://docs.tabalyst.com/reference/types-and-patterns.md) for
the complete list of inferred types, semantic types and detectors.

```console
tabalyst report customers.csv
```

A dataset is the rows of a CSV file, one selected JSON collection, the object
lines of a JSONL file, or one selected Excel sheet or named table. In JSON, a
column is usually a scalar field path such as `address.city`. Section counts
refer to this dataset. The optional numeric, date, string, detector and limits
sections appear only when they have results to show.

## Dataset overview

**Purpose:** establish the size and completeness of the dataset before
investigating individual fields.

| Indicator | What it measures | What to check |
| --- | --- | --- |
| Rows | Records analyzed, excluding a CSV or Excel header. The note counts empty rows. | Compare with the expected source size. Excluded malformed records appear as a separate quality observation. |
| Columns | Fields analyzed. The note counts cell positions: rows × columns for a table; field occurrences plus absent positions under parent objects for JSON. | Check whether the expected fields were found. |
| Missing cells | Missing positions divided by all analyzed cell positions, with the count below. Empty or whitespace-only values, configured markers and absent JSON members can count as missing. | Check the affected columns and the effective missing-value rules. |
| Duplicate rows | Records equal to an earlier record by exact raw values across columns, excluding the first occurrence. | `≥` is a lower bound after the tracking limit; `–` means the check was disabled. |

The **Quality observations** card lists issues with warning or information
severity; a signal such as a constant column can be intentional. **Column
types** counts columns by inferred type and semantic type, not individual
cells. The date card sums ambiguous values across date columns, so one record
can contribute more than once. See the [issue codes](https://docs.tabalyst.com/report/json-profile.md#issues).

## Columns

**Purpose:** locate fields that need investigation and compare their value
shape, detected meaning and completeness. Each row is one field. **With
issues** filters to fields with missing values, a `mixed` inferred type or
ambiguous dates.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Column | Name and source position. JSON paths identify nested fields; names in a table may be blank or repeated. | Use position or the JSON profile's stable `id` when a name is ambiguous. |
| Missing | Missing positions as a count and percentage of this field's positions. Absent JSON members count in the denominator. | Check whether the missing markers in **Analysis settings** match your data. |
| Distinct | Different present values after analytical normalization (Unicode composition, trim and whitespace collapse when enabled). | `limited` means counting stopped; it is not an estimate. |
| Inferred type | Value family inferred from present values, such as `number`, `date` or `mixed`. | A `mixed` field did not meet the configured confidence threshold. See [all inferred types](https://docs.tabalyst.com/reference/types-and-patterns/types.md#inferred-types). |
| Semantic type | One primary detected meaning, such as `email` or `postal_code`. | A dash means there is no unique qualifying interpretation. Other detectors may still have matched. See [semantic types](https://docs.tabalyst.com/reference/types-and-patterns/types.md#semantic-types). |
| Error | Count and percentage of present values outside the inferred type. | This is type disagreement, not malformed records or detector failures. It is unavailable for `mixed`. |
| Examples | A few retained values; the tooltip gives more values and counts. | A distribution can be complete or a bounded sample. Sensitive values can be masked or hidden. |

Missing values do not lower type confidence. For example, `not available` can
disagree with a numeric type if it was not configured as a missing marker. An
email field can have inferred type `text` and semantic type `email`: these
answer different questions. Add `--details` to link each column to its own
page with scan evidence and value counts:

```console
tabalyst report customers.csv --details
```

## JSON structure

**Purpose:** show the paths and containers behind the flattened columns. This
table appears for JSON and JSONL, including objects and arrays that are not
themselves scalar columns.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Path, Depth, Role | Location in each record, number of path segments and structural role. | Follow paths such as `orders[].total` back to their parent array. |
| Native types | JSON types encountered at the path, including containers and `null`. | Find fields that vary in shape between records. |
| Present, Absent | Occurrences and missing members relative to the path's parent. | Distinguish an absent field from an explicit JSON `null`. |
| Array length, Empty arrays | Array-size evidence when the path is an array. | Check whether empty or unusually sized arrays affect the data you expect. |

**Records**, **Paths** and **Maximum depth** summarize the structure above the
table. Field and depth limits appear under **Limits and diagnostics**.

## Transformations and variant groups

**Purpose:** reveal values written in several ways without changing the source
or the raw **Data sample**. Stages run in order: Unicode composition (`nfc`),
trim, internal whitespace collapse, case folding and accent removal.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Unicode composed, Trimmed, Whitespace collapsed | Number and percentage of occurrences changed by each enabled analytical stage. | Compare with the raw values when a stage changes many cells. |
| Case folded, Accents removed | Occurrences changed by the comparison stages. | These stages can group spellings without changing the analytical values. |
| Distinct | Distinct present values after the analytical stages. | Compare with **Compared distinct** to see the effect of comparison. |
| Compared distinct | Distinct keys after all enabled comparison stages. | A smaller count suggests case or accent variants. |
| Variant groups | Groups with at least two raw spellings that share a comparison key. | Open the groups table to inspect the actual spellings. |

In the **Variant groups** table, **Comparison key** is the value used for
comparison, **Occurrences** totals its appearances, **Spellings** counts raw
forms, and **Variants** lists bounded forms with counts. For example,
`Montréal`, `montreal` and `MONTREAL` can group together when the relevant
stages are enabled. A group is an analytical equivalence, not proof of equal
meaning. Counts or lists can be limited; the [normalization fields](https://docs.tabalyst.com/report/json-profile.md#normalization)
record their statuses.

## Numeric analysis

**Purpose:** summarize accepted numeric values in inferred `integer` or
`number` columns. The table appears when numeric statistics are available.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Range | Maximum minus minimum. | A large span may suggest an outlier or mixed units. |
| Minimum, Maximum | Smallest and largest accepted numbers. | Compare with expected bounds. |
| Mean, Median | Arithmetic mean and middle value of accepted numbers. | A blank median means its measure reached a scan limit. |
| Distinct, Examples | Normalized value count and retained value examples. | These describe value representation, not the calculation of numeric statistics. |

Configured number conventions can accept comma decimals such as `12,5`. Check
type errors in **Columns** before treating these statistics as representative
of every present value.

## Date analysis

**Purpose:** separate recognized date and time values, ambiguous readings,
invalid date-shaped values and other present values.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Status | Summary of the field's date results, such as valid, ambiguous, invalid, mixed or multiple formats. | Use the counts before assuming the whole field is a date. |
| Breakdown | Counts by valid format and by ambiguous, invalid or non-date category. | See which form creates uncertainty. |
| Valid | Unambiguous recognized date, date-time or time values. | Percentages use eligible present values, not all rows. |
| Ambiguous | Values such as `02/03/2026` that fit both DMY and MDY. | Other values in the column do not resolve their order automatically. |
| Invalid | Date-shaped values with impossible content, such as `2026-02-30`. | Check the source value and the configured rules. |
| Other | Present values that did not match as dates or were not tested. | See whether a missing marker or another format was expected. |
| Distinct, Formats | Normalized distinct values and recognized date formats. | Open the formats tooltip for counts; ambiguity evidence is in the column details. |

Only an explicit `scan.detectors.date.ambiguous_order` setting resolves an
ambiguous order. The report shows unambiguous DMY/MDY evidence but does not
apply it to ambiguous values.

## String analysis

**Purpose:** inspect the lengths of inferred `text` columns. Missing values
do not contribute to lengths.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Class | `very_short`, `short`, `medium`, `long` or `very_long`, based on the maximum observed length and configured thresholds. | This is a length class, not a semantic type. |
| Fixed | One length when minimum and maximum agree. | A fixed width alone does not prove a field is an identifier. |
| Min, Max, Mean, Median | Length statistics for present text. | These cells are blank for a fixed-length field because **Fixed** already conveys its length. |
| Lengths | Frequency of each length, when the maximum fits the configured distribution limit. | The tooltip includes bounded examples per length. |
| Distinct, Examples | Value count and retained examples. | These concern values, not the number of distinct lengths. |

## Detectors and formats

**Purpose:** inspect the evidence behind a possible semantic type. Each row
pairs one column with a detector that found evidence or failed. See
[Recognized patterns](https://docs.tabalyst.com/reference/types-and-patterns/patterns.md#built-in-detectors) for
the complete detector catalog.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Column, Detector | Field and detector ID. `primary` marks the interpretation shown as semantic type in **Columns**. | Several detectors may appear for one field. |
| Matched | Recognized values as a count and percentage of eligible values. | A match checks the detector's rules, not real-world existence. |
| Ambiguous, Invalid | Values with multiple readings or a recognized shape with invalid content. | These are separate from **Matched** and from column type errors. |
| Formats | Recognized forms with counts and percentages in the tooltip. | Formats describe syntax; their set varies by detector and configuration. |

A `failed` detector has no usable coverage for that field. Adaptive detection
can skip values after a warm-up; inspect tested and not-tested counts in the
[JSON profile](https://docs.tabalyst.com/report/json-profile.md#detectors) or a standalone column page before
treating a low match rate as full-field coverage.

## Limits and diagnostics

**Purpose:** make incomplete measures and technical scan events visible.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Field, Measure | Where and which measure stopped. A dataset-level measure has no field. | Check whether the affected measure is needed for your conclusion. |
| Reason, Limit | Why it stopped and the configured bound. | Adjust the relevant [scan limit](https://docs.tabalyst.com/reference/configuration.md#limits) if needed. |
| At least | A proven lower bound, when one is available. | Do not treat it as an estimate of the final count. |

The section also lists **Diagnostics** (code, level, count and bounded record
locations), **Untracked observations** beyond the field limit, and
**Truncated by depth** observations beyond the reader depth limit.

## Data sample

**Purpose:** inspect the first records in source order. The default is 10,
controlled by `scan.records.preview`. This preview does not limit the full
analysis. Sorting and filtering its HTML table only change the display.

| Report column | Meaning | What to check |
| --- | --- | --- |
| Row | Record number in the dataset, starting at 1. | A CSV or Excel header is excluded. |
| Each field | Raw source value under its column name and inferred type. | Sensitive values can be masked or hidden; `absent` in JSON differs from explicit `null`. |

## Analysis settings

**Purpose:** reproduce and interpret the report. This final section gives the
effective scope, missing markers, normalization, type and date rules, detector
thresholds, exposure policy, value representation, preview size and source
SHA-256. Consult it whenever a count or match differs from your expectation;
the [configuration reference](https://docs.tabalyst.com/reference/configuration.md) explains how to
change the settings.

---

# Report CSV files

> Create HTML reports and JSON profiles from one or many CSV files with tabalyst report, choose output names, use column pages and understand the batch safety rules.

Source: https://docs.tabalyst.com/report/csv/ (Tabalyst 0.6.2)

`tabalyst report` analyzes a CSV file and writes an HTML report and a JSON
profile. The simplest command keeps the source name:

```console
tabalyst report customers.csv
```

```text
customers.csv
customers.html
customers.json
executions.json
```

Use `-o` to choose a different HTML filename for one source. The profile takes
the same name with `.json`:

```console
tabalyst report customers.csv -o customer-analysis.html
```

## Report several files

```console
tabalyst report *.csv
```

Each report is created beside its source and keeps the source stem:
`customers.csv` gives `customers.html`, `orders.csv` gives `orders.html`.

Use `-d` to place every report in one directory:

```console
tabalyst report *.csv -d reports/
```

```text
reports/
├── customers.html
├── customers.json
├── orders.html
├── orders.json
└── executions.json
```

`-o` names one output file and accepts only one input. `-d` names an output
directory and accepts one or many inputs. `tabalyst report *.csv -o report.html`
is rejected before any file is processed. Quotes are only needed when a path
contains spaces. On Windows, Tabalyst expands wildcards itself.

## One page per column

```console
tabalyst report customers.csv --details
```

The **Columns** table of the report then links to one self-contained HTML page
per column, with its values, counts and detected formats. The pages are in a
subfolder named after the HTML file without `.html` (`customers/`, or `report/`
for `-o report.html`), named `col-01-` followed by a shortened slug of the
column name, in source order. Details are off by default. With `--force`,
`--no-details` removes the column pages generated earlier and keeps other files
of the folder.

## Safe batch behavior

Before processing begins, Tabalyst resolves every input and every planned HTML
and JSON output. It stops the whole batch when two outputs collide or when an
output already exists. With `--details`, each column page is checked after its
profile is built and before that report's files are written. Replace existing
reports only on purpose:

```console
tabalyst report *.csv -d reports/ --force
```

If one CSV is malformed, Tabalyst reports the error, continues with the other
files and returns a non-zero exit code at the end.

Interactive terminals show file and phase progress on standard error:

```text
[2/8] orders.csv - Analyzing
```

Progress is turned off outside a terminal, and by `--no-progress` or
`--quiet`.

## Useful options

```console
tabalyst report data.csv --delimiter ";"
tabalyst report data.csv --encoding cp1252
tabalyst report data.csv --config tabalyst.json
tabalyst report data.csv --verbose
tabalyst report data.csv --workers 4
```

Files of 16 MiB or more are analyzed by several worker processes with the same
result; `--workers 1` keeps one process. All options are in the
[command line reference](https://docs.tabalyst.com/reference/cli/report.md), and the
analysis settings in the [configuration reference](https://docs.tabalyst.com/reference/configuration.md).

## Reuse a scan

`tabalyst report` reuses the stored scan of a source when its content and the
requested scan settings are current, and creates or replaces it otherwise. To
build a report from a scan document without reading the source again:

```console
tabalyst report --scan customers.scan.json
```

Tabalyst refuses a scan whose source changed since it was written. See
[Scan CSV and JSON files](https://docs.tabalyst.com/scan/files.md).

---

# Report JSON and JSONL files

> Turn a JSON file into an interactive HTML report and a JSON profile with tabalyst report, one view per collection of records.

Source: https://docs.tabalyst.com/report/json/ (Tabalyst 0.6.2)

`tabalyst report` reads JSON files as well as CSV files:

```console
tabalyst report orders.json
```

This creates `orders.report.html` (the report), `orders.report.json` (the
profile) and `executions.json` beside `orders.json`. The `.report` part keeps
the profile from replacing the source. `-o` and `-d` work as for CSV files.

## JSONL files

Files ending in `.jsonl` or `.ndjson` hold one JSON object per line. They are
reported the same way, as one dataset `$[]` whose records are the lines:

```console
tabalyst report events.jsonl
```

This creates `events.report.html` and `events.report.json`. A line that is not
valid JSON, is not an object, holds an object with a duplicate key or is longer
than `scan.limits.max_line_bytes` is excluded and counted; the report lists it
in its *excluded records* issue, with the physical line numbers. The report
is then partial and the command prints a warning. Set `errors.policy` to
`strict`, in the `config` of an [Inspect file](https://docs.tabalyst.com/inspect/json.md#jsonl-and-ndjson-files)
or in `scan.errors.policy`, to stop at the first such line instead. Blank
lines are ignored, and a file without any record line is an error.

## Dataset

The report analyzes one **collection** of records, chosen by
[Tabalyst Inspect](https://docs.tabalyst.com/inspect/json.md): a top-level array, or an array inside
a top-level object, such as `customers` in `{"customers": [...]}`. It is the
dataset of the report, such as `$.customers[]`. When the source has no Inspect
file, Tabalyst inspects it by itself and uses the collection it selects. The
values outside the collection are not analyzed.

When several arrays are equally plausible, or the file has no collection of
objects, the report stops with exit code `2` before analyzing anything and lists
the candidates. Run `tabalyst inspect data.json` and set
`config.structure.dataset_path` in the file it writes, or pass the collection
with `--collection`, which outranks the Inspect file:

```console
tabalyst report data.json --collection products
tabalyst report data.json --collection '$.catalog.products[]'
```

The command that stops lists one ready-to-run command per collection. To report
all of them in one command, use `--all-collections`, which writes
`data.<collection>.html` and `data.<collection>.json` for each array (add `-d` to
choose the folder):

```console
tabalyst report data.json --all-collections -d reports
```

A collection is written as a path, `$.catalog.products[]`, or in the short
form without `$` and `[]`, `catalog.products`: the keys that lead to the array.
The root array of a file is `$[]`, or `.` in the short form. A key made of
digits, such as the category in `groups.121`, is accepted as it is; any other
key that is not a plain identifier is quoted, as in `["my key"]`.

A JSON file whose only content is a single object, a
number or an array of plain values has no collection.

To report on several collections of one file, pass `--collection` once for
each, or list them in the `scan.json.collections` setting of a
[configuration file](https://docs.tabalyst.com/reference/configuration.md) passed with `--config`. The
report then has one dataset per collection, and a **Dataset** selector in the
report navigation switches between them. Each dataset has its own overview,
columns, issues and data sample. Rules set in the Inspect file, such as the
flatten depth, apply to the source whichever way the collections are chosen.

## Columns

Every field holding strings, numbers, booleans or nulls is a column, named by
its path from the record:

| JSON record | Columns |
| --- | --- |
| `{"id": 1, "address": {"city": "Paris"}}` | `id`, `address.city` |
| `{"tags": ["a", "b"]}` | `tags[]` |
| `{"orders": [{"total": 3}]}` | `orders[].total` |

Objects and arrays themselves are not columns, unless the flatten depth
(`config.flatten.max_depth` of the Inspect file, or the scan setting
`json.flatten.max_depth`) keeps them whole: at that depth, an object or an array
is one column of type `complex` and its content is not analyzed. A field absent from a record is
missing, like a null or an empty string. In the data sample, an absent field
shows `absent`, and the values of a field under an array are joined with `, `.

## Structure

The **JSON structure** section lists every path of the records, objects and
arrays included, such as `orders`, `orders[]` and `orders[].total`, with:

- the JSON types found at the path;
- presence: the share of parent objects holding the field, and how many do
  not;
- for arrays, their lengths (minimum, maximum, mean) and how many are empty;
- the role of the path: a column, a container, or a collection analyzed as
  its own dataset.

CSV reports have no structure section: every row has the same columns.

## Detectors and formats

The **Detectors** section lists, for each column of CSV and JSON reports, what
the scan detectors recognized: emails, phone numbers, dates, numbers and the
other built-in detectors, with the share of values they matched and the
formats they found. The detector marked **primary** gives the column its
semantic type.

The structure of the profile is described in the
[JSON profile reference](https://docs.tabalyst.com/report/json-profile.md).

---

# Report Excel workbooks

> Turn an Excel workbook (.xlsx, .xlsm) into an interactive HTML report and a JSON profile with tabalyst report, one table per report, chosen automatically or with --collection.

Source: https://docs.tabalyst.com/report/excel/ (Tabalyst 0.6.2)

`tabalyst report` reads Excel workbooks as well as CSV and JSON files:

```console
tabalyst report sales.xlsx
```

This creates `sales.report.html` (the report), `sales.report.json` (the profile)
and `executions.json` beside `sales.xlsx`. The `.report` part keeps the profile
from replacing a source with the same stem. `-o` and `-d` work as for CSV files,
and `.xlsx` and `.xlsm` files are accepted. The workbook is never modified.

## Which table is analyzed

A report analyzes **one table**: a sheet, or a named Excel table inside a sheet.
When the workbook has one sheet with one table, the command needs nothing else.
Tabalyst finds the header of the sheet, even under a title or blank rows, and
reports the rows under it.

When several tables are plausible, the command stops with exit code `2` before
analyzing anything and lists them, as for JSON files. Choose one with
`--collection`, once, or set it in the Inspect file:

```console
tabalyst report shop.xlsx --collection Costs
tabalyst report shop.xlsx --collection '$.Sales.Orders'
```

`Costs` is a sheet, `$.Sales.Orders` the named table `Orders` of the sheet
`Sales`. A sheet name that is not a plain identifier is quoted:
`'$["Q1 2026"]'`. The command that stops lists one ready-to-run command per
table, so you only copy the one you want and run it again:

```text
Choose the table to analyze:
  tabalyst report shop.xlsx --collection Costs
  tabalyst report shop.xlsx --collection '$["Sales Q1"]'
Or report every table: tabalyst report shop.xlsx --all-collections
```

`tabalyst inspect shop.xlsx` prints the same lines. [Inspect Excel
workbooks](https://docs.tabalyst.com/inspect/excel.md) explains how the table is chosen and how to set
the header row when the detection is wrong.

## Every table of a workbook

To report all the tables in one command, add `--all-collections`:

```console
tabalyst report shop.xlsx --all-collections
tabalyst report shop.xlsx --all-collections -d reports
```

Each table gets its own pair of files, named `<file>.<collection-slug>`:
`shop.costs.html` and `shop.costs.json` for the sheet `Costs`, and
`shop.sales-q1.html` and `shop.sales-q1.json` for `Sales Q1` (lower case, without
accents, every other character a hyphen). A named table of a sheet gives
`shop.sales-orders.html`. Empty sheets, sheets without a header and hidden sheets
are left out, the last with a warning: report one with `--collection`.

`-d` chooses the folder and is optional for one workbook, which writes beside the
source; with several files it is required. `--all-collections` cannot be used
with `-o`, `--collection` or `--scan`, and existing files stop the command unless
you pass `--force`. See [tabalyst report](https://docs.tabalyst.com/reference/cli/report.md) for every
option.

## Columns

The columns of the report are the columns of the header, named by their header
cell. A repeated or empty header gets a distinct name (`id#1`, `id#2`) and an
`ambiguous_headers` issue. Every filled row under the header is a row of the
report; blank rows are skipped, and an empty cell is a missing value.

Cells keep their Excel type. Numbers are integers or decimals, booleans are
booleans, and dates are read as dates, so the report shows their formats and
ranges. Excel has one number type: a whole number is an integer, even in a column
of decimal amounts. A formula is read as its last calculated value. Merged cells
only hold their value in the top-left cell.

The report is built on [Tabalyst Scan](https://docs.tabalyst.com/scan/files.md), so the detectors, the
masking of sensitive values and every other analysis work as for CSV files. A
report of a workbook has no encoding or delimiter, and its profile `source.format`
is `excel`.

## Limits

One table per report, no recognition of blocks separated by blank rows or of
total rows under the data, and error cells read as empty. The
[known limitations](https://docs.tabalyst.com/about/limitations.md) list them with the size limits.

---

# JSON profile

> Structure of the JSON profile written by tabalyst report, with its top-level fields, column profiles, issue codes and format versioning.

Source: https://docs.tabalyst.com/report/json-profile/ (Tabalyst 0.6.2)

Each `tabalyst report` run writes a JSON profile next to the HTML report, with
the same name: `orders.html` comes with `orders.json`. For a JSON source, the
default names are `orders.report.html` and `orders.report.json`, so the profile
never replaces the source. The profile holds the
complete analysis result; the HTML report is rendered from it. Use it to read
Tabalyst results from scripts or other tools.

The format is **experimental**: it can change incompatibly between releases.
Always check `format_version` and `format_revision` first. This page describes
format `0.2.0`, revision `1`.

## Example

A shortened profile for a five-row `orders.csv`:

```json
{
  "format_version": "0.2.0",
  "format_revision": 1,
  "generated_at": "2026-10-06T12:00:00Z",
  "processing_seconds": 0.0098,
  "source": {
    "filename": "orders.csv",
    "format": "csv",
    "size_bytes": 274,
    "sha256": "7d426449...",
    "encoding": "utf-8-sig",
    "delimiter": ","
  },
  "config": { "...": "effective report and scan settings" },
  "datasets": [
    {
      "id": "rows",
      "kind": "table",
      "summary": {
        "row_count": 5,
        "column_count": 6,
        "missing_count": 2,
        "duplicate_row_count": 1,
        "with_issues_column_count": 2
      },
      "date_summary": { "...": "present when a column contains dates" },
      "columns": [
        {
          "id": "column_3",
          "name": "amount",
          "path": "amount",
          "position": 3,
          "inferred_type": "mixed",
          "type_counts": { "number": 3, "text": 1 },
          "type_confidence": 0.75,
          "missing_count": 1,
          "missing_percent": 20.0,
          "with_issues": true,
          "distinct_count": 3,
          "examples": ["25.00", "12.50", "not available"],
          "semantic_type": null,
          "detectors": [
            {
              "id": "number",
              "status": "complete",
              "primary": false,
              "eligible_count": 4,
              "matched_count": 3,
              "matched_percent": 75.0,
              "ambiguous_count": 0,
              "invalid_count": 0,
              "formats": []
            }
          ]
        }
      ],
      "issues": [
        {
          "code": "duplicate_rows",
          "severity": "warning",
          "message": "Duplicate rows beyond their first occurrence",
          "count": 1,
          "column_ids": [],
          "row_numbers": [4]
        }
      ],
      "preview": [
        {
          "row_number": 1,
          "values": [
            "001", "Alice", "12.50", "2026-01-01", "true", "First order"
          ],
          "absent": []
        }
      ],
      "limits": {
        "measures": [],
        "untracked_observations": 0,
        "depth_truncated_observations": 0,
        "diagnostics": []
      },
      "structure": null
    }
  ]
}
```

## Top-level fields

| Field | Type | Content |
| --- | --- | --- |
| `format_version` | string | Experimental format family, `"0.2.0"` |
| `format_revision` | integer | Revision within the family, increased for each structural or semantic change |
| `generated_at` | string | UTC date and time of the analysis (ISO 8601) |
| `processing_seconds` | number | Scan and profile time, in seconds; for a report built from a scan document with `--scan`, the scan duration recorded in the document plus the time to read it and build the profile |
| `source` | object | The analyzed file (see below) |
| `config` | object | The effective settings, after merging defaults, the configuration files and command options: `string_analysis`, `value_examples` and `scan`, the complete scan configuration; for a report built from a scan document, `scan` is the configuration recorded in it. See the [configuration](https://docs.tabalyst.com/reference/configuration.md) |
| `datasets` | array | One profile per dataset of the file (see below) |

## `source`

| Field | Content |
| --- | --- |
| `filename` | File name, without folder |
| `format` | `csv`, `json`, `jsonl` or `excel` |
| `size_bytes` | File size in bytes |
| `sha256` | SHA-256 hash of the file, to check that two profiles describe the same file |
| `encoding` | Text encoding used to read the file; `null` for an Excel workbook |
| `delimiter` | Field delimiter used to read the file; `null` for JSON, JSONL and Excel files |

## `datasets`

A CSV file has one dataset, `rows`, whose records are the rows of the file;
so has an Excel workbook, whose rows are those of the table analyzed. A
JSON file has one dataset per collection analyzed, such as `$.customers[]`,
usually one: the collection selected by [Inspect](https://docs.tabalyst.com/inspect/json.md)
or named with `--collection`. A JSONL file has one dataset, `$[]`. Reports
built by the commands have no `$` dataset for the rest of the document. See the
[scan format](https://docs.tabalyst.com/scan/format.md) for the datasets of a scan.

| Field | Type | Content |
| --- | --- | --- |
| `id` | string | Dataset identifier: `rows`, `$` or a collection path such as `$.customers[]` |
| `kind` | string | `table` (CSV), `document` or `collection` (JSON) |
| `summary` | object | Dataset-level counts (see below) |
| `date_summary` | object or `null` | Dataset-level date counts, when at least one column contains date values |
| `columns` | array | One profile per column, in file order for CSV and in order of discovery for JSON |
| `issues` | array | Detected problems (see below) |
| `preview` | array | The first records, with raw values; values of sensitive columns are masked or hidden |
| `limits` | object | Measures stopped by a scan limit, structural truncation and scan diagnostics (see below) |
| `structure` | object or `null` | Paths of a JSON dataset, containers included (see below); `null` for CSV files |

In a JSON dataset, a column is a field that holds strings, numbers, booleans
or nulls, named by its path: `email`, `address.city`, `orders[].total`.
Objects and arrays themselves are not columns, except the ones kept whole at
the `scan.json.flatten.max_depth` limit, which are columns of type `complex`.
A row is a record of the dataset.

## `summary`

Counts over the whole dataset:

- size: `row_count` (records), `column_count`, `cell_count` (the places a
  value could be: one per row and column for CSV; for JSON, the occurrences
  of each field plus the objects where it is absent);
- missing values: `missing_count`, `missing_percent`;
- normalization: `trim_count`, `collapse_internal_whitespace_count`;
- structure: `duplicate_row_count`, `empty_row_count`, `empty_column_count`,
  `constant_column_count`, `with_issues_column_count`;
- `duplicate_row_status`: `complete`; `limited` when more distinct rows than
  `scan.limits.max_tracked_records` were read, `duplicate_row_count` then
  being a lower bound; or `disabled` when `scan.records.duplicates` is
  `false`, `duplicate_row_count` then being `null`;
- column families: `numeric_column_count`, `date_column_count`,
  `string_column_count`;
- type distributions: `inferred_type_counts`, `inferred_type_percents`,
  `semantic_type_counts`, `semantic_type_percents`.

Percentages are numbers from 0 to 100.

## `columns`

Each column profile always contains:

| Field | Content |
| --- | --- |
| `id` | Stable identifier from the position, such as `column_3`. Use it rather than `name`, which can be blank or repeated |
| `name` | Header text for CSV; the field path for JSON |
| `path` | Field path as displayed by the scan: the header text for CSV, a path such as `orders[].total` for JSON |
| `position` | Position in the file for CSV, in order of discovery for JSON, starting at 1 |
| `inferred_type` | `empty`, `complex`, `boolean`, `integer`, `number`, `date`, `text` or `mixed`. A column of dates is `date` even with several formats or ambiguous values. `complex` is a JSON column whose present values are all objects or arrays kept whole at the flatten limit; `type_counts` then counts `object` and `array` |
| `type_counts` | Number of present values of each type |
| `type_confidence` | Share of present values accepted by the inferred type, from 0 to 1. For `mixed` columns, the share of the largest type family; `null` for `empty` columns |
| `type_error_count`, `type_error_percent` | Values outside the inferred type; `null` for `mixed` columns |
| `missing_count`, `missing_percent` | Missing cells; for JSON, also the objects where the field is absent |
| `with_issues` | `true` when the column has missing values, its type is `mixed` or it holds ambiguous dates |
| `normalization` | Values changed by each normalization stage and the variant groups (see below); missing cells are not counted |
| `distinct_count` | Number of distinct values after normalization, even for a masked column; `null` when a scan limit stopped the count |
| `examples` | A few representative values |
| `value_profile` | Value occurrences, complete or sampled (see `selection`) |
| `semantic_type` | `date` for date columns, otherwise the id of the scan's primary interpretation, such as `enumeration`, `email`, `phone` or `postal_code`, or `null` |
| `exposure` | `mask`, `hide` or `show` for a sensitive column, the way its values appear in `examples`, `value_profile` and `preview`; `null` otherwise |
| `detectors` | The detectors that recognized values of the column, in scan order (see below); empty when none did |
| `scan_details` | Bounded field evidence retained from the exposed Scan result: presence, native types, missing breakdown, first and last values, string characteristics and lengths, numeric statistics, boolean counts and temporal ranges. See below |

`value_profile.selection` is `complete` when all available frequencies are
listed. A primary `enumeration` keeps this complete list even above
`value_examples.full_distribution_max_distinct`, if the Scan frequency measure
is complete. Otherwise the profile uses a bounded diverse or random sample.

Depending on the column, these objects are also present (otherwise `null`):

| Field | Present for | Content |
| --- | --- | --- |
| `numeric` | `integer` and `number` columns with finite values | `minimum`, `maximum`, `range`, `mean`, `median` (`null` when a scan limit stopped it), over every number of the column, including decimal commas such as `12,5` |
| `date_profile` | Columns containing date values, whatever their type | Valid, ambiguous and invalid counts, detected formats and their breakdown, and `ambiguity_evidence`: the number of unambiguous values per day-month order (`DMY`, `MDY`), shown but never applied. `resolved_ambiguous_order` is set only by `scan.detectors.date.ambiguous_order` |
| `string_profile` | `text` columns | Length statistics, length distribution and representative examples |

## `normalization`

What the scan's normalization did to the present values of the column. Raw
values are never changed.

| Field | Content |
| --- | --- |
| `trim_count`, `trim_percent` | Values changed by trimming surrounding whitespace |
| `collapse_internal_whitespace_count`, `collapse_internal_whitespace_percent` | Values changed by collapsing repeated internal whitespace |
| `stages` | One item per stage, in order: `raw`, `nfc`, `trim`, `collapse_whitespace`, `casefold`, `strip_accents` (see below) |
| `variant_group_count` | Normalized values written in at least two raw ways; `null` unless `variant_group_status` is `complete` |
| `variant_group_status` | `complete`, `limited` when a scan limit stopped the count, or `not_applicable` without text values |
| `variant_groups` | The largest groups (`scan.limits.max_variant_groups`), each with its comparison `key`, its occurrence `count`, `distinct_count` (raw spellings), `variants` (`value` and `count`, at most `scan.limits.max_variants_per_group`) and `truncated`; masked for a sensitive column, empty when hidden |
| `variant_groups_truncated` | `true` when more groups exist than are listed |

Each item of `stages` has:

| Field | Content |
| --- | --- |
| `stage` | Stage name; `raw` describes the values as read |
| `enabled` | `false` when the stage is turned off in `scan.normalization` |
| `changed_count`, `changed_percent` | Values the stage changed, given the previous enabled stage; `null` for `raw` and disabled stages |
| `distinct_count` | Distinct values after the stage; `null` unless `distinct_status` is `complete` |
| `distinct_status` | `complete`, `limited`, `not_applicable` (no values) or `disabled` |

A variant group is an analytical equivalence, such as `Montréal`,
`montreal` and `MONTREAL`, not proof that the values mean the same thing.

## `detectors`

Each item describes what one scan detector recognized in the column. The
detectors that matched values or found ambiguous or invalid ones are listed,
as well as failed ones.

| Field | Content |
| --- | --- |
| `id` | Detector id, such as `email`, `date` or `pattern:order_id` |
| `status` | `complete`, or `failed` when the detector stopped with an error |
| `primary` | `true` for the scan's primary interpretation, shown as `semantic_type` |
| `eligible_count` | Values the detector could examine |
| `matched_count`, `matched_percent` | Values recognized, and their share of the eligible values |
| `ambiguous_count` | Values with more than one reading, such as `01/02/2026` |
| `invalid_count` | Values with the right shape but an impossible content, such as `2026-02-30` |
| `formats` | Formats found, each with `format`, `count` and `percent` of the eligible values |
| `coverage` | Complete Scan coverage, including tested, not tested and not matched counts; `null` for a failed detector |
| `evidence` | Bounded exposed examples for matched, ambiguous, invalid and not matched values; `null` for a failed detector |
| `details` | Detector-specific, exposure-controlled Scan metadata |
| `adaptive` | Adaptive detection skip and probe metadata, when applicable |

## `scan_details`

The standalone column pages use these Scan facts when present. `first` and
`last` retain the record number and scalar type; their values, detector
evidence and detector details use the same `mask`, `hide` or `show` policy as
the rest of the profile. `string_lengths`, `numeric`, `booleans` and `temporal`
are `null` when the corresponding Scan measure is unavailable. The complete
Scan document has other fields and statuses; `scan_details` is a bounded
selection, not a copy of the Scan document.

## `issues`

Each issue has a `code`, a `severity` (`warning` or `info`), an English
`message`, a `count`, the affected `column_ids` and up to 10 `row_numbers`.
Row numbers count the records of the dataset from 1; a CSV header is not
counted.

| `code` | Severity | Meaning |
| --- | --- | --- |
| `duplicate_rows` | warning | Rows repeating an earlier row, beyond its first occurrence |
| `missing_values` | warning | Missing cells |
| `empty_columns` | warning | Columns without any present value |
| `mixed_types` | warning | Columns with mixed value types |
| `ambiguous_dates` | warning | Date values matching more than one day-month order |
| `ambiguous_headers` | warning | Blank or repeated column names |
| `constant_columns` | info | Columns with a single distinct present value |
| `trimmed_cells` | info | Cells changed by trimming surrounding whitespace |
| `collapsed_whitespace` | info | Cells changed by collapsing repeated internal whitespace |
| `limited_measures` | info | Columns with measures stopped by a scan limit |
| `excluded_records` | warning | Records excluded by the `tolerant` error policy, not analyzed |
| `variant_groups` | info | Values written in several ways that normalization compares as equal |

An issue is listed only when its count is above zero, except `trimmed_cells`
and `collapsed_whitespace`, which are always listed.

## `preview`

The first records of the dataset (10 by default, set by `scan.records.preview`
in the [configuration](https://docs.tabalyst.com/reference/configuration.md)), each with its `row_number` and raw `values`
in column order. The preview size does not affect the analysis, which always
reads every record. In sensitive columns, values are masked (`mask`) or `null`
(`hide`), as given by the column's `exposure`; missing cells stay as read.

For JSON, a field absent from the record is `null` and its position (from 0)
is listed in `absent`. A field under an array, such as `tags[]`, joins the
values of the record with `, `. A JSON `null` is the text `null`.

## `limits`

What the scan could not measure completely in the dataset. A limited measure
is never estimated.

| Field | Content |
| --- | --- |
| `measures` | One item per limited measure: `column_id` and `path` of its field (both `null` for a measure of the dataset), `measure` (its place in the scan result, such as `values.cardinality`, `numeric.median` or `structure.paths`), `reason` (such as `distinct_limit`, `global_budget`, `value_too_long`, `field_limit` or `record_budget`), `limit` and `lower_bound`, a proven minimum or `null` |
| `untracked_observations` | Values under paths beyond `scan.limits.max_fields`, counted but not analyzed |
| `depth_truncated_observations` | Values deeper than `scan.limits.max_depth`, counted but not analyzed |
| `diagnostics` | The scan diagnostics of this dataset and of the whole scan: `code`, `level` (`error` or `warning`), `message`, `count`, `dataset` (`null` for the whole scan), `path`, `detector` and up to `scan.errors.max_locations` `locations` (`record`, and `line` for CSV or `element` for JSON) |

## `structure`

The shape of a JSON dataset; `null` for CSV files.

| Field | Content |
| --- | --- |
| `record_types` | Number of records of each native type, such as `{"object": 60}` |
| `path_count` | Number of paths; `null` when `path_status` is `limited` |
| `path_status` | `complete`, or `limited` when the dataset has more paths than `scan.limits.max_fields` |
| `max_depth_seen` | Depth of the deepest analyzed value |
| `fields` | One item per path, in order of discovery, containers included (see below) |

Each item of `fields` has:

| Field | Content |
| --- | --- |
| `id` | Scan field id |
| `path` | Path from the record, such as `orders[].total` |
| `depth` | Number of path segments: `orders[].total` has depth 3 |
| `parent` | Path of the parent field, `null` at the record root |
| `native_types` | Occurrences of each JSON type: `object`, `array`, `string`, `integer`, `number`, `boolean`, `null` |
| `occurrences` | Values found at the path |
| `parent_type` | `record`, `object`, or `array` for array items |
| `parent_count`, `present_count`, `absent_count` | Parents that could hold the field, those that do and those that do not; `absent_count` is `null` for array items |
| `present_percent` | Share of the parents holding the field; `null` for array items, which count elements |
| `collection` | For an array analyzed as its own dataset, the id of that dataset |
| `arrays` | For arrays: `count`, `empty_count`, `minimum_length`, `maximum_length`, `mean_length` and `item_count`; `null` otherwise |
| `column` | `true` when the field is also listed in `columns` |

## Versioning

- `format_version` names the experimental format family. The current value is
  `"0.2.0"`.
- `format_revision` increases for each structural or semantic change. It does
  not change for report styling or performance improvements.
- Revisions can be incompatible, and no migration is provided. See the
  [JSON profile changelog](https://docs.tabalyst.com/report/profile-changelog.md).

The profile contains values from the source file, in `examples`,
`value_profile` and `preview`; only sensitive columns are masked. Share it as
you would share the data.

---

# JSON profile changelog

> Current format version and revisions of the JSON profile written by tabalyst report.

Source: https://docs.tabalyst.com/report/profile-changelog/ (Tabalyst 0.6.2)

The JSON profile written by `tabalyst report` identifies its experimental
contract with two fields:

```json
{
  "format_version": "0.2.0",
  "format_revision": 1
}
```

`format_version` names the compatibility family. `format_revision` increases
for a meaningful structural or semantic change within that family. Styling,
documentation, performance improvements and corrections that preserve the
JSON contract do not change it. Beta revisions may be incompatible; automatic
migration is not provided.

## Version 0.2.0, revision 1 — 2026-10-06

This is the starting point for the current [JSON profile](https://docs.tabalyst.com/report/json-profile.md).
It describes the existing Report profile, including CSV, JSON, JSONL and Excel
sources, effective settings, dataset summaries, column analysis, normalization,
detectors, issues, record previews, limits and JSON structure.

---

# Tabalyst Inspect

> Overview of Tabalyst Inspect, the tool that finds which array of a JSON, JSONL or NDJSON file, or which sheet or table of an Excel workbook, holds its records and writes an editable Inspect file.

Source: https://docs.tabalyst.com/inspect/ (Tabalyst 0.6.2)

A JSON file can hold several arrays, and an Excel workbook several sheets.
**Tabalyst Inspect** reads a JSON, JSONL, NDJSON or Excel (`.xlsx`, `.xlsm`) file
once, finds which array, sheet or table holds the records and writes its answer
in `<source>-inspect.json` beside the source:

```console
tabalyst inspect orders.json
tabalyst inspect sales.xlsx
```

```text
Inspect: orders.json-inspect.json
Selection: $.customers[] (the only eligible collection)
```

The `config` section of that file holds the rules that `tabalyst scan` and
`tabalyst report` apply to the source: the collection, the flattening of nested
objects, the array mode and the error policy. Edit it, then scan or report as
usual. Inspect is optional: `scan` and `report` inspect a JSON file themselves
when it has no Inspect file, and stop with exit code `2` when several arrays are
equally plausible. Inspect reads JSON, JSONL and Excel files; CSV files are
refused.

## Pages

- [Inspect JSON and JSONL files](https://docs.tabalyst.com/inspect/json.md): detection, selection rule, editing
  the `config`, JSONL files and messages.
- [Inspect Excel workbooks](https://docs.tabalyst.com/inspect/excel.md): sheets and named tables, the header
  row, selection, editing the `config` and messages.
- [Inspect format](https://docs.tabalyst.com/inspect/format.md): the structure of the Inspect file, and its
  [changelog](https://docs.tabalyst.com/inspect/format-changelog.md).
- [Report JSON and JSONL files](https://docs.tabalyst.com/report/json.md) and [Scan CSV and JSON files](https://docs.tabalyst.com/scan/files.md):
  the tools that apply the Inspect file.

---

# Inspect JSON and JSONL files

> Let tabalyst inspect find the collection of records of a JSON, JSONL or NDJSON file, then edit the Inspect file to choose how scan and report read it.

Source: https://docs.tabalyst.com/inspect/json/ (Tabalyst 0.6.2)

Use `tabalyst inspect` to see how Tabalyst understands a JSON or JSONL file
before you scan it, and to change that understanding when it is wrong.
**Tabalyst Inspect** reads the file once, finds which array holds the records,
and writes the answer in a small JSON file beside the source:

```console
tabalyst inspect orders.json
```

```text
Inspect: orders.json-inspect.json
Selection: $.customers[] (the only eligible collection)
```

The Inspect file is named after the full source name plus `-inspect.json`:
`orders.json-inspect.json`, `events.jsonl-inspect.json`,
`events.ndjson-inspect.json`. Its last section, `config`, holds the rules that
`tabalyst scan` and `tabalyst report` apply to this source. The other sections
describe what Inspect found; Tabalyst replaces them each time it inspects. The
structure of the file is in the [Inspect format](https://docs.tabalyst.com/inspect/format.md).

`tabalyst inspect` accepts files ending in `.json`, `.jsonl` and `.ndjson`, or
non-recursive patterns, and, for workbooks, `.xlsx` and `.xlsm`
(see [Inspect Excel workbooks](https://docs.tabalyst.com/inspect/excel.md)). Nothing else is accepted: a CSV file
is refused with exit code `2`. It never modifies the source.

## Inspect is optional

`tabalyst scan` and `tabalyst report` work on a JSON file without it. When the
source has no Inspect file, they inspect it themselves, keep the result in
Tabalyst's local storage and use the collection it selects:

```console
tabalyst report orders.json
```

Run `tabalyst inspect` when you want to see or change the rules, or when
Inspect cannot choose.

## When Inspect cannot choose

A JSON file can hold several arrays. Inspect selects a collection only when
that choice is clear:

- the root of the file is an array;
- only one array holds objects; or
- one array holds at least 10 times more objects than the next one.

Otherwise nothing is selected, and `tabalyst scan` and `tabalyst report` stop
with exit code `2` before analyzing anything. For a file with a `customers` and
an `orders` array of the same size:

```text
Error [shop.json]: 2 collections of shop.json are equally plausible. Nothing was analyzed.
Candidates:
  $.customers[] (2 elements)
  $.orders[] (2 elements)
Pass --collection, or run `tabalyst inspect shop.json` and set config.structure.dataset_path in the file it writes.
Choose the collection to analyze:
  tabalyst report shop.json --collection customers
  tabalyst report shop.json --collection orders
Or report every collection: tabalyst report shop.json --all-collections
```

`report` and `scan` print one ready-to-run command per collection, and so does
`tabalyst inspect shop.json`. A short form such as `--collection orders` stands
for `$.orders[]`, and the commands use it whenever the name is plain.
`--all-collections` reports each collection, as `shop.customers.html` and
`shop.orders.html` with their `.json` profiles.

To make the choice stick for `report` too, open `shop.json-inspect.json` and
set the collection in `config`:

```json
"config": {
  "structure": {"dataset_path": "$.orders[]"},
  "flatten": {"enabled": true, "separator": ".", "max_depth": null},
  "arrays": {"mode": "preserve"},
  "errors": {"policy": "strict"}
}
```

`tabalyst report shop.json` then analyzes `$.orders[]`. Tabalyst never picks a
collection by the name of its property, such as `results` or `data`, and never
picks one to let the work go on.

To analyze a collection once without editing a file, pass it on the command
line; `--collection` outranks the Inspect file:

```console
tabalyst scan shop.json --collection "$.customers[]"
```

A file with no usable collection, such as a single object, a number or an
array of plain values, is also unresolved: the Inspect file says so in its
`warnings`, and Tabalyst does not turn a lone object into a one-row dataset.

## Change how the file is read

Everything you may edit is in `config`. Omit a key to keep the default:

| Key | Default | Effect |
| --- | --- | --- |
| `structure.dataset_path` | the selection, or `null` | The collection analyzed, such as `$.customers[]`. For JSONL it is always `$[]`. |
| `flatten.enabled` | `true` | `false` keeps nested objects whole as complex values. |
| `flatten.separator` | `.` | One character joining nested keys in field names, such as `address.city`. |
| `flatten.max_depth` | `null` | Depth, in keys and `[]`, from which an object is kept whole. `null` means no limit. |
| `arrays.mode` | `preserve` | The only supported value: arrays never add records. |
| `errors.policy` | `strict` for JSON, `tolerant` for JSONL | What to do with a malformed record. |

For example, `"max_depth": 2` makes `address` and `address.city` fields and
keeps deeper objects whole, so `tabalyst scan orders.json` lists 13 fields
instead of 23. A value that is not supported is an error that names the key:

```text
Error [orders.json]: Invalid Inspect file orders.json-inspect.json: config.arrays.mode: Value error, Array mode 'explode' is not supported in this version; only 'preserve' is
```

Unknown keys and values of the wrong type are errors too, never ignored. An
Inspect file that cannot be read stops `scan` and `report` with exit code `2`;
Tabalyst never replaces your file with an automatic choice. The
[configuration reference](https://docs.tabalyst.com/reference/configuration.md#scan-settings)
describes each setting.

## Inspect a file again

Run `tabalyst inspect` again after the source changes. The detection and
warnings are rewritten; your `config` is kept:

```text
Inspect: shop.json-inspect.json
Selection: none (several collections are equally plausible)
Configured collection: $.orders[] (kept from the existing file)
```

- `--reset-config` replaces `config` with the detected one.
- `--force` replaces a file that cannot be kept, such as one written for
  another version of the format or with an invalid `config`.
- `--config FILE` seeds a new file from the `scan` section of a
  [configuration file](https://docs.tabalyst.com/reference/configuration.md).

If the source changes and the configured collection disappears, `scan` and
`report` stop with exit code `2`, before writing anything, and use no other
collection:

```text
Error [shop.json]: The collection $.orders[] set in the Inspect file shop.json-inspect.json is not an array of shop.json, so nothing was analyzed. Set config.structure.dataset_path in that file to a collection that exists; `tabalyst inspect shop.json` lists the candidates.
```

A source that changed while the collection still exists keeps its `config`;
`scan` and `report` mention that the file was written for another version of
the source, and scan again: a stored scan is reused only when the source
content, the effective settings and the version of Tabalyst are the same.

## Which setting wins

From lowest to highest priority:

1. Tabalyst defaults;
2. the collection Inspect detected, when the source has no Inspect file;
3. the `scan` section of each `--config` file;
4. the `config` of the Inspect file beside the source;
5. `--collection`, `--delimiter` and `--encoding`.

## JSONL and NDJSON files

A `.jsonl` or `.ndjson` file has one dataset, `$[]`: its records are the lines.
Inspect counts the lines instead of choosing a collection:

```console
tabalyst inspect web-events.jsonl --verbose
```

```text
Inspect: web-events.jsonl-inspect.json
Selection: $[] (the lines of the file)
Format: jsonl
Candidates: 1
  $[] (300 elements)
Warning [web-events.jsonl]: 2 lines are not valid JSON or are too long.
Warning [web-events.jsonl]: 1 line is valid JSON but not an object.
```

Inspect never fails on a bad line: it counts them in `detection.lines` and lists
the first ones, by physical line number, in `warnings`. A scan then follows the
error policy of `config`:

- `tolerant`, the default for JSONL, excludes the bad lines, counts them and
  finishes with status `partial`. The command succeeds with a warning:
  `partial scan, 3 records excluded (invalid_line: 2, not_object: 1).`
- `strict` stops at the first bad line, with exit code `4`:
  `Record 121 (line 121) of web-events.jsonl is not valid JSON (...)`.

Blank lines are ignored. `scan` and `report` do not inspect a JSONL file by
themselves, since there is nothing to choose; an Inspect file only sets the
other rules, such as the policy.

## Messages and exit codes

`--verbose` lists every candidate collection and the informative warnings;
`--quiet` keeps only warnings and errors; `--no-progress` turns off progress.
Messages use standard error.

| Code | Meaning |
| --- | --- |
| `0` | The inspection worked, including when it selected nothing. |
| `2` | Configuration problem: unsupported extension, invalid Inspect file, unresolved collection or missing configured collection (in `scan` and `report`). |
| `4` | The source cannot be read: invalid JSON, invalid UTF-8 or an empty source. No file is written. |
| `1` | Another failure, such as an output that cannot be written. |

A pattern skips files named `*-inspect.json`, and an Inspect file given as an
input is refused: give the source it describes. A source whose own name ends in
`-inspect.json` must be renamed first. Wildcards are resolved by Tabalyst,
including on Windows, so `tabalyst inspect *.json` skips the Inspect files too.

## Python API

```python
import tabalyst

result = tabalyst.inspect("orders.json")
print(result.path)
print(result.document.detection.selection.path)

batch = tabalyst.generate_inspections(["data/*.json", "logs/*.jsonl"])
```

`tabalyst.inspect()` inspects one source as the command does, writes the
Inspect file and returns an `InspectResult`, with the `source`, the `path` of
the file and the `document` as written. It raises an error derived from
`tabalyst.TabalystError` when the inspection fails. `tabalyst.generate_inspections()`
inspects several sources and returns the plan, the successes and the failures.
Both accept `config_path`, `reset_config` and `force`.

---

# Inspect Excel workbooks

> Let tabalyst inspect find which sheet or table of an Excel workbook (.xlsx, .xlsm) holds the data, then edit the Inspect file to choose the table and the header row that scan and report use.

Source: https://docs.tabalyst.com/inspect/excel/ (Tabalyst 0.6.2)

A workbook can hold several sheets and several tables. Use `tabalyst inspect`
to see how Tabalyst understands an `.xlsx` or `.xlsm` workbook before you scan
it, and to choose the table when Tabalyst cannot. **Tabalyst Inspect** reads the
workbook once, lists its candidate tables, proposes one and writes the answer in
a small JSON file beside the source:

```console
tabalyst inspect sales.xlsx
```

```text
Inspect: sales.xlsx-inspect.json
Selection: $.Orders (much larger than $.Regions)
```

The Inspect file is named after the full source name plus `-inspect.json`:
`sales.xlsx-inspect.json`. Its last section, `config`, holds the rules that
`tabalyst scan` and `tabalyst report` apply to this workbook: the table, and
optionally its header row. The other sections describe what Inspect found;
Tabalyst replaces them each time it inspects. The structure of the file is in
the [Inspect format](https://docs.tabalyst.com/inspect/format.md#inspect-files-of-workbooks-kind-excel).

Tabalyst reads `.xlsx` and `.xlsm` files (macros are ignored). It never modifies
the workbook. Older `.xls` files, `.xlsb` and `.ods` files are refused with exit
code `2`: save them as `.xlsx` first.

## One table: nothing to choose

A workbook with one sheet holding one table needs no Inspect file. `tabalyst
scan` and `tabalyst report` inspect it themselves, keep the result in Tabalyst's
local storage and read the table it selects:

```console
tabalyst report sales.xlsx
```

Run `tabalyst inspect` when you want to see or change the rules, or when
Inspect cannot choose.

## What is a candidate table

A candidate is a **sheet**, or a **named table** (an Excel table, also called a
list object) inside a sheet. Candidates are written as paths with the workbook as
root:

| Path | Meaning |
| --- | --- |
| `$.Sales` | The sheet `Sales`. |
| `$.Sales.Orders` | The named table `Orders` of the sheet `Sales`. |
| `$["Q1 2026"]` | A sheet whose name is not a plain identifier is quoted. |

A sheet that holds named tables is not a candidate itself: its tables are. The
author declared them, and a table guessed beside them would duplicate or
contradict them. Inspect lists such a sheet as not eligible (`has_tables`).

For a sheet, Inspect looks for the **header row** in its first 50 rows: the first
row that fills at least half the width of the sheet, holds only text, repeats few
names and is followed by data. Title lines and blank rows above the header are
skipped, and the table starts at the first and ends at the last filled column of
the header. The data rows are the filled rows under it; blank rows are not
records. A named table gives its range and header exactly, with no guessing.

| `ineligible_reason` | A candidate is not eligible when |
| --- | --- |
| `no_cells` | The sheet is empty, or is a chart sheet. |
| `no_header` | No row looks like a header followed by data: a note, a block of numbers, a header row alone. |
| `no_data_rows` | A named table has no data row. |
| `has_tables` | The sheet holds named tables, which are listed on their own. |

## When Inspect cannot choose

Inspect selects a table only when that choice is clear, with the rule that it
uses for [JSON files](https://docs.tabalyst.com/inspect/json.md#when-inspect-cannot-choose): only one table is
eligible, or the largest one has at least 10 times more data rows than the next
one. A hidden sheet is selected only when no visible table is eligible. Sheet
names play no part. Otherwise nothing is selected, and `scan` and `report` stop
with exit code `2` before analyzing anything:

```text
Error [shop.xlsx]: 2 tables of shop.xlsx are equally plausible. Nothing was analyzed.
Candidates:
  $.Sales (8 rows)
  $.Costs (5 rows)
Pass --collection with a sheet or table path such as '$.Sheet', or run `tabalyst inspect shop.xlsx` and set config.structure.dataset_path in the file it writes.
Choose the table to analyze:
  tabalyst report shop.xlsx --collection Sales
  tabalyst report shop.xlsx --collection Costs
Or report every table: tabalyst report shop.xlsx --all-collections
```

`report` and `scan` print one ready-to-run command per table, and so does
`tabalyst inspect shop.xlsx`: copy the one you want and run it. The commands use
the short form, the name alone (`--collection Costs`, or `--collection
Sales.Orders` for a named table), whenever the name is plain. A sheet name that
is not a plain identifier needs the quoted path (`'$["Q1 2026"]'`), which the
commands print as it must be typed. A workbook is read one table at a time: pass
`--collection` once, or use `tabalyst report --all-collections` to report every
table, one pair of files per table (see [Report Excel
workbooks](https://docs.tabalyst.com/report/excel.md#every-table-of-a-workbook)). To make the choice
stick, open `shop.xlsx-inspect.json` and set it in `config`:

```json
"config": {
  "structure": {"dataset_path": "$.Costs", "header_row": null}
}
```

`tabalyst report shop.xlsx` then analyzes `$.Costs`. `--collection` outranks the
Inspect file, and a notice says so when they differ.

## Change how the table is read

Everything you may edit is in `config`. Omit a key to keep the default:

| Key | Default | Effect |
| --- | --- | --- |
| `structure.dataset_path` | the selection, or `null` | The table analyzed: `$.Sheet` or `$.Sheet.Table`. |
| `structure.header_row` | `null` | The 1-based row of the sheet that holds the header, instead of the detected one. Use it when the detection picks the wrong row, or finds none. |

`header_row` applies to a sheet; a named table has its own header. The row must
hold at least one value. A sheet with no recognizable header makes `scan` and
`report` stop with exit code `4` and say to set it. Unknown keys and values of
the wrong type are errors that name the key, never ignored, and `flatten`,
`arrays` and `errors` do not exist for a workbook.

## Types, dates and empty cells

Cells keep their Excel type: a number is a number, a boolean a boolean, a text a
string. Excel has a single number type, so a whole number such as `21` is read as
an integer, even in a column of amounts such as `10.5`. Dates and times are read
as ISO text (`2026-01-31`), which the date detector recognizes, so a date column
is typed `date`. A formula is read as its last calculated value. An empty cell
is a missing value, like an empty CSV cell.

## Warnings

Inspect warns about what a scan will have to live with:

| Code | When |
| --- | --- |
| `blocks_not_split` | Blank rows lie among the data. Blocks separated by blank rows are not split into several tables; the blank rows are skipped. |
| `duplicate_headers`, `blank_headers` | The header repeats a name or leaves a cell empty. The columns keep their position, and the report names them `id#1`, `id#2`. |
| `merged_cells` | Merged ranges lie in the data: only the top-left cell holds a value. |
| `multi_level_header` | Header cells are merged, as in a header on two levels. Only the last row is read as the header. |

Merged cells above the header, such as a title, are ignored. The informative
`candidate_not_eligible` entries explain why a sheet was left out.

## Inspect a workbook again

Run `tabalyst inspect` again after the workbook changes. The detection and
warnings are rewritten; your `config` is kept. `--reset-config` replaces it by
the detected one, and `--force` replaces a file that cannot be kept, such as one
written for another version of the format, one with an invalid `config`, or an
Inspect file written for a JSON source. `scan` and `report` refuse such a file of
another kind, with exit code `2`, instead of reading it with the wrong rules. If
the configured sheet or table is no longer in the workbook, the new file warns
(`configured_path_not_found`), and `scan` and `report` stop with exit code `2`
and list the sheets that exist.

## Which setting wins

A new Inspect file starts from the layers below it: `scan.excel.header_row` of a
`--config` file is written into its `config`, so the file does not undo what
the configuration asked for.

From lowest to highest priority:

1. Tabalyst defaults;
2. the table Inspect detected, when the workbook has no Inspect file;
3. the `scan.excel` section of each `--config` file (`dataset_path`,
   `header_row`);
4. the `config` of the Inspect file beside the workbook;
5. `--collection`.

## Limits

- A pattern such as `*.xlsx` skips the lock files that Excel keeps beside an open
  workbook (`~$book.xlsx`), as it skips Inspect files.

- One table per run. A workbook is not scanned sheet by sheet, and relations
  between sheets are not analyzed.
- Blocks separated by blank rows, and total or note rows under the data, are
  not recognized: a total row counts as a record.
- A real error cell (`#DIV/0!`) and a formula that was never calculated read as
  empty cells, and cannot be counted.
- Calamine reads a sheet into memory, about 0.9 byte per byte of its XML. A
  workbook with a sheet whose XML exceeds 1 GiB is refused. Reading a sheet
  of a million rows of eight columns takes about 4 seconds and 360 MB.
- Password-protected workbooks are refused with exit code `4`.

## Messages and exit codes

`--verbose` lists every candidate and the informative warnings; `--quiet` keeps
only warnings and errors. Messages use standard error.

| Code | Meaning |
| --- | --- |
| `0` | The inspection worked, including when it selected nothing. |
| `2` | Configuration problem: unsupported spreadsheet format, invalid Inspect file, unresolved table, or a configured sheet that is gone (in `scan` and `report`). |
| `4` | The workbook cannot be read: corrupt, password-protected, empty, too large, or no header in the sheet. No file is written. |
| `1` | Another failure, such as an output that cannot be written. |

## Python API

```python
import tabalyst

result = tabalyst.inspect("sales.xlsx")
print(result.path)
print(result.document.detection.selection.path)
```

`tabalyst.inspect()` and `tabalyst.generate_inspections()` accept workbooks as
they accept JSON files, with `config_path`, `reset_config` and `force`.
`tabalyst.scan()` reads the table named by `ScanConfig(excel={"dataset_path":
"$.Sales"})`; without it, a workbook is an error that says to choose.

---

# Inspect format

> Structure of the Inspect file written by tabalyst inspect, with its zones, detection, warnings, the config you may edit, and how Tabalyst reads it.

Source: https://docs.tabalyst.com/inspect/format/ (Tabalyst 0.6.2)

`tabalyst inspect data.json` writes `data.json-inspect.json` beside the source:
a JSON document that says how Tabalyst understands the source and holds the
rules to read it. This page describes format `tabalyst.inspect`, version
`0.2.0`, revision `1`, kinds `json` and `excel`. The format is **experimental**: it can
change incompatibly between releases. Changes are listed in the
[Inspect format changelog](https://docs.tabalyst.com/inspect/format-changelog.md). For the commands, see
[Inspect JSON and JSONL files](https://docs.tabalyst.com/inspect/json.md) and [Inspect Excel workbooks](https://docs.tabalyst.com/inspect/excel.md).
The kind `json` is described first; [the kind `excel`](#inspect-files-of-workbooks-kind-excel)
follows the same top level with its own `detection` and `config`.

## Top level

```json
{
  "format": "tabalyst.inspect",
  "format_version": "0.2.0",
  "format_revision": 1,
  "inspect": {
    "kind": "json",
    "tabalyst_version": "0.6.2",
    "generated_at": "2026-10-06T12:00:00Z",
    "note": "Edit only the \"config\" section. ..."
  },
  "source": {
    "name": "orders.json",
    "format": "json",
    "size_bytes": 60720,
    "sha256": "ece53017..."
  },
  "detection": {"...": "see below"},
  "warnings": [],
  "config": {
    "structure": {"dataset_path": "$.customers[]"},
    "flatten": {"enabled": true, "separator": ".", "max_depth": null},
    "arrays": {"mode": "preserve"},
    "errors": {"policy": "strict"}
  }
}
```

The sections always come in this order, with `config` last. The file is UTF-8
with two-space indentation. It holds no absolute path: `source.name` is a file
name, and the other paths point inside the source, such as `$.customers[]`.

| Section | Written by | You edit it | Effect on scans |
| --- | --- | --- | --- |
| `format`, `format_version`, `format_revision` | Tabalyst | No | A version Tabalyst does not read is refused. |
| `inspect` | Tabalyst | No | None. |
| `source` | Tabalyst | No | None: it describes the source when it was inspected. |
| `detection` | Tabalyst | No | None. |
| `warnings` | Tabalyst | No | None. |
| `config` | Tabalyst once, then you | Yes | The rules `tabalyst scan` and `tabalyst report` apply. |

For the same source, the same version of Tabalyst and the same settings, two
inspections write the same file except `inspect.generated_at`.

## `inspect` and `source`

| Key | Meaning |
| --- | --- |
| `inspect.kind` | The kind of Inspect: `json` for JSON, JSONL and NDJSON sources, `excel` for workbooks. |
| `inspect.tabalyst_version` | The version of Tabalyst that wrote the file. |
| `inspect.generated_at` | UTC time of the inspection, the only value that varies. |
| `inspect.note` | A reminder to edit only `config`. |
| `source.name` | File name of the source, with its extension. |
| `source.format` | `json`, `jsonl` for `.jsonl` and `.ndjson` files, or `excel` for `.xlsx` and `.xlsm` files. |
| `source.size_bytes` | Bytes read. |
| `source.sha256` | SHA-256 of every byte read. |

`source.sha256` records the source at inspection time. It is never used to
decide whether a stored scan can be reused.

## `detection`

| Key | Meaning |
| --- | --- |
| `scope.structure` | Always `complete`: candidates, element counts and element types come from reading the whole source. |
| `scope.detail` | Always `bounded`: fields, depths and nesting come from the first records of each candidate. |
| `scope.limits` | The bounds applied: `records` (first records observed per candidate, 1,000) and `fields` (distinct fields tracked per candidate, 1,000). |
| `scope.discovery_max_depth` | How many keys deep arrays were searched, from `scan.json.discovery_max_depth` (default 3). |
| `scope.candidates` | `truncated` when more than 100 arrays were found. Absent otherwise. |
| `root.type` | Type of the root: `object`, `array`, `string`, `number`, `boolean`, `null`, or `lines` for JSONL. |
| `candidates` | The candidate collections, in document order. |
| `selection` | The collection proposed, or none. |
| `lines` | JSONL only. Exact counts of lines: `read` (non-blank), `blank`, `objects`, `invalid`, `not_object`. `invalid` includes lines longer than `scan.limits.max_line_bytes`. |

A **candidate collection** is the root array, or an array reachable from the
root through object keys only, at most `scan.json.discovery_max_depth` keys
deep. Arrays inside the records of a candidate are not candidates: they stay
fields of the records. A JSONL file has one candidate, `$[]`.

| Key of a candidate | Meaning |
| --- | --- |
| `path` | Absolute path, such as `$.customers[]`. |
| `elements` | Exact number of elements; for JSONL, the number of object lines. |
| `element_types` | Exact number of elements per type, among `object`, `array`, `string`, `number`, `boolean`, `null`. |
| `eligible` | `true` when the candidate has at least one element and every element is an object. |
| `ineligible_reason` | Only when not eligible: `empty` or `non_object_elements`. |
| `observation` | Detail from the first records: `records` observed, distinct `fields`, `max_depth` (keys and `[]`), `nested_objects`, `arrays`, and `complete`, `false` when a bound cut the observation. |

A field that first appears after the observed records is absent from
`detection`; the scan finds it. Observations are never exhaustive when
`complete` is `false`.

### Selection

`selection.path` is the collection proposed, or `null`. `selection.basis` says
why:

| `basis` | Meaning |
| --- | --- |
| `root_array` | The root is an array of objects. |
| `jsonl_records` | The source is JSONL and has at least one object line. |
| `only_eligible_candidate` | One candidate is eligible. |
| `dominant_candidate` | Several are eligible and the largest has at least 10 times the elements of the next one, which is given in `selection.over`. |
| `ambiguous` | Several are eligible and none stands out. Nothing is selected. |
| `no_eligible_candidate` | No candidate is eligible, or the root is not an object or an array. Nothing is selected. |
| `candidates_truncated` | More than 100 arrays exist, so the list cannot prove that one stands out. Nothing is selected. |

The names of properties play no part. A tie never selects. When nothing is
selected, `config.structure.dataset_path` is `null` and `tabalyst scan` and
`tabalyst report` stop until you set it.

## `warnings`

A list of entries with `code`, `level` (`warning` or `info`) and `message`,
plus the keys that apply: `path`, `reason`, `count`, and `locations`, a list of
`{"record": n, "line": n}` of at most `scan.errors.max_locations` entries (the
`count` is always complete). Warnings never change how a scan reads the source.

| Code | Level | When | Keys |
| --- | --- | --- | --- |
| `ambiguous_collections` | warning | `basis` is `ambiguous` | `count` of eligible candidates |
| `no_collection` | warning | `basis` is `no_eligible_candidate` | `reason`: `scalar_root`, `no_array` or `no_eligible_array` |
| `candidates_truncated` | warning | The candidate list was cut at 100 | `count` kept |
| `candidate_not_eligible` | info | One per ineligible candidate, for the first 10 only | `path`, `reason` (`empty`, `non_object_elements`) |
| `candidate_not_eligible_truncated` | info | More than 10 candidates are ineligible | `count` of ineligible candidates |
| `invalid_lines` | warning | JSONL lines that are not valid JSON, or too long | `count`, `locations` |
| `non_object_lines` | warning | JSONL lines that are valid JSON but not objects | `count`, `locations` |
| `configured_path_not_found` | warning | After a new inspection, the kept `dataset_path` is not an array of the source | `path` |
| `source_name_mismatch` | info | `source.name` of the existing file differs from the source inspected | `path`: the recorded name |

`no_collection` reasons: `scalar_root` (the root is not an object or an array),
`no_array` (an object with no array reachable through keys) and
`no_eligible_array` (arrays exist and none is eligible, such as an array of
plain values). `configured_path_not_found` is only raised for a path the
candidates can judge: one reachable through keys alone within the discovery
depth. A deeper path, or one that crosses an array, is checked by the scan.

## `config`

The part you edit. Omit a key to keep the default of the layer below
(see [Which setting wins](https://docs.tabalyst.com/inspect/json.md#which-setting-wins)).

| Key | Type | Default when omitted | Written by Inspect |
| --- | --- | --- | --- |
| `structure.dataset_path` | string or `null` | `null`: not decided | The selection, or `null` |
| `flatten.enabled` | boolean | `true` | The effective value |
| `flatten.separator` | string | `"."` | The effective value |
| `flatten.max_depth` | integer or `null` | `null`: no limit | The effective value |
| `arrays.mode` | string | `"preserve"` | `"preserve"` |
| `errors.policy` | `"strict"`, `"tolerant"` or `null` | `null`: the default of the format | The resolved value, never `null` |

Rules:

- `structure.dataset_path` is an absolute path that starts with `$` and ends
  with `[]`, such as `$.customers[]` or `$["a.b"][]`. For a JSONL source it must
  be `$[]` or `null`. Equal paths in different spellings are the same path.
  Writing `null` does not select the automatic discovery of `tabalyst.scan()`.
- `flatten.separator` is exactly one character that is not a letter, a digit,
  `_`, whitespace, a control character or one of `[ ] " \ $`. It changes the
  names of fields, never the paths of collections, which always use `.`. A key
  that holds the separator is written `["a/b"]`, so a key `a.b` and the nested
  keys `a` then `b` stay two distinct fields.
- `flatten.max_depth` is `null` or an integer from 1 to 1,000. Depth counts
  keys and `[]`: `address` is 1, `address.city` is 2, `orders[].amount` is 3. An
  object or array at that depth is kept whole as a complex value. `enabled:
  false` is a depth of 1.
- `arrays.mode` accepts `preserve` only. `ignore` and `explode` are refused.
- `errors.policy` is `strict` for JSON and `tolerant` for JSONL when it is
  `null`.

A `config` is kept as you wrote it when Inspect runs again: the same keys and
values, nothing added. Every setting that influences the dataset, the fields or
the records analyzed takes part in the identity of a scan.

## Reading an Inspect file

For the file beside a source, Tabalyst validates the top-level `format`,
`format_version` and `format_revision`, `inspect.kind` and `config`; the other
sections are informative, so a damaged `detection` never stops a scan. It
refuses, with exit code `2` and the name of the file:

| Case | Message names |
| --- | --- |
| Not a JSON object, or another `format` | The file |
| A `format_version` or `format_revision` it does not read | Both versions, and the two ways out: edit the file or `tabalyst inspect --force` |
| An unknown or missing `inspect.kind` | The kinds it reads |
| A missing `config`, an unknown key at any depth, a value of the wrong type (a number or boolean written as a string included) or a value outside the supported set | The key, such as `config.flatten.sepator` |

There is no migration during the beta, and Tabalyst never replaces an invalid
file with an automatic choice.

## Stored copy

When a JSON source or a workbook has no Inspect file and nothing on the command line or in a
`--config` file names its collection, `tabalyst scan` and `tabalyst report`
inspect it and keep a copy of the result in Tabalyst's local storage, next to
the stored scan. The copy is rebuilt whenever the source content, the format
version, the version of Tabalyst or the discovery depth (JSON) or header search (Excel)
differ. It is
disposable and is never read when an Inspect file exists beside the source.

## Inspect files of workbooks (kind `excel`)

`tabalyst inspect sales.xlsx` writes `sales.xlsx-inspect.json` with the same
top level, the same zones and the same reading rules as above. Only
`inspect.kind` (`excel`), `source.format` (`excel`), `detection` and `config`
differ.

```json
{
  "format": "tabalyst.inspect",
  "format_version": "0.2.0",
  "format_revision": 1,
  "inspect": {
    "kind": "excel",
    "tabalyst_version": "0.6.2",
    "generated_at": "2026-10-06T12:00:00Z",
    "note": "..."
  },
  "source": {
    "name": "sales.xlsx",
    "format": "excel",
    "size_bytes": 13344,
    "sha256": "35657227..."
  },
  "detection": {
    "scope": {"structure": "complete", "header_scan_rows": 50},
    "workbook": {"sheets": 3, "tables": 0},
    "candidates": [
      {
        "path": "$.Orders", "kind": "sheet", "sheet": "Orders",
        "visible": true,
        "range": "A4:I124", "header_row": 4, "elements": 120, "eligible": true,
        "observation": {
          "columns": 9, "column_names": ["order_id", "..."],
          "blank_rows": 0, "merged_ranges": 0
        }
      },
      {"path": "$.Notes", "kind": "sheet", "sheet": "Notes", "visible": true,
       "elements": 0, "eligible": false, "ineligible_reason": "no_header"}
    ],
    "selection": {
      "path": "$.Orders", "basis": "dominant_candidate", "over": "$.Regions"
    }
  },
  "warnings": [],
  "config": {"structure": {"dataset_path": "$.Orders", "header_row": null}}
}
```

### `detection` of a workbook

| Key | Meaning |
| --- | --- |
| `scope.structure` | Always `complete`: every sheet and table is read. |
| `scope.header_scan_rows` | Rows searched for the header of a sheet, from its first filled row (50). |
| `scope.candidates` | `truncated` when more than 100 candidates were found. Absent otherwise. |
| `workbook` | The number of `sheets` and of named `tables`. |
| `candidates` | The sheets and named tables, in workbook order; the tables of a sheet follow it. |
| `selection` | The table proposed, or none. |

| Key of a candidate | Meaning |
| --- | --- |
| `path` | `$.Sheet` for a sheet, `$.Sheet.Table` for a named table; a name that is not a plain identifier is quoted, `$["Q1 2026"]`. |
| `kind` | `sheet` or `table`. |
| `sheet`, `table` | The names; `table` only for a named table. |
| `visible` | `false` for a hidden sheet, and for the tables on it. |
| `range` | A1 range of the header and the data, such as `A4:I124`. Absent without a table. |
| `header_row` | The 1-based sheet row of the header. Absent without a table. |
| `elements` | Filled data rows; `0` for a candidate with none. |
| `eligible` | `true` when the candidate has a header and at least one data row. |
| `ineligible_reason` | Only when not eligible: `no_cells`, `no_header`, `no_data_rows` or `has_tables`. |
| `observation` | `columns`, `column_names` (the first 100), `blank_rows` among the data and `merged_ranges` in the table. Absent without a table. |

A sheet that holds named tables is not eligible (`has_tables`): its tables
are. `selection.basis` takes the values of the JSON kind that make sense for a
table: `only_eligible_candidate`, `dominant_candidate` (the largest eligible
table has at least 10 times the data rows of the next one, given in
`selection.over`), `ambiguous`, `no_eligible_candidate` and
`candidates_truncated`. Hidden sheets compete only when no visible table is
eligible. Sheet names play no part, and a tie never selects.

### Warnings of a workbook

The entries have the keys of the JSON kind. `ambiguous_collections`,
`no_collection` (`reason` `no_eligible_table`), `candidates_truncated`,
`candidate_not_eligible`, `candidate_not_eligible_truncated`,
`configured_path_not_found` and `source_name_mismatch` are raised as for
JSON. Five codes are specific to workbooks, all at level `warning`, with the
`path` of the table and a `count`:

| Code | When |
| --- | --- |
| `blocks_not_split` | Blank rows lie among the data of a sheet; `count` of blank rows. |
| `duplicate_headers` | The header repeats names; `count` of repeated names. |
| `blank_headers` | The header leaves cells empty; `count` of empty cells. |
| `merged_cells` | Merged ranges lie in the data; `count` of ranges. |
| `multi_level_header` | Merged cells lie on the header row; `count` of ranges. |

### `config` of a workbook

| Key | Type | Default when omitted | Written by Inspect |
| --- | --- | --- | --- |
| `structure.dataset_path` | string or `null` | `null`: not decided | The selection, or `null` |
| `structure.header_row` | integer or `null` | `null`: the detected row | `null` |

- `structure.dataset_path` is an absolute path with the workbook as root and one
  or two keys: `$.Sales` for a sheet, `$.Sales.Orders` for a named table of that
  sheet. Equal paths in different spellings are the same path
  (`$["Sales"]["Orders"]` is `$.Sales.Orders`). A path that ends with `[]`,
  has three keys or does not start with `$` is refused.
- `structure.header_row` is a 1-based row number (an integer from 1; a string, a
  boolean or `0` is refused) that replaces the detected header row of a sheet.
  It does not apply to a named table. A new file takes it from `scan.excel.header_row`
  of a `--config` file, and writes `null` when there is none.
- `flatten`, `arrays` and `errors` do not exist for a workbook, and an unknown key
  is an error that names it, as in the JSON kind.

`tabalyst scan` and `tabalyst report` apply `config` as the settings
`excel.dataset_path` and `excel.header_row` of the
[scan configuration](https://docs.tabalyst.com/reference/configuration.md#scan-settings). An Inspect
file written for another kind than the source, such as a JSON Inspect file
beside a workbook, is not replaced silently: `tabalyst inspect --force` replaces
it.

---

# Inspect format changelog

> Current format version and revisions of the JSON file written by tabalyst inspect.

Source: https://docs.tabalyst.com/inspect/format-changelog/ (Tabalyst 0.6.2)

An Inspect file written by `tabalyst inspect` identifies its experimental
contract with three fields:

```json
{
  "format": "tabalyst.inspect",
  "format_version": "0.2.0",
  "format_revision": 1
}
```

`format` identifies the document kind. `format_version` names the compatibility
family, and `format_revision` increases for meaningful structural or semantic
changes within it. [Scan](https://docs.tabalyst.com/scan/format-changelog.md) and the
[JSON profile](https://docs.tabalyst.com/report/profile-changelog.md) have their own versions and
changelogs. Beta revisions may be incompatible; automatic migration is not
provided. Tabalyst refuses a file of another version, and
`tabalyst inspect --force` writes a new one.

## Version 0.2.0, revision 1 — 2026-10-06

This is the starting point for the current [Inspect format](https://docs.tabalyst.com/inspect/format.md). It
describes the `json` and `excel` kinds, their source identity, detection,
warnings and editable configuration.

---

# Tabalyst Scan

> Overview of Tabalyst Scan, the analysis engine that describes every field of a CSV, JSON, JSONL or Excel file in one JSON document, with commands and pages.

Source: https://docs.tabalyst.com/scan/ (Tabalyst 0.6.2)

**Tabalyst Scan** is the analysis engine of Tabalyst. It reads a CSV, JSON,
JSONL or Excel file once and describes every field in one JSON document:
presence, native types, missing values, frequencies, exact statistics,
normalization variants, technical type and the result of every detector.

```console
tabalyst scan customers.csv
tabalyst scan orders.json --collection "$.customers[]"
tabalyst scan events.jsonl
tabalyst scan sales.xlsx
```

Without `-o` or `-d`, the scan is stored for reuse in Tabalyst's local storage.
`-o` or `-d` writes a standalone `<stem>.scan.json` document instead.
[Tabalyst Report](https://docs.tabalyst.com/report.md) is built on it, and reuses a stored scan
when the source has not changed.

## What it detects

Numbers with decimal commas, dates, booleans, enumerations, email addresses,
URLs, phone numbers, postal codes, currency amounts, percentages, quantities,
UUIDs and IP addresses. Values of sensitive fields are masked by default.
Memory depends on configurable limits, not on the number of records. Results
are written atomically: an interrupted scan never leaves a partial file.

## Pages

- [Types and patterns](https://docs.tabalyst.com/reference/types-and-patterns.md): native,
  inferred and semantic types, plus every recognized pattern.
- [Scan CSV and JSON files](https://docs.tabalyst.com/scan/files.md): commands, JSON, JSONL and Excel
  sources, reuse, settings and the Python API.
- [Scan format](https://docs.tabalyst.com/scan/format.md): the structure of the scan document, and its
  [changelog](https://docs.tabalyst.com/scan/format-changelog.md).
- [Manage query caches](https://docs.tabalyst.com/scan/cache.md): `tabalyst cache info` and
  `tabalyst cache clean`.
- [Configuration](https://docs.tabalyst.com/reference/configuration.md): the `scan` settings.
- [Tabalyst Inspect](https://docs.tabalyst.com/inspect/json.md): how JSON sources are
  read before a scan, and [for workbooks](https://docs.tabalyst.com/inspect/excel.md).

---

# Scan CSV and JSON files

> Describe every field of a CSV, JSON, JSONL or Excel file in a complete JSON scan document with tabalyst scan, in one streaming pass.

Source: https://docs.tabalyst.com/scan/files/ (Tabalyst 0.6.2)

Use `tabalyst scan` to describe a CSV, JSON, JSONL or Excel file in a JSON document: every
field, its presence, values, statistics, formats and detected meanings, such
as email addresses, dates or amounts.

```console
tabalyst scan customers.csv
```

Without `-o` or `-d`, whatever the format, this stores a reusable `scan.json` in
Tabalyst's local storage directory, under `workspaces/local/scans/` in a
directory keyed by the source path. It does not build a DuckDB database. Set `TABALYST_HOME` to choose the
storage root. The source file is never modified. **Tabalyst Scan** reads the whole file in one streaming pass,
so memory depends on the configured limits, not on the number of records. The
structure of the result is described in the
[scan format reference](https://docs.tabalyst.com/scan/format.md).

## Scan JSON files

Files ending in `.json` are read as JSON, files ending in `.jsonl` or `.ndjson`
(in any letter case) as JSONL, files ending in `.xlsx` or `.xlsm` as Excel
workbooks, and every other file as CSV.

```console
tabalyst scan orders.json
```

A scan analyzes one **collection** of records: a top-level array, or an array
inside a top-level object, such as `customers` in `{"customers": [...]}`.
Arrays inside records, such as the `orders` of each customer, stay fields of
their records. Tabalyst finds the collection with
[Inspect](https://docs.tabalyst.com/inspect/json.md), which runs by itself when the source has no
Inspect file, and keeps the choice in the Inspect file beside the source when
you ran `tabalyst inspect`. When several arrays are equally plausible, the scan
stops with exit code `2` and lists them; choose one in the Inspect file, or
pass `--collection`:

```console
tabalyst scan orders.json --collection "$.customers[]"
```

Repeat `--collection` to analyze several collections in one scan, one dataset
each:

```console
tabalyst scan orders.json --collection customers --collection products
```

A collection path starts with `$`, the document root, and ends with `[]`, the
elements of an array. The short form lists the keys that lead to the array:
`customers` is `$.customers[]` and `data.items` is `$.data.items[]`; `.` is the
root array, `$[]`, and a key made of digits, such as `groups.121`, is accepted
as it is. Nested fields are written with dots, such as
`orders[].amount`. `--collection` outranks the Inspect file. A collection set in
an Inspect file that no longer exists in the source stops the scan with exit
code `2`; `--collection` or a `--config` collection that is not found is a
warning, with an empty dataset.

A JSONL source has one dataset, `$[]`: its records are the lines. An invalid
line, or one that is not an object, is excluded and counted under the default
`tolerant` policy, and the scan is `partial`.

## Scan Excel workbooks

```console
tabalyst scan sales.xlsx
tabalyst scan shop.xlsx --collection Costs
```

A scan analyzes **one table** of a workbook: a sheet, or a named Excel table. The
table is chosen as for JSON collections, with [Inspect](https://docs.tabalyst.com/inspect/excel.md),
which runs by itself when the source has no Inspect file, or with
`--collection`, once (`Costs`, `'$.Costs'` or `'$.Sales.Orders'`). When several
tables are equally plausible, the scan stops with exit code `2` and lists them.
The header row is detected, and `excel.header_row` of the
[configuration](https://docs.tabalyst.com/reference/configuration.md#scan-settings) or the Inspect file
sets it. The scan has one dataset, `rows`, of kind `table`, whose fields are the
columns of the header. Cells keep their Excel type, and dates are read as ISO
text that the date detector recognizes. The [scan format](https://docs.tabalyst.com/scan/format.md#excel-sources)
records the table read.

## Choose output locations

Use `-o` to export a standalone scan document with a complete output filename
when scanning one source. The name
must end in `.json`:

```console
tabalyst scan customers.csv -o scans/customers.json
```

Wildcards are resolved by Tabalyst, including on shells that do not expand
them. Use `-d` to export the scans of one or more sources in a directory:

```console
tabalyst scan data/*.csv data/*.json -d scans/
```

The outputs are named `customers.scan.json`, `orders.scan.json`, and so on.
`-o` accepts only one resolved input; `-o` and `-d` cannot be combined.
Recursive `**` patterns are not supported.

## Safe batch behavior

Before writing standalone scans, Tabalyst resolves every input and output and
rejects the whole batch when:

- two sources map to the same output, such as `data.csv` and `data.json`;
- an output would replace an input or a configuration file;
- an output already exists and `--force` is not given.

Each result is written to a temporary file in the target directory, then moved
into place: an interrupted scan never leaves a partial result. When one source
fails, for example with invalid JSON, Tabalyst reports the error, continues
with the other sources and returns a non-zero exit code at the end.

A pattern such as `data/*.json` also matches earlier standalone results such as
`data/orders.scan.json`. Write scans to another directory with `-d` to keep
them apart from the sources. Patterns skip Inspect files, named
`*-inspect.json`, and an Inspect file given as an input is refused. Wildcards are resolved by
Tabalyst, including on Windows, so the same pattern skips them there too.

## Configure the scan

Scan settings live in the `scan` section of a JSON
[configuration file](https://docs.tabalyst.com/reference/configuration.md#scan-settings):

```json
{
  "scan": {
    "values": {"null_markers": ["N/A", "NULL"]},
    "errors": {"policy": "tolerant"},
    "exposure": {"sensitive_values": "hide"}
  }
}
```

```console
tabalyst scan customers.csv --config tabalyst.json
```

Repeat `--config` to combine several files: later files override earlier ones.
`--delimiter`, `--encoding` and `--collection` override every file. Unknown
settings are errors, so a misspelled name never passes silently.

The CSV delimiter is a comma and the encoding accepts UTF-8 with or without a
byte order mark by default:

```console
tabalyst scan data.csv --delimiter ";" --encoding cp1252
```

## Errors and partial scans

By default, a CSV record with the wrong number of fields, or a JSON object with
a duplicate key, stops the scan of that file with an error. A JSONL file is
tolerant by default: a line that is not valid JSON or not an object is excluded.
With `"errors": {"policy": "tolerant"}`, such records are excluded and counted,
and the scan finishes with status `partial`; `"strict"` stops at the first one.
The command then succeeds and prints a warning:

```text
Warning [data.csv]: partial scan, 2 records excluded (width_mismatch: 2).
```

Exit codes: `0` for success, including partial scans; `2` for configuration
errors; `4` for unreadable or invalid inputs; `1` for other failures, such as
an existing output without `--force`.

## Sensitive values

Fields holding email addresses, phone numbers or IP addresses are sensitive.
By default, their values are masked in the result: letters become `A` or `a`
and digits `9`, so `jane@example.com` is listed as `aaaa@aaaaaaa.aaa`. Counts
and formats stay exact. Set `exposure.sensitive_values` to `hide` to remove
these values, or to `show` to keep them. Values of other fields are always
shown: share a scan as you would share the data.

## Large files

Files of 16 MiB or more are analyzed by several processes: Tabalyst reads the
file once and hands the values of each field to a worker process, one per
spare processor, at most 8. The scan document is the same whatever the
number of workers. Choose the number with `--workers`, or keep the scan in
one process with `--workers 1`, for example on a shared machine:

```console
tabalyst scan big.csv --workers 4
```

Memory stays bounded by the scan limits in each process. `tabalyst report`
accepts `--workers` too, except with `--scan`, which does not read the source.

## Progress and messages

Interactive terminals show the file and the share of it already read:

```text
[2/8] orders.json - Reading 42%
```

Progress and messages use standard error. Progress is disabled outside a
terminal and with `--no-progress` or `--quiet`. `--verbose` adds the detected
format, encoding, delimiter, status and number of diagnostics.

## Report from a scan

`tabalyst report customers.csv` uses the stored scan, and so does a report on
a JSON or JSONL file. If none exists, it creates one. A scan is reused when the
source content is unchanged, its effective scan settings match and the same
version of Tabalyst wrote it. For a JSON file, the settings include the
collection, so editing `config.structure.dataset_path` in the
[Inspect file](https://docs.tabalyst.com/inspect/json.md) rescans. A changed source causes an atomic replacement of
`scan.json`. Existing DuckDB projects from 0.4.3 are left untouched. Before
reusing a scan, Report checks the source's SHA-256, whatever its size and
modification time. When no scan settings are requested, Report keeps the
settings recorded by the existing scan.

Build the HTML report from a standalone scan document instead of reading the source
again:

```console
tabalyst report --scan customers.scan.json
```

This writes `customers.html` and `customers.json` beside the scan document,
named after the source it records; a JSON source `orders.json` gives
`orders.report.html`. The report is the one `tabalyst report customers.csv`
writes with the same scan settings, since the scan document holds everything
it needs, including duplicate rows and the preview. `-o`, `-d`, `--force` and
wildcards work as for sources:

```console
tabalyst report --scan scans/*.scan.json -d reports/
```

Tabalyst refuses a scan that no longer describes its source. It looks for the
source beside the scan document, under the name the scan recorded:

- a source of another size, or whose content changed (compared by SHA-256,
  whatever its modification time), is an error: scan it again;
- a source that is not beside the scan document is accepted, since the scan
  stands on its own, with a warning that it was not checked. This is the case
  for scans written with `-d` to another directory;
- a scan written by another version of Tabalyst is reported, with a warning;
- for a JSON or JSONL source, an Inspect file beside the source that asks for
  settings different from the scan's, or that cannot be read, is reported with
  a warning. The report still follows the scan document.

The report uses the scan settings recorded in the document. Configuration
files passed with `--config` give the presentation settings; settings given
in their `scan` object must have the values recorded in the document,
otherwise the report stops with exit code `2`. Settings they do not give, such
as a `--delimiter` passed to `tabalyst scan`, are taken from the document.
`--delimiter` and `--encoding` cannot be used with `--scan`. A document that
cannot be read, or was written by another format revision, fails without
stopping the other reports of the batch; scan its source again.

## Python API

```python
import tabalyst

result = tabalyst.scan("orders.json")
print(result.scope.records_analyzed)

batch = tabalyst.generate_scans(["data/*.csv"], output_dir="scans")
reports = tabalyst.generate_reports(["scans/*.scan.json"], from_scan=True)
```

`tabalyst.scan()` reads one source and returns the result without writing
anything; `result.model_dump(mode="json")` gives the document. It works at the
level of the engine: it does not read an Inspect file and, for a JSON file with
no `json.collections` setting, discovers every array reachable through objects
as a collection, plus the document itself as dataset `$`. Use
`tabalyst.generate_scans()` or `tabalyst.generate_reports()` to apply Inspect. Pass
`config=tabalyst.ScanConfig(...)` to change settings, and `workers=` to choose
the number of worker processes as `--workers` does. `tabalyst.generate_scans()`
writes standalone scan documents by default and returns the plan, successes and
failures. Pass `project_storage=True` to use the command's default scan-only
storage for CSV files.
`tabalyst.generate_reports(..., from_scan=True)` builds reports from scan
documents, as `tabalyst report --scan` does.

## Query caches

`tabalyst cache` inspects and cleans the disposable query caches of existing
DuckDB projects. See [Manage query caches](https://docs.tabalyst.com/scan/cache.md).

---

# Scan format

> Structure of the JSON document written by tabalyst scan, with its top level, datasets, fields, measure envelopes, detectors and diagnostics.

Source: https://docs.tabalyst.com/scan/format/ (Tabalyst 0.6.2)

`tabalyst scan data.csv` writes a reusable `scan.json` by default;
`-o` and `-d` export a standalone `<stem>.scan.json`. Both are JSON documents that
describe every field of the source. This page describes format
`tabalyst.scan`, version `0.2.0`, revision `1`. The format is
**experimental**: it can change incompatibly between releases. Always check
`format`, `format_version` and `format_revision` first; changes are listed in
the [scan format changelog](https://docs.tabalyst.com/scan/format-changelog.md).

## Top level

```json
{
  "format": "tabalyst.scan",
  "format_version": "0.2.0",
  "format_revision": 1,
  "engine": {
    "version": "0.6.2",
    "normalization_version": 1,
    "detectors": {"number": 1, "date": 1, "email": 1, "...": 1}
  },
  "status": "complete",
  "started_at": "2026-10-06T12:00:00Z",
  "duration_seconds": 0.0023,
  "source": {
    "format": "csv",
    "name": "orders.csv",
    "size_bytes": 63,
    "modified_at": "2026-09-27T05:06:40.930099Z",
    "sha256": "fe7a56e5...",
    "encoding": "utf-8-sig",
    "csv": {"delimiter": ",", "header": ["id", "amount", "email"]}
  },
  "config": {"...": "effective scan configuration"},
  "config_sha256": "030617af...",
  "scope": {
    "collections": null,
    "records_read": 3,
    "records_analyzed": 3,
    "records_excluded": 0,
    "exclusions": {}
  },
  "datasets": [],
  "diagnostics": []
}
```

- `engine`: the Tabalyst version, the normalization version and the version of
  each detector that ran.
- `status`: `complete` when every record in scope was analyzed, `partial` when
  records were excluded under the `tolerant` error policy. A file that cannot
  be read produces no document.
- `source`: the file as read. `size_bytes` and `sha256` cover every byte read.
  `encoding` is the CSV encoding, or `utf-8` or `utf-8-sig` for JSON, and
  `null` for an Excel workbook. `csv` is `null` for sources that are not CSV.
  `excel` is `null` unless the source is a workbook: it gives the table read,
  as described [below](#excel-sources).
- `config`: the complete effective configuration, with every default, as in
  the `scan` section of a [configuration file](https://docs.tabalyst.com/reference/configuration.md#scan-settings).
  `config_sha256` is the SHA-256 of its canonical JSON (sorted keys, no
  whitespace, UTF-8): two scans with the same value used the same settings.
- `scope`: records read, analyzed and excluded, with exclusions counted by
  reason (`width_mismatch`, `duplicate_key`, `record_too_large`, and for JSONL
  `invalid_line`, `not_object` and `line_too_long`).
  `collections` is `null` for CSV, JSONL and Excel; for JSON it gives the `mode`
  (`auto` or `explicit`) and the `requested` collection paths.

## Excel sources

A workbook is read one table at a time, the one `config.excel.dataset_path`
names. `source.excel` records which:

```json
"excel": {
  "dataset_path": "$.Orders",
  "sheet": "Orders",
  "table": null,
  "range": "A4:I124",
  "header_row": 4,
  "header": ["order_id", "order_date", "customer"]
}
```

`dataset_path` is `$.Sheet` or `$.Sheet.Table`; `table` is the name of a named
table, or `null` for a sheet. `range` is the A1 range of the header and the data
actually read, `header_row` the 1-based sheet row of the header, and `header` the
header cells, which name the fields. The records are the filled rows under the
header, with `location.line` the 1-based sheet row. Values keep their Excel
type: whole numbers are `integer`, other numbers `number`, booleans `boolean`;
dates and times are `string` values in ISO 8601 that the date detector
recognizes. `config.excel` holds `dataset_path` and `header_row`, and takes part
in `config_sha256`.

## Datasets

A CSV source has one dataset, `rows`, and so has an Excel source: its table.
A JSON source has one dataset per
collection of records, and one `document` dataset when its root is not an
array and the collections were discovered automatically (`mode` `auto`). A
JSONL source has one dataset, `$[]`. The commands choose the collection with
[Inspect](https://docs.tabalyst.com/inspect/json.md) and name it explicitly, so a scan
made by `tabalyst scan` or `tabalyst report` has `mode` `explicit` and no
`document` dataset; only `tabalyst.scan()` without `json.collections` discovers
collections.

```json
{
  "id": "$.customers[]",
  "kind": "collection",
  "collection_path": [{"key": "customers"}, {"items": true}],
  "record_count": 4,
  "record_types": {"object": 4},
  "structure": {
    "paths": {"status": "complete", "value": 5},
    "untracked_observations": 0,
    "depth_truncated_observations": 0,
    "max_depth_seen": 3
  },
  "records": {"...": "record facts, described below"},
  "fields": []
}
```

- `kind`: `table` (CSV and Excel), `collection` or `document` (JSON).
- `record_count`: analyzed records; `record_types`: records per JSON type.
- `structure`: the number of distinct field paths, and what the `max_fields`
  and `max_depth` limits left out.
- `records`: facts about whole records, described in the next section.

## Records

Each dataset describes its records as a whole: records with missing values,
empty records, duplicate records and a preview of the first records.

```json
{
  "with_missing": {"count": 2, "records": [2, 5]},
  "empty": {"count": 1, "records": [5]},
  "duplicates": {
    "count": {"status": "complete", "value": 1},
    "records": [3]
  },
  "preview": [
    {
      "record": 1,
      "values": {"column_1": ["1"], "column_2": ["aaa@aaaaaaa.aaa"]}
    }
  ]
}
```

- The values of a record are its strings, numbers, booleans and nulls: every
  cell of a CSV row. A JSON field absent from a record is not a value of it.
- `with_missing`: records with at least one missing value, under the
  `values.missing` setting. `empty`: records whose values are all missing.
- `duplicates`: records equal to an earlier record of the same dataset,
  beyond its first occurrence. Records are equal when every value, its field
  and its native type are equal, raw values compared, before normalization.
  `count` is a measure envelope: `limited` with reason `record_budget` when
  more distinct records than `limits.max_tracked_records` were read, its
  `lower_bound` being the duplicates found; `disabled` when
  `records.duplicates` is `false`.
- `records` lists record numbers, at most `limits.max_listed_records` of
  them. Record numbers are those of `first_record` and diagnostics.
- `preview`: the first `records.preview` records, each with its values per
  field identifier, as text. The values of an items path are listed in order;
  a JSON null is `null`; a field absent from the record has no key. Sensitive
  values go through the field's exposure; a cell holding a hidden value is
  `null`. Empty strings, blanks and null markers are shown as read.

## Fields

Fields are listed in order of discovery. A field is a path inside the records:
a CSV column, a JSON key, or the elements `[]` of an array.

| Key | Content |
| --- | --- |
| `id`, `path`, `display`, `name`, `parent` | Identity: an identifier such as `f4`, the path as segments, its readable form such as `orders[].amount` (keys joined with `json.flatten.separator`), the last segment and the parent field identifier. |
| `collection` | For a JSON array promoted to its own dataset, the identifier of that dataset. |
| `first_record`, `occurrences` | The first record with a value, and the number of values at this path. |
| `presence` | `parent_count`, `present` and `absent`: how often the field exists where it could exist. |
| `native_types` | Values per native type: `string`, `integer`, `number`, `boolean`, `null`, `object`, `array`. |
| `strings` | Strings per category: `empty`, `blank` (whitespace only), `marker` (a configured null marker) and `content`. |
| `missing` | The missing count, the categories included in it and each component. |
| `arrays` | For array fields: count, empty arrays, minimum, maximum and total length. |
| `values` | Analyzable values: `count`, `cardinality`, `frequencies`, `samples`, `first` and `last`. |
| `string_characteristics`, `string_lengths` | Case, non-ASCII characters, line breaks and whitespace; length statistics and histogram. |
| `numeric`, `booleans`, `temporal` | Exact numeric statistics, boolean counts, date range and formats. |
| `normalization` | For each normalization stage, how many values it changed and the distinct values left, plus groups of variants that normalize to the same value. |
| `technical_type` | `integer`, `number`, `date`, `boolean`, `text`, `mixed` or `empty`, with `confidence`, `counts` and `outside_count`. |
| `detectors` | One result per detector, described below. |
| `interpretations` | `candidates`, the detectors that matched at least 95% of the values, and `primary`, the only candidate when there is exactly one. |
| `sensitive`, `exposure` | Whether the field holds sensitive values, and how they are exposed: `mask`, `hide`, `show`, or `null` for other fields. |

Values are listed as their text with their native type, for example
`{"value": "12.50", "type": "string", "count": 1}`. Listings use the analytical
value: surrounding whitespace is trimmed and repeated spaces are collapsed.

## Measure envelopes

A measure that can be limited, disabled or inapplicable is wrapped in an
envelope with a `status`:

```json
{"status": "complete", "value": 42}
{"status": "limited", "reason": "distinct_limit",
 "limit": 100000, "lower_bound": 100001}
{"status": "not_applicable", "reason": "no_values"}
{"status": "disabled"}
{"status": "failed", "reason": "detector_error", "diagnostic": 3}
```

- `complete`: exact for its population.
- `limited`: a configured limit stopped the measure. `lower_bound`, when
  present, is a proven minimum; no estimate is ever given as a value.
- `not_applicable`: meaningless here, for example without values.
- `disabled`: turned off by configuration, or hidden because the field is
  sensitive.
- `failed`: a technical failure; `diagnostic` is its index in `diagnostics`.

A scan can be `complete` while some of its measures are `limited`: the scan
status is about records, the envelope about one measure.

## Detectors

```json
{
  "id": "number",
  "version": 1,
  "status": "complete",
  "coverage": {
    "eligible": 2, "tested": 2, "matched": 2, "ambiguous": 0,
    "invalid": 0, "not_matched": 0, "not_tested": 0,
    "share_tested": 1.0, "share_eligible": 1.0
  },
  "formats": [{"format": "0", "count": 1}, {"format": "0.0", "count": 1}],
  "evidence": {
    "matched": ["12.50", "7"], "ambiguous": [],
    "invalid": [], "not_matched": []
  },
  "details": {},
  "adaptive": null
}
```

Every detector is listed for every field, with status `complete`,
`not_applicable`, `disabled` or `failed`.

- `coverage`: `eligible` values of an accepted type, split into `matched`,
  `ambiguous` (several readings, such as `01/02/2026`), `invalid` (the right
  shape with a wrong content, such as a 13th month), `not_matched` and
  `not_tested`. Shares are rounded to four decimals.
- `formats`: the forms matched, such as `YYYY-MM-DD` or `#,##0.0`, by count.
- `evidence`: the first distinct values in each state.
- `details`: detector-specific counts, such as email domains or ambiguity
  evidence.
- `adaptive`: `null` when the detector tested every value of the field.
  When it recognized none of the first `detection.warmup_values` distinct
  values, or at most `detection.rare_share` of them and none in the second
  half of that warm-up, it is skipped for the other values except probes, and
  `adaptive` is
  `{"skipped_after": 10000, "warmup_reactions": 0, "not_tested": 1200, "diagnostic": null}`:
  the warm-up size, the warm-up values it recognized, the values it did not
  test (included in `coverage.not_tested`), and the index of a
  `detector_skipped_reacted` warning when probes were recognized more often
  than the warm-up allowed, meaning its counts are incomplete. `number` and
  `date` are never skipped.

The built-in detectors are `number`, `date`, `boolean`, `enumeration`,
`email`, `url`, `phone`, `postal_code`, `currency`, `percentage`, `quantity`,
`uuid` and `ip_address`. Patterns of the configuration appear as
`pattern:<id>`. Detectors check syntax, never real-world existence.

Tabalyst never resolves an ambiguity by guessing from other values of the
field: the evidence is published, and only the configuration resolves it,
for example with `detectors.date.ambiguous_order`.

## Sensitive values

A field is sensitive when a sensitive detector (`email`, `phone`,
`ip_address`, or a pattern declared `sensitive`) matched one of its values.
With the default `mask` exposure, its listed values, evidence and variant
groups show masks: uppercase letters become `A`, other letters `a` and digits
`9`. Equal masks are merged. Under `mask` and `hide`, the `numeric` and
`temporal` blocks of a sensitive field are `disabled`, since a minimum or a
date range is itself a value. Counts are never masked.

## Diagnostics

```json
{
  "code": "csv_width_mismatch",
  "level": "error",
  "message": "...",
  "count": 2,
  "dataset": "rows",
  "field": null,
  "detector": null,
  "locations": [{"record": 2, "line": 3}, {"record": 3, "line": 4}]
}
```

Diagnostics describe technical events, not data quality: excluded records,
reached limits (`field_limit`, `depth_limit`, `measures_limited`,
`global_budget`, `record_budget`), collections not found and detector
failures. `level` is
`error` or `warning`. `count` is always complete; `locations` lists at most
`errors.max_locations` places, with `record` and `line` for CSV and JSONL,
`record` and `element` for JSON. For JSONL, `line` is the physical line number,
blank lines included, and `record` counts the non-blank lines.

---

# Scan format changelog

> Current format version and revisions of the JSON document written by tabalyst scan.

Source: https://docs.tabalyst.com/scan/format-changelog/ (Tabalyst 0.6.2)

A scan document written by `tabalyst scan` identifies its experimental
contract with three fields:

```json
{
  "format": "tabalyst.scan",
  "format_version": "0.2.0",
  "format_revision": 1
}
```

`format` identifies the document kind. `format_version` names the compatibility
family, and `format_revision` increases for meaningful structural or semantic
changes within it. The [JSON profile](https://docs.tabalyst.com/report/profile-changelog.md) has its
own version and changelog. Beta revisions may be incompatible; automatic
migration is not provided. `tabalyst report --scan` reads the current Scan
version only: scan the source again to use an older document.

## Version 0.2.0, revision 1 — 2026-10-06

This is the starting point for the current [Scan format](https://docs.tabalyst.com/scan/format.md). It
describes CSV, JSON, JSONL and Excel sources, their effective configuration,
datasets, fields, detectors, record facts, limits and diagnostics.

---

# Manage query caches

> Inspect and clean the disposable query caches of Tabalyst DuckDB projects with tabalyst cache info and tabalyst cache clean.

Source: https://docs.tabalyst.com/scan/cache/ (Tabalyst 0.6.2)

`tabalyst cache` shows how much disk space the disposable query caches of your
Tabalyst projects use, and removes them. A cache never holds anything that
cannot be rebuilt.

```console
tabalyst cache info
tabalyst cache clean
```

```text
Cache: 0 directories in 0 projects (0 bytes).
Cleaned 0 cache directories in 0 projects (0 bytes).
```

## What a cache is

A **project** is the storage Tabalyst keeps for one source file in its local
storage directory (`workspaces/local/projects/`, under `TABALYST_HOME` when
that variable is set). A DuckDB project holds a scan and a database. Its
**query caches** are directories of intermediate query results that speed up
later reads of the database.

In 0.5.0, the commands `tabalyst scan` and `tabalyst report` store a plain
`scan.json` and do not build a DuckDB database. They create no query cache,
so `tabalyst cache` reports 0 directories unless projects built by 0.4.3 exist
on your computer. Those projects are left untouched by other commands.

## Show the size of the caches

```console
tabalyst cache info
tabalyst cache info customers.csv
```

Without an argument, `info` covers every project. With a CSV file, it covers
the project of that source and fails with an error when the source has no
project. It prints the number of managed cache directories, the number of
projects and their logical size, followed by `Retained:` lines for entries
Tabalyst cannot manage and therefore never removes.

## Clean the caches

```console
tabalyst cache clean
tabalyst cache clean customers.csv
```

`clean` removes only the cache directories that Tabalyst owns. It never
removes the scan or the database of a project, and it does not modify your
source files. A project that is being read is skipped, with a warning on
standard error; run the command again once the reader has closed.

The summary goes to standard error, so `tabalyst cache clean 2>$null` hides it
in PowerShell.

## Exit codes

| Code | Meaning |
| --- | --- |
| `0` | Success, even when nothing was found to remove |
| `4` | The source file named in the command has no Tabalyst project |

See [Command line (CLI)](https://docs.tabalyst.com/reference/cli/cache.md) for all commands.

---

# Tabalyst Sample

> Create smaller first, last, random or stratified CSV samples with Tabalyst.

Source: https://docs.tabalyst.com/sample/ (Tabalyst 0.6.2)

Use `tabalyst sample` to create a smaller CSV without changing the source file.
Choose exactly one size option: `--rows` for a fixed count or `--percent` for a
percentage.

```console
tabalyst sample customers.csv --sample-method random --rows 1000
```

This creates `customers.sample.csv` beside `customers.csv`. The output keeps the
header, column order, raw cell values and configured encoding. Existing files
are not replaced unless you pass `--force`; the source can never be overwritten.

## Choose a method

Keep the first or last records:

```console
tabalyst sample customers.csv --sample-method first --rows 1000
tabalyst sample customers.csv --sample-method last --rows 1000
```

Select records uniformly from the entire file. Add `--seed` when another run
must select the same records:

```console
tabalyst sample customers.csv --sample-method random --percent 5 --seed 42
```

Approximately preserve the distribution of a field with proportional
stratified sampling. Empty field values form their own stratum:

```console
tabalyst sample customers.csv --sample-method stratified --field province --rows 1000 --seed 42
```

Percentages round up to the next whole row. A 100% sample contains every data
record. A header-only CSV therefore produces a header-only output at 100%.

## Choose output locations

Use `-o` for the complete output filename when sampling one source:

```console
tabalyst sample customers.csv --sample-method first --rows 100 -o test.csv
```

Wildcards are resolved by Tabalyst, including on shells that do not expand
them. Use `-d` to put samples from one or more sources in a directory:

```console
tabalyst sample *.csv --sample-method random --percent 5 -d samples/
```

The outputs are named `customers.sample.csv`, `orders.sample.csv`, and so on.
`-o` accepts only one resolved input; `-o` and `-d` cannot be combined.
Recursive `**` patterns are not supported.

## Read other CSV formats

The default delimiter is a comma and the default encoding accepts UTF-8 with or
without a byte-order mark. Override either setting when necessary:

```console
tabalyst sample data.csv --sample-method random --rows 100 --delimiter ";" --encoding cp1252
```

`--config` reads the same strict JSON configuration used by `tabalyst report`;
`--delimiter` and `--encoding` override its CSV settings.

Sampling validates the header, quoting and field count of every CSV record. It
uses bounded memory: random sampling stores only the requested rows, while
stratified sampling additionally stores one count per distinct field value.

---

# Tabalyst Generate

> How to create synthetic CSV datasets from a YAML definition or the packaged CRM and insurance examples.

Source: https://docs.tabalyst.com/generate/ (Tabalyst 0.6.2)

Tabalyst Generate creates synthetic CSV datasets for tests, demonstrations and
teaching. Install Tabalyst 0.6.2 or later to use `tabalyst generate`.

```console
tabalyst generate --example crm -o crm-contacts.csv
tabalyst generate --example insurance --rows 3000 -d insurance-output/
```

The CRM example creates one contacts CSV. Insurance creates three branch CSVs
and one combined CSV. With `--rows 3000`, the branches contain 2,182, 273 and
545 rows. These are synthetic records; the current public insurance report
uses a separate dataset and remains unchanged.

## Use a definition

Pass a YAML definition instead of `--example`:

```console
tabalyst generate examples/generate/input/crm.yaml -o contacts.csv
```

The definition must use `definition_version: 1` and declare its datasets and
columns. See the editable [CRM and insurance definitions](https://github.com/loribel-labs/tabalyst/tree/main/examples/generate/input)
for complete examples. The definition format is experimental during beta.

## Create an anomaly variant

An anomaly profile adds an altered CSV, a cell change log and a JSON summary
beside an untouched clean CSV. The insurance profile changes only the combined
dataset:

```console
tabalyst generate --example insurance --rows 3000 --anomaly-profile dirty_realistic -d insurance-dirty/
```

Choose `-o` only when the result is one clean CSV. Use `-d` for multiple CSVs
or an anomaly profile. Existing output files require `--force`. Nothing is
written beside the definition or into the current directory by default.

Use `--seed` and `--as-of-date YYYY-MM-DD` to override the definition defaults.
Generation is reproducible with the same definition, options and environment.
The [CLI reference](https://docs.tabalyst.com/reference/cli/generate.md) lists all options.

---

# Reference

> Reference pages of Tabalyst - command line, Python API, configuration file, execution history and glossary, with the JSON format references of each tool.

Source: https://docs.tabalyst.com/reference/ (Tabalyst 0.6.2)

Reference pages describe exactly what Tabalyst accepts and produces.

- [Command line (CLI)](https://docs.tabalyst.com/reference/cli.md): the shared conventions and one page per
  command ([report](https://docs.tabalyst.com/reference/cli/report.md), [scan](https://docs.tabalyst.com/reference/cli/scan.md),
  [inspect](https://docs.tabalyst.com/reference/cli/inspect.md), [sample](https://docs.tabalyst.com/reference/cli/sample.md), [cache](https://docs.tabalyst.com/reference/cli/cache.md))
  with every option, examples and exit codes.
- [Python API](https://docs.tabalyst.com/reference/python-api.md): the public functions and their
  arguments.
- [Configuration](https://docs.tabalyst.com/reference/configuration.md): the JSON configuration file and every
  `scan` setting.
- [Types and patterns](https://docs.tabalyst.com/reference/types-and-patterns.md): native, inferred and
  semantic [types](https://docs.tabalyst.com/reference/types-and-patterns/types.md), plus built-in and configurable
  [patterns](https://docs.tabalyst.com/reference/types-and-patterns/patterns.md) shared by Scan and Report.
- [Execution history](https://docs.tabalyst.com/reference/history.md): the `executions.json` file written next to each report.
- [Glossary](https://docs.tabalyst.com/reference/glossary.md): the terms used in the documentation and the report.

## Formats of the generated files

- [JSON profile](https://docs.tabalyst.com/report/json-profile.md) and its
  [changelog](https://docs.tabalyst.com/report/profile-changelog.md), for `tabalyst report`.
- [Scan format](https://docs.tabalyst.com/scan/format.md) and its
  [changelog](https://docs.tabalyst.com/scan/format-changelog.md), for `tabalyst scan`.
- [Inspect format](https://docs.tabalyst.com/inspect/format.md) and its
  [changelog](https://docs.tabalyst.com/inspect/format-changelog.md), for `tabalyst inspect`.

---

# Command line (CLI)

> The tabalyst commands (report, scan, inspect, sample, generate, cache) and their shared conventions.

Source: https://docs.tabalyst.com/reference/cli/ (Tabalyst 0.6.2)

The `tabalyst` command has six subcommands. `python -m tabalyst` accepts the
same ones. Each has its own page with every option and examples.

| Command | Purpose | Options and examples |
| --- | --- | --- |
| `tabalyst report` | HTML report and JSON profile of CSV, JSON, JSONL or Excel files | [tabalyst report](https://docs.tabalyst.com/reference/cli/report.md) |
| `tabalyst scan` | Complete JSON description of every field | [tabalyst scan](https://docs.tabalyst.com/reference/cli/scan.md) |
| `tabalyst inspect` | Find how to read a JSON, JSONL or Excel file | [tabalyst inspect](https://docs.tabalyst.com/reference/cli/inspect.md) |
| `tabalyst sample` | Smaller CSV file from a larger one | [tabalyst sample](https://docs.tabalyst.com/reference/cli/sample.md) |
| `tabalyst cache` | Inspect and clean disposable query caches | [tabalyst cache](https://docs.tabalyst.com/reference/cli/cache.md) |
| `tabalyst generate` | Synthetic CSV datasets from YAML | [tabalyst generate](https://docs.tabalyst.com/reference/cli/generate.md) |

```console
tabalyst --version
tabalyst --help
tabalyst report --help
```

## Conventions

- **Inputs.** `report`, `scan`, `inspect` and `sample` take one or more files or
  non-recursive glob patterns, such as `*.csv`. Tabalyst expands wildcards
  itself, so they behave the same in PowerShell, `cmd` and POSIX shells. It
  does not search subfolders.
- **Generate inputs.** `generate` takes one YAML definition or `--example`.
  It requires `-o` or `-d`; see its [command reference](https://docs.tabalyst.com/reference/cli/generate.md).
- **Outputs.** `-o` names one output file and accepts a single input. `-d`
  names an output directory and accepts one or many inputs. They are mutually
  exclusive.
- **Safety.** The whole batch is checked before any work: colliding outputs,
  an output that overwrites a source and existing outputs stop the batch.
  `--force` allows replacing outputs, never sources. Files are written
  atomically.
- **Options by format.** `--delimiter` and `--encoding` read CSV files;
  `--collection` and `--all-collections` choose among the collections of a JSON
  file or the tables of a workbook. On a file that cannot use an option, the
  option is ignored and a warning says so, such as `--delimiter is ignored:
  sales.xlsx is not a CSV file.` Each command page has a table of the options
  by format.
- **Failures.** A failed file does not stop the others; the command returns a
  non-zero exit code at the end.
- **Progress.** Progress and diagnostics use standard error. They are off
  outside a terminal, and with `--no-progress` or `--quiet`.
- **Configuration.** No configuration file is read unless `--config` is given.
  Explicit options override the configuration file, which overrides the
  defaults. See [Configuration](https://docs.tabalyst.com/reference/configuration.md).

## Choosing a collection or a table {#collection}

A JSON file can hold several arrays, and an Excel workbook several sheets and
tables. Tabalyst chooses one when the choice is clear, and otherwise stops with
exit code `2` and lists what you can choose from, as one command per choice, ready
to copy and run:

```text
Choose the table to analyze:
  tabalyst report shop.xlsx --collection Costs
  tabalyst report shop.xlsx --collection '$["Sales Q1"]'
Or report every table: tabalyst report shop.xlsx --all-collections
```

`scan`, `report` and `inspect` all print them. `tabalyst report` also takes
`--all-collections` to report every table in one command. The `--collection`
option takes the short form when the name is plain, and the absolute path
otherwise:

| Source | Value | Meaning |
| --- | --- | --- |
| JSON | `orders` | The array `$.orders[]` |
| JSON | `data.items` | The array `$.data.items[]` |
| JSON | `.` | The root array, `$[]` |
| JSON | `'$.data.items[]'` | The same array, as an absolute path |
| Excel | `Costs` | The sheet `Costs` |
| Excel | `Sales.Orders` | The named table `Orders` of the sheet `Sales` |
| Excel | `'$["Sales Q1"]'` | A sheet whose name is not a plain identifier (space, dot, accent) |

Put an absolute path in single quotes, so that the shell leaves `$` and `"`
alone in PowerShell and in POSIX shells. A JSON file accepts several
`--collection` options, a workbook one. [Inspect JSON files](https://docs.tabalyst.com/inspect/json.md)
and [Inspect Excel workbooks](https://docs.tabalyst.com/inspect/excel.md) explain how the choice is
made and how to keep it in an Inspect file.

## Exit codes

| Code | Meaning |
| --- | --- |
| `0` | Success |
| `1` | Analysis or output failure |
| `2` | Invalid configuration or command, or a JSON or Excel source whose collection or table must be chosen (see [Inspect](https://docs.tabalyst.com/inspect/json.md) and [Inspect Excel workbooks](https://docs.tabalyst.com/inspect/excel.md)) |
| `4` | Input error: missing, unreadable or invalid source |

When a batch has several kinds of failure, `1` wins over `2`, which wins over
`4`.

## Environment

| Variable | Effect |
| --- | --- |
| `TABALYST_HOME` | Root of Tabalyst's local storage for stored scans and projects |

---

# CLI: tabalyst report

> Every option of tabalyst report and examples for CSV, JSON and Excel files, including how to choose a table with --collection and report every table with --all-collections.

Source: https://docs.tabalyst.com/reference/cli/report/ (Tabalyst 0.6.2)

```console
tabalyst report INPUT... [OPTIONS]
```

Analyzes CSV, JSON, JSONL or Excel (`.xlsx`, `.xlsm`) files and writes, for each
one, an interactive HTML report and a JSON profile, plus an `executions.json`
history in the same folder. With `--scan`, the inputs are scan documents. The
sources are never modified. See [Tabalyst Report](https://docs.tabalyst.com/report.md) for what
the report contains.

## Output files

| Source | Report | Profile |
| --- | --- | --- |
| CSV: `data.csv` | `data.html` | `data.json` |
| JSON, JSONL, NDJSON or Excel: `data.json`, `data.xlsx` | `data.report.html` | `data.report.json` |
| With `--all-collections`: `shop.xlsx` | `shop.<collection>.html` | `shop.<collection>.json` |
| With `-o report.html` | `report.html` | `report.json` |

The files go beside the source, or in the folder of `-d`. `.report` keeps the
profile of a JSON or Excel file from replacing a source with the same stem. With
`--details`, the column pages are written in a folder named like the report, for
example `data/`.

## Options

`INPUT...` is required: one or more files, scan documents with `--scan`, or
non-recursive glob patterns. The table at the end of this section shows which
options each kind of file uses.

### Options for every format

| Option | Description |
| --- | --- |
| `-o`, `--output PATH` | HTML filename ending in `.html`, for a single input. The profile takes the same name with `.json`. Not with `-d` or `--all-collections` |
| `-d`, `--output-dir PATH` | Folder for the reports, named after their sources. Required by `--all-collections` with several inputs |
| `-c`, `--config PATH` | JSON [configuration file](https://docs.tabalyst.com/reference/configuration.md) |
| `-f`, `--force` | Replace existing report files |
| `--details`, `--no-details` | One standalone HTML page per column (off by default) |
| `-q`, `--quiet` | Suppress success messages; warnings and errors remain |
| `-v`, `--verbose` | Also show the profile and history paths, and the encoding and delimiter of a CSV file |
| `--no-progress` | Disable the progress line |
| `--workers N` | Worker processes that analyze values: `1` for one process. Files of 16 MiB or more use one per spare processor by default. Not with `--scan` |
| `--help` | Show the options and exit |

### Options for CSV files

| Option | Description |
| --- | --- |
| `--delimiter TEXT` | One-character delimiter, instead of the detected one |
| `--encoding TEXT` | Text encoding, instead of the detected one |

### Options for JSON and Excel files

| Option | Description |
| --- | --- |
| `--collection TEXT` | The part of the file to analyze. JSON: an array, such as `data.items` or `'$.data.items[]'`; `.` is the root array; repeatable. Excel: a sheet or table, such as `Sales`, `Sales.Orders` or `'$["Q1 2026"]'`; once. See [Choosing a collection or a table](https://docs.tabalyst.com/reference/cli.md#collection). Not with `--scan` or `--all-collections` |
| `--all-collections` | Report every visible collection of each file, as `<name>.<collection>.html` and `.json`. Not with `-o`, `--collection` or `--scan` |

A JSONL file has one dataset, its lines, so it has no collection to choose.

### Option for scan documents

| Option | Description |
| --- | --- |
| `--scan` | Build the reports from scan documents written by `tabalyst scan`, without reading the sources again. Not with `--delimiter`, `--encoding`, `--collection`, `--all-collections` or `--workers` |

### Options by format

| Option | CSV | JSON | JSONL | Excel |
| --- | :---: | :---: | :---: | :---: |
| `-o`, `-d`, `-c`, `-f`, `--details`, `-q`, `-v`, `--no-progress`, `--workers` | yes | yes | yes | yes |
| `--delimiter`, `--encoding` | yes | - | - | - |
| `--collection` | - | yes, repeatable | - | yes, once |
| `--all-collections` | - | yes | - | yes |
| `--scan` (the inputs are scan documents) | yes | yes | yes | yes |

An option that a kind of file cannot use is ignored, with a warning, and the
report is still written: `report sales.xlsx --delimiter ";"` prints
`Warning [sales.xlsx]: --delimiter is ignored: sales.xlsx is not a CSV file.`

## Examples

### CSV files

```console
tabalyst report customers.csv
tabalyst report data/*.csv -d reports
tabalyst report customers.csv -o out/customers-2026.html --details --force
tabalyst report semicolon.csv --delimiter ";" --encoding cp1252
```

### Excel workbook with one table

```console
tabalyst report sales.xlsx
```

When Tabalyst can tell which table to analyze, a workbook needs nothing else:
only one table is eligible, or the largest has at least 10 times the rows of the
next one. This writes `sales.report.html`, `sales.report.json` and
`executions.json`.

### Excel workbook with several tables

When several tables are equally plausible, `report` analyzes nothing, exits with
code `2` and offers the choices:

```console
tabalyst report shop.xlsx
```

```text
Error [shop.xlsx]: 2 tables of shop.xlsx are equally plausible. Nothing was analyzed.
Candidates:
  $.Costs (40 rows)
  $["Sales Q1"] (50 rows)
  $.Notes (0 rows), empty
Pass --collection with a sheet or table path such as '$.Sheet', or run `tabalyst inspect shop.xlsx` and set config.structure.dataset_path in the file it writes.
Choose the table to analyze:
  tabalyst report shop.xlsx --collection Costs
  tabalyst report shop.xlsx --collection '$["Sales Q1"]'
Or report every table: tabalyst report shop.xlsx --all-collections
```

Copy the line you want and run it:

```console
tabalyst report shop.xlsx --collection Costs
tabalyst report shop.xlsx --collection '$["Sales Q1"]'
tabalyst report shop.xlsx --collection Costs -d reports
```

The first one writes `shop.report.html` and `shop.report.json`. A named table of a
sheet is chosen with `--collection Sales.Orders`.

### Every table of a workbook

```console
tabalyst report shop.xlsx --all-collections
```

```text
Report: shop.xlsx -> D:\work\shop.costs.html
Report: shop.xlsx -> D:\work\shop.sales-q1.html
2 succeeded, 0 failed
```

Each eligible table gets its own pair of files, named
`<file>.<collection-slug>`: `shop.costs.html`, `shop.costs.json`,
`shop.sales-q1.html` and `shop.sales-q1.json`. The slug is the sheet (and table)
name in lower case, without accents, with every other character replaced by a
hyphen, and a number is added when two names give the same slug. Only the visible
sheets that [Inspect](https://docs.tabalyst.com/reference/cli/inspect.md) finds eligible are reported: empty sheets and
sheets without a header are left out, and a hidden sheet is left out with a
warning (choose it with `--collection`).

A single file can go to a folder, and `-d` is required for several files:

```console
tabalyst report shop.xlsx --all-collections -d reports
tabalyst report shop.xlsx orders.xlsx --all-collections -d reports
tabalyst report "exports/*.xlsx" --all-collections -d reports --force
```

A source with a single dataset, such as a CSV file, keeps its usual name in the
same command. A workbook that has nothing to report fails alone, with exit code
`2`, and does not stop the others.

### JSON files

```console
tabalyst report orders.json
tabalyst report shop.json --collection orders
tabalyst report shop.json --all-collections -d reports
```

For a JSON file with a `customers` and an `orders` array, the last command writes
`shop.customers.html` and `shop.orders.html` with their `.json` profiles.

### Reusing scans

```console
tabalyst scan data/*.csv -d scans
tabalyst report scans/*.scan.json --scan -d reports
```

### Configuration

```console
tabalyst report customers.csv --config settings.json
```

## Exit codes

| Code | When |
| --- | --- |
| `0` | Every report was written |
| `1` | A report could not be analyzed, rendered or written |
| `2` | Invalid options, or a JSON or Excel source whose collection or table must be chosen |
| `4` | A source is missing, unreadable or invalid |

See also [Report Excel workbooks](https://docs.tabalyst.com/report/excel.md),
[Report JSON files](https://docs.tabalyst.com/report/json.md) and [Inspect](https://docs.tabalyst.com/reference/cli/inspect.md).

---

# CLI: tabalyst scan

> Every option of tabalyst scan and examples for CSV, JSON, JSONL and Excel files, including how to choose a JSON collection or an Excel table with --collection.

Source: https://docs.tabalyst.com/reference/cli/scan/ (Tabalyst 0.6.2)

```console
tabalyst scan INPUT... [OPTIONS]
```

Describes CSV, JSON, JSONL or Excel (`.xlsx`, `.xlsm`) files in a JSON scan
document: every field, its presence, values, statistics and detected meanings.
Without `-o` or `-d`, the scan is stored in Tabalyst's local storage, ready to be
reused by `tabalyst report`; with them, a standalone `<stem>.scan.json` is
written. The sources are never modified. See [Scan CSV and JSON
files](https://docs.tabalyst.com/scan/files.md) for what a scan analyzes.

## Options

`INPUT...` is required: one or more files or non-recursive glob patterns. The
table at the end of this section shows which options each kind of file uses.

### Options for every format

| Option | Description |
| --- | --- |
| `-o`, `--output PATH` | Scan filename ending in `.json`, for a single input. Not with `-d` |
| `-d`, `--output-dir PATH` | Folder for standalone scans named `<stem>.scan.json` |
| `-c`, `--config PATH` | JSON [configuration file](https://docs.tabalyst.com/reference/configuration.md); repeat to merge several, in order |
| `-f`, `--force` | Replace existing scan files |
| `-q`, `--quiet` | Suppress success messages; warnings and errors remain |
| `-v`, `--verbose` | Show the format, encoding, delimiter or table, status, diagnostics and duration |
| `--no-progress` | Disable the progress line |
| `--workers N` | Worker processes that analyze values: `1` for one process. Files of 16 MiB or more use one per spare processor by default |
| `--help` | Show the options and exit |

### Options for CSV files

| Option | Description |
| --- | --- |
| `--delimiter TEXT` | One-character delimiter, instead of the detected one |
| `--encoding TEXT` | Text encoding, instead of the detected one |

### Option for JSON and Excel files

| Option | Description |
| --- | --- |
| `--collection TEXT` | The part of the file to analyze. JSON: an array, such as `data.items` or `'$.data.items[]'`; `.` is the root array; repeatable, one dataset each. Excel: a sheet or table, such as `Costs`, `Sales.Orders` or `'$["Q1 2026"]'`; once. See [Choosing a collection or a table](https://docs.tabalyst.com/reference/cli.md#collection) |

A JSONL file has one dataset, its lines, so it has no collection to choose.

### Options by format

| Option | CSV | JSON | JSONL | Excel |
| --- | :---: | :---: | :---: | :---: |
| `-o`, `-d`, `-c`, `-f`, `-q`, `-v`, `--no-progress`, `--workers` | yes | yes | yes | yes |
| `--delimiter`, `--encoding` | yes | - | - | - |
| `--collection` | - | yes, repeatable | - | yes, once |

An option that a kind of file cannot use is ignored, with a warning, and the
scan is still made: `scan sales.xlsx --delimiter ";"` prints
`Warning [sales.xlsx]: --delimiter is ignored: sales.xlsx is not a CSV file.`

## Examples

### CSV and JSON files

```console
tabalyst scan customers.csv
tabalyst scan data/*.csv -d scans
tabalyst scan customers.csv -o scans/customers-2026.scan.json --force
tabalyst scan orders.json
```

### Choosing a collection

```console
tabalyst scan orders.json --collection customers
tabalyst scan orders.json --collection customers --collection products
```

When several arrays are equally plausible, `scan` stops with exit code `2` and
offers one command per array:

```text
Choose the collection to analyze:
  tabalyst scan shop.json --collection customers
  tabalyst scan shop.json --collection orders
```

### Excel workbooks

```console
tabalyst scan sales.xlsx
tabalyst scan shop.xlsx --collection Costs
tabalyst scan shop.xlsx --collection '$["Sales Q1"]' -d scans
```

A scan analyzes one table of a workbook. When several tables are equally
plausible, `scan` offers the same choices as `report`:

```text
Choose the table to analyze:
  tabalyst scan shop.xlsx --collection Costs
  tabalyst scan shop.xlsx --collection '$["Sales Q1"]'
```

To report every table of a workbook, use
[`tabalyst report --all-collections`](https://docs.tabalyst.com/reference/cli/report.md).

### Reuse in a report

```console
tabalyst scan data/*.csv -d scans
tabalyst report scans/*.scan.json --scan -d reports
```

## Exit codes

| Code | When |
| --- | --- |
| `0` | Every scan was written |
| `1` | A scan could not be analyzed or written |
| `2` | Invalid options, or a JSON or Excel source whose collection or table must be chosen |
| `4` | A source is missing, unreadable or invalid |

---

# CLI: tabalyst inspect

> Every option of tabalyst inspect and examples for JSON, JSONL and Excel files, including the ready-to-run report commands it prints when several collections or tables are possible.

Source: https://docs.tabalyst.com/reference/cli/inspect/ (Tabalyst 0.6.2)

```console
tabalyst inspect INPUT... [OPTIONS]
```

Reads JSON, JSONL, NDJSON or Excel (`.xlsx`, `.xlsm`) files once and writes an
Inspect file, `<source>-inspect.json`, beside each source. The file lists the
collections or tables it found, says which one it selects and why, and holds the
`config` section you can edit. CSV files, and the spreadsheet formats `.xls`,
`.xlsb` and `.ods`, are refused with exit code `2`. See [Inspect JSON
files](https://docs.tabalyst.com/inspect/json.md) and [Inspect Excel
workbooks](https://docs.tabalyst.com/inspect/excel.md).

## Options

| Option | Description |
| --- | --- |
| `INPUT...` | One or more JSON, JSONL, NDJSON or Excel files, or non-recursive glob patterns (required) |
| `-c`, `--config PATH` | JSON [configuration file](https://docs.tabalyst.com/reference/configuration.md) whose `scan` section seeds a new Inspect file; repeat to merge several, in order |
| `--reset-config` | Replace the `config` of an existing Inspect file by the detected one |
| `-f`, `--force` | Replace an existing Inspect file that cannot be kept |
| `-q`, `--quiet` | Suppress success messages; warnings and the commands to choose from remain |
| `-v`, `--verbose` | List every candidate collection, sheet or table, with its size and why it is not eligible |
| `--no-progress` | Disable the progress line |
| `--help` | Show the options and exit |

An existing Inspect file keeps its `config` when you inspect again. Inspect has
no `-o` or `-d`: the file always goes beside its source.

## Examples

### A workbook with one clear table

```console
tabalyst inspect sales.xlsx -v
```

```text
Inspect: D:\work\sales.xlsx-inspect.json
Selection: $.Orders (much larger than $.Regions)
Format: excel
Candidates: 3
  $.Orders (120 elements)
  $.Regions (5 elements)
  $.Notes (0 elements, no_header)
Warning [sales.xlsx]: $.Notes has no header row that Tabalyst can recognize.
```

`tabalyst report sales.xlsx` needs no choice.

### Several tables: the commands to copy

```console
tabalyst inspect shop.xlsx
```

```text
Inspect: D:\work\shop.xlsx-inspect.json
Selection: none (several collections are equally plausible)
Warning [shop.xlsx]: 2 tables are equally plausible: $.Costs (40), $["Sales Q1"] (50). None was selected: set config.structure.dataset_path to the one to analyze.
Choose the table to analyze:
  tabalyst report shop.xlsx --collection Costs
  tabalyst report shop.xlsx --collection '$["Sales Q1"]'
Or report every table: tabalyst report shop.xlsx --all-collections
```

Copy a line to run it. The short form (`Costs`) is used whenever the name is
plain, and the quoted path otherwise. A JSON file works the same way:

```console
tabalyst inspect shop.json
```

```text
Choose the collection to analyze:
  tabalyst report shop.json --collection customers
  tabalyst report shop.json --collection orders
Or report every collection: tabalyst report shop.json --all-collections
```

### Several files, other settings

```console
tabalyst inspect "exports/*.xlsx" "logs/*.jsonl"
tabalyst inspect shop.json --config settings.json
tabalyst inspect shop.json --reset-config
```

## Exit codes

| Code | When |
| --- | --- |
| `0` | Every Inspect file was written, even when no collection could be selected |
| `2` | Invalid options, a source Inspect does not read, or an invalid Inspect file |
| `4` | A source is missing, unreadable or invalid |

---

# CLI: tabalyst sample

> Every option of tabalyst sample and examples of first, last, random and stratified CSV samples, with fixed sizes, percentages and seeds.

Source: https://docs.tabalyst.com/reference/cli/sample/ (Tabalyst 0.6.2)

```console
tabalyst sample INPUT... --sample-method METHOD [OPTIONS]
```

Creates `<stem>.sample.csv` beside each CSV source, or in the folder of `-d`,
without modifying the source. Choose exactly one size: `--rows` or `--percent`.
See [Tabalyst Sample](https://docs.tabalyst.com/sample.md).

## Options

| Option | Description |
| --- | --- |
| `INPUT...` | One or more CSV files or non-recursive glob patterns (required) |
| `--sample-method METHOD` | Required: `first`, `last`, `random` or `stratified` |
| `--rows N` | Number of data rows to select |
| `--percent X` | Percentage of data rows to select, rounded up to a whole row. Exclusive with `--rows` |
| `--field TEXT` | Field that defines the strata, required by `stratified` |
| `--seed N` | Seed that makes a random or stratified selection reproducible |
| `-o`, `--output PATH` | Output CSV filename, for a single input. Not with `-d` |
| `-d`, `--output-dir PATH` | Folder for samples named `<stem>.sample.csv` |
| `-c`, `--config PATH` | JSON [configuration file](https://docs.tabalyst.com/reference/configuration.md) |
| `--delimiter TEXT` | One-character CSV delimiter |
| `--encoding TEXT` | CSV text encoding |
| `-f`, `--force` | Replace an existing sample file; a source is never replaced |
| `--help` | Show the options and exit |

## Examples

```console
tabalyst sample customers.csv --sample-method first --rows 1000
tabalyst sample customers.csv --sample-method last --rows 1000
tabalyst sample customers.csv --sample-method random --percent 5 --seed 42
tabalyst sample customers.csv --sample-method stratified --field province --rows 1000 --seed 42
tabalyst sample customers.csv --sample-method first --rows 100 -o test.csv
tabalyst sample *.csv --sample-method random --percent 5 -d samples/
tabalyst sample data.csv --sample-method random --rows 100 --delimiter ";" --encoding cp1252
```

Sample accepts CSV files only; there is no Excel or JSON sample.

## Exit codes

| Code | When |
| --- | --- |
| `0` | Every sample was written |
| `1` | A sample could not be written |
| `2` | Invalid options |
| `4` | A source is missing, unreadable or invalid |

---

# CLI: tabalyst generate

> The inputs, outputs and options of the tabalyst generate command.

Source: https://docs.tabalyst.com/reference/cli/generate/ (Tabalyst 0.6.2)

`tabalyst generate` validates a YAML definition and writes synthetic CSV data.
It is available in Tabalyst 0.6.2 and later. See
[Tabalyst Generate](https://docs.tabalyst.com/generate.md) for a walkthrough.

```text
tabalyst generate DEFINITION.yaml (-o FILE.csv | -d DIR) [options]
tabalyst generate --example crm (-o FILE.csv | -d DIR) [options]
```

Provide exactly one definition source: a YAML file or `--example crm` or
`--example insurance`. Exactly one destination is required.

| Option | Meaning |
| --- | --- |
| `--example NAME` | Use the packaged `crm` or `insurance` definition. |
| `-o`, `--output FILE` | Write one clean CSV. |
| `-d`, `--output-dir DIR` | Write all planned artifacts to a directory. |
| `--rows N` | Override the generated row count; for insurance, allocate the combined total across its branches. |
| `--seed INTEGER` | Override the root random seed. |
| `--as-of-date YYYY-MM-DD` | Override the reference date. |
| `--anomaly-profile NAME` | Produce the selected anomaly variant, cell change log and summary. Requires `-d`. |
| `-f`, `--force` | Replace existing generated artifacts. |

Explicit options override YAML defaults. Paths are checked before generation;
the definition and packaged resources cannot be overwritten. A selected
anomaly profile keeps the clean CSV alongside its altered variant.

---

# CLI: tabalyst cache

> The tabalyst cache info and tabalyst cache clean commands, which show and remove the disposable query caches of Tabalyst projects.

Source: https://docs.tabalyst.com/reference/cli/cache/ (Tabalyst 0.6.2)

```console
tabalyst cache info [SOURCE]
tabalyst cache clean [SOURCE]
```

`info` shows the number and size of the disposable query caches of Tabalyst
projects, and `clean` removes them. A cache never holds anything that cannot be
rebuilt, and `clean` never removes a scan or a database. See [Manage query
caches](https://docs.tabalyst.com/scan/cache.md).

## Arguments and options

| Argument | Description |
| --- | --- |
| `SOURCE` | Optional CSV file whose project is selected. Without it, every project is |
| `--help` | Show the options and exit |

`cache` has no other options and takes no glob pattern.

## Examples

```console
tabalyst cache info
tabalyst cache info customers.csv
tabalyst cache clean
tabalyst cache clean customers.csv
```

```text
Cache: 0 directories in 0 projects (0 bytes).
Cleaned 0 cache directories in 0 projects (0 bytes).
```

## Exit codes

| Code | When |
| --- | --- |
| `0` | Success, even when nothing was found to remove |
| `4` | The source file named in the command has no Tabalyst project |

---

# Python API

> The public Python functions of Tabalyst for reports, scans, Inspect and sampling, with the conventions they share - batch results, errors, configuration files and progress callbacks.

Source: https://docs.tabalyst.com/reference/python-api/ (Tabalyst 0.6.2)

Everything the command line does is available from Python. Import `tabalyst`
and call the functions described in the pages below. Names not documented here
are internal and may change without notice during beta.

```python
import tabalyst

batch = tabalyst.generate_reports(["*.csv"], output_dir="reports")
```

## Functions by task

| To do this | Use | Page |
| --- | --- | --- |
| Write the report and profile of one CSV file | `tabalyst.analyze()` | [Python API: reports](https://docs.tabalyst.com/reference/python-api/report.md) |
| Write reports for CSV, JSON, JSONL or Excel files | `tabalyst.generate_reports()` | [Python API: reports](https://docs.tabalyst.com/reference/python-api/report.md) |
| Get a profile or its HTML without writing files | `tabalyst.analyze_csv()`, `tabalyst.analyze_scan()`, `tabalyst.render_report()` | [Python API: reports](https://docs.tabalyst.com/reference/python-api/report.md) |
| Scan one source in memory | `tabalyst.scan()` | [Python API: scans](https://docs.tabalyst.com/reference/python-api/scan.md) |
| Write scan documents | `tabalyst.generate_scans()` | [Python API: scans](https://docs.tabalyst.com/reference/python-api/scan.md) |
| Find how to read a JSON, JSONL or Excel file | `tabalyst.inspect()`, `tabalyst.generate_inspections()` | [Python API: Inspect](https://docs.tabalyst.com/reference/python-api/inspect.md) |
| Write a smaller CSV file | `tabalyst.sample_csv()`, `tabalyst.generate_samples()` | [Python API: sampling](https://docs.tabalyst.com/reference/python-api/sample.md) |

## Conventions

### Batch functions

The `generate_*` functions take `input_specs`, a list of files and
non-recursive glob patterns, as the command line does, and process the whole
batch. They return a batch result with the same four members:

- `plan`: the jobs that were planned, and `plan.warnings`, a tuple of
  `(source, sentence)` pairs about what planning left out, such as an option that
  a source cannot use.
- `successes`: one entry per source that was processed, with its `job` and its
  `result`.
- `failures`: one entry per source that failed, with its `job` and its `error`.
  A failed source does not stop the others.
- `succeeded`: `True` when there is no failure.

The whole batch is rejected before any work when an output is unsafe, such as
two sources that map to one file or an output that would replace a source.

```python
batch = tabalyst.generate_reports(["data/*.csv"], output_dir="reports")
for failure in batch.failures:
    print(failure.job.source, failure.error)
```

### Errors

Expected failures derive from `tabalyst.TabalystError`: `ConfigurationError`
(invalid options or configuration, or a collection that must be chosen),
`InputError` (a missing, unreadable or invalid source) and `ReportError`
(a file that cannot be rendered or written). A source whose collection must be
chosen raises a `ConfigurationError`, as exit code `2` does on the command line.
Functions for one source raise the error; batch functions list it among the
failures.

### Existing files

Existing outputs are never replaced silently. Pass `force=True` when
replacement is intended. A source is never replaced.

### Configuration files

`config_path` is one JSON [configuration file](https://docs.tabalyst.com/reference/configuration.md) or a
sequence of files merged in order. Explicit arguments override the files, which
override the defaults.

### Progress

`on_progress` is a function that receives a progress event for each phase of
each source. It is optional, and is what the command line uses to print its
progress line.

## Related pages

- [Command line (CLI)](https://docs.tabalyst.com/reference/cli.md), for the same operations as commands.
- [Configuration](https://docs.tabalyst.com/reference/configuration.md), for the files that `config_path` reads
  and the `ScanConfig` settings.
- [Scan CSV and JSON files](https://docs.tabalyst.com/scan/files.md) and
  [Inspect JSON and JSONL files](https://docs.tabalyst.com/inspect/json.md).

---

# Python API: reports

> The Python functions that write or build Tabalyst reports - analyze, generate_reports, analyze_csv, analyze_scan and render_report - with their parameters, return values and examples for CSV, JSON and Excel files.

Source: https://docs.tabalyst.com/reference/python-api/report/ (Tabalyst 0.6.2)

Reports are written by `tabalyst.analyze()` for one CSV file and by
`tabalyst.generate_reports()` for any number of CSV, JSON, JSONL or Excel files.
`analyze_csv()`, `analyze_scan()` and `render_report()` build a profile or its
HTML without writing files. The command is
[`tabalyst report`](https://docs.tabalyst.com/reference/cli/report.md).

```python
import tabalyst

result = tabalyst.analyze("customers.csv", "customers.html", separator=";")
batch = tabalyst.generate_reports(["data/*.csv"], output_dir="reports")
batch = tabalyst.generate_reports(["shop.xlsx"], collections=["$.Costs"])
batch = tabalyst.generate_reports(["shop.xlsx"], all_collections=True)
batch = tabalyst.generate_reports(["scans/*.scan.json"], from_scan=True)
```

## tabalyst.generate_reports()

Plans and writes the reports of several sources: an HTML report and a JSON
profile for each, with an `executions.json` history in the same folder.

```python
tabalyst.generate_reports(
    input_specs,
    *,
    output=None,
    output_dir=None,
    separator=None,
    encoding=None,
    collections=None,
    config_path=None,
    force=False,
    details=False,
    on_progress=None,
    from_scan=False,
    workers=None,
    all_collections=False,
)
```

Returns a [batch result](https://docs.tabalyst.com/reference/python-api.md#batch-functions) whose successes hold the
profile as a dictionary in `result`. A successful job also has `warnings`.

- `input_specs`: files or non-recursive glob patterns, or scan documents with
  `from_scan=True`.
- `output`: HTML filename for a single source; the profile takes the same name
  with `.json`. Not with `output_dir` or `all_collections`.
- `output_dir`: folder for the reports, named after their sources. Required with
  `all_collections` when there are several sources.
- `separator`, `encoding`: CSV delimiter and text encoding, instead of the
  detected ones. Not with `from_scan`.
- `collections`: what to analyze in a JSON file or workbook, as absolute paths:
  arrays such as `["$.customers[]"]` for JSON, one table such as `["$.Sales"]` or
  `["$.Sales.Orders"]` for Excel. It outranks the Inspect file and the
  configuration. Not with `from_scan` or `all_collections`.
- `all_collections`: `True` reports every visible collection of each JSON file
  or workbook, one report per collection, named
  `<stem>.<collection-slug>.html`. See
  [Report Excel workbooks](https://docs.tabalyst.com/report/excel.md#every-table-of-a-workbook).
- `config_path`, `force`, `on_progress`: see the [conventions](https://docs.tabalyst.com/reference/python-api.md#conventions).
- `details`: also write one HTML page per column.
- `from_scan`: the sources are scan documents written by `tabalyst scan`; the
  reports are built without reading the sources again. Not with `separator`,
  `encoding`, `collections`, `all_collections` or `workers`.
- `workers`: number of analysis processes; `1` for one. Files of 16 MiB or more
  use one per spare processor by default.

An option that a source cannot use is ignored and listed in `batch.plan.warnings`:

```python
batch = tabalyst.generate_reports(["sales.xlsx"], separator=";")
for source, notice in batch.plan.warnings:
    print(source, notice)
# sales.xlsx --delimiter is ignored: sales.xlsx is not a CSV file.
```

## tabalyst.analyze()

Analyzes one CSV file, writes its report, its sibling JSON profile and the
execution history, and returns the profile as a dictionary.

```python
tabalyst.analyze(
    csv_path,
    report_path,
    separator=None,
    encoding=None,
    config_path=None,
    force=False,
    details=False,
)
```

- `csv_path`: the CSV file.
- `report_path`: HTML filename ending in `.html`; the profile is written beside
  it with the extension `.json`.
- `separator`, `encoding`: override the delimiter and the text encoding.
- `config_path`, `force`: see the [conventions](https://docs.tabalyst.com/reference/python-api.md#conventions).
- `details`: also write one HTML page per column.

Raises the error of the analysis, such as an existing output without `force`.

## tabalyst.analyze_csv()

Scans a file and builds its profile without writing anything.

```python
tabalyst.analyze_csv(path, config=None, on_progress=None, workers=None)
```

Returns a `ReportProfile` model. `config` is a `tabalyst.ReportConfig`; its
`scan` setting is a `tabalyst.ScanConfig`.

## tabalyst.analyze_scan()

Builds the profile of a scan document written by `tabalyst scan`, without
reading its source again.

```python
tabalyst.analyze_scan(scan_path, config_path=None, on_progress=None)
```

Returns a `ReportProfile`. The presentation settings come from `config_path`.
The `scan` settings given there must be those of the document, and a source found
beside the document must not have changed since the scan; both raise a
`ConfigurationError` otherwise.

## tabalyst.render_report()

Renders the HTML of a report.

```python
tabalyst.render_report(profile, *, column_links=None)
```

Returns the HTML as a string. `profile` is a `ReportProfile`, such as the one
`analyze_csv()` returns. `column_links` maps `(dataset id, column id)` to the URL
of a column page.

```python
profile = tabalyst.analyze_csv("customers.csv")
html = tabalyst.render_report(profile)
```

---

# Python API: scans

> The Python functions that scan a CSV, JSON, JSONL or Excel source - scan and generate_scans - with their parameters, return values and examples.

Source: https://docs.tabalyst.com/reference/python-api/scan/ (Tabalyst 0.6.2)

`tabalyst.scan()` scans one source in memory and writes nothing.
`tabalyst.generate_scans()` scans several sources and writes their scan
documents. The command is [`tabalyst scan`](https://docs.tabalyst.com/reference/cli/scan.md); the result is
described in the [scan format](https://docs.tabalyst.com/scan/format.md).

```python
import tabalyst

scan = tabalyst.scan("orders.json")
scans = tabalyst.generate_scans(["data/*.json"], output_dir="scans")
sheet = tabalyst.scan(
    "sales.xlsx",
    config=tabalyst.ScanConfig(excel={"dataset_path": "$.Orders"}),
)
```

## tabalyst.scan()

Reads one source completely and returns its scan result. It does not read an
Inspect file: choose the collection or the table with `config`.

```python
tabalyst.scan(
    source,
    *,
    config=None,
    registry=None,
    on_progress=None,
    on_record=None,
    workers=None,
)
```

Returns a `tabalyst.ScanResult`.

- `source`: the file to scan. A file ending in `.json` is read as JSON,
  `.jsonl` or `.ndjson` as JSONL, `.xlsx` or `.xlsm` as an Excel workbook, and
  any other as CSV.
- `config`: a `tabalyst.ScanConfig` that changes the settings, such as
  `ScanConfig(excel={"dataset_path": "$.Orders"})` for a table or
  `ScanConfig(json={"collections": ["$.customers[]"]})` for JSON collections. See
  [Configuration](https://docs.tabalyst.com/reference/configuration.md).
- `workers`: number of processes that analyze values; `1` scans in this process.
  By default, large sources use one per spare processor. The result does not
  depend on it.
- `on_progress`: see the [conventions](https://docs.tabalyst.com/reference/python-api.md#conventions).
- `registry`, `on_record`: advanced. `registry` replaces the detectors (and scans
  in this process); `on_record` receives each analyzed record, in reading order,
  and must not modify it.

## tabalyst.generate_scans()

Scans several sources and writes one scan document each. It applies Inspect to
JSON, JSONL and Excel sources, as the command does.

```python
tabalyst.generate_scans(
    input_specs,
    *,
    output=None,
    output_dir=None,
    config_path=None,
    delimiter=None,
    encoding=None,
    collections=None,
    force=False,
    on_progress=None,
    workers=None,
    project_storage=False,
)
```

Returns a [batch result](https://docs.tabalyst.com/reference/python-api.md#batch-functions) whose successes hold the
`ScanResult` in `result`. A successful job also has `warnings`.

- `input_specs`: files or non-recursive glob patterns.
- `output`: scan filename ending in `.json`, for a single source.
- `output_dir`: folder for standalone scans named `<stem>.scan.json`.
- `delimiter`, `encoding`: CSV delimiter and text encoding.
- `collections`: JSON collections as absolute paths such as
  `["$.customers[]"]`, or the one Excel table such as `["$.Sales"]`.
- `project_storage`: `True` stores the scans in the reusable storage of the
  command, as `tabalyst scan` does without `-o` or `-d`, instead of writing
  standalone documents.
- `config_path`, `force`, `on_progress`: see the [conventions](https://docs.tabalyst.com/reference/python-api.md#conventions).
- `workers`: as for `scan()`.

A source whose collection or table cannot be decided fails alone with a
`ConfigurationError`; an option that a source cannot use is listed in
`batch.plan.warnings`.

---

# Python API: Inspect

> The Python functions that inspect a JSON, JSONL or Excel file - inspect and generate_inspections - with their parameters, return values and examples.

Source: https://docs.tabalyst.com/reference/python-api/inspect/ (Tabalyst 0.6.2)

`tabalyst.inspect()` inspects one source and `tabalyst.generate_inspections()`
several. Each writes the visible Inspect file, `<source>-inspect.json`, beside
its source. The command is [`tabalyst inspect`](https://docs.tabalyst.com/reference/cli/inspect.md); the file is
described in the [Inspect format](https://docs.tabalyst.com/inspect/format.md).

```python
import tabalyst

inspection = tabalyst.inspect("orders.json")
inspections = tabalyst.generate_inspections(["data/*.json", "logs/*.jsonl"])
workbook = tabalyst.inspect("sales.xlsx")
print(inspection.document.detection.selection.path)
```

## tabalyst.inspect()

```python
tabalyst.inspect(
    source,
    *,
    config_path=None,
    reset_config=False,
    force=False,
    on_progress=None,
)
```

Returns an `InspectResult` with `source`, `path` (the Inspect file that was
written) and `document` (what it holds, including `detection`, `warnings` and
`config`). Raises the error of the inspection.

- `source`: a JSON, JSONL, NDJSON or Excel (`.xlsx`, `.xlsm`) file.
- `config_path`: the `scan` section of these files seeds the `config` of a new
  Inspect file. See the [conventions](https://docs.tabalyst.com/reference/python-api.md#conventions).
- `reset_config`: replace the `config` of an existing Inspect file by the
  detected one. Without it, an existing file keeps its `config`.
- `force`: replace an existing Inspect file that cannot be kept.
- `on_progress`: see the [conventions](https://docs.tabalyst.com/reference/python-api.md#conventions).

## tabalyst.generate_inspections()

```python
tabalyst.generate_inspections(
    input_specs,
    *,
    config_path=None,
    reset_config=False,
    force=False,
    on_progress=None,
)
```

Returns a [batch result](https://docs.tabalyst.com/reference/python-api.md#batch-functions) whose successes hold an
`InspectResult` each. It lists the error of a source among the failures, where
`inspect()` raises it. Parameters are those of `inspect()`, with `input_specs`
for files or non-recursive glob patterns.

A source with several equally plausible collections is not a failure: its
`document.detection.selection.path` is `None` and
`document.detection.selection.basis` is `"ambiguous"`. Choose one with the
`collections` argument of `generate_reports()` or `generate_scans()`.

---

# Python API: sampling

> The Python functions that write a smaller CSV file - sample_csv and generate_samples - with their parameters, return values and examples of first, last, random and stratified samples.

Source: https://docs.tabalyst.com/reference/python-api/sample/ (Tabalyst 0.6.2)

`tabalyst.sample_csv()` samples one CSV file and `tabalyst.generate_samples()`
several. Sampling reads CSV files only. The command is
[`tabalyst sample`](https://docs.tabalyst.com/reference/cli/sample.md).

```python
import tabalyst

sample = tabalyst.sample_csv(
    "customers.csv", method="random", rows=1000, seed=42
)
print(sample.output, sample.sample_rows)
batch = tabalyst.generate_samples(
    ["*.csv"], method="first", rows=100, output_dir="samples"
)
```

## tabalyst.sample_csv()

```python
tabalyst.sample_csv(
    source,
    *,
    method,
    rows=None,
    percent=None,
    field=None,
    seed=None,
    output=None,
    delimiter=",",
    encoding="utf-8-sig",
    force=False,
)
```

Returns a `tabalyst.SampleResult` with `source`, `output`, `method`,
`source_rows`, `sample_rows`, `sample_rate`, `field`, `seed`, `encoding` and
`delimiter`.

- `source`: the CSV file.
- `method`: `"first"`, `"last"`, `"random"` or `"stratified"`, or a
  `tabalyst.SampleMethod`.
- `rows`, `percent`: the size of the sample, as a number of data rows or a
  percentage rounded up to a whole row. Give exactly one.
- `field`: the field that defines the strata; required by `"stratified"`.
- `seed`: makes a random or stratified selection reproducible.
- `output`: the output filename; by default `<stem>.sample.csv` beside the source.
- `delimiter`, `encoding`: how the CSV file is read and written.
- `force`: replace an existing sample file. A source is never replaced.

## tabalyst.generate_samples()

```python
tabalyst.generate_samples(
    input_specs,
    *,
    method,
    rows=None,
    percent=None,
    field=None,
    seed=None,
    output=None,
    output_dir=None,
    delimiter=None,
    encoding=None,
    config_path=None,
    force=False,
)
```

Returns a [batch result](https://docs.tabalyst.com/reference/python-api.md#batch-functions) whose successes hold a
`SampleResult` each. `input_specs` are CSV files or non-recursive glob patterns,
`output` names the file of a single source and `output_dir` the folder of the
samples, named `<stem>.sample.csv`. `delimiter` and `encoding` default to the
configuration files in `config_path`, then to the built-in defaults. The other
parameters are those of `sample_csv()`.

---

# Types and patterns

> Understand the native, inferred and semantic types and the value patterns shared by Tabalyst Scan and Report.

Source: https://docs.tabalyst.com/reference/types-and-patterns/ (Tabalyst 0.6.2)

**Tabalyst Scan** reads source values, infers a technical type for each field
and runs detectors for possible meanings. **Tabalyst Report** presents the
same results as an inferred type, an optional semantic type and a table of
detected formats. These are separate answers to three questions:

| Question | Result | Example |
| --- | --- | --- |
| How was the value stored? | Native source type | `"12.5"` is a CSV string; a JSON `12.5` is a native number. |
| What kind of values does the field contain? | Inferred technical type | Strings such as `"12.5"` can give a field inferred type `number`. |
| What recognizable meaning or form do the values have? | Semantic type and detector evidence | A `text` field of email addresses can have semantic type `email`. |

Read [Types](https://docs.tabalyst.com/reference/types-and-patterns/types.md) for every native, inferred and semantic type and the
selection rules. Read [Recognized patterns](https://docs.tabalyst.com/reference/types-and-patterns/patterns.md) for every built-in
detector, configured patterns, format labels and coverage states. The
[report guide](https://docs.tabalyst.com/report/read-report.md) explains the HTML tables; the
[Scan format](https://docs.tabalyst.com/scan/format.md) and
[Report JSON profile](https://docs.tabalyst.com/report/json-profile.md) give the machine-readable
fields.

---

# Native, inferred and semantic types

> Define every source, inferred and semantic type and how Tabalyst Scan and Report select them.

Source: https://docs.tabalyst.com/reference/types-and-patterns/types/ (Tabalyst 0.6.2)

**Tabalyst Scan** records each field's native types, infers a technical type
from present values and chooses a primary interpretation only when one
detector qualifies. **Tabalyst Report** calls the technical type the
**inferred type** and displays the optional **semantic type** in its
[Columns table](https://docs.tabalyst.com/report/read-report.md#columns).

## Native source types

`native_types` in a Scan field counts values as read, before technical
inference. CSV cells start as strings; JSON and Excel can carry typed values.
The native type alone does not decide the inferred type.

| Native type | Source value |
| --- | --- |
| `string` | Text, including text that may later parse as a number or date. |
| `integer`, `number` | A source number, with `integer` for whole numbers. |
| `boolean` | A source true/false value. |
| `null` | An explicit JSON null. |
| `object`, `array` | JSON containers; they can form paths or stay whole at a flatten limit. |

See `native_types` and `missing` in the [Scan field](https://docs.tabalyst.com/scan/format.md#fields)
to distinguish source representation from missing-value rules.

## Inferred types

The technical type is chosen from **present** values, after missing values
are removed. Native scalar types contribute directly. Strings are classified
with date, boolean and number rules. Tabalyst chooses the first family that
reaches `scan.types.minimum_confidence`: `integer`, `number` (which also
accepts integers), `date`, `boolean`, then `text`. Otherwise it chooses
`mixed`. The report records `type_counts`, `type_confidence` and
`type_error_count`; its **Analysis settings** gives the effective threshold.

| Inferred type | When it appears | Important distinction |
| --- | --- | --- |
| `empty` | No present analyzable value. | A missing-only field is not inferred as text. |
| `complex` | JSON objects or arrays kept whole at the configured flatten depth. | Their contents are not scalar columns at that depth. |
| `boolean` | Native booleans or technical true/false strings meet the threshold. | The separate boolean detector can recognize more configured word pairs. |
| `integer` | Whole-number values meet the threshold. | A leading-zero identifier stays text. |
| `number` | Numeric values, including integers and non-integers together, meet the threshold. | Configured decimal conventions decide which strings parse. |
| `date` | Calendar dates, including ambiguous day/month forms, meet the threshold. | Date-times and times can match the date detector without becoming calendar-date technical values. |
| `text` | Remaining text values meet the threshold. | Text can still have a semantic type such as `email`. |
| `mixed` | No family reaches the threshold. | No type error count is available because no type was accepted. |

**Error** in the report counts present values outside the accepted inferred
type. Missing values are excluded from its denominator. `mixed` reports the
largest family as confidence but has no type error count.

## Semantic types

Detectors become candidate interpretations when their matched share of
eligible values reaches `scan.detection.minimum_share`. A field gets a
**primary** interpretation only when exactly one detector qualifies. The
report shows `date` for an inferred date field; otherwise it shows the unique
primary detector ID when it adds meaning beyond a technical type. Technical
`number` and `boolean` are not repeated as semantic types. A dash in the
report means no single qualifying meaning, even when detectors have matches.

| Semantic type | Meaning | Scope or condition |
| --- | --- | --- |
| `date` | Date, date-time or time syntax. | A calendar-date inferred field is `date`; the date detector can also be primary for another field. |
| `enumeration` | A bounded set of recurring values. | Field-level rule: enough present values and no more than the configured distinct maximum. |
| `email` | Bare email address syntax. | Does not verify mailbox or domain ownership. |
| `url` | URL syntax and host. | Does not visit the URL or resolve its host. |
| `phone` | Supported phone syntax. | Does not check assigned numbers. |
| `postal_code` | Canadian or US postal-code syntax. | Does not check an assigned address. |
| `currency` | Number with currency symbol or code. | A bare number is not currency evidence. |
| `percentage` | Number followed by `%`. | The parsed value remains as written, such as 12 for `12%`. |
| `quantity` | Number followed by a unit-shaped suffix. | Units are not validated as physical units. |
| `uuid` | UUID text form. | Does not verify that an ID was issued. |
| `ip_address` | IPv4 or IPv6 text form. | Does not check allocation or reachability. |
| `pattern:<id>` | Full-value match to a configured regular expression. | Exists only for a configured pattern. |

Two qualifying detectors do not produce an arbitrary semantic type. Inspect
all their evidence in [Recognized patterns](https://docs.tabalyst.com/reference/types-and-patterns/patterns.md) and in the report's
**Detectors and formats** table.

## Measures by value family

Scan retains measures independently of its final technical type. For example,
a `mixed` field can still have numeric evidence for the values that parsed as
numbers. Report shows its **Numeric analysis** table only for inferred
`integer` and `number` fields with usable statistics, and its **String
analysis** table only for inferred `text` fields. The `--details` column page
also exposes a bounded selection of Scan measures. Always check the measure's
population and status before comparing it with a column's total rows.

### All fields

| Measure | Population and meaning | Where to read it |
| --- | --- | --- |
| Present and missing | Values found at a field versus missing values or absent JSON members under a parent. | Scan `presence` and `missing`; Report **Columns → Missing**. |
| Distinct values | Different present analytical values after enabled Unicode composition, trim and whitespace collapse. | Scan `values.cardinality` and normalization stages; Report **Columns → Distinct**. |
| Frequencies and examples | Occurrence counts for listed values and a bounded sample of distinct values. | Scan `values.frequencies` and `samples`; Report **Examples** and its tooltip. |
| Type confidence and error | Share of present values in the accepted family and count outside it. | Scan `technical_type`; Report **Inferred type** and **Error**. |
| Variant groups | Raw spellings that share a final comparison key after enabled case and accent stages. | Scan `normalization`; Report **Transformations**. |

Distinct counts and frequency lists can be limited; examples are a selection,
not necessarily a complete distribution. A `mixed` field has no type error
count because no family was accepted.

### Numeric values

The Scan `numeric` block measures every accepted numeric value in the field,
including native numbers and text parsed under the configured number rules.
Its `count`, `native_count` and `text_count` identify this population. Report
shows five summary measures for inferred numeric fields:

| Measure | Definition | Availability |
| --- | --- | --- |
| Minimum, Maximum | Smallest and largest accepted numeric value. | Scan and Report. |
| Range | Maximum minus minimum. | Derived in Report from the two bounds. |
| Mean | Sum divided by numeric count. | Scan and Report. |
| Median | Middle numeric value, averaging the two middle values when needed. | Scan measure; `null` in Report when its tracking limit was reached. |
| Sum | Total of accepted numeric values. | Scan and the standalone Report column page. |
| Population variance, standard deviation | Spread around the mean, using the numeric population. | Scan and the standalone Report column page. |
| Positive, negative, zero | Counts of numeric occurrences greater than, less than, or equal to zero. Missing and nonnumeric values are excluded. | Scan and the standalone Report column page. |
| Integral decimals | Decimal values whose numeric value is whole. | Scan and the standalone Report column page. |

The numeric population can be smaller than the field's present-value count if
some values did not parse as numbers. In the Report table, **Distinct** and
**Examples** describe normalized values, not the numeric calculation.
When numeric statistics are complete, `positive + negative + zero` equals
`numeric.count`. For example, `10`, `-2.5`, `0`, `4.5`, and `1.0` yield
3 positive, 1 negative, and 1 zero value. A precision limit can make the
numeric measure unavailable instead of showing partial sign counts.

### Text values and lengths

Scan measures the length of each present **analytical string** after enabled
Unicode composition, trim and internal whitespace collapse. Report's string
profile uses this measure only for an inferred `text` field.

| Measure | Definition | Availability |
| --- | --- | --- |
| Minimum, Maximum length | Shortest and longest observed analytical string, in characters. | Scan `string_lengths`; Report **Min** and **Max**, or **Fixed** if equal. |
| Mean, Median length | Average and middle character count, weighted by occurrences. | Scan and Report; Report leaves these cells blank when **Fixed** already gives the one length. |
| Fixed length | Minimum equals maximum; the shared length is displayed. | Derived in Report, not a claim about the field's meaning. |
| Length distribution | Exact occurrence count for each observed length. | Scan histogram; Report **Lengths** only when the maximum fits `string_analysis.length_distribution_max_length`. |
| Distinct lengths | Number of different lengths observed. | Report JSON profile even when the HTML length distribution is omitted. |
| Length class | `very_short`, `short`, `medium`, `long` or `very_long`, chosen from the maximum length and configured thresholds. | Report **Class**. |

Length examples come from bounded listed or sampled values. **Distinct** in
the String analysis table counts values, not lengths. Scan also records string
characteristics such as case, non-ASCII content, line breaks and whitespace.

### Dates and times

The date detector separates unambiguous matches, ambiguous readings, invalid
date-shaped values and other present values. Report's **Date analysis** table
appears when it finds date evidence, even if the field's inferred type is not
`date`.

| Measure | Definition | Availability |
| --- | --- | --- |
| Valid, ambiguous, invalid, other | Counts and percentages of eligible present values by outcome. | Report **Date analysis** and detector coverage in Scan. |
| Formats | Counts for each recognized format, including numeric orders, date-times and times. | Scan detector and Report **Formats**. |
| Ambiguity evidence | Unambiguous values per DMY and MDY order; evidence is shown but not applied to ambiguous values. | Scan detector details and Report date profile. |
| Temporal range | Minimum and maximum per kind: `date`, naive date-time, aware date-time or `time`. Aware and naive values are not compared together. | Scan `temporal` and the standalone Report column page. |
| Years | Count per year when applicable to a temporal kind. | Scan `temporal` and the standalone Report column page. |

Only `scan.detectors.date.ambiguous_order` can explicitly resolve a supported
ambiguous day/month order. Ambiguous values do not contribute to the temporal
minimum and maximum.

### Boolean values and other meanings

Scan `booleans` counts native and recognized technical true/false values
separately as `true` and `false`; the standalone Report column page displays
those counts. The configurable boolean detector can additionally recognize
word pairs such as `yes/no` and `oui/non`; its matched counts and formats are
detector evidence, not the same as the technical boolean counts.

For `email`, `phone`, `postal_code`, `url`, `currency`, `percentage`,
`quantity`, `uuid`, `ip_address`, `enumeration` and configured patterns, the
shared measures are detector coverage, format counts and bounded examples.
Some detectors add details such as domains, hosts, units or UUID versions.
Those details are in [Recognized patterns](https://docs.tabalyst.com/reference/types-and-patterns/patterns.md) and the
[Scan detector result](https://docs.tabalyst.com/scan/format.md#detectors); they do not imply
external verification.

### Limits and exposure

Scan wraps measures with a `status`: `complete`, `limited`, `not_applicable`,
`disabled` or `failed`. A limited measure is never estimated; a lower bound,
when present, is only a proven minimum. Sensitive fields can have numeric or
temporal measures disabled under masking or hiding, while counts remain
available. See [measure envelopes](https://docs.tabalyst.com/scan/format.md#measure-envelopes)
and [exposure settings](https://docs.tabalyst.com/reference/configuration.md#sensitive-values-and-sampling).

---

# Recognized patterns

> Catalog every built-in Tabalyst detector, its accepted forms, configured patterns and evidence states.

Source: https://docs.tabalyst.com/reference/types-and-patterns/patterns/ (Tabalyst 0.6.2)

**Tabalyst Scan** applies detectors to eligible field values and records
matched, ambiguous, invalid, unmatched and untested counts. **Tabalyst
Report** shows detectors with evidence in **Detectors and formats**. A pattern
match means the value fits the detector's rule; it does not prove that an
address, phone number or identifier exists in an external system. The
[Types page](https://docs.tabalyst.com/reference/types-and-patterns/types.md) explains how one detector can become a semantic type.

## Built-in detectors

The table lists every built-in detector ID. Its examples show representative
forms, not the complete grammar. Defaults can change through
[`scan.detectors`](https://docs.tabalyst.com/reference/configuration.md#detectors). A field can have evidence
from several detectors at once.

| Detector ID | What is recognized | Representative forms and limits |
| --- | --- | --- |
| `number` | Integers, decimals and scientific notation under configured dot/comma conventions. | `42`, `12.5`, `12,5`, `1e3`; `1,234` may be ambiguous until a convention is set. |
| `date` | Numeric dates, ISO date-times, times and English/French month-name dates. | `2026-10-06`, `02/03/2026`, `2026-10-06T12:30Z`, `13:45`, `26 September 2026`. Orders and separators are configurable; impossible dates are invalid, and ambiguous DMY/MDY values stay unresolved by default. |
| `boolean` | Configured pairs of textual truth values. | Defaults: `true/false`, `yes/no`, `y/n`, `oui/non`, `vrai/faux`; native JSON booleans are already technical values. |
| `enumeration` | A low-cardinality field, considered as a whole. | Defaults: at least 500 eligible values and at most 49 distinct analytical values; case-sensitive by default. |
| `email` | Bare `local@domain` with a dot-atom local part and valid domain syntax. | `name@example.com`; display names and addresses embedded in URLs do not match. Local part is limited to 64 characters. |
| `url` | Absolute URLs with configured schemes, or `www.` addresses. | Defaults: `http`, `https`, `ftp` and `www.`; syntax, host and port are checked without network access. |
| `phone` | North American (`nanp`) and French (`fr`) phone forms. | Defaults include national and supported international prefixes; punctuation and leading digits are checked, but assigned ranges are not. |
| `postal_code` | Canadian postal codes and US ZIP / ZIP+4. | Defaults: `ca`, `us`; `K1A 0B1`, `12345`, `12345-6789`; valid shape does not prove an assigned postal code. |
| `currency` | Number with supported currency symbol or ISO code before or after it. | `$12.50`, `12,50 EUR`; number conventions still apply. |
| `percentage` | Signed number with optional space and `%`. | `12%`, `-2.5 %`; the numeric part is not divided by 100. |
| `quantity` | Number followed by a bounded unit-shaped suffix. | `10m`, `10 Go`, `90 km/h`; currency codes and ordinal endings are excluded, and units are not semantically validated. |
| `uuid` | Hyphenated, braced or `urn:uuid:` 128-bit hexadecimal form. | Case variants are recorded; version and variant are summarized. |
| `ip_address` | IPv4 dotted decimal and IPv6 text, including compressed and embedded-IPv4 forms. | Invalid octets, leading zeros, compression and group counts can be reported. |

`email`, `phone` and `ip_address` are sensitive detectors. When they match a
field, its examples, value profile and preview are masked by default. A
configured pattern can also be sensitive. `postal_code` is not sensitive by
default. See [exposure settings](https://docs.tabalyst.com/reference/configuration.md#sensitive-values-and-sampling).

## Configured patterns

`scan.patterns` adds organization-specific detectors named `pattern:<id>`.
The Python regular expression must match the **whole** value. For example:

```json
{
  "scan": {
    "patterns": [
      {
        "id": "customer_number",
        "regex": "C-\\d{4}",
        "description": "Customer number",
        "accepts": ["string"],
        "sensitive": false,
        "max_input_length": 256
      }
    ]
  }
}
```

This recognizes `C-0042` but cannot validate that the customer exists.
`accepts` selects native types (`string`, `integer`, `number`, `boolean`);
`sensitive` controls exposure when the pattern matches; and
`max_input_length` skips longer values. The
[pattern settings](https://docs.tabalyst.com/reference/configuration.md#patterns) specify limits and validation.

## Reading detector evidence

| Result | Meaning | Practical reading |
| --- | --- | --- |
| Matched | The detector recognized the value. | It contributes to a possible semantic-type interpretation. |
| Ambiguous | Several supported readings remain. | It is separate from an unambiguous match; date and number settings can resolve some cases. |
| Invalid | The value has a relevant shape but impossible or disallowed content. | Inspect the value and rule; this is separate from a column type error. |
| Not matched | The detector tested the value and its rule did not fit. | Another detector may still match. |
| Not tested | The value was skipped, for example after adaptive warm-up. | Do not read its match percentage as a complete test of all present values. |
| Failed | Detector processing failed for the field. | The scan diagnostic gives the reason. |

In Report, **Matched** is a count and a percentage of eligible values;
**Ambiguous** and **Invalid** are separate counts. **Formats** gives syntax
labels and counts, which vary by detector and configuration. Adaptive
detection can skip values after its warm-up, while `number` and `date` always
test every eligible value. The [Scan format](https://docs.tabalyst.com/scan/format.md#detectors)
records every detector's complete coverage and bounded evidence; the
[Report JSON profile](https://docs.tabalyst.com/report/json-profile.md#detectors) retains its
exposed result. The HTML detector table only lists detectors with evidence or
a failure.

---

# Configuration

> Reference for the JSON configuration file read by tabalyst report, tabalyst sample, tabalyst scan and the Python API.

Source: https://docs.tabalyst.com/reference/configuration/ (Tabalyst 0.6.2)

Tabalyst reads strict JSON configuration files. JSON does not allow comments;
this page is the descriptive reference for supported settings. Commands load
the files passed with `--config`, and the Python API those passed with
`config_path`; no file is loaded automatically. `tabalyst.json` is only a
conventional name.

One file can configure every command. The `scan` object holds the analysis
settings, read by `tabalyst scan` and `tabalyst report`, described in
[Scan settings](#scan-settings). The top level holds the presentation settings
of the report and the CSV settings of `tabalyst sample`:

```json
{
  "string_analysis": {"examples_per_length": 5},
  "csv": {"delimiter": ";"},
  "scan": {
    "csv": {"delimiter": ";"},
    "values": {"null_markers": ["N/A"]},
    "records": {"preview": 20}
  }
}
```

Every command validates the whole file, including the sections it does not
use, so a misspelled or unknown setting is always an error. Explicit
command-line values, such as `--delimiter` and `--encoding`, are the final
override.

## Report settings

The report is built on Tabalyst Scan: it reuses a current stored scan when
possible, or reads the CSV with the effective `scan` settings, then presents
the result with the settings below.

```json
{
  "string_analysis": {
    "very_short_max_length": 5,
    "short_max_length": 20,
    "medium_max_length": 50,
    "long_max_length": 255,
    "length_distribution_max_length": 50,
    "examples_per_length": 10
  },
  "value_examples": {
    "full_distribution_max_distinct": 50,
    "short_text_max_length": 20,
    "short_text_percentile": 0.95,
    "short_text_result_size": 20,
    "long_text_result_size": 20,
    "long_text_truncate_at": 30,
    "truncation_suffix": "...",
    "inline_display_size": 3
  }
}
```

### Preview

The report shows the preview of the scan: its first `scan.records.preview`
records, 10 by default (see [Records](#records)). It never limits analysis of
the complete file. Values of sensitive columns are masked or hidden in the
preview as in the rest of the report (see
[How the report uses the scan settings](#how-the-report-uses-the-scan-settings)).

### String analysis

Present values in `text` columns are classified from their maximum length as
`very_short`, `short`, `medium`, `long` or `very_long`. A fixed length is
recorded separately when minimum and maximum match. Missing values do not
contribute to lengths. Mean and median length are calculated for every text
column; the report leaves them blank when the fixed length already conveys the
same information.

When the column maximum is at most `length_distribution_max_length`, JSON also
contains the exact occurrence count of every observed length. Each length lists
up to `examples_per_length` example values, taken from the most frequent values
of the column, then from its sampled values. `distinct_length_count` remains
available for every text column, including columns whose distribution is
omitted. The report orders length groups by occurrence and shows three examples
per length in its String analysis tooltip.

### Examples and value profiles

- `full_distribution_max_distinct`: through this limit, every present distinct
  value and its occurrence count are retained in JSON. At `50`, the
  distribution is still complete; at `51`, Tabalyst selects values among a
  reproducible sample of the distinct values, whose size is
  `scan.limits.max_samples` and whose seed is `scan.random_seed`.
- `short_text_max_length` and `short_text_percentile`: a column is considered
  short text when this percentile of its sampled lengths does not exceed the
  configured length.
- `short_text_result_size`: **number of values retained for the tooltip when a
  short-text column has more than 50 distinct values.** Its default is `20`.
- `long_text_result_size`: equivalent limit for long text. Its default is also
  `20`.
- `long_text_truncate_at` and `truncation_suffix`: display limit for retained
  long-text values in both JSON and the report.
- `inline_display_size`: number of values shown directly in the `Examples` cell.
  They are always the most frequent retained values.

The `Examples` cell shows `+N` for a complete distribution. For a sampled profile,
it shows `++`; the tooltip then shows, for example, `20 / 2,992`, meaning retained
values / actual distinct values. Tooltip values are sorted by descending occurrence
count. For a sensitive column, the cell shows `masked` or `hidden` instead, and
the tooltip lists masked values.

### How the report uses the scan settings

| Report behavior | Scan settings |
| --- | --- |
| Reading the CSV | `scan.csv`, `--delimiter`, `--encoding` |
| Missing cells | `scan.values`: `null_markers`, `null_markers_case_sensitive` and `missing` |
| Normalization counts, distinct values and variant groups | `scan.normalization`: `nfc`, `trim`, `collapse_whitespace`, `casefold`, `strip_accents` |
| Inferred types | `scan.types.minimum_confidence`, with the `number` and `date` detectors |
| Date analysis | `scan.detectors.date` |
| Semantic types | `scan.detection.minimum_share` and every detector |
| Masked values | `scan.exposure.sensitive_values` |
| Preview | `scan.records.preview` |
| Duplicate rows | `scan.records.duplicates` and `scan.limits.max_tracked_records` |
| Limits and diagnostics | `scan.limits` and `scan.errors` |

- **Dates.** A column whose present values are dates is `date`, even with
  several formats or ambiguous values. Ambiguous values such as `02/03/2025`
  are never resolved from other values of the column: the report counts them
  as ambiguous, shows how many unambiguous values use each order and, when only
  one order appears, suggests setting `scan.detectors.date.ambiguous_order`.
  The `ambiguous_dates` issue counts them.
- **Semantic types.** `date` for date columns, otherwise the scan's primary
  interpretation: the only detector matching at least
  `scan.detection.minimum_share` of the present values, such as `enumeration`,
  `email`, `phone` or `postal_code`. `number` and `boolean` are not repeated as
  semantic types.
- **Sensitive values.** Columns where a sensitive detector matches, such as
  email addresses and phone numbers, are masked by default in examples, value
  profiles and the preview: `jane@example.com` becomes `aaaa@aaaaaaa.aaa`. Set
  `scan.exposure.sensitive_values` to `show` to keep the values, or `hide` to
  remove them.
- **Limits.** When a scan limit stops a measure, such as the distinct values
  of a column with more than `scan.limits.max_distinct_per_field` of them, the
  report shows `limited` instead of a number and lists the column in the
  `limited_measures` issue.
- **Duplicate rows.** With more distinct rows than
  `scan.limits.max_tracked_records`, the duplicate count is a lower bound,
  shown as `≥ 12`. With `scan.records.duplicates` set to `false`, duplicates
  are not checked and the report shows `–`.

The report reads the `scan.csv` settings, not the top-level `csv` settings. When
a file sets both with different values, the report stops with an error, since
the file would otherwise be read with settings it does not expect; a value
given with `--delimiter` or `--encoding` always wins. The report also needs
every column: a CSV wider than `scan.limits.max_fields` is an error.

A report built from a scan document with `tabalyst report --scan` uses the
settings recorded in the document. Its configuration files still give the
presentation settings; settings given in their `scan` object must have the
values recorded in the document, otherwise the report stops with an error (see [Report from a scan](https://docs.tabalyst.com/scan/files.md#report-from-a-scan)).

### Settings moved to `scan`

These top-level settings are rejected with a message naming their new location:

| Former setting | New location |
| --- | --- |
| `missing_values` | `scan.values.null_markers` for markers such as `"N/A"`; empty and whitespace-only cells are missing by default (`scan.values.missing`) |
| `normalization.trim` | `scan.normalization.trim` |
| `normalization.collapse_internal_whitespace` | `scan.normalization.collapse_whitespace` |
| `date_detection` | `scan.detectors.date` |
| `type_inference.minimum_confidence` | `scan.types.minimum_confidence` |
| `enum_detection` | `scan.detectors.enumeration`: `maximum_distinct_values` is `maximum_distinct`, and `minimum_row_count` is `minimum_values`, which counts present values, not rows |
| `value_examples.candidate_sample_size` | `scan.limits.max_samples` |
| `value_examples.random_seed` | `scan.random_seed` |
| `preview_rows` | `scan.records.preview`, now up to 1,000 |

## Sample settings

```json
{
  "csv": {
    "encoding": "utf-8-sig",
    "delimiter": ","
  }
}
```

- `csv.encoding`: source-file encoding for `tabalyst sample`. The default
  accepts UTF-8 with or without a BOM; `cp1252` is useful for some
  Windows-produced files.
- `csv.delimiter`: single-character separator, a comma by default.

## Scan settings

`tabalyst scan`, `tabalyst report` and their Python functions read the `scan`
object of each configuration file. The complete object with its defaults is:

```json
{
  "scan": {
    "csv": {"encoding": "utf-8-sig", "delimiter": ","},
    "json": {
      "collections": null, "discovery_max_depth": 3,
      "flatten": {"enabled": true, "separator": ".", "max_depth": null},
      "arrays": {"mode": "preserve"}
    },
    "excel": {"dataset_path": null, "header_row": null},
    "errors": {"policy": null, "max_locations": 10},
    "values": {
      "null_markers": [],
      "null_markers_case_sensitive": true,
      "missing": ["absent", "null", "empty", "blank", "marker"]
    },
    "normalization": {
      "nfc": true, "trim": true, "collapse_whitespace": true,
      "casefold": true, "strip_accents": true
    },
    "limits": {
      "max_fields": 10000, "max_depth": 64, "max_record_observations": 100000,
      "max_line_bytes": 16777216,
      "max_distinct_per_field": 100000, "max_tracked_values": 2000000,
      "max_stored_value_length": 1000, "max_listed_frequencies": 100,
      "max_samples": 100, "max_variant_groups": 100,
      "max_variants_per_group": 20, "max_evidence_examples": 10,
      "max_tracked_records": 2000000, "max_listed_records": 10
    },
    "records": {"preview": 10, "duplicates": true},
    "types": {"minimum_confidence": 0.95},
    "detection": {
      "minimum_share": 0.95, "warmup_values": 10000, "probe_interval": 100,
      "rare_share": 0.001
    },
    "detectors": {"number": {"enabled": true}},
    "patterns": [],
    "exposure": {"sensitive_values": "mask"},
    "random_seed": 42
  }
}
```

Scans and reports do not read the top-level `csv` settings: set the delimiter
and encoding in `scan.csv`, or pass `--delimiter` and `--encoding`.

### Layers and merge rules

From lowest to highest priority: built-in defaults, for a JSON source with no
Inspect file the collection that [Inspect](https://docs.tabalyst.com/inspect/json.md)
detects, the `scan` object of each `--config` file in the order given, the
`config` of the [Inspect file](https://docs.tabalyst.com/inspect/format.md#config) beside the source, then
`--delimiter`, `--encoding` and `--collection`. For a workbook, the layers are
the same with `excel.dataset_path` in place of `json.collections`.

- Objects merge key by key, including `detectors.<id>`: a file that sets
  `{"detectors": {"number": {"enabled": false}}}` keeps the other number
  settings.
- Lists replace the previous list entirely, such as `null_markers`,
  `patterns` or `collections`.
- Unknown keys are errors.
- `null` is accepted only where a setting allows it; there is no syntax to
  delete a setting.

The scan document embeds the effective configuration and its SHA-256
(`config_sha256`). A stored scan is reused only when the source content, that
configuration and the version of Tabalyst are all the same.

### Sources and errors

- `csv.encoding`, `csv.delimiter`: as for reports. The first record is the
  header.
- `json.collections`: for JSON files, a list of absolute collection paths such
  as `"$.customers[]"` or `"$.customers[].orders[]"`. Paths must end with `[]`,
  be unique and not contain one another, and are stored in canonical spelling
  (`$["orders"][]` becomes `$.orders[]`). `null` means that no collection is
  named: `tabalyst scan` and `tabalyst report` then use the collection that
  [Inspect](https://docs.tabalyst.com/inspect/json.md) selects, or stop with exit code `2`
  when it cannot select one. Only `tabalyst.scan()` turns `null` into automatic
  discovery of every array. For a JSONL file it is `null` or `["$[]"]`. A file
  that lists `json.collections` is shared by sources of every format, so it is
  ignored for JSONL and Excel sources in a mixed batch.
- `excel.dataset_path`: for Excel files (`.xlsx`, `.xlsm`), the table to read: a
  sheet, `"$.Sales"`, or a named table of a sheet, `"$.Sales.Orders"`. A name that
  is not a plain identifier is quoted, `"$[\"Q1 2026\"]"`. It is stored in canonical
  spelling. `null` means that no table is named: `tabalyst scan` and
  `tabalyst report` then use the table that
  [Inspect](https://docs.tabalyst.com/inspect/excel.md) selects, or stop with exit code `2` when it
  cannot select one. `tabalyst.scan()` has no automatic mode: a workbook without
  `excel.dataset_path` is an error.
- `excel.header_row`: the 1-based row of a sheet that holds the header, instead
  of the detected one; `null` (default) detects it in the first 50 rows. A strict
  integer from 1.
- `json.discovery_max_depth`: how deep Inspect, and automatic discovery, look
  for arrays under a root object.
- `json.flatten`: how nested objects become fields. `enabled` (default `true`),
  `separator` (default `.`) joins the keys of a field name such as
  `address.city`; it is one character that is not a letter, digit, `_`,
  whitespace or one of `[ ] " \ $`. `max_depth` (default `null`, no limit) counts
  segments from the root of the record, a key or `[]` each: `address` is 1,
  `address.city` is 2, `orders[].amount` is 3. A container (object or array)
  at that depth is kept whole as a complex value and its content is not
  analyzed: nothing is lost, no warning is raised, and the report shows it as a
  column of type `complex`. `enabled: false` is a depth of 1. The separator
  changes only the names shown: a key holding it is written `["a/b"]`, and
  dataset paths such as `$.customers[]` always use `.`. This is not
  `limits.max_depth`, which protects the reader and truncates with a warning.
  Content below the flatten depth is not read for duplicate keys.
- `json.arrays` (`mode`, only `preserve`): arrays never add records. `ignore`
  and `explode` are refused.
- `errors.policy`: `null` (default) is the default of the source format,
  `strict` for CSV and JSON, `tolerant` for JSONL. `strict` stops at the first
  malformed record, such as a CSV record with the wrong number of fields, a JSON
  object with a duplicate key or a JSONL line that is not valid JSON. `tolerant`
  excludes such records, counts them and marks the scan `partial`. Invalid
  syntax in a JSON file, undecodable text and unreadable files stop the scan in
  both policies.
- `errors.max_locations`: how many record locations each diagnostic lists.

The `json.collections`, `json.flatten`, `json.arrays` and `errors.policy`
settings are the ones an [Inspect file](https://docs.tabalyst.com/inspect/format.md#config) edits, under
the names `structure.dataset_path`, `flatten`, `arrays` and `errors.policy`.
For a workbook, they are `excel.dataset_path` and `excel.header_row`, under the
names `structure.dataset_path` and `structure.header_row`.

### Values and missing values

- `values.null_markers`: strings that stand for a missing value, such as
  `"N/A"`, compared after trimming surrounding whitespace.
- `values.null_markers_case_sensitive`: set to `false` to match `n/a` too.
- `values.missing`: the categories counted as missing: `absent` (the key or
  column does not exist in a record), `null`, `empty` (`""`), `blank`
  (whitespace only) and `marker`. Every component stays listed in the result.

### Normalization

Each stage can be turned off. They run in this order: `nfc` (Unicode
composition), `trim`, `collapse_whitespace`, `casefold` and `strip_accents`.
The first three produce the analytical value used by listings and detectors;
all stages count the values they change and group variants that become equal,
such as `Montréal` and `MONTREAL`. Raw values are never modified.

### Limits

Limits bound memory and output size. A measure that reaches a limit is marked
`limited` in the result, never estimated. Values above the maximum are
rejected before the scan starts.

| Setting | Default | Maximum | When reached |
| --- | --- | --- | --- |
| `max_fields` | 10,000 | 1,000,000 | New field paths of a dataset are counted but not analyzed. |
| `max_depth` | 64 | 1,000 | Deeper JSON content is counted but not analyzed. |
| `max_record_observations` | 100,000 | 100,000,000 | The record is an error, or excluded under `tolerant`. |
| `max_line_bytes` | 16,777,216 (16 MiB) | 268,435,456 (256 MiB) | JSONL only: a longer line is not parsed. It is an error, or excluded under `tolerant`. The memory needed to parse a line is about 7 to 13 times its size. |
| `max_distinct_per_field` | 100,000 | 50,000,000 | Distinct values of the field stop being stored; frequencies and cardinality become limited. |
| `max_tracked_values` | 2,000,000 | 500,000,000 | Distinct values stored for the whole scan; the largest field is released first. |
| `max_stored_value_length` | 1,000 | 1,000,000 | Longer values are counted, not stored. |
| `max_listed_frequencies` | 100 | 100,000 | Most frequent values listed per field. |
| `max_samples` | 100 | 10,000 | Sample values listed per field. |
| `max_variant_groups` | 100 | 100,000 | Normalization variant groups listed per field. |
| `max_variants_per_group` | 20 | 10,000 | Variants listed per group. |
| `max_evidence_examples` | 10 | 1,000 | Example values per detector state. |
| `max_tracked_records` | 2,000,000 | 500,000,000 | Distinct records stored for the whole scan to find duplicates, about 80 bytes each; later records are still compared with the stored ones, so duplicate counts become lower bounds. |
| `max_listed_records` | 10 | 10,000 | Record numbers listed per records block. |

`max_distinct_per_field` cannot exceed `max_tracked_values`. Counters,
statistics and detector coverage stay exact after a limit is reached.

### Records

- `records.preview`: how many of the first records each dataset keeps in its
  preview, from 0 to 1,000. Sensitive values are exposed as elsewhere.
- `records.duplicates`: set to `false` to skip duplicate detection, which
  stores one digest per distinct record, up to `limits.max_tracked_records`.
  The duplicate count is then `disabled`.

### Types and interpretations

- `types.minimum_confidence`: share of values that must agree for a field to
  get a technical type such as `integer` or `date`, otherwise `mixed`.
- `detection.minimum_share`: share of the values that a detector must match to
  become a candidate interpretation of the field.
- `detection.warmup_values`: default 10,000, up to 1,000,000. The first
  distinct values of each field go through every detector. A detector that
  recognized none of them, not even as invalid or ambiguous, is then skipped
  for the other values of the field, which count as `not_tested` in its
  coverage. `number` and `date` are never skipped. A field with fewer
  distinct values is always analyzed by every detector. Set it to `0` to test
  every value with every detector, as before this setting existed.
- `detection.rare_share`: default 0.001, from 0 to 1. A detector that
  recognized at most this share of the warm-up values, and none of those in
  the second half of the warm-up, is skipped too: its reactions were rare and
  have stopped. The default allows 10 values of a 10,000-value warm-up. A
  detector still recognizing values at the end of the warm-up stays, as in a
  sorted column where matches become frequent, and so does a sensitive
  detector that found only invalid values. `0` skips only the detectors that
  recognized nothing.
- `detection.probe_interval`: default 100, up to 1,000,000. After the
  warm-up, about one distinct value in this number, chosen by a hash of the
  value, still goes through every detector. The scan reports a
  `detector_skipped_reacted` warning, meaning that the counts of a skipped
  detector for the field are incomplete, when it recognizes one of these
  probes after recognizing nothing in the warm-up, or, for a rare detector,
  when it recognizes at least 10 probed values, more often than
  `detection.rare_share` allows. `0` disables probes.

A detector skipped after the warm-up may miss rare values that appear later
in the file, including sensitive ones such as an email address in a comment
column: such a value would not be masked. Set `detection.warmup_values` to
`0` when every sensitive value must be found.

### Detectors

Every detector has `enabled` (default `true`) and the settings below. Other
detector ids are errors.

| Detector | Settings and defaults |
| --- | --- |
| `number` | `conventions`: `["dot", "comma"]`, the decimal separators accepted; `ambiguous_convention`: `null`, or `dot` or `comma` to read values such as `1,234` one way. |
| `date` | `orders`: `["YMD", "MDY", "DMY"]`; `separators`: `["-", "/", "."]`; `ambiguous_order`: `null`, or `MDY` or `DMY` to read values such as `02/03/2025` one way; `month_languages`: `["en", "fr"]`. |
| `boolean` | `pairs`: `[["true", "false"], ["yes", "no"], ["y", "n"], ["oui", "non"], ["vrai", "faux"]]`. |
| `enumeration` | `minimum_values`: 500; `maximum_distinct`: 49; `case_sensitive`: `true`. |
| `email` | `max_tracked_domains`: 10,000; `max_listed_domains`: 20. |
| `url` | `schemes`: `["http", "https", "ftp"]`; `www`: `true`, accept addresses starting with `www.`; `max_tracked_hosts`: 10,000; `max_listed_hosts`: 20. |
| `phone` | `regions`: `["nanp", "fr"]`; `nanp` covers Canada and the United States. |
| `postal_code` | `regions`: `["ca", "us"]`, Canadian postal codes and ZIP codes. |
| `currency` | `conventions` and `ambiguous_convention`, as for `number`. |
| `percentage` | `conventions` and `ambiguous_convention`, as for `number`. |
| `quantity` | `conventions` and `ambiguous_convention`, as for `number`; `max_tracked_units`: 10,000; `max_listed_units`: 20. |
| `uuid` | No other setting. |
| `ip_address` | `versions`: `["ipv4", "ipv6"]`. |

Ambiguous values are never resolved from other values of the field: only
`ambiguous_convention` and `ambiguous_order` resolve them.

### Patterns

`patterns` adds your own detectors, reported as `pattern:<id>`:

```json
{
  "scan": {
    "patterns": [
      {
        "id": "customer_number",
        "regex": "C-\\d{4}",
        "description": "Customer number",
        "accepts": ["string"],
        "sensitive": false,
        "max_input_length": 256
      }
    ]
  }
}
```

- `id`: 1 to 64 letters, digits, `_` or `-`, unique.
- `regex`: a Python regular expression of at most 1,000 characters that must
  match the whole value.
- `accepts`: native types tested, among `string`, `integer`, `number` and
  `boolean`; default `["string"]`.
- `sensitive`: `true` masks the fields where the pattern matches.
- `max_input_length`: longer values are not tested; default 256, at most
  10,000.

At most 200 patterns. Regular expressions run without a timeout: use only
patterns you trust, since some expressions can take a very long time on some
values.

### Sensitive values and sampling

- `exposure.sensitive_values`: how values of sensitive fields (email
  addresses, phone numbers, IP addresses and sensitive patterns) appear in the
  result. `mask` (default) replaces letters with `A` or `a` and digits with
  `9`; `hide` removes the values and keeps the counts; `show` keeps them.
- `random_seed`: seed of the value samples, so that two scans of the same file
  with the same settings are identical.

---

# Execution history

> Reference for executions.json, the cumulative performance history written next to each report.

Source: https://docs.tabalyst.com/reference/history/ (Tabalyst 0.6.2)

Every successful Tabalyst analysis updates `executions.json` in the HTML
report's output folder. The file provides a lightweight performance history for
comparing reports as Tabalyst and its configuration evolve.

Given this command:

```console
tabalyst report examples/input/basic.csv -o examples/output/basic/report.html --config examples/config.json
```

Tabalyst writes `report.html`, `report.json`, and `executions.json` in
`examples/output/basic`. A later named report in the same folder adds another item
to the existing history. Only filenames are stored because all generated files
share one folder.

## Format

```json
{
  "schema_version": "1.0",
  "executions": [
    {
      "timestamp": "2026-09-18T14:32:10Z",
      "tabalyst_version": "0.1.0",
      "source_file": "data.csv",
      "html_file": "report1.html",
      "json_file": "report1.json",
      "rows": 3000,
      "columns": 34,
      "analysis_seconds": 0.6527,
      "total_seconds": 0.9314,
      "git": {
        "available": true,
        "commit": "a1b2c3d4e5f6...",
        "dirty": true,
        "state": "a1b2c3d4+working"
      }
    }
  ]
}
```

`analysis_seconds` covers CSV ingestion and profiling. `total_seconds` also covers
JSON serialization and HTML rendering. For a report built from a scan document
with `--scan`, both include the scan duration recorded in the document. The Tabalyst package version is always
recorded. Git metadata is included when the command runs inside a checkout; an
installed production run outside Git uses `available: false` and null values.

The complete history is rewritten through a temporary file and atomically replaced
after each successful analysis. Failed analyses do not add entries. Incompatible
schema changes are reserved for a future minor release.

---

# Glossary

> Terms used in the Tabalyst documentation and report, with their French equivalents, and the names that are never translated.

Source: https://docs.tabalyst.com/reference/glossary/ (Tabalyst 0.6.2)

This glossary fixes the vocabulary of the Tabalyst documentation. English is the
source language; the French terms below are used when Tabalyst is translated, for example on the website.

## Never translated

Keep these exactly as written in every language:

- the product names `Tabalyst`, `Tabalyst Report`, `Tabalyst CSV Report`,
  `Tabalyst Scan`, `Tabalyst Inspect` and `Tabalyst Generate`;
- commands and options, such as `tabalyst report`, `-o`, `--output-dir` or
  `--config`;
- JSON keys and values, such as `format_version`, `with_issues` or `mixed`;
- file names and paths, such as `report.json` or `executions.json`;
- Python modules, functions and parameters, such as `tabalyst.analyze()`.

## Terms

| English | French | Meaning |
| --- | --- | --- |
| toolkit | boîte à outils | What Tabalyst is: a set of tools for unfamiliar data |
| report | rapport | The interactive HTML file produced by `tabalyst report` |
| profile, JSON profile | profil, profil JSON | The `report.json` file describing a dataset |
| column profile | profil de colonne | The part of the profile describing one column |
| dataset | jeu de données | The tabular data being analyzed |
| source file | fichier source | The CSV file given to Tabalyst |
| dataset overview | vue d'ensemble du jeu de données | The report section summarizing the whole dataset |
| data sample | échantillon de données | The first rows shown in the report |
| raw preview | aperçu brut | Source values before normalization |
| missing value | valeur manquante | An empty value or a configured missing marker |
| issue | anomalie | A detected data quality problem |
| with issues | avec anomalies | A column with missing values, ambiguous dates or of inferred type `mixed` |
| value normalization | normalisation des valeurs | Cleaning applied before analysis, such as whitespace trimming |
| type inference | inférence de type | How Tabalyst decides the type of a column |
| inferred type | type inféré | The type deduced from the values of a column |
| physical type | type physique | The storage type of the values: number, date, text |
| semantic type | type sémantique | The meaning of a column, such as an email or an identifier |
| date analysis | analyse des dates | The report section on date columns |
| numeric analysis | analyse numérique | The report section on numeric columns |
| string analysis | analyse des textes | The report section on text columns |
| analysis settings | paramètres d'analyse | The effective configuration used for a report |
| configuration file | fichier de configuration | The JSON file passed with `--config` |
| delimiter | séparateur | The character separating CSV fields |
| encoding | encodage | The character encoding of the source file |
| execution history | historique d'exécution | The cumulative `executions.json` file |
| format revision | révision du format | The `format_revision` number of the profile format |
| batch | lot | Several source files processed by one command |
| output directory | dossier de sortie | The folder given with `-d` / `--output-dir` |
| alpha | alpha | Early stage where interfaces and formats may change |
| beta | bêta | Pre-1.0 stage where interfaces and formats may still change |
| scan | analyse | The complete description of a source written by `tabalyst scan` |
| scan document | document d'analyse | The project's `scan.json` or a standalone `.scan.json` export written by `tabalyst scan` |
| stale scan | analyse périmée | A scan document whose source or settings changed since it was written |
| duplicate record | enregistrement en double | A record equal to an earlier record of its dataset |
| record | enregistrement | One CSV row, one element of a JSON collection, one object line of a JSONL file or one data row of an Excel table |
| collection | collection | A JSON array whose elements are analyzed as records; a JSONL file is one collection, `$[]` |
| field | champ | A CSV column or a path inside JSON records, such as `orders[].amount` |
| detector | détecteur | A rule that recognizes a kind of value, such as email addresses or dates |
| interpretation | interprétation | A meaning proposed for a field by the detectors that match its values |
| sensitive value | valeur sensible | A value identifying a person, such as an email address, masked by default |
| mask | masque | A value with letters replaced by `A` or `a` and digits by `9` |
| measure envelope | enveloppe de mesure | The `status` wrapper saying whether a measure is complete or limited |
| diagnostic | diagnostic | A technical event of a scan, such as an excluded record or a reached limit |
| configuration layer | couche de configuration | One level of settings: defaults, each configuration file, then options |
| JSONL | JSONL | A file with one JSON object per line, ending in `.jsonl` or `.ndjson` |
| workbook | classeur | An Excel file, `.xlsx` or `.xlsm`, with one or more sheets |
| sheet | feuille | One tab of a workbook; a candidate table of its own unless it holds named tables |
| named table | tableau nommé | An Excel table (list object) defined in a sheet, with its own header and range |
| header row | ligne d'en-tête | The row of a sheet whose cells name the columns, detected or set by `excel.header_row` |
| inspect, Tabalyst Inspect | inspection, Tabalyst Inspect | The step that reads a JSON, JSONL or Excel source, finds its collection or table and writes the Inspect file; the command is `tabalyst inspect` |
| Inspect file | fichier Inspect | The `<source>-inspect.json` file beside a source, with the detection and the `config` you may edit |
| detection | détection | The section of an Inspect file that describes what Inspect found |
| candidate collection | collection candidate | An array, or for a workbook a sheet or named table, that Inspect could analyze as the dataset of a source |
| eligible candidate | candidat éligible | A candidate collection that has at least one element, all of them objects |
| selection | sélection | The collection Inspect proposes, or none |
| ambiguous selection, unresolved selection | sélection ambiguë, sélection non résolue | Several collections are equally plausible, or none is usable, so nothing is selected and `tabalyst scan` and `tabalyst report` stop |
| flatten | aplatissement | Reading nested objects as fields named by their path, such as `address.city` |
| complex value | valeur complexe | An object or array kept whole instead of being developed into fields |
| error policy | politique d'erreurs | `strict` or `tolerant`: whether a malformed record stops the scan or is excluded |
| partial scan | analyse partielle | A scan with status `partial`, which excluded some records under the `tolerant` policy |

---

# About Tabalyst

> What Tabalyst is and where to read more - concepts, known limitations, release notes, and how to contribute or report an issue.

Source: https://docs.tabalyst.com/about/ (Tabalyst 0.6.2)

Tabalyst is a beta toolkit for understanding unfamiliar structured data. It
analyzes files on your computer, with no telemetry and no remote processing.

- [Concepts](https://docs.tabalyst.com/about/concepts.md): how the toolkit and its tools fit together, from a
  file to a report, and what stays local.
- [Known limitations](https://docs.tabalyst.com/about/limitations.md): what Tabalyst does not do yet during the
  beta.
- [Release notes](https://docs.tabalyst.com/about/releases.md): what changed in each release.
- [Contributing and issues](https://docs.tabalyst.com/about/contributing.md): where to report a defect, ask for
  a feature and find the source code and licenses.

---

# Concepts

> How Tabalyst works - the toolkit and its four tools, the workflow from a file to a report, local-first processing, masking of sensitive values, experimental JSON formats and beta status.

Source: https://docs.tabalyst.com/about/concepts/ (Tabalyst 0.6.2)

## The toolkit and its tools

**Tabalyst** is an open-source, local-first toolkit for understanding
unfamiliar structured data. You install Tabalyst, never a single tool, and it
provides:

| Tool | Command | Role |
| --- | --- | --- |
| [Tabalyst Report](https://docs.tabalyst.com/report.md) | `tabalyst report` | HTML report and JSON profile |
| [Tabalyst Scan](https://docs.tabalyst.com/scan.md) | `tabalyst scan` | Complete JSON description of every field |
| [Tabalyst Inspect](https://docs.tabalyst.com/inspect/json.md) | `tabalyst inspect` | Finds how to read a JSON, JSONL or Excel file |
| [Tabalyst Sample](https://docs.tabalyst.com/sample.md) | `tabalyst sample` | Smaller CSV files |

## The workflow

```text
CSV, JSON, JSONL or Excel
  -> Scan -> scan document -> Report -> HTML report and JSON profile
        ^
        Inspect (JSON, JSONL and Excel sources only):
        which collection or table, how to flatten
```

- **Scan** reads the file once, as a stream, and records what it finds about
  each field.
- **Report** builds the profile from the scan, then renders the HTML from the
  profile alone. The profile is the canonical result, for scripts and other
  tools.
- **Inspect** decides how a JSON, JSONL or Excel file is read: which array,
  sheet or table holds the records, and for JSON how nested objects become
  columns named by their path, such as `address.city`. Scan and Report run it by themselves when needed, and use
  your edits of its file when you made some.
- **Sample** is independent: it cuts a CSV file down to a manageable size.

A scan is reused when the source content, the effective configuration and the
Tabalyst version are unchanged; otherwise it is replaced.

## Local-first

Tabalyst analyzes files on your computer, with no telemetry and no remote
processing. The HTML report works without a network connection. Source files
are never modified: Tabalyst only writes new files, and refuses to replace
existing ones without `--force`.

## Sensitive values

Generated JSON and HTML contain values from your source, including a preview of
the first rows. Values of fields detected as sensitive, such as email
addresses and phone numbers, are masked by default. Values of every other field
are listed, so share the outputs as you would share the data.

## Descriptive, not corrective

Detected types, formats and meanings are hints that describe the values.
Tabalyst never converts, repairs or validates your data against business rules,
and ambiguous values, such as the date `02/03/2025`, stay ambiguous unless you
configure how to read them.

## Bounded memory

Reports, scans, samples and Inspect read files as a stream. Memory is bounded
by configurable limits, not by the file size. When a limit is reached, results
show proven lower bounds instead of failing. The detail is in
[Known limitations](https://docs.tabalyst.com/about/limitations.md) and the
[configuration reference](https://docs.tabalyst.com/reference/configuration.md).

## Beta and experimental formats

Tabalyst is in beta until its first stable 1.0 release. Commands and options
may change, and the JSON formats are experimental: the
[JSON profile](https://docs.tabalyst.com/report/json-profile.md), the
[scan format](https://docs.tabalyst.com/scan/format.md) and the
[Inspect format](https://docs.tabalyst.com/inspect/format.md) may change incompatibly between
releases, without migration support. Each has a changelog and carries a
`format_version` and a monotonic `format_revision`; check them before reading a
file. Update with `pip install --upgrade tabalyst`: this documentation describes
the latest release.

---

# Known limitations

> What Tabalyst does not do yet during its beta, including JSON report limits, the duplicate detection budget and experimental JSON formats.

Source: https://docs.tabalyst.com/about/limitations/ (Tabalyst 0.6.2)

Tabalyst is in beta. This page lists what it does not do yet, so you can decide
whether it fits your data. Limitations are removed from this page when a
published release lifts them.

## Input

- **CSV, JSON, JSONL and Excel only.** `tabalyst report` and `tabalyst scan`
  read CSV, JSON, JSONL (`.jsonl`, `.ndjson`) and Excel (`.xlsx`, `.xlsm`) files;
  `tabalyst inspect` reads JSON, JSONL and Excel files; `tabalyst sample` reads
  CSV files only. The older `.xls`, `.xlsb` and `.ods` spreadsheets, XML,
  GeoJSON, compressed JSON, Parquet and databases are not supported; export
  them to CSV, or save a spreadsheet as `.xlsx`, first.
- **One table per Excel workbook.** Tabalyst analyzes the one sheet or named
  table that [Inspect](https://docs.tabalyst.com/inspect/excel.md) selects. When several are equally
  plausible, `scan` and `report` stop until you choose one with `--collection` or
  in the Inspect file. Blocks of a sheet separated by blank rows are not split
  into several tables, and total or note rows under the data count as records.
  The header is found in the first 50 rows of a sheet, by a rule measured on
  synthetic workbooks only; set `excel.header_row` when it picks the wrong row.
- **Excel cells.** Excel has one number type, so a whole number is read as an
  integer, even in a column of decimals. Dates are read as ISO text that the
  date detector recognizes, and a formula as its last calculated value. A real
  error cell (`#DIV/0!`) and a formula that was never calculated read as empty
  cells and cannot be counted. Merged cells keep their value in the top-left
  cell only, and a header on two levels is read as its last row. A workbook is
  read sheet by sheet into memory, about 0.9 byte per byte of sheet XML: a
  sheet above 1 GiB of XML is refused, and a million rows of eight columns takes
  about 4 seconds and 360 MB to read. Password-protected workbooks are refused.
- **One collection per JSON source by default.** Tabalyst analyzes the one
  collection of records that [Inspect](https://docs.tabalyst.com/inspect/json.md) selects.
  When several arrays are equally plausible, `scan` and `report` stop until you
  choose one with `--collection` or in the Inspect file. A JSON file that is a
  single object, a scalar or an array of plain values has no collection and is
  not turned into a one-row dataset. Inspect looks for arrays only through
  object keys, at most `scan.json.discovery_max_depth` (3) deep, and when it
  finds more than 100 it selects none. The relations between collections are not
  analyzed.
- **Inspect selects by size, not by name.** A collection is selected on its own
  when it is the only array of objects, or has at least 10 times more elements
  than the next one. A large table next to a much larger one, as in a relational
  export, can be selected without being the one you want: check the `selection`
  in the Inspect file.
- **Inspect observes the first records.** Its fields, depths and nesting come
  from the first 1,000 records of each collection; a field that appears later is
  found by the scan, not listed in the Inspect file. Inspect reads the whole
  source, so its time grows with the file size; on synthetic benchmarks, roughly
a tenth of the time of a scan.
- **No array explosion, no JSONPath.** Arrays never add records
  (`arrays.mode` is `preserve`), and collection paths use a limited syntax: keys
  from the root, ending with `[]`.
- **Inspect is for JSON, JSONL and Excel.** `tabalyst inspect` refuses CSV files.
- **JSON reports show scalar fields.** A JSON report lists the fields holding
  strings, numbers, booleans or nulls; the structure of objects and arrays
  (nesting, array lengths) appears only in `tabalyst scan` results. A record
  with a missing value is counted in `missing_values` only for nulls and empty
  values, not for absent fields.
- **JSON integers.** Integers of more than 4,300 digits, the Python limit, make
  a JSON file invalid.
- **Invalid JSON stops everything.** A syntax error anywhere in a `.json` file,
  even after the records Inspect observed, fails the inspection and the scan:
  Tabalyst does not recover a damaged document. In a JSONL file, only the
  affected lines are excluded.
- **JSONL lines.** A line is read as UTF-8 and parsed as a whole, up to
  `scan.limits.max_line_bytes` (16 MiB, at most 256 MiB); parsing a line needs
  about 7 to 13 times its size in memory. A file of compressed or non-UTF-8
  text is not read.
- **One header row.** The first record is always the header.
- **Strict structure.** A record with too few or too many fields, or a blank
  line inside the data, stops the analysis of that file with an error.
  `tabalyst scan` and `tabalyst report` can exclude such records instead with
  the `tolerant` error policy (`scan.errors.policy`); the report then lists
  them in its `excluded_records` issue.
- **No recursive search.** `tabalyst report *.csv` reads the matching files of
  one folder; it does not search subfolders. The same applies to `sample` and
  `scan`.

## Size and performance

- **Duplicate rows have a budget.** Reports and scans read the file once as a
  stream, with memory bounded by the scan limits. Duplicate row detection
  stores about 80 bytes per distinct row, up to
  `scan.limits.max_tracked_records` (2,000,000 rows by default). Beyond it,
  later rows are compared only with the stored ones, so the report shows a
  lower bound such as `≥ 12`. Raise the limit for an exact count, or set
  `scan.records.duplicates` to `false` to skip the detection.
- **No row-level progress.** Report and scan progress show the share of the
  file read, not a number of rows.
- **One reader per file.** Files of 16 MiB or more are analyzed by worker
  processes, but each file is read by a single process, which bounds the
  speed on very large files. Several files are processed one after another,
  not in parallel.

## Analysis

- **Types are hints.** Detected types describe the values; Tabalyst never
  converts or validates data against business rules.
- **Detector evidence not in reports.** The report shows what each detector
  recognized, with its formats, and the normalization variant groups of each
  column. The evidence examples and details of each detector are only in
  `tabalyst scan` documents.
- **Ambiguous dates stay ambiguous.** Values such as `02/03/2025` are never
  resolved from other values of the column; set
  `scan.detectors.date.ambiguous_order` to read them one way.
- **Syntax only.** Detectors check the form of values, never whether an
  address, number or code exists.
- **Rare late values on large columns.** On a column with more than
  `scan.detection.warmup_values` distinct values (10,000 by default), a
  detector that recognized none of the first ones, or only a few early ones
  (`scan.detection.rare_share`), stops testing the others, except about one
  value in 100. A rare email address, phone number or code that appears only
  later can go unnoticed, and a sensitive one is then not masked. Set
  `scan.detection.warmup_values` to `0` to test every value.
- **Missing values.** By default only empty and whitespace-only cells are
  missing. `NA`, `NULL` or `NaN` stay text unless you declare them in
  `scan.values.null_markers` in a [configuration file](https://docs.tabalyst.com/reference/configuration.md).

## Output and interfaces

- **Experimental JSON formats.** The [JSON profile](https://docs.tabalyst.com/report/json-profile.md), the
  [scan format](https://docs.tabalyst.com/scan/format.md) and the [Inspect format](https://docs.tabalyst.com/inspect/format.md) may
  change incompatibly between releases. No migration tool is provided: an
  Inspect file of another version is refused, and `tabalyst inspect --force`
  writes a new one. Check `format_version` and `format_revision` before reading
  a profile, a scan or an Inspect file.
- **Inspect files are yours to keep.** `tabalyst scan` and `tabalyst report`
  never write or change the Inspect file beside a source; only
  `tabalyst inspect` does. A source whose own name ends in `-inspect.json`
  cannot be used until renamed.
- **Changing commands.** Commands and options may change incompatibly while
  Tabalyst is in beta. Update often with `pip install --upgrade tabalyst`: this
  documentation describes the latest release.
- **English report.** The HTML report is only available in English.
- **Raw data in outputs.** The report, the JSON profile and scan documents
  contain values from the source file, including a preview of the first rows
  in reports. Values of sensitive fields, such as email addresses and phone
  numbers, are masked by default, but values of every other field are listed.
  Share them as you would share the data.
- **Patterns are trusted.** Regular expressions of scan patterns run without a
  timeout.

---

# Release notes

> A short summary of what changed in each Tabalyst release from 0.1.x to 0.6.2.

Source: https://docs.tabalyst.com/about/releases/ (Tabalyst 0.6.2)

## 0.6.2

Adds [Tabalyst Generate](https://docs.tabalyst.com/generate.md) for synthetic CSV datasets from
YAML definitions, with packaged CRM and insurance examples and optional logged
anomaly variants.

## 0.6.1

Makes choosing an Excel table or collection easier. Reorganizes the documentation.

## 0.6.0

Adds support for Excel workbooks.

## 0.5.1

Chooses a JSON collection from `report` and from `inspect`'s own output.

## 0.5.0

Adds Tabalyst Inspect and JSON Lines support.

## 0.4.4

Fast default CSV scans.

## 0.4.3

First beta release.

## 0.4.2

Makes scans several times faster on large files.

## 0.4.1

Adds adaptive detection for columns with many distinct values and a report time
that includes the scan.

## 0.4.0

Introduces Tabalyst Scan, a streaming analysis engine for CSV and JSON.

## 0.3.0

Introduces Tabalyst Sample.

## 0.2.0

Introduces a command-oriented toolkit: `tabalyst report INPUT... [OPTIONS]`.

## 0.1.x

First packaged releases.

---

# Contributing and issues

> Where to report a defect, ask for a feature, find the source code, and the license of Tabalyst and of its documentation.

Source: https://docs.tabalyst.com/about/contributing/ (Tabalyst 0.6.2)

## Report a defect or request a feature

Use the
[GitHub issue tracker](https://github.com/loribel-labs/tabalyst/issues). Include
the Tabalyst version (`tabalyst --version`), your operating system and Python
version, the command you ran and the message you got. Do not attach real
sensitive data: reduce the file with
[Tabalyst Sample](https://docs.tabalyst.com/sample.md) or replace its values first.

## Source code

Tabalyst is developed in the open at
[github.com/loribel-labs/tabalyst](https://github.com/loribel-labs/tabalyst).
The package is on [PyPI](https://pypi.org/project/tabalyst/).

## License

The Tabalyst code is released under the Mozilla Public License 2.0 (MPL-2.0). The contents of this
documentation are licensed under CC BY 4.0.

Created by Gregory Borelli. Open source project by Loribel Labs.