Skip to content
Beta · version 0.7.4 · commands and the JSON format may still change.

Glossary

This glossary fixes the vocabulary of the Tabalyst documentation. English is the source language; the French terms below are used when Tabalyst is translated, for example on the website.

Keep these exactly as written in every language:

  • the product names Tabalyst, Tabalyst Report, Tabalyst CSV Report, Tabalyst Scan, Tabalyst Inspect, Tabalyst Dedup and Tabalyst Generate;
  • commands and options, such as tabalyst report, -o, --output-dir or --config;
  • JSON keys and values, such as format_version, with_issues or mixed;
  • file names and paths, such as report.json or executions.json;
  • Python modules, functions and parameters, such as tabalyst.analyze().
EnglishFrenchMeaning
toolkitboîte à outilsWhat Tabalyst is: a set of tools for unfamiliar data
reportrapportThe interactive HTML file produced by tabalyst report
profile, JSON profileprofil, profil JSONThe report.json file describing a dataset
column profileprofil de colonneThe part of the profile describing one column
datasetjeu de donnéesThe tabular data being analyzed
source filefichier sourceThe CSV file given to Tabalyst
dataset overviewvue d’ensemble du jeu de donnéesThe report section summarizing the whole dataset
findingconstatWhat a rule found in a column: a count of one unit, out of a population, with its accuracy. An Error, an Issue or an Info
ErrorerreurA finding that a rule proves invalid, such as a date that does not exist in the calendar
Issuepoint d’attentionA finding that is an ambiguity or a deviation worth a look, never proof of an error. The report has no Warning class
signalsignalA tile of the dataset overview that sums one kind of finding over the columns that have it
classificationclassificationThe family and the type of a column (for example Identifier / ID, Text / Short), decided from the evidence of the scan and refined by the second pass
Unique IDidentifiant uniqueAn identifier column with no missing value and an exact count proving that every value is distinct across all the rows (raw uniqueness)
identifier uniquenessunicité de l’identifiantThe report’s reading of an identifier column that separates the repetitions of the ID that exact copies of whole rows explain from those that remain between distinct records (identifier_uniqueness in the JSON profile)
raw uniquenessunicité bruteEvery value of the ID is distinct across all the source records, copies of rows included; the Unique ID of the Scan
unique after row deduplicationunique après déduplication des lignesEvery value of the ID is distinct among the distinct rows: the only repetitions come from exact copies of whole rows. It is not a Unique ID, which is the raw reading
unique among present IDsunique parmi les ID présentsThe reading of an identifier that lacks a value in some records: after the exact copies of rows are set aside, no ID repeats between the distinct rows that have one. It is never a Unique ID, because the records without an ID remain
near-uniquequasi-uniqueA non-binding reading of an identifier whose remaining repetitions between distinct rows are few for their number (at most the smaller of 1% of them, their square root and 1,000). It is never a guarantee and never makes the column a Unique ID
explained repetitionrépétition expliquéeA repetition of an ID that an exact extra copy of a whole row accounts for; the unexplained ones are residual repetitions, between distinct records
data sampleéchantillon de donnéesThe rows of the row sample shown in the report: the start, sampled rows and the end
raw valuesvaleurs brutesSource values before normalization
missing valuevaleur manquanteAn empty value or a configured missing marker
quality issueanomalie de qualitéAn observation that is a confirmed issue or a candidate anomaly; a remark is not one
with quality issuesavec anomalies de qualitéA column with a confirmed issue or a candidate anomaly, such as an inferred type mixed or ambiguous dates
observationobservationA finding of a scan about the data, with a status: informational, candidate anomaly or confirmed issue
limitlimiteWhat a scan could not establish, such as a count stopped by a scan limit; never a defect of the data
value normalizationnormalisation des valeursCleaning applied before analysis, such as whitespace trimming
type inferenceinférence de typeHow Tabalyst decides the type of a column
inferred typetype inféréThe type deduced from the values of a column
physical typetype physiqueThe storage type of the values: number, date, text
semantic typetype sémantiqueThe meaning of a column, such as an email or an identifier
date analysisanalyse des datesThe report section on date columns
numeric analysisanalyse numériqueThe report section on numeric columns
string analysisanalyse des textesThe report section on text columns
analysis settingsparamètres d’analyseThe effective configuration used for a report
configuration filefichier de configurationThe JSON file passed with --config
delimiterséparateurThe character separating CSV fields
encodingencodageThe character encoding of the source file
execution historyhistorique d’exécutionThe cumulative executions.json file
format revisionrévision du formatThe format_revision number of the profile format
batchlotSeveral source files processed by one command
output directorydossier de sortieThe folder given with -d / --output-dir
alphaalphaEarly stage where interfaces and formats may change
betabêtaPre-1.0 stage where interfaces and formats may still change
scananalyseThe complete description of a source written by tabalyst scan
scan documentdocument d’analyseThe project’s scan.json or a standalone .scan.json export written by tabalyst scan
stale scananalyse périméeA scan document whose source, settings or pattern discovery changed since it was written
stratum, stratastrate, stratesA group of rows that share the value of the stratification field, or the combination of values of several fields
stratification adviceaide à la stratificationWhat tabalyst scan --plan --stratify adds to a plan: which columns make good strata and how risky a combination of columns is
smart sampleéchantillon smartA sample that aims to cover the diversity and the rare cases of a file; its proportions do not estimate the population
targetcibleA thing a smart sample aims to cover: a value, a pattern, an extreme, a period, a problem the scan found. It has a weight and is mandatory or optional
smart sample planplan d’échantillon smartWhat tabalyst scan --plan --smart adds to a plan: the targets, their weights, the sizes, the reserve and the readings of the source
coverage (of a smart sample)couvertureWhat a smart sample holds of the targets of its plan, by family and weighted by their weights. It is the coverage of the catalog of targets, not of every phenomenon of the data
reserveréserveThe ordinary rows of a smart sample, chosen by a seeded hash of their position, so that it does not hold only rare cases. It gives way to mandatory targets
distinct sampleéchantillon de valeurs distinctesThe distinct original values of one field kept in values.samples, all of them or the bottom-k of smallest priority, each with its exact count over the whole field. It is not a ranking
row sampleéchantillon de lignesWhole rows of a dataset kept in records.sample (the start, the end and a reproducible choice of the interior), so that two fields can be related on the same rows. It is not a uniform sample of the file
estimated figurevaleur estiméeA figure with accuracy: "estimated", such as the KMV estimate of a distinct count, with its expected error. It is never quoted as a count
limited resultrésultat limitéA measure stopped by a configured limit, written with its reason and a proven lower_bound, never with an estimate in place of the value
fingerprint-based identityidentité par empreinteidentity_accuracy: "fingerprint_based": two long texts are the same entry when their 128-bit digests are equal, not when the full texts were compared
analysis planplan d’analysesWhat tabalyst scan --plan shows: the analyses a scan could receive, their cost and the decision on each
forced planplan forcéA plan that takes every applicable analysis within the hard memory budget, whatever the limits of an automatic choice (tabalyst scan --plan --deep)
confirmationconfirmationThe question a costly run asks before it starts; a plan says whether it would, and why
second passsecond passageThe reading of a source after its scan, to add analyses such as histograms, patterns counted on every value and dates per period (tabalyst scan --deep, and by itself in tabalyst report); its result is the second pass document
validated patternpattern validéAn inferred pattern whose occurrences were counted on every value of the file by the second pass, not only on a sample of the distinct values
formatformatThe exact form of one value, a symbol per character (AA-9999)
patternpatternThe values of one structure and one set of lengths, with their constants ('AK-'9999); a discovered pattern never has a range
familyfamilleThe patterns of one structure that differ only by lengths or constants; shown as the list of its patterns with their counts
character mapcarte des caractèresThe most frequent characters of a field, their scripts and Unicode categories, the case of the ends of the values and the emoji; counted on the scan’s samples, and on every value by the second pass
facetfacetteA dimension along which a family is specialized: composition, lengths, or the content of a token (head, tail, whole)
nodenœudA pattern plus a constraint on a facet, with its counts in distinct values and occurrences
deep.modemode du second passageThe setting that tells tabalyst report whether to plan a second pass by itself (auto), never (off) or always, as --deep does (force)
second pass documentdocument de second passageThe deep.json or <stem>.deep.json written by tabalyst scan --deep, beside the scan, which it never changes
pair of columnspaire de colonnesTwo numeric columns of one dataset analyzed together. The planner scores the pairs on the row sample, lists the best and the second pass computes their correlation over every row
common rowslignes communesThe rows where both columns of a pair have a number; the means and the coefficient of a pair are those of these rows only (pairwise_complete)
undefined coefficientcoefficient non définiA correlation that cannot be computed, because too few common rows or a column that does not vary; the result says why and never writes 0
categorycatégorieA column of a few repeated values, such as a region, whose value is the row of a table; its exact frequencies come from the scan
correlation ratiorapport de corrélationThe share of the variance of a number that the categories of another column explain, from 0 to 1 (eta_squared). The second pass also gives it corrected for the number of categories (epsilon_squared), and the planner rates a pair by its square root
skewnessasymétrieWhether the long tail of a numeric column is on the right (positive) or on the left (negative); the third standardized moment. The second pass gives it with the excess kurtosis, which grows with the weight of the tails
outlier of Tukeyvaleur aberrante de TukeyA number beyond a fence at 1.5 (mild) or 3 (far) interquartile ranges below the first quartile or above the third; it is a candidate anomaly to look at, not an error
featurecaractéristique dérivéeWhat a table reads of a column to make its classes: the pattern that reads the value, its length, its prefix or its value (another category)
table of a category against a featuretable croiséeThe exact count of the rows of each category in each class of a feature of another column, computed by the second pass over every row (category_crosstab)
classclasseA column of a table: a pattern, a length or a prefix; the last one is always other, what the others do not take
puritypuretéThe share of the classified rows of a category that sit in its dominant class; the purity of a table is that of all its categories together, and it differs from category to class and from class to category
exceptionexceptionA row outside the dominant class of its category, counted exactly and shown with its first rows
rank correlationcorrélation de rangThe Spearman coefficient of a pair: the Pearson coefficient of the ranks, which sees a relation that always goes up without being a line. The second pass estimates it from a grid of bins and gives certain bounds
provided rulerègle fournieA rule you wrote for a table, such as “region east takes plan basic”, checked as it is against the counts with its counterexamples; it is never proposed by itself
candidate rulerègle candidateA relation learned from the data, such as “region QC reads pattern XX9999”, that nobody validated; it is not a rule of the data
Cramér’s VV de CramérThe strength of the association of two columns in a table, from 0 to 1, computed from the counts; the corrected form does not grow with the number of classes and is what Tabalyst ranks by. It is not defined, never 0, when a single category or class has rows
exact distinct countnombre exact de valeurs distinctesThe number of different original values of a field, counted exactly by the second pass when the scan only estimated it; redundant is the occurrences beyond the first of each value
insight cardcarte d’analyseWhat a Report prints for one analysis of the second pass and one column: figures, distributions and tables, each with its own accuracy label (exact, estimated, sample, at least). It is derived from the document when the page is rendered, never stored
slicetrancheOne tenth of the records of a dataset, in file order, with the same number of records (one more or fewer): the unit in which the second pass follows a column along the file
drift along the filedérive selon la positionA change of the missing values or of the values of a column from one slice of the file to the next, counted exactly; it shows that the column was filled or exported differently, not why
set of keysensemble de clésThe keys an object of a JSON file has; objects with the same keys are one set, counted exactly by the second pass
fingerprintempreinteA 128-bit digest that tells two long values apart; two values are taken for one only if their digests collide, which is about as unlikely as 2^-128
maskmasqueA value with each letter shown as a or A and each digit as 9; the form of a sensitive value that the document may hold
floor (forms)seuilThe number of values (deep.sensitive.min_count) under which a mask or a class is summed up, so no figure is given about so few values
missing togethermanquants simultanésColumns that lack a value in the same records, counted exactly by the second pass; the share where both do among the records where either does ranks the pairs
histogramhistogrammeThe count of the values of a numeric field per integer or per bin, exact, with a median and quantiles that are exact or estimated
already satisfieddéjà satisfaiteAn analysis whose answer the scan already holds, so the plan does not select it
duplicate recordenregistrement en doubleA record equal to an earlier record of its dataset
row identityidentité de ligneIn Dedup, the list of the decoded cells of a CSV record, compared without any normalization: case, spaces and leading zeros count
distinct rowligne distincteOne row identity; Dedup counts it once, and writes it once to unique.csv
duplicate groupgroupe de doublonsAll the occurrences of one row that occurs at least twice; records.duplicate_groups counts the groups
repeated rowligne répétéeThe row of a duplicate group, written once to duplicates.csv
extra copyoccurrence supplémentaireAn occurrence after the first of its row, counted by records.extra_copies; a row that occurs three times has two
first occurrencepremière occurrenceThe first record of a row in the order of the source; the one Dedup keeps
singletonligne uniqueA row that occurs exactly once
maximum frequencyfréquence maximalerecords.max_occurrences, the most occurrences of one row; three or more makes Dedup write duplicates_all.csv
group occurrencesoccurrences des groupesEvery occurrence of the rows of the duplicate groups, first ones included; the rows of duplicates_all.csv
managed outputsortie géréeA file under an output name of Dedup that a Dedup report proves it published
foreign outputsortie étrangèreA file under an output name of Dedup that no report proves it published; Dedup replaces it only with --force
recovery reportrapport de repriseThe report with the status in_progress that Dedup writes before it changes the files of an earlier result
verification levelniveau de vérificationmetadata (size and modification time) or digest (SHA-256): how far Dedup checked that a skipped source is unchanged
storage modemode de stockagememory or disk: where Dedup kept the rows it had seen
spillrepli sur disqueThe automatic switch of Dedup from memory to disk when the memory budget is reached
recordenregistrementOne CSV row, one element of a JSON collection, one object line of a JSONL file or one data row of an Excel table
collectioncollectionA JSON array whose elements are analyzed as records; a JSONL file is one collection, $[]
fieldchampA CSV column or a path inside JSON records, such as orders[].amount
detectordétecteurA rule that recognizes a kind of value, such as email addresses or dates
interpretationinterprétationA meaning proposed for a field by the detectors that match its values
sensitive valuevaleur sensibleA value identifying a person, such as an email address; shown by default, maskable
maskmasqueA value with letters replaced by A or a and digits by 9
measure envelopeenveloppe de mesureThe status wrapper saying whether a measure is complete or limited
diagnosticdiagnosticA technical event of a scan, such as an excluded record or a reached limit
configuration layercouche de configurationOne level of settings: defaults, each configuration file, then options
JSONLJSONLA file with one JSON object per line, ending in .jsonl or .ndjson
workbookclasseurAn Excel file, .xlsx or .xlsm, with one or more sheets
sheetfeuilleOne tab of a workbook; a candidate table of its own unless it holds named tables
named tabletableau nomméAn Excel table (list object) defined in a sheet, with its own header and range
header rowligne d’en-têteThe row of a sheet whose cells name the columns, detected or set by excel.header_row
inspect, Tabalyst Inspectinspection, Tabalyst InspectThe step that reads a JSON, JSONL or Excel source, finds its collection or table and writes the Inspect file; the command is tabalyst inspect
Inspect filefichier InspectThe <source>-inspect.json file beside a source, with the detection and the config you may edit
detectiondétectionThe section of an Inspect file that describes what Inspect found
candidate collectioncollection candidateAn array, or for a workbook a sheet or named table, that Inspect could analyze as the dataset of a source
eligible candidatecandidat éligibleA candidate collection that has at least one element, all of them objects
selectionsélectionThe collection Inspect proposes, or none
ambiguous selection, unresolved selectionsélection ambiguë, sélection non résolueSeveral collections are equally plausible, or none is usable, so nothing is selected and tabalyst scan and tabalyst report stop
flattenaplatissementReading nested objects as fields named by their path, such as address.city
complex valuevaleur complexeAn object or array kept whole instead of being developed into fields
error policypolitique d’erreursstrict or tolerant: whether a malformed record stops the scan or is excluded
partial scananalyse partielleA scan with status partial, which excluded some records under the tolerant policy