Data Models API
This API allows managing data models in the system. It provides endpoints to create, read, update, and delete data models, as well as to perform queries with filtering and pagination options.
The DataModel Object
The DataModel object represents a data model in the system.
Attributes
| Attribute | Description | Type | Required |
|---|---|---|---|
| id | Unique identifier of the data model | String | Yes |
| path | Path of the data model | String | Yes |
| key | Unique key of the data model | String | Yes |
| label | Descriptive label of the model | String | Yes |
| entityType | Entity type of the model | String | Yes |
| priority | Priority of the model | Integer | No |
| order | Order of the model | Integer | No |
| allowedValues | Allowed values for the field | Array | No |
| createdAt | Creation date and time | String | Yes |
| updatedAt | Last update date and time | String | No |
| isSensitive | Marks an identity DataModel as sensitive (PII) | Boolean | No |
| metadata | Additional metadata of the model | Object | No |
| processing | Processing definition of a document data model; null or absent when none is defined. See Document Processing | Object | No |
Valid Entity Types
The entityType field must be one of the following values:
| Entity Type | Description |
|---|---|
identity | Identity records (national ID, passport, etc.) |
contact | Contact information fields |
document | Document type definitions |
borrower | Borrower entity configuration |
borrower_field | Custom fields for borrowers |
point_of_contact | Point of contact fields |
authorization | Authorization type definitions |
step | Workflow step definitions |
metric | Metric definitions |
signal | Signal definitions written by the conversational Goal node |
goal | Goal definitions counted by Conversation Insights (label is the name, metadata.description the description, order optional; a PATCH with "order": null clears it). The key cannot be changed after creation |
deal_field | Custom fields for deals |
deal_step | Deal workflow step definitions |
deal_role | Deal role definitions |
decision | Decision type definitions |
accounting_document | Accounting document type definitions |
asset_field | Custom fields for assets |
asset_group | Asset group definitions for categorizing assets |
Available Operations
Create a New Data Model
Creates a new data model in the system.
Input Parameters:
| Parameter | Description | Type | Required |
|---|---|---|---|
| path | Path of the data model | String | Yes |
| key | Unique key of the model | String | Yes |
| label | Descriptive label of the model | String | Yes |
| entityType | Entity type of the model | String | Yes |
| priority | Priority of the model | Integer | No* |
| order | Order of the model | Integer | No |
| allowedValues | Allowed values for the model | Array | No |
| metadata | Additional metadata of the model | Object | No |
| isSensitive | Marks an identity DataModel as sensitive (PII) | Boolean | No |
*For entityType: "identity", priority is required and must be greater than or equal to -1.
Request Example:
Successful Response:
Update a Data Model
Updates an existing data model.
Path Parameters:
| Parameter | Description |
|---|---|
| :data_model_id | ID of the data model to update |
Input Parameters:
| Parameter | Description | Type | Required |
|---|---|---|---|
| path | New path of the data model | String | No |
| key | New key of the model | String | No |
| label | New label of the model | String | No |
| order | New order of the model | Integer | No |
| priority | New priority of the model | Integer | No |
| allowedValues | Allowed values for the model | Array | No |
| metadata | New metadata of the model | Object | No |
Note: isSensitive cannot be updated via PATCH. Use the dedicated make-sensitive endpoint below.
Request Example:
Successful Response:
The response will be the updated DataModel object.
Make an Identity Data Model Sensitive
Marks an existing identity DataModel as sensitive and encrypts existing identities for that key.
Path Parameters:
| Parameter | Description |
|---|---|
| :data_model_id | ID of the data model to update |
Successful Response:
The response will be the updated DataModel object (with isSensitive: true).
Save the Processing Definition of a Document Data Model
Saves the processing block of a document data model: the fields to extract, the validation conditions with what is
expected of each one, and, for advanced configurations, several document types, validations, logic rules and document
rules. In the Hub these are the Details tab (fields to extract) and the Validation tab of the data model. The
Document Processing guide explains each part, maps every Hub name to its key and
describes the rule grammar. This is the only way to write processing; POST and PATCH never touch it.
Requires the configurations.write permission. Only data models with entityType: "document" accept it.
Path Parameters:
| Parameter | Description |
|---|---|
| :data_model_id | ID of the document data model |
Input Parameters:
| Parameter | Description | Type | Required |
|---|---|---|---|
| documentTypes | The document types, 1 to 20. A configuration made in the Hub has one. With several, order matters: when a file looks like two types, the first one wins | Array | Yes |
| documentRules | acceptedTypes (anyOf or allOf over type keys) and document-level validations | Object | No |
| enabled | false stops the document from being processed and keeps existing results. Default true | Boolean | No |
| expectedVersion | The definitionVersion you edited. If the stored one is different, the save is rejected with 409 | Integer | No |
Each item of documentTypes:
| Parameter | Description | Type | Required |
|---|---|---|---|
| key | Lowercase letters, digits and _, starting with a letter, up to 64 characters | String | Yes |
| label | Name shown in the Hub, up to 120 characters | String | Yes |
| description | What the document is, up to 1000 characters. Files are recognized by it. Required with two or more types. With a single type it can be absent, null or blank (stored as null): then there is no recognition step and every file is read directly as that type | String | With 2+ types |
| minConfidence | Minimum type confidence, 0 to 1. Default 0.6 | Number | No |
| fields | 1 to 40 fields | Array | Yes |
| validations | Up to 50 validations: key, label, severity (blocking or warning), rules | Array | No |
| logic | Up to 50 logic rules: key, label, rules | Array | No |
Each item of fields is a row of the Details tab, of the Validation tab, or of both:
| Parameter | Description | Type | Required |
|---|---|---|---|
| key | Same pattern as the type key, unique within the type | String | Yes |
| label | Name shown in the Hub, up to 120 characters. The Hub shows the key when absent | String | No |
| type | string, number, integer, date, boolean or list | String | Yes |
| description | What to extract or to check, up to 500 characters. It is sent to the model; for a boolean row it is the question | String | Yes |
| purpose | extraction (a field to extract), validation (a condition) or both (a field to extract that is also a condition). When absent, the row is a field to extract, as before this key existed | String | No |
| expected | What a condition expects; see below. Required when purpose is validation or both, and not allowed otherwise. It is never sent to the model | Object | Depends |
| required | Advanced options only. Whether the file is not valid without this value. Default false. Not allowed as true on a row with purpose: use {"op": "exists"} instead | Boolean | No |
| minConfidence | Minimum confidence, 0 to 1. Default 0.7, 0.5 for boolean | Number | No |
| format | numberFormat (auto, latam, us) on number and integer fields, dateOrder (auto, dmy, mdy, ymd) on date fields | Object | No |
expected is {"op": ..., "value": ...}. Which op each row type accepts:
op | value | Row types | Meaning |
|---|---|---|---|
exists | none | all | A value was read |
equals | a value of the row's type (dates as YYYY-MM-DD, text up to 500 characters) | string, number, integer, date | Text ignores case and accents, not punctuation; numbers compare numerically; dates by day |
equals | {"ref": "borrower.label"} | string | The customer's name, by similarity from 0.85 |
equals | {"ref": "identity.<key>"} | string, number, integer, date | One of the customer's identities, compared like equals. Sensitive identities are rejected |
is_true, is_false | none | boolean | Expected Yes / No |
greater_than, greater_than_or_equal, less_than, less_than_or_equal | a number (whole on integer rows) | number, integer | Exact comparison |
after, before | a date, YYYY-MM-DD | date | Strictly after or before |
one_of | 1 to 200 non-empty strings | string | Equal to any of them, compared like equals |
A condition's key does not need to be among the fields to extract. Keys are unique within a type, so a value that is
extracted and checked is one row with purpose: "both".
Request Example:
Two fields to extract and two conditions, on a single type without a description:
An advanced configuration, with a described type, required fields, a validation and a logic rule:
Successful Response:
The updated DataModel object. Its processing holds the definition with the defaults filled in. A field to extract is
stored without purpose, so it is the same object as a field saved before this key existed.
The server also derives two things from the conditions, which you see when you read the data model back:
- each row with
purposevalidationorbothbecomes a blocking validation of its type, with the key__expected_<field key>, added after your own validations; - with at least one condition,
documentRules.unrecognizedFilesisinvalid: a file that matches none of the types isinvalidinstead ofneeds_review. It only applies when there is a recognition step.
Both are managed by the server. They are dropped from any body you send and derived again, so a GET followed by a
PUT of the same processing stores the same definition, and a hand-edited __expected_* validation is replaced by
the one its row produces. A user key can never start with _, so these never collide with yours.
Four more fields are managed by the server (they are ignored if you send them):
| Field | Description |
|---|---|
| definitionVersion | Starts at 1 and goes up only when the types or rules change. Toggling enabled or saving the same definition keeps it |
| definitionHash | Fingerprint of the types, rows and rules. Results store it, so you can tell when they are older than the definition. A definition saved before purpose and expected existed keeps its hash |
| updatedAt | When the definition last changed |
| updatedBy | Who changed it |
Validation errors:
Every problem is reported at once, in a single 400 with errorSubCode: "PROCESSING_DEFINITION_INVALID". Each
entry of details.errors has a code, a path (a JSON pointer into the body you sent) and a message:
code | When |
|---|---|
PROCESSING_SCHEMA_INVALID | Wrong JSON type, missing required property, unknown property, or text too long. Also a purpose that is not one of the three values, required: true on a row with purpose, or a type without description when there are two or more types |
PROCESSING_INVALID_KEY | A key does not match the allowed pattern |
PROCESSING_DUPLICATE_KEY | A repeated type key, field key within a type, validation or logic key within a type, or document validation key. A validation and a logic rule of the same type cannot share a key either |
PROCESSING_LIMIT_EXCEEDED | Over a limit: 20 types, 40 fields, 50 validations, 50 logic rules, 50 rules per tree, 200 in_list or one_of values, or more than five levels of nesting. A type's validations plus its conditions count together against the 50 (path /documentTypes/i/fields) |
PROCESSING_UNKNOWN_TYPE | acceptedTypes, an operand or a field: reference names a type that is not defined |
PROCESSING_UNKNOWN_FIELD | A rule or a field: reference names a field that is not defined |
PROCESSING_UNKNOWN_OPERATOR | The operator does not exist (is_pending and the authentication operators are not available) |
PROCESSING_OPERATOR_TYPE_MISMATCH | The operator does not apply to the field's type |
PROCESSING_INVALID_VALUE | value is missing or present against what the operator expects, has the wrong type, is a date that is not YYYY-MM-DD, is a negative day count, or threshold is outside 0 to 1 or used with an operator other than similar_to |
PROCESSING_INVALID_REF | A malformed reference. In expected, also a reference other than borrower.label or identity.<key>, or borrower.label on a row that is not string |
PROCESSING_REF_UNKNOWN_IDENTITY | identity.<key> names an identity data model that does not exist |
PROCESSING_REF_SENSITIVE_IDENTITY | identity.<key> names a sensitive identity, which is never decrypted for a comparison |
PROCESSING_INVALID_MIN_CONFIDENCE | A minConfidence outside 0 to 1 |
PROCESSING_INVALID_FORMAT | A format key that does not apply to the field's type, or an unknown value |
PROCESSING_INVALID_RULE | A rule node that is not exactly one of all, any, not or a rule; an empty group; or a document validation operand without its type |
PROCESSING_INVALID_EXPECTED | expected is missing on a condition, present on a field to extract, not an object, has an unknown op or one that does not apply to the row's type, or its value is missing, extra, of the wrong type, an empty string, a date that is not YYYY-MM-DD, a fraction on an integer row, or an empty list. path points at .../fields/j/expected, its op or its value |
Other errors of this endpoint:
| Status | errorSubCode | When |
|---|---|---|
| 404 | The data model does not exist | |
| 409 | PROCESSING_DEFINITION_VERSION_CONFLICT | expectedVersion does not match. details.currentVersion has the stored one; reload before saving again |
| 422 | DATA_MODEL_PROCESSING_NOT_DOCUMENT | The data model is not a document data model |
Test a Processing Definition
Runs a definition against one uploaded file and returns the result. Nothing is stored: no document, no results and no images. It is what Test with a file does in the Hub.
Requires the configurations.write permission. Unlike the PUT, this route takes the data model key, not its
id; it is the same key as the documents it applies to.
Path Parameters:
| Parameter | Description |
|---|---|
| :data_model_key | Key of the document data model |
Form Parameters (multipart/form-data):
| Parameter | Description | Type | Required |
|---|---|---|---|
| file | The file to test, up to 20 MB | File | Yes |
| definition | A definition as JSON text, with the same body as the PUT above. When absent, the stored definition is used | String | No |
| borrowerId | A customer whose name and identities are used for borrower.label and identity.* references | String | No |
| context | JSON text {"borrower": {"label": "..."}, "identities": {"<key>": "..."}}, used when borrowerId is absent | String | No |
Successful Response:
file has the same shape as one attachment's result in the
Documents API (section "Get the Processing Results of a Document"), except that
page images come inline as base64 and there are no crops. At most 10 page images come back: first the pages where a
value was found (values to review before the rest), then the lowest-numbered pages, all in page order. pageCount, in file
and at the top, is the number of pages in the file, and pagesTruncated is true when there were more than 10
page images.
mode, extraction and validation are the same as for one attachment in the Documents API, computed with the tested
definition: mode is extraction, validation or both, and a block is null when the definition has no part of
that kind. current is always true, and pageUrl and cropUrl in the blocks are null, because the pages come
inline. With a single file, a document that needs several types will usually not be satisfied.
Errors:
| Status | errorSubCode | When |
|---|---|---|
| 400 | PROCESSING_TEST_FILE_EMPTY | The file is empty |
| 400 | PROCESSING_TEST_FILE_TOO_LARGE | The file is over 20 MB |
| 400 | PROCESSING_TEST_CONTEXT_CONFLICT | Both borrowerId and context were sent |
| 400 | PROCESSING_DEFINITION_INVALID | The definition sent is not valid (same details.errors as the PUT) |
| 404 | No document data model with that key | |
| 422 | DOCUMENT_PROCESSING_NOT_CONFIGURED | No definition was sent and the data model has none stored |
| 503 | DOCUMENT_PROCESSING_* | The processing service could not complete the run. See "Processing errors" in the Documents API |
Get a Data Model
Retrieves the information of a specific data model.
Path Parameters:
| Parameter | Description |
|---|---|
| :data_model_id | ID of the data model to retrieve |
Successful Response:
The response will be a complete DataModel object.
Delete a Data Model
Deletes a data model from the system.
Path Parameters:
| Parameter | Description |
|---|---|
| :data_model_id | ID of the data model to delete |
Successful Response:
Status code 204 (No Content) if the deletion was successful.
List Data Models
Retrieves a paginated list of data models with filtering and sorting options.
Query Parameters:
| Parameter | Description | Type |
|---|---|---|
| search | Text to search in data models | String |
| page | Page number (default: 1) | Integer |
| per-page | Items per page (default: 10) | Integer |
| entity-type | Filter by entity type | String |
| sort-by | Field to sort by | String |
| sort-direction | Sorting direction (asc or desc) | String |
Successful Response:
The response will be a paginated list of DataModel objects.
Error Handling
The API may return the following error codes:
| Code | Description |
|---|---|
| 400 | Bad Request |
| 401 | Unauthorized |
| 403 | Forbidden |
| 404 | Not Found |
| 409 | Conflict (processing definition version) |
| 422 | Unprocessable Entity (processing) |
| 500 | Internal Server Error |
| 503 | Service Unavailable (processing test) |
Errors will include a descriptive message in the response body.