> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cominty.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Sources
> Turn local files into an indexed source with an API key, give it to an agent, and read its citations
A source is a set of files that Cominty indexes, so an agent can search them and
cite them. With an API key, you create a source from files on your machine. The
SDKs do it in one call. Read
[Custom agents](https://docs.cominty.ai/guides/custom-agents.md#what-an-api-key-can-do) first for what an
API key can and cannot see.
Each snippet has a tab for the [Python SDK](https://docs.cominty.ai/sdk/overview.md), the
[JavaScript SDK](https://docs.cominty.ai/js-sdk/overview.md), and HTTP. Both SDKs have the same methods
on `client.sources`. Only the casing of the longer names differs:
| Action | Python SDK | JavaScript SDK | HTTP |
| - | - | - | - |
| Wait for a sync to end | `wait_synced` | `waitSynced` | Poll `GET /dc/sources` until `status` is `synced` |
| Add files | `add_files` | `addFiles` | Upload to the source's drive, then `PUT /dc/sources/{pid}/status?operation=sync` |
| Remove a file | `remove_file` | `removeFile` | `DELETE /files/{file_id}`, then the same sync |
| List the files | `list_files` | `listFiles` | `GET /drives/{drive_id}` |
| Download a file | `download_file` | `downloadFile` | `GET /files/{file_id}` |
> [!NOTE]
>
> Only **drag-and-drop** sources can be created with a key. Google Drive and
> SharePoint sources need the console's OAuth flow. They can show up in
> `sources.list()` as organization sources, but you cannot create them here.
## How it works
```mermaid theme={null}
flowchart TD
files["Local files"] --> drive["Drive: the container for the files"]
drive --> upload["Each file goes straight to storage"]
upload --> source["Source: private, drag-and-drop"]
source --> sync["First sync starts"]
sync --> synced["Status synced: documents are searchable"]
synced --> agent["Agent with source_ids"]
agent --> answer["Answer with citations"]
answer --> document["documents.get resolves a citation"]
```
`sources.create` runs the first four steps and returns while the source is
`pending` or `syncing`. Wait until it is `synced` before you search it.
## Create a source
> [!WARNING]
>
> Your files are not available to the agent as soon as the source is created and
> the upload ends. The platform must first index every file. Until the source
> is `synced`, the agent finds nothing, or only some of the files. Wait for
> `synced` before you start a chat. Each snippet below waits for it.
> [tab] Python SDK
>
> ```python theme={null}
> from pathlib import Path
>
> source = await client.sources.create(Path("docs"), name="Product docs")
> source = await client.sources.wait_synced(source.pid, timeout=420, interval=3)
> print(source.model_dump_json(indent=2))
> ```
> [tab] JavaScript SDK
>
> ```ts theme={null}
> import { readdir, readFile } from 'node:fs/promises'
> import { join } from 'node:path'
>
> // The folder's top-level files, hidden ones skipped, as File objects.
> const entries = await readdir('docs', { withFileTypes: true })
> const files = await Promise.all(
> entries
> .filter((entry) => entry.isFile() && !entry.name.startsWith('.'))
> .map(async (entry) => new File([await readFile(join('docs', entry.name))], entry.name)),
> )
>
> let source = await client.sources.create(files, { name: 'Product docs' })
> source = await client.sources.waitSynced(source.pid, { timeout: 420_000, interval: 3_000 })
> console.log(JSON.stringify(source, null, 2))
> ```
> [tab] HTTP
>
> Creating a source is a 5-step flow. Steps 2 to 4 are the same
> [presigned upload](https://docs.cominty.ai/api-reference.md#files-and-exports) as for chat files.
>
> ```bash theme={null}
> BASE_URL=https://ds.cominty.com
> AUTH="x-cominty-token: $COMINTY_API_KEY"
>
> # 1. Create a drive
> DRIVE_ID=$(curl -s -X POST "$BASE_URL/drives" -H "$AUTH" | jq -r .id)
>
> # 2. Ask for an upload permission
> PERMIT=$(curl -s -G "$BASE_URL/drives/$DRIVE_ID/upload" -H "$AUTH" \
> --data-urlencode filename=facts.txt \
> --data-urlencode mimetype=text/plain)
>
> # 3. Upload the file to the storage host. Do not send the API key there.
> URL=$(echo "$PERMIT" | jq -r .url)
> FORM=()
> while IFS= read -r field; do FORM+=(-F "$field"); done < <(
> echo "$PERMIT" | jq -r '.fields | to_entries[] | "\(.key)=\(.value)"'
> )
> ETAG=$(curl -s -D - -o /dev/null "${FORM[@]}" \
> -F "file=@facts.txt;type=text/plain" "$URL" \
> | tr -d '\r' | awk -F': ' 'tolower($1)=="etag"{print $2}')
>
> # 4. Confirm the upload
> KEY=$(echo "$PERMIT" | jq -r .fields.key)
> curl -s -X POST "$BASE_URL/drives/upload" -H "$AUTH" \
> -H "Content-Type: application/json" \
> -d "{\"etag\": $ETAG, \"key\": \"$KEY\"}"
>
> # 5. Create the source over the drive. The first sync starts.
> curl -s -X POST "$BASE_URL/dc/sources" -H "$AUTH" \
> -H "Content-Type: application/json" \
> -d "{\"name\": \"Product docs\", \"store\": \"draganddrop\",
> \"scope\": \"private\", \"config\": {\"drive_id\": \"$DRIVE_ID\"}}"
> ```
>
> The file field must come last in step 3. Send the `ETag` with its quotes
> in step 4. Poll `GET /dc/sources` until the source has `status` `synced`.
```json theme={null}
{
"id": 2698,
"pid": "oGCnsJeH-fyO",
"created_at": "2026-11-04T11:22:21.956451Z",
"name": "Product docs",
"scope": "private",
"created_by": "user_31HPTBuBvX20xlQNAbvxjOxPbKB",
"store": "draganddrop",
"status": "synced",
"size": 111,
"embedded_size": 60,
"config": {"meta": {"status": "ok", "errors": null}, "values": {"drive_id": "NgvpdTDFHCeW"}},
"documents_synced": 2,
"documents_failed": 0,
"oversized_documents": 0,
"documents_embedded": 1,
"last_synced_at": "2026-11-04T11:22:27.200315Z",
"archived_at": null
}
```
In Python, `files` can be one file `Path`, a folder `Path` (its top-level files
only, no recursion, hidden files skipped), or an iterable of file `Path`s. The
SDK refuses these before any upload with `InvalidParams`:
* a plain `str`: wrap a path in `Path(...)`;
* an empty list, or a file that does not exist;
* two files with the same name;
* more than 50 files.
In JavaScript, `files` is one `File`, one `{ filename, content }` object whose
`content` is a `Blob`, `ArrayBuffer`, or `Uint8Array`, or an array of them.
There is no path input: read the files yourself, as in the snippet above. A
string, an empty array, two files with the same name, or more than 50 files
throw `InvalidParams` before any upload. See
[`client.sources`](https://docs.cominty.ai/js-sdk/reference#client-sources).
`name` is optional. Without it, the server generates one.
### Two ids
A source has two ids. Do not mix them up.
| Field | Type | Where it goes |
| - | - | - |
| `source.pid` | String | Every `client.sources` method, and every HTTP path |
| `source.id` | Number | The source filter of the `indexed_documents` capability and scope. See [Filters](https://docs.cominty.ai/guides/capabilities.md#filters). |
### Wait for the sync
The wait method polls the source until it is `synced`. By default, it gives up
after 10 minutes and polls every 5 seconds. The Python SDK takes seconds
(`timeout=600.0, interval=5.0`). The JavaScript SDK takes milliseconds
(`{ timeout: 600_000, interval: 5_000 }`).
| Status it sees | What it does |
| - | - |
| `synced` | Returns the source |
| `pending`, `syncing`, `out_of_sync`, `cancelling`, `pausing` | Keeps polling |
| Anything else (`error`, `paused`, `archived`, `daily_quota_reached`, `invalid_configuration`, ...) | Fails with `SourceSyncFailed`. Its `source` holds the source as last read. |
| Still in progress after the timeout | Fails with `asyncio.TimeoutError` in Python, and with `SourceSyncTimeout` in JavaScript. `SourceSyncTimeout` holds the last read in `source` (`null` if no read finished). |
> [tab] Python SDK
>
> ```python theme={null}
> from cominty_sdk import SourceSyncFailed
>
> try:
> await client.sources.wait_synced(source.pid, timeout=60, interval=2)
> except SourceSyncFailed as exc:
> print(exc) # source 'oGCnsJeH-fyO' stopped in status 'paused' instead of reaching 'synced'
> print(exc.source.status.value) # paused
> ```
> [tab] JavaScript SDK
>
> ```ts theme={null}
> import { SourceSyncFailed } from '@cominty-ai/sdk'
>
> try {
> await client.sources.waitSynced(source.pid, { timeout: 60_000, interval: 2_000 })
> } catch (error) {
> if (error instanceof SourceSyncFailed) {
> console.log(error.message) // source 'oGCnsJeH-fyO' stopped in status 'paused' instead of reaching 'synced'
> console.log(error.source.status) // paused
> } else {
> throw error
> }
> }
> ```
## Give the source to an agent
Put the source's number `id` in an `IndexedDocumentsCapability`.
> [tab] Python SDK
>
> ```python theme={null}
> from cominty_sdk import (
> AgentCapabilities,
> IndexedDocumentsCapability,
> WebCapability,
> )
>
> agent = await client.agents.create(
> name="Docs assistant",
> instructions=(
> "Answer only from the company documents, always search them first. "
> "Cite the documents you used."
> ),
> capabilities=AgentCapabilities(
> indexed_documents=IndexedDocumentsCapability(
> activation="always", source_ids=[source.id]
> ),
> web=WebCapability(activation="never"),
> ),
> )
> ```
> [tab] JavaScript SDK
>
> ```ts theme={null}
> const agent = await client.agents.create({
> name: 'Docs assistant',
> instructions:
> 'Answer only from the company documents, always search them first. ' +
> 'Cite the documents you used.',
> capabilities: {
> indexedDocuments: { activation: 'always', sourceIds: [source.id] },
> web: { activation: 'never' },
> },
> })
> ```
> [tab] HTTP
>
> ```bash theme={null}
> curl -X POST "https://ds.cominty.com/agents" \
> -H "x-cominty-token: $COMINTY_API_KEY" \
> -H "Content-Type: application/json" \
> -d '{
> "name": "Docs assistant",
> "instructions": "Answer only from the company documents, always search them first. Cite the documents you used.",
> "capabilities": {
> "indexed_documents": {"activation": "always", "source_ids": [2698]},
> "web": {"activation": "never"}
> }
> }'
> ```
You can also pass the source in the thread capabilities on `chat.start`, or
narrow it for one message with the message scope. See
[Capabilities](https://docs.cominty.ai/guides/capabilities.md#find-your-ids).
### Chat as the key owner
A source created with a key is **private to the key owner**. A chat that searches
it must run as the owner: create the client with the owner's id as its user id.
The owner is `agent.owner` on any agent the key created.
> [tab] Python SDK
>
> ```python theme={null}
> from cominty_sdk import AsyncCominty
>
> async with AsyncCominty(user_id=agent.owner) as owner_client:
> run = await owner_client.chat.start(
> agent_id=agent.id, message="What is the code word?"
> )
> message = await run.result()
> print(message.content)
> # The code word is **tangerine-4f2a**.
> ```
> [tab] JavaScript SDK
>
> ```ts theme={null}
> import { Cominty } from '@cominty-ai/sdk'
>
> const ownerClient = new Cominty({ userId: agent.owner })
> const run = await ownerClient.chat.start({
> agentId: agent.id,
> message: 'What is the code word?',
> })
> const message = await run.result()
> console.log(message.content)
> // The code word is **tangerine-4f2a**.
> ```
> [tab] HTTP
>
> Send the owner's id in `options.user_id` on `POST /chat`.
With any other user id, the agent's document tool reports `You have no
authorized document sources.` The agent then says it found nothing. The
`COMINTY_USER_ID` of your client may not be the key owner, so read `agent.owner`.
## Read a citation
An answer cites its documents inline, as
``. The id is the
source id, a `#`, and the document's own identifier. It is not always a number.
Resolve it to its source and file:
> [tab] Python SDK
>
> ```python theme={null}
> import re
>
> cites = re.findall(r' document = await client.documents.get(cites[0])
> print(document.model_dump_json(indent=2))
> ```
> [tab] JavaScript SDK
>
> ```ts theme={null}
> const cites = [...message.content.matchAll(/ m[1]!)
> const document = await client.documents.get(cites[0]!)
> console.log(JSON.stringify(document, null, 2))
> ```
> [tab] HTTP
>
> The id contains a `#`, so URL-encode it.
>
> ```bash theme={null}
> curl -G "https://ds.cominty.com/documents" \
> -H "x-cominty-token: $COMINTY_API_KEY" \
> --data-urlencode "id=2698#10778"
> ```
```json theme={null}
{
"document_id": "2698#10778",
"source_id": 2698,
"source_pid": "oGCnsJeH-fyO",
"source_type": "draganddrop",
"document_name": "facts.txt",
"document_uri": "https://app.cominty.com/cominty_drive/files/ZsripjwG2ohN",
"document_extension": "txt",
"document_mime_type": "text/plain"
}
```
* An unknown id raises `NotFoundError` (404). There is no call that lists
documents. An empty id is refused before any request (`ValueError` in
Python, `InvalidParams` in JavaScript).
* `document_uri` can be null. For a drag-and-drop source, it ends with the id
of the file, the one the download method takes.
To read the original file, list the source's files and download one:
> [tab] Python SDK
>
> ```python theme={null}
> for file in await client.sources.list_files(source.pid):
> print(file.id, file.name, file.size, file.mimetype)
>
> content = await client.sources.download_file(file.id) # bytes
> ```
> [tab] JavaScript SDK
>
> ```ts theme={null}
> const sourceFiles = await client.sources.listFiles(source.pid)
> for (const file of sourceFiles) {
> console.log(file.id, file.name, file.size, file.mimetype)
> }
>
> const content = await client.sources.downloadFile(sourceFiles.at(-1)!.id) // Uint8Array
> ```
> [tab] HTTP
>
> ```bash theme={null}
> curl "https://ds.cominty.com/files/$FILE_ID" \
> -H "x-cominty-token: $COMINTY_API_KEY"
> ```
>
> The answer is a JSON string: a download URL, valid for 5 minutes. Fetch it
> without the API key.
The SDKs fetch the presigned URL for you, without sending the key. They hold the
whole file in memory.
## Keep a source up to date
> [!NOTE]
>
> New files follow the same rule. After you add or remove a file, the source is
> indexed again, and the change reaches the agent only once the source is
> `synced` again.
> [tab] Python SDK
>
> ```python theme={null}
> added = await client.sources.add_files(source.pid, [Path("extra.csv")])
> source = await client.sources.wait_synced(source.pid)
> await client.sources.remove_file(source.pid, added[0].id)
> source = await client.sources.wait_synced(source.pid)
> source = await client.sources.update(source.pid, name="Product docs (2026)")
>
> source = await client.sources.sync(source.pid) # force a sync now
> source = await client.sources.pause(source.pid) # cancels the running sync
> source = await client.sources.archive(source.pid) # terminal
> ```
> [tab] JavaScript SDK
>
> ```ts theme={null}
> import { readFile } from 'node:fs/promises'
>
> const added = await client.sources.addFiles(source.pid, [
> new File([await readFile('extra.csv')], 'extra.csv'),
> ])
> source = await client.sources.waitSynced(source.pid)
> await client.sources.removeFile(source.pid, added[0]!.id)
> source = await client.sources.waitSynced(source.pid)
> source = await client.sources.update(source.pid, { name: 'Product docs (2026)' })
>
> source = await client.sources.sync(source.pid) // force a sync now
> source = await client.sources.pause(source.pid) // cancels the running sync
> source = await client.sources.archive(source.pid) // terminal
> ```
> [tab] HTTP
>
> ```bash theme={null}
> curl -X PUT "https://ds.cominty.com/dc/sources/$PID/status?operation=sync" \
> -H "x-cominty-token: $COMINTY_API_KEY"
> ```
>
> `operation` is `start`, `pause`, `sync`, or `archive`.
* **Adding files** uploads them into the source's drive, then requests a sync
if the source's status accepts one. If a sync is already running, the new
files are left for it or for the daily sync. The limit of 50 files counts the
files already in the source.
* **Removing a file** deletes it, then requests a sync under the same rule. An
empty file id is refused before any request (`ValueError` in Python,
`InvalidParams` in JavaScript).
* `update` only changes the name. The drive of a drag-and-drop source cannot
change.
* `list()` returns the key owner's sources and the organization's sources,
archived ones included. `get(pid)` reads the list and picks, because a key has
no single-source read. An unknown `pid` raises `NotFoundError`.
Each operation is valid from some statuses only. Any other combination is an
`APIError` with status 400 and `detail` `Operation is not valid`.
| Operation | Valid from |
| - | - |
| `start` | `paused` |
| `pause` | A running sync, which it cancels |
| `sync` | `synced`, `error`, `out_of_sync`, `daily_quota_reached` |
| `archive` | Any status but `archived` |
`pause` returns the source as `pausing`. It reaches `paused` a moment later, and
only then does `start` resume it. A `start` while the source is still `pausing`
fails with the same 400.
> [!WARNING]
>
> `archive` removes the indexed documents and the files. The source stays in the
> list with `status` `archived`. There is no delete, and no way back.
## Limits
* A source holds at most **50 files**. The SDK checks it before it uploads.
* A user has at most **100 drives**. Each `sources.create` makes one, and a drive
cannot be deleted. Reuse a source and add files to it instead of creating a
new one for every batch.
* A presigned download URL is valid for 5 minutes.
* Uploads go straight to storage. The API key never reaches the storage host,
and presigned signatures never appear in error messages.
### Supported file types
The indexing pipeline reads more than 130 file formats. These are the most
common ones.
| Category | Common formats | Also supported |
| - | - | - |
| Documents | `pdf`, `docx`, `doc`, `pptx`, `ppt`, `rtf`, `txt`, `epub` | `docm`, `dot`, `dotm`, `pptm`, `pot`, `potm`, `potx`, `key`, `pages`, `hwp`, `wpd`, `wps`, `xml`, `vsd`, `vsdx`, and other office and legacy formats |
| Images | `jpg`, `jpeg`, `png`, `gif`, `bmp`, `tiff`, `webp`, `heic`, `heif` | `svg`, `htm`, `html` |
| Spreadsheets | `xlsx`, `xls`, `csv`, `tsv`, `numbers`, `ods` | `xlsm`, `xlsb`, `xlw`, `dif`, `slk`, `dbf`, `fods`, and other legacy formats |
| Audio | `mp3`, `wav`, `m4a`, `aac`, `ogg`, `oga`, `opus`, `flac`, `wma`, `aiff`, `amr` | Any other common audio format |
| Video | `mp4`, `mov`, `avi`, `mkv`, `webm`, `m4v`, `wmv`, `flv`, `mpeg`, `mpg`, `3gp`, `ts` | Any other common video format |
| Text | Plain text and text-like files: `text/*` types, `application/json`, and similar | - |
Audio and video files are **transcribed** by the indexing pipeline. Almost every
common audio and video format works. The agent searches
and reads the transcript. For a video, it does not see the images, only what is
said.
### What the agent can do with an indexed file
Indexing lets the agent **find** a file and **read its content**. Two limits
apply today:
* **Spreadsheets are not part of the search.** An Excel file (or another
spreadsheet) is registered in the source with a document id, but the lexical
and semantic search never returns its content. The agent can still list the
documents of a source, spreadsheets included, and open one by its document
id. It then reads the content extracted from the spreadsheet, as long as it is
not too large. For a large spreadsheet, or to work on its data, attach the
file to the chat instead, with the upload method of the
[Python SDK](https://docs.cominty.ai/sdk/reference.md#client-chat) or the
[JavaScript SDK](https://docs.cominty.ai/js-sdk/reference.md#client-chat).
* **The agent cannot download the raw file.** It reads the indexed content, but it
cannot fetch the original file to transform it, for example to edit a
document or to compute on a spreadsheet. To get the original, download it
from your own code, as in [Read a citation](#read-a-citation).
## Next steps
> [card] [Custom agents](https://docs.cominty.ai/guides/custom-agents.md)
>
> Create and manage the agent that searches the source.
> [card] [Capabilities](https://docs.cominty.ai/guides/capabilities.md)
>
> Narrow the sources and documents of a thread or a message.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.