@kinedb/embedded (0.0.0-gbf535bab3c21)
Installation
@kinedb:registry=https://git.kinedb.com/api/packages/kinedb/npm/npm install @kinedb/embedded@0.0.0-gbf535bab3c21"@kinedb/embedded": "0.0.0-gbf535bab3c21"About this package
@kinedb/embedded
The embedded engine of kinedb: the database runs inside your process, on wasm. There is no server, no socket and no network.
import { KineDB } from '@kinedb/embedded';
const db = new KineDB('shop');
const r = await db.sql('SELECT id, bal FROM acct');
await db.flush();
await db.close();
The two shapes of every kinedb SDK
kinedb ships each SDK twice, in every language. The names are fixed, and the API surface is identical, so an application changes ONE import line and nothing else.
| role | Rust crate | npm | status |
|---|---|---|---|
| wire types + TLV/frame codec | kinedb-protocol |
@kinedb/protocol |
the column-code table only |
| direct client (remote server) | (none yet) | @kinedb/client |
built |
| embedded engine + sync | kinedb |
@kinedb/embedded |
this package |
import { KineDB } from '@kinedb/client'; // talks to a remote kinedb-server
import { KineDB } from '@kinedb/embedded'; // talks to a local engine
sql() answers the SAME result on both packages, cell for cell. That is the
whole point of the pair, and the tests hold it still.
Install
The packages live on the forge, under the kinedb organisation:
https://git.kinedb.com/kinedb/-/packages. There is no npmjs.com copy.
.npmrc:
@kinedb:registry=https://git.kinedb.com/api/packages/kinedb/npm/
The registry needs no token: the kinedb organisation is public. A token without the
package scope gets 401 Unauthorized. To send a token, add the line
//git.kinedb.com/api/packages/kinedb/npm/:_authToken=<gitea token with package read scope>.
A build is named by the commit that produced it, like a docker image:
npm install @kinedb/embedded@ga1b2c3d4e5f6 # the commit
npm install @kinedb/embedded@pr-740 # the head of a pull request
The package ships the .wasm inside it, so there is nothing else to fetch.
Runtimes
The package needs bun 1.4.2 or newer, or Node 22.3 or newer. A browser needs WebAssembly, and OPFS for a durable database.
Use
import { KineDB } from '@kinedb/embedded';
const db = new KineDB('shop');
await db.sql('CREATE TABLE acct (id INT PRIMARY KEY, bal INT)');
await db.sql('INSERT INTO acct (id, bal) VALUES (1, 100)');
const r = await db.sql('SELECT id, bal FROM acct');
// r.type 'rows'
// r.columns ['id', 'bal']
// r.types [{code: 2, name: 'Int', param: 0}, {code: 2, name: 'Int', param: 0}]
// r.data [[1, 100]]
// r.count 1
await db.close(); // each commit is already durable — see below
The constructor is SYNCHRONOUS and LAZY. The first await db.sql(...) loads
the .wasm and opens the database, because sql() is already async in
@kinedb/client and the import line must stay one line.
Options
new KineDB(name, { dir, wasm, host, memory, memoryBudget });
| option | what it does |
|---|---|
dir |
the directory the default node:fs host roots at. Default: the working directory, so new KineDB('shop') is ./shop. |
wasm |
where the module is: a path, a URL, an ArrayBuffer/Uint8Array, or a compiled WebAssembly.Module. A path is a POSIX path, a Windows path (a drive path, a UNC path, a root-relative or a drive-relative path), or a path relative to the working directory. A runtime with no working directory (a browser) refuses a relative path: pass a URL, for example new URL('k.wasm', import.meta.url). Default: the .wasm shipped in this package. |
host |
an explicit host, which overrides the default choice. |
memory |
true opens an in-memory database. It writes NO file. |
memoryBudget |
the bytes the engine may use (B1429, B1610). The block cache takes 1/3, the two footer-window caches 1/16 and 1/64, and the tree-node cache 1/144 (at least 1 MiB), the embedded plan of MemoryPlan in kinedb-cache. Minimum: 4 MiB. A wasm memory never shrinks, so a large budget keeps the page large. Default for a named database: 32 MiB. Default for memory: true: no budget; see An in-memory database. |
durability |
'commit' (the default) or 'manual'. See Durability: two modes. Another value is an error at open. |
Durability: two modes
'commit' (the default). A statement that commits answers only after its
data and the HEAD file are on disk. A power cut, a crash of the process or a
closed tab after the answer loses no committed write. This includes the offline
writes that sync() keeps for the next push(): the op-log row and the table
row are one local commit, so they reach the disk together. A
transaction([...]) call syncs once, at its commit. A statement that reads
does not sync.
'manual' (opt-in, fast). A write stays in memory until you call
flush(). A crash loses EVERY write since the last flush, not only the last few
rows, and the offline op-log with them. Use it when the application accepts that
loss, for example a bulk import that it can run again:
const db = await KineDB.open('import', { durability: 'manual' });
for (const row of rows) await db.sql(insertOf(row));
await db.flush();
The cost. Each durable commit is one sync of the disk. On a native disk
(Linux, ext4) 500 single-row INSERTs took about 13 ms each in 'commit' mode and
about 3.5 ms each in 'manual' mode. Put many rows in one transaction([...])
call to pay the sync once.
If the sync fails in 'commit' mode, the statement answers an error that
says the write is applied but not durable. flush() tries the sync again.
A failed fsync of a data segment file is different (B2960): the disk can lose
the data of a failed fsync, so kinedb does not try again. The database stops.
On wasm32 the panic traps the module. Restart the process.
In a browser, the 'commit' mode writes HEAD.tmp on each commit, so the
OPFS host needs its pool of spare files (the default poolSize is 8). An OPFS
host with poolSize: 0 opens only in 'manual' mode.
flush()
flush() makes every write durable. In 'commit' mode each commit already is,
and flush() has no work. In 'manual' mode, call it when a batch of writes is
done, and before a tab closes:
await db.sql("INSERT INTO acct (id, bal) VALUES (1, 100)");
await db.flush();
close() does not flush, on purpose: in 'manual' mode durability is a call
you make, so discarding a scratch database stays possible.
One opener per database
A database has one opener (B2150, B2160). A second open of the same database fails with
B2150: the database directory … is already open. This is true for a second Node process,
a second KineDB in one process, a second Web Worker and a second browser tab. Call
close() before another opener takes the database.
- Node: the host writes
<dir>/<name>/LOCKwith its pid, its host name, the start time of the process and its pid namespace.close()and a normal exit remove the file. After a crash, the next open takes over theLOCKof a process that is gone. - Browser: the host takes a Web Lock (
navigator.locks). The browser frees it when the tab or the worker closes.
A host of your own (host:) must supply the imports fs_lock_dir and fs_unlock_dir. See
src/node-host.js for the contract. The repairs are in
web/docs/troubleshooting.md, "the data directory is already open".
In a browser
The engine needs a synchronous filesystem, and the only one a browser has is
the Origin Private File System through createSyncAccessHandle, which exists
in a Web Worker and not on the main thread. So run this package in a worker.
OPFS reaches a file with an ASYNCHRONOUS call, and a wasm import must be
synchronous, so the OPFS host answers "not yet" and records what it needs.
KineDB.open() drives that retry loop for the open:
// inside a Web Worker
import { KineDB } from '@kinedb/embedded';
const db = await KineDB.open('shop'); // drives host.prepare() for you
await db.sql("INSERT INTO acct (id, bal) VALUES (1, 100)");
await db.flush();
After the open, no call needs a prepare. The host keeps a pool of spare files
per database in <name>/.opfs-pool/ (8 by default), and a file the database
creates later — the HEAD.tmp of a durable commit, the next 64 MiB segment — takes a
spare at once. The package refills the pool before every call, and putFile
refills it between its 1 MiB slices. A name map in
the same directory, with a checksum per record, keeps each file under its name
across a reload. createOpfsHost(root, { poolSize }) sets the size; 0 turns
the pool off.
A rename is atomic, also after a crash (B2839). OPFS has no synchronous rename,
so when the name map cannot move a name, the host copies the bytes. The copy
runs under a redo journal, <name>/.opfs-rename: the host writes an intent
record before the copy and clears it after. The next open finishes or redoes a
copy that a crash stopped, so a file such as HISTORY_FLOOR or DICTS holds
its old record or its new record, never a torn mix. The host hides the journal
from the database, as it hides the pool.
A reopen of a durable database creates no file: the open continues the last segment (B1725). So a page that opens its database at every load keeps the same files. A pool that older versions grew stops growing. Its extra spares are 0-byte files, and the next creates use them before the pool makes a new spare.
One host serves one database. Several databases can share one OPFS root, each with its own host, in one worker or in several. A host opens sync access handles only of the files of its own database, so it never blocks the open of another database under the same root (B2184).
One statement that creates more new files than the pool holds (more than 512 MiB of segments, with the default of 8) fails, and so does every create with the pool off. The message names the paths. Open them and repeat the call:
await db.host.prepare([...db.host.pending]);
A local statement is NEVER retried for you: a failed INSERT that partly applied
and then ran again would write the row twice. (A statement on a connect()
connection is different: see the hold retry in "Connect to a server".)
make test-browser (from the repository root) runs this package in a real
headless Chrome, on a real OPFS (B1155). It writes, flushes and reads back in a
new tab. It kills the browser with SIGKILL and reads the flushed rows after the
relaunch. It stores a 72 MiB file across a segment rotation and reads it back
byte-exact after a restart. It starts a kinedb server and syncs an empty
database from it: the first sync is a pack, and the rows stay after a restart
of the browser (B3501). It needs Chrome or Chromium, the staged wasm
(make wasm, then bun run build here) and a server built from the tree
(KINEDB_SERVER_BIN, else target/release/kinedb). The gate does not run it, because the
gate has no browser (owner 2026-09-30, B1468: not for now). The check serves
the files as they are, with no bundler, so it also proves the next paragraph.
The package loads in a browser WITHOUT a bundler (B1469): a Web Worker can
import src/index.js as a plain ES module, from a CDN or through an import
map. No module of the package imports a node: module statically.
src/node-host.js reaches node:fs through process.getBuiltinModule, so
Node needs 22.3 or newer. src/no-bundler.test.js walks the static import
graph as such a browser walks it.
An in-memory database
const db = new KineDB('scratch', { memory: true });
It writes no file. The database lives in the wasm memory and goes away with it.
With no memoryBudget, the memory keeps every row. Give a budget to set a limit:
const db = new KineDB('scratch', { memory: true, memoryBudget: 16 * 1024 * 1024 });
The rows may then use the budget less the tree-node cache and less 1/3 for the work of the statements. A write past that share fails with an error that names the budget, and the rows committed before it stay readable.
The result of sql()
A statement that writes:
{ type: 'created' } // CREATE, and a transaction([...]) call
{ type: 'inserted', count: 1 } // also 'updated', 'deleted'
A statement that reads:
{ type: 'rows',
columns: ['id', 'name'],
types: [{ code: 2, name: 'Int', param: 0 }, { code: 4, name: 'Text', param: 0 }],
data: [[1, 'ada']],
count: 1,
schemaId: null }
types[i].code is the frozen table COLUMN_CODES, which this package exports.
The JS value of a cell, by code:
| code | name | the value you get |
|---|---|---|
| 0 | Null | null — every cell of the column is NULL |
| 1 | Bool | boolean |
| 2 | Int | Number, or an exact BigInt outside ±(2^53−1) |
| 3 | Float | Number, including NaN and both infinities |
| 4 | Text | string |
| 5 | Bytes | Uint8Array |
| 6 | Timestamp | Number, or an exact BigInt outside ±(2^53−1) |
| 7 | Json | the PARSED value; the raw text when it will not parse |
| 8 | Blob | Array<Uint8Array(32)> — one BLAKE3 chunk hash each |
| 9 | Vector | Float32Array of types[i].param elements |
An INT past 2^53 comes back as an exact BigInt, not as a rounded Number, so
every integer this engine stores reads back with all of its bits.
schemaId is always null. It is the cache key of the WebSocket schema cache
in @kinedb/client; an embedded reply carries its header every time, so there
is no cache to key.
Parameters
sql(text, params) takes the values of $1, $2, … apart from the text. The
engine binds them, so a value is never parsed as SQL. The same call works on
a synced table and on a Connection.
const name = "it's'); DELETE FROM t; --";
await db.sql('INSERT INTO t (id, name) VALUES ($1, $2)', [2, name]); // one row, a string
await db.sql('SELECT id FROM t WHERE name = $1', [name]);
The type table is the one of @kinedb/client: an integer number or a
bigint → INT, a fractional number → FLOAT, string → TEXT, boolean →
BOOL, Date → TIMESTAMP (microseconds), Uint8Array → BYTES, Float32Array
→ VECTOR, null or undefined → NULL. Any other value throws a TypeError or
a RangeError before the statement runs. src/params.js is a verbatim copy of
the client file, and src/params.test.js fails when the two copies differ.
Transactions
A transaction is ONE call (B2174). There is no open transaction: sql('BEGIN'),
sql('COMMIT') and sql('ROLLBACK') throw, and the error names
transaction(). The same call works on @kinedb/client.
await db.transaction([
['INSERT INTO acct (id, bal) VALUES ($1, $2)', [1, 100]],
['UPDATE acct SET bal = bal - $1 WHERE id = $2', [10, 2]],
'INSERT INTO audit (id) VALUES (7)',
], { token: 'transfer-42' }); // { type: 'created' }
- An item is a statement (a string,
[sql]or[sql, params]) or a file ({ putFile: [collection, name, data] }, see Files). - Every item commits in one commit, or none of them. The error of the item that failed is the error of the call.
tokenmakes a retry safe: a call whose token committed before applies nothing and answerscreated.- A read in the list runs against the committed state and gives no rows back. DDL cannot be an item.
Describe a table
describe(table) answers the schema of one table: the columns, the primary key
and the indexes. It runs DESCRIBE, SHOW INDEXES FROM and DESCRIBE again
through sql(), so it uses the database and the branch of the connection.
const t = await db.describe('users');
// { table: 'users', schema_version: 42, primary_key: ['id'],
// columns: [{ name: 'email', type: 'Text', not_null: true, default: null,
// primary_key: false, unique: true, indexes: ['users_email'] }, …],
// indexes: [{ name: 'users_email', columns: ['email'], unique: true }, …] }
- The names are snake_case.
schema_versionis an integer. The value 0 means unknown. uniqueis true for one column. A UNIQUE index of ONE column sets it. A composite unique index does not set it.defaultis SQL text. The text'it''s'is a default with a quote. The value is null when the column has no default.- The schema can change during the read. When the two
DESCRIBEreads show two versions, the call starts again. After 3 reads, it throws aKineDBErrorwith the wordsschema changed. - A name that is not a plain identifier goes in brackets.
describe('my table')sendsDESCRIBE [my table].
describe() is on the database and on a Connection. src/describe.js is a
verbatim copy of the file of @kinedb/client, checked by src/params.test.js.
Call a function
call(name, args) calls a function on the server with typed arguments. It is
on a Connection and on the database. The rules are the ones of
@kinedb/client (src/call.js is a verbatim copy, checked by
src/params.test.js):
- the connection reads
SHOW FUNCTIONSone time and keeps every signature; - each argument is converted to the type of its parameter (
2into a FLOAT is a FLOAT, an object into a ROW is JSON); a value that does not fit throws before anything is sent; - a stale signature (the server refusal
signature) gives one new read and one new send; RETURNS <type>gives the value,RETURNS TABLE (…)gives row objects.
const conn = await db.connect('ws://127.0.0.1:8080/ws', { user: 'root', password: '…' });
await conn.call('add_points', [1, 50]); // 51
await db.call('add_points', { player: 1, points: 50 });
await db.call('add_points', [1, 50], { offline: 'refuse' });
db.call() is ONLINE when the database has a live connect() connection: the
call goes through it. The sync session does not count, because it has no
statement link to the server. OFFLINE, the embedded database cannot run a
function yet (B2175), so the offline option only picks the words of the
refusal:
offline |
offline result |
|---|---|
'preview' (the default) |
a KineDBError: the database cannot run or queue a function yet (B2175) |
'queue' |
the same KineDBError (the queue is B2175) |
'refuse' |
a KineDBError: call(<name>) needs the server: the database is offline |
Any other offline value throws a TypeError, online and offline.
Errors
A failed statement rejects with a KineDBError carrying the engine's own
message:
import { KineDBError } from '@kinedb/embedded';
try {
await db.sql('DESCRIBE absent');
} catch (err) {
err instanceof KineDBError; // true
err.message; // 'NotFound: table not found: absent'
}
A SELECT from a table that does not exist throws table not found: <name>, the
same error as UPDATE, DELETE and DESCRIBE. An empty table answers zero rows.
On a connection, a KineDBOutcomeUnknownError (a subclass of KineDBError)
tells you that the server sent the statement on, but the reply was lost (B2304).
The write MAY have applied. The error has outcomeUnknown true, retryable
false and response.refusal.kind outcome_unknown. The connection never sends
the statement again. Read the data back before you send it again.
Rows as objects
import { rowsAsObjects } from '@kinedb/embedded';
const r = await db.sql('SELECT id, name FROM people');
rowsAsObjects(r); // [{ id: 1, name: 'ada' }]
The same function @kinedb/client exports, with the same behaviour: a result
that is not rows answers [], so it is safe on whatever sql() returned.
Files
putFile and getFile have the shape of @kinedb/client:
await db.sql('CREATE TABLE photos (name TEXT PRIMARY KEY)');
await db.sql('CREATE TABLE albums (id INT PRIMARY KEY, photo TEXT)');
const put = await db.putFile('photos', 'beach.jpg', bytes); // a Uint8Array
// put.size the byte length
// put.hash the blake3 of the whole file, hex
// put.chunks the chunk hashes, hex, in file order
// A file and a row in ONE transaction: they land together, or not at all.
const done = await db.transaction([
{ putFile: ['photos', 'sea.jpg', bytes] },
['INSERT INTO albums (id, photo) VALUES ($1, $2)', [1, 'sea.jpg']],
]);
// done.type 'created'
// done.files the record of each file item, in list order: [{size, hash, chunks}]
const back = await db.getFile('photos', 'beach.jpg'); // a Uint8Array
The engine cuts the file into the same content-defined chunks as the server
and @kinedb/client (BLAKE3, at most 128 KiB each). The file goes into a TABLE
with a TEXT primary key (the file name is the key), or into a schemaless
collection.
A FILE AND A ROW COMMIT IN ONE TRANSACTION (B2174). A { putFile: [collection, name, data] } item of transaction([...]) commits with the other items of the
call. An error of any item commits nothing: no file and no row. The call reads
the source and stores the chunks first, and the commit names them; a failed
call leaves only orphan chunks, which the next sweep removes.
getFile checks every chunk against its hash and the whole file against its
hash. A missing file throws a KineDBError with the message
file not found: "<name>" in <table>, as @kinedb/client does.
A file is durable when its write commits, the same as a row (in 'manual' mode:
after flush()).
A file larger than the memory (B1429)
A file moves through the engine ONE PIECE AT A TIME, so the wasm memory holds a
chunk and never the whole file. putFile takes a stream as well as bytes:
// a Uint8Array, an ArrayBuffer, a Blob, a ReadableStream<Uint8Array>,
// or an (async) iterable of Uint8Array
const put = await db.putFile('videos', 'talk.mp4', file.stream());
const stream = await db.getFileStream('videos', 'talk.mp4'); // or null
for await (const chunk of stream) {
// one verified chunk (at most 128 KiB) per pull
}
putFile never concatenates the pieces, and a piece larger than 1 MiB goes into
the engine in 1 MiB slices. The answer is the same {size, hash, chunks} as for
the same bytes in one Uint8Array. When the source fails, or a piece is not
bytes, putFile rejects, nothing lands, and a ReadableStream or Blob source
is cancelled. A { putFile: [...] } item of transaction([...]) takes the
same sources.
getFileStream answers null for a missing file. It reads a chunk only when
you pull one, and checks it against its hash. The pull of the LAST chunk also
checks the size and the whole-file hash, and ERRORS the stream instead of
giving that chunk when either fails, so a damaged file never ends as if it were
whole. cancel() releases the reader, and so does the garbage collector for a stream you drop unread; close() of the database releases every
reader and every unfinished putFile. getFile reads through the same path,
so the wasm memory does not hold the file there either; its answer does.
Measured (R4 of B1429, bun 1.4.2, the node:fs host): a 256 MiB file streams in
and back out with the wasm memory at 16.4 MiB at its peak.
Connect to a server
connect() opens a WebSocket to a running kinedb-server, presents the
credential, and answers a connection. With a user and a password, a user and a
key, or a token, it answers only after the server GRANTS the login, so the
connection you hold is one you can use. With {anonymous: true} it presents
nothing and there is no grant to wait for — see the four shapes below.
const conn = await db.connect('ws://127.0.0.1:8080/ws', {
user: 'root',
password: '…',
});
const r = await conn.sql('SELECT id, bal FROM acct');
// The SAME result shape `db.sql()` answers, key for key.
await conn.close();
It takes ONE of four credential shapes. Name the one you want:
await db.connect(url, { user: 'root', password: '…' }); // a user and a password
await db.connect(url, { user: 'alice', key: signer }); // a key pair
await db.connect(url, { token }); // a session bearer
await db.connect(url, { anonymous: true }); // a permissive server
| shape | what the connection presents |
|---|---|
{user, password} |
the login the server checks against that user. |
{user, key} |
a signature of a new server challenge, made by the private key of that user. |
{token} |
a session bearer, for example the token POST /login answers. |
{anonymous: true} |
nothing. The connection never logs in. |
connect(url, {}) THROWS, and that is deliberate. A caller who leaves the
credentials out gets the error AT THE CALL, instead of an unauthenticated
connection that fails at its first statement. So anonymous access is a shape you
ask for by name, and it is never the default.
Two things a reader does not guess, and both matter:
- The password shape and the token shape ride the SAME login frame. A token login sends an empty user name and an empty password, because the server reads the token alone and never looks at the other two fields.
{anonymous: true}sends NO frame at all. A permissive server serves a connection that never logs in, and an enforcing server refuses it. "No credentials" and "empty credentials" are different things on the wire: a login frame that carries an empty user name and an empty password is refused withinvalid username or password, whatever the server's mode.
Log in with a key pair
A user with a public key (ALTER USER alice ADD PUBLIC KEY '<point or PEM>') logs
in with the private key. The private key never goes to the server, and this
package never holds it: you give a signer, and the signer holds the key.
import { keySignerFromPkcs8 } from '@kinedb/client';
// The .pk8.pem file of the web console, or the PKCS#8 DER bytes.
const signer = await keySignerFromPkcs8(pem);
const conn = await db.connect(url, { user: 'alice', key: signer });
await db.sync('orders', { url, user: 'alice', key: signer });
- The signer is
{keyId, sign(bytes) → Promise<Uint8Array>}.keySignerFromPkcs8of@kinedb/clientis one source of a signer: it imports the ECDSA P-256 key with WebCrypto as a NOT extractableCryptoKey. This package does not import@kinedb/client, and any object of that shape works. keyIdis the lower-case hex SHA-256 of the 65-byte public point (0x04 || x || y), 64 characters.signreturns the rawr || ssignature (64 bytes) of SHA-256 overbytes.- The credential is checked at the call. A key without a 64-hex
keyId, without asignfunction or withoutuserthrows. A key next to apasswordor atokenis two shapes, and it throws too. - Each login asks the server for a new challenge, with a new 32-byte nonce from
crypto.getRandomValues, and signs it once. A signature is never sent twice. The package does not retry a refused login. reconnect()and every new sync session sign a NEW challenge with the same signer. The package keeps the signer, never a signature.- Errors: a wrong key and an unknown user both get the server text
invalid username or password. A server with no key login givesthe server is too old for a key login; update the server. A signer that throws givesthe key did not sign; the signer's own error is inerr.causeonly, never in the message. A signature that is not 64 bytes is refused. In both cases nothing more goes to the server. - The signer has 30 s to answer (the challenge validity of the server). After
that, the login fails with
the key did not sign in time, and a later signature is dropped, never sent. console.log(err)and similar tools printerr.cause, which can hold the error text of the signer. Do not log the error object to a shared sink. Logerr.message.- No error text contains a nonce, a signature or key bytes.
A connection carries four methods.
| method | what it does |
|---|---|
sql(statement) |
runs one statement on the server and answers the shape db.sql() answers. |
watch(table, callback) |
subscribes to a table. It answers when the server acknowledges, and the callback then gets every change notification. |
reconnect() |
opens a new socket, presents the SAME credential again, and re-issues every subscription. |
close() |
shuts the socket. Every later statement rejects. |
A statement or a transaction on the connection that gets a typed hold refusal
(refusal.kind write_hold or not_ready) is sent again for you (B2298). The
server sent the refusal before the write took effect, so nothing was written.
The connection waits the retry_ms of the server between two attempts, for up
to 20 s from the first send. A transaction is sent again WHOLE. Every other
error rejects at once, as before. retry: false turns the hold retry off, for
the connection or for one transaction:
const conn = await db.connect(url, { user, password }, { retry: false });
await conn.transaction(statements, { retry: false });
watch() answers an object with the server's acknowledgement and a cancel():
const sub = await conn.watch('orders', (n) => {
console.log(n.table, 'changed', n.changed, 'row(s)');
});
sub.ack.table; // 'orders'
await sub.cancel(); // sends UNWATCH and stops the callback
cancel() stops the callback at once and sends UNWATCH to the server, so the
table stays cancelled across a later reconnect().
A reconnect() RESUMES every subscription that is still live. It does not
replay what changed while the socket was down: a notification names a table and
a row count, so read the table again after a reconnect if you need the rows.
A reconnect() also RE-PRESENTS the credential the first connect() took, and
the same one. A connection opened with a token logs in again with that token, a
connection opened with a key signs a new challenge with the same signer, and
an anonymous connection sends no frame again. So reconnect() never asks the
caller for a credential a second time.
The socket comes from the host. Both hosts this package ships carry the bridge,
so a default database connects with no setup. An in-memory database gets a host
that reaches nothing, by design — pass host: when you want one that connects.
Both hosts take the socket class as an option (B1460). The default is
globalThis.WebSocket. Pass a class of your own to wrap the sockets of ONE
database, for example to drop the link on purpose in a demo or a test:
const host = createNodeHost('./data', { WebSocket: MySocket });
// in a Web Worker: createOpfsHost(await navigator.storage.getDirectory(), { WebSocket: MySocket })
const db = new KineDB('shop', { memory: true, host });
The class needs the surface the bridge uses: new MySocket(url), binaryType,
send, close, and the four handlers onopen, onmessage, onclose and
onerror.
db.close() closes every connection the database opened.
How flush() reaches the disk
This section and the two below sit OUTSIDE the use region on purpose.
web/docs/clients/javascript.md includes the install and use regions
verbatim, and memory/public-docs-web-docs.md (owner 2026-09-11, B863) keeps a
defect, a limit and a "not built yet" list off a public page.
In 'manual' mode a write stays in the WASM MEMORY until flush(). A closed
tab drops that memory, so a database that was never flushed keeps nothing. In
'commit' mode (B1882) the engine runs the same two steps after each statement
that moved the head (Db::sync_committed in crates/kinedb/src/lib.rs).
flush() runs flush_buffer() and then store().force_sync(), and
force_sync is the ONLY call that writes the HEAD file:
BlockStore::set_head updates the head in memory alone
(DiskBlockStore::set_head and DiskBlockStore::force_sync in
crates/kinedb-storage/src/disk_store.rs).
So a database directory with no HEAD file was never flushed, whatever it
wrote — that is the first thing to check when rows do not survive a reopen.
How a bundler reaches a browser
Nothing changes for a caller: in a browser you still write
import { KineDB } from '@kinedb/embedded' inside a Web Worker, and
KineDB.open() chooses the OPFS host. This section is for whoever bundles it.
src/node-host.js took NAMED imports from node:fs and node:path before
B1469. A bundler replaced those with a browser stub whose only export is a
default object, and a NAMED import of that stub is a LINK ERROR, so a browser
build of this package used to fail outright:
"resolve" is not exported by "__vite-browser-external",
imported by "sdk/js/embedded/src/node-host.js"
package.json answers it with a browser field that maps
./src/node-host.js to ./src/node-host.browser.js, whose createNodeHost
throws and names the OPFS host. So the node host is not in the browser bundle
AT ALL — not merely silenced. web/demos/lib/browser-bundle.test.js builds
the graph and reads the emitted chunks, because absence is a claim only a
build can check. A bundler with no browser support needs the equivalent
alias; node:fs in the graph is the symptom.
The exact integers
An INT past ±(2^53−1) rides as a decimal STRING and comes back as a BigInt.
A JSON number past 2^53 loses its low bits, which is the bug B607 exists to
fix; this package must never reintroduce it. src/result.test.js holds both
sides of the boundary.
What a connection does not do yet
connect() is wired to the connection ABI: B1146 (#1231) brought the wire
protocol to wasm32, and B1162 (#1259) built this package's wrapper over it —
src/connection.js and src/ws-bridge.js.
A reconnect RE-ISSUES every subscription but does not REPLAY the gap. B861 (#781) separates the two, and the replay needs a version token that says where the subscriber stopped; that token is B1147 (#1232).
The sync
sync(), push(), freshness(), refused(), discardRefused() and
queryCoverage() are the whole surface. B1194 (#1307) built the first five
over the kdb_sync_* ABI — src/sync.js. db.sync(table, {url, user, password}) fills the local table from a remote kinedb and answers when the
fill is COMPLETE; after it the database answers reads on its own, with the
server stopped. A local write to a
synced table RUNS and appends its op-log entry in the same commit as the row
(B1149, #1234), and push() sends that log back.
THE SYNC NAMES A SERVER DATABASE (B3027). db.sync(table, {url, …, database}) syncs from the server database database; the default is
default. A value that is not a non-empty string is a TypeError before any
socket opens. The session is the one source of the target: each Query,
SyncOps and file-chunk frame carries it. The local database stores the
server database it syncs with (the identity row server_db, written at the
first sync, kdb_sync_set_target). A later call that asks for another one is
refused with sync target mismatch: this local database syncs with server database '<bound>'; the call asks for '<asked>'. Use a new local database to sync '<asked>'. A call with a url or a database asks for database ?? 'default'. A call with neither asks for the database of the session (live or
offline); with no session, for the stored record (kdb_sync_target); with no
record, for default. An open session refuses another name before any frame
of the session. With an offline session, a wire call first opens a socket and
logs in; then the module refuses before any other frame. A call that opens a
session waits for the open of another call, and then uses that session.
A record with no sync state behind it is bound again to a new name after the
session closes. So a refused first sync (a typo) can be done again with the
right name after db.close() and a reopen. The reconnect supervisor
opens the new session with the database of the lost one, before
kdb_sync_adopt. Use a new local database to sync another server database
(B3032 for a rebind of a record with synced rows). A .wasm without
kdb_sync_set_target (before B3027) syncs default only. There is no branch
option (B3031).
THE SYNC STATE IS ONE ROW FOR EACH SUBSCRIPTION (B2203). The table
kinedb_sync_subscriptions of the local database holds it, keyed by the
table and the subscription id; sync() uses the id "". A row holds the
predicate, the token, the state and last_used (for the leases of B2206). An
old kinedb_sync_coverage table moves at the first sync open and keeps its
sync state. Both names are reserved. db.queryCoverage(sql, options) answers
{ state } for one local SELECT, where state is covered,
offline_stale or not_covered; when the check is not sure, the answer is
not_covered (see web/docs/clients/javascript.md). B2203 built it over
kdb_sync_query_coverage. The query answer does not carry the state; ask
queryCoverage(sql).
THE SYNC KEEPS A LIVE TAIL (B1529, #1982). sync() subscribes with
WATCH t SINCE … LIVE by default, and after the fill the server sends each
change of the table as a WatchTail diff from the token of the diff before.
The module applies it inside kdb_sync_pump, from the typed payload, with the
token in the same local commit, and calls onChange(table, report, error) —
an option of sync(), kept per table. A diff whose base is not the local
token is dropped and counted (session.tailDropped). A lost socket leaves the
tables offline_stale; the session that the next sync() or push() with
credentials opens re-runs the live sync of every live table, after the push.
{live: false} is the one-shot fill.
THE FIRST FILL OF AN EMPTY DATABASE COMES AS A PACK (B3431, B3490). A
persistent open of a new or empty name DEFERS kdb_open (kdb_pack_name_empty)
and holds the lock of the directory (B2160). The first sync() of such a
database, with a user login and no pack: false, sends ONE PackDb frame
(B3506: it names the server database of the call) with the subscription of the call
(SELECT * FROM <t>, plus WHERE <predicate>), and writes the files of the
answer under the name, through the pool of the OPFS host
(kdb_pack_begin, kdb_pack_pump: a pump creates at most poolSize - 2
files, and the package refills the pool before each pump). Then
kdb_pack_open opens the files (the identity of this device, then the
marker goes), the session binds, and the first resume runs: report.pack
is {commit}, and upserts is 0 when nothing changed since the pack.
Any other call before the first sync opens the database normally.
A PACK IS NEVER A LOCK-OUT. A PackRefused, a server that does not know
the PackDb frame (unknown msg_type: 0x2F), a pack error (a refused query,
a node with no pack cache; one console.warn) and a pack that does not open
each give the normal fill in the same call, and report.pack is null. A
closed socket during the pack deletes the files, the sync() rejects as
retryable, and the next sync() asks for the pack again. Until each table
of the pack has resumed once, a query of the app (also getFile) is
refused with a "fresh pack" error. Each persistent open runs
kdb_pack_recover: a marker that a pool file of the OPFS host holds is seen
there too.
A RESUME CAN ANSWER A GAP (B2201). sync() adds ALLOW GAP to its WATCH.
When the server does not send the changes of a resume (more changed rows than
cluster.watch.max_fill_rows, or a token beyond the retention horizon), it
sends one gap frame with no rows, and the watch is live from the server HEAD.
A gap never keeps the local rows (B3443): a row that the person lost must not
stay on the device. The module drops the set of the table in one commit and
fills again from START, once. The pending writes stay. A refill with no gap
replaces the set, and report.gap is null. A gap of that refill leaves the
table empty, moves the token to HEAD and records the gap. report.gap names it (reason, since, limit, changedRowsAtLeast), and
freshness() says gapped with the gap until a full fill replaces the local
set. For a server_limit gap, report.gap.path names the read that counted
(table_walk, index_lookup or token_diff, B3233); the key is absent when
the server names no path. The path is not stored: freshness().gap has none. A gap is not an error: the page decides when to fill again, with
db.sync(table, {full: true}). That sync asks SINCE START, replaces the
local set and removes the gap.
THE SYNC OPENS A CONNECTION OF ITS OWN, beside the one connect() answers.
It must: a sync begins with UNWATCH <table>, the server removes watchers BY
CONNECTION, and on a shared connection that call would cancel the page's own
live subscription in silence. Any of the six methods may carry the url and ONE
credential shape; they open the session when there is none. The four shapes are
connect()'s own: {url, user, password}, {url, user, key}, {url, token} and
{url, anonymous: true}.
THIS SLICE SYNCS A TABLE, NOT A DOCUMENT COLLECTION. db.sync() on a
collection has never worked on any client, and two of its four breaks are on
the SERVER — the fill header carries zero columns, and a fill row decodes
empty. That is B1206 (#1325), and three of the four share the root filed as
B1205 (#1324). It is not a regression: B1148 and B1149 synced a table.
A FILE ON A SYNCED TABLE RIDES THE PUSH (B1421, #1780). putFile on a table
that sync() covers writes the file and appends ONE op-log entry that names
its chunks, in one local commit. push() then asks the server which chunks it
already holds, sends only the others, and sends the entry last. A push that
dies half way loses nothing: the next push sends only the chunks the server
does not hold yet, and a reply lost after the server applied the entry comes
back as replayed, never as a second file. push() reports the chunk leg as
chunksTotal, chunksHeld, chunksSent and chunkBytesSent. putFile over
a file the server already holds records the version you read, as a row write
does: when another device changed the file first, the push answers a conflict
and the server keeps its file (B1424, #1784). getFile and getFileStream
read the local store.
A FILE ON A SYNCED TABLE RIDES THE PULL TOO (B1425, #1785). A fill names the
manifest of each file row it brings, and sync() fetches only the chunks the
device does not hold, checks each one against its hash, and writes the row and
the manifest in one local commit. getFile then reads the file on the device.
A pull that dies half way keeps the chunks that arrived, so the next sync()
fetches only the rest. A file the server deleted, or a full fill that no longer
names it, no longer reads back. sync() reports the leg as chunksTotal,
chunksHeld, chunksFetched and chunkBytesFetched. A file that the server
deletes or changes during the pull does not fail sync(): the device skips it
(no row is written, and the older local row goes), counts it in
filesSkippedChanged, and the next sync() brings the delete or the new file.
A TRANSACTION ON A SYNCED TABLE STAYS ONE TRANSACTION (B1423, #1783; B2174).
Each statement and each { putFile: [...] } item of one transaction([...])
call records its op-log entry inside the transaction, so an error of any item
drops the rows, the files and the entries together. The entries of one
transaction form one GROUP: push() sends a group in one
frame, and the server applies every entry of it or none. One refused entry
refuses the whole group, and refused() lists every member. sql() records a
write to a synced table also when no sync session is open (after a reload).
A STREAMED FILE RIDES THE PUSH TOO (B1429). Every putFile source — a
ReadableStream, a Blob, an (async) iterable, bytes — goes through the one
stream path, and on a synced table its end writes the manifest and the op-log
entry in one commit, or, as an item of transaction([...]), stages both in the
transaction of the call. The transaction
keeps the file's chunk hashes, never its bytes. The refusals are checked when
the stream begins (fail fast) AND when it ends, because your code can run a row
write between two pieces; a refusal at the end drops the stream and nothing
lands. The push reads one chunk at a time from the store. Measured (bun 1.4.2,
the node:fs host): a 128 MiB streamed file on a synced table and its push
keep the wasm memory at 15.9 MiB.
Build and test
The .wasm is a build product and is gitignored. Build it, then stage it:
make wasm # from the repository root
cd sdk/js/embedded
bun run build # copies the .wasm into src/kinedb.wasm
bun test src/
src/result.test.js, src/column-types.test.js and src/node-host.test.js
need no .wasm at all. src/sql.test.js and src/files.test.js drive the real
engine and need one.
The web console links this package, so make web performs the two steps above
before it builds the bundle (scripts/build-web.ts, B1232).
What is in here
| file | what it is |
|---|---|
src/index.js |
the KineDB class — the public surface |
src/result.js |
the module's JSON → the result @kinedb/client answers |
src/column-types.js |
COLUMN_CODES and columnTypeName, a gated copy of the client's |
src/wasm.js |
loading, instantiation, the result buffer, the C ABI |
src/connection.js |
the Connection — the wrapper over the kdb_remote_* ABI |
src/sync.js |
the SyncSession — the wrapper over the kdb_sync_* ABI |
src/ws-bridge.js |
the WebSocket bridge both hosts carry, and the ws.listen seam |
src/node-host.js |
the node:fs host of the kinedb_host imports |
src/node-host.browser.js |
what the browser field maps the above to, so a browser bundle carries no node:fs |
src/opfs-host.js |
the browser host, over OPFS, for a Web Worker |
src/stage-wasm.ts |
the build script |
The C ABI this package calls is crates/kinedb-wasm/src/lib.rs. That is the
truth; this package follows it.