From 0e765bc90cdacbbcda26cf424cb449c1e8c86263 Mon Sep 17 00:00:00 2001 From: lyrathorpe Date: Mon, 6 Jul 2026 22:29:47 +0100 Subject: [PATCH] =?UTF-8?q?docs(reference):=2005-filesystem-dbf=20?= =?UTF-8?q?=E2=80=94=20SIBO/Workabout=20MX=20programming=20reference?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/reference/05-filesystem-dbf.md | 725 ++++++++++++++++++++++++++++ 1 file changed, 725 insertions(+) create mode 100644 docs/reference/05-filesystem-dbf.md diff --git a/docs/reference/05-filesystem-dbf.md b/docs/reference/05-filesystem-dbf.md new file mode 100644 index 0000000..f8f333b --- /dev/null +++ b/docs/reference/05-filesystem-dbf.md @@ -0,0 +1,725 @@ +# Psion SIBO / Workabout MX — File System and DBF Database API Reference + +Source: *PLIB Reference*, chapters **Files** (ch. 11) and **Database Files** (ch. 14), with +corroborating structural detail from *EPOC O/S System Services*, chapter 20 (*Database File +Management*). Everything below is drawn from those manuals; page-anchored citations are given as +`[PLIB p.NNN]` / `[SysSvc §20]`. OCR-garbled items are flagged explicitly. + +> **OCR caveat.** The source is scanned OCR text. Identifiers written `p_open`, function-name +> casing, and hex constants have been normalised to their obvious intended form. Where a +> *signature* itself is corrupted or truncated in the source, this is called out inline. Do not +> treat a normalised name as a guarantee of exact header spelling — verify against `p_file.h` / +> `p_dbf.h` before compiling. + +--- + +## Part 1 — The File System + +### 1.1 The file server and file systems (nodes) + +- All file operations are performed by a high-priority system process, the **file server** + (process name `SYS$FSRV`). A process must connect to it before use; the C startup module + normally does this automatically. Sending a message to the file server without connecting + panics with panic number 41. [PLIB p.117] +- The file server supports multiple **file systems**, also called **nodes**. `p_open` can list + them. Three were implemented at the time of writing [PLIB p.117]: + - `LOC::` — the local filing system, with the RAM drive `M:` and SSD drives `A:`, `B:`, … + (count depends on hardware). + - `REM::` — the remote filing system (present only while connected to a remote file server). + - `ROM::` — the ROM filing system for ROM-based files; normally invisible to the user; does + **not** support devices or directories. +- Within `LOC::` (and `REM::` when the remote end is a PC or another SIBO machine), the device + and directory structure is MSDOS-compatible. [PLIB p.117] + +**Media types (SSDs).** RAM SSDs (battery-backed static RAM, block-structured, not buffered on +SIBO) and Flash SSDs (linked variable-length records). Overwriting or deleting on Flash consumes +space that is only reclaimed by reformatting; a byte can be physically overwritten with **no** +storage penalty only if the new value is derived from the old by clearing bits to zero. This is +the property the DBF layer exploits. [PLIB pp.118–120, 195] + +### 1.2 File specification structure + +A full file specification has the form [PLIB p.120]: + +``` + +``` + +Example: `LOC::B:\NOTES\OLD\PLANS.TPD`, where: + +| Component | Meaning | Example | +|-----------|----------------------|-----------------| +| `` | file system node | `LOC::` | +| ``| device name | `B:` | +| `` | directory name | `\NOTES\OLD\` | +| `` | file name | `PLANS` | +| `` | extension name | `.TPD` | + +Rules [PLIB p.120]: + +- A file specification never exceeds `P_FNAMESIZE` (**128**) bytes including the zero terminator. +- The `` component is always `P_FSYSNAMESIZE` bytes long (excluding any zero terminator). +- Beyond `` and the `P_FNAMESIZE` total, make **no** assumptions about component sizes; the + syntax of ``/``/``/`` is owned by the node's file-system code (foreign + remote systems can map onto this model, e.g. VMS or Mac paths). Use `p_fparse`/`p_chdir` rather + than manipulating specs by hand. + +**Default path.** The file server stores a per-client default path (``) plus a +system-wide default assigned to new clients. Manipulate with: +`p_setpth` / `p_setpthasync` (set this process's default), `p_getpth` (get it), +`p_getpthbyid` (get another process's), `p_setdefaultpath` (set the system-wide default). +[PLIB pp.121, 126] + +**Parsing / directory manipulation.** +- `INT p_fparse(TEXT *name, TEXT *related, TEXT *full, P_FPARSE *perk);` — builds a full spec into + `full` (reserve `P_FNAMESIZE` bytes; `P_FNAMESIZE` bytes are always written). Components are + taken from `name`, then `related` (may be `NULL`), then the default path, in that order of + precedence. Output is upper-cased. `perk` (may be `NULL`) receives a `P_FPARSE` struct with the + lengths of each component and a wildcard-flags byte (`P_PWILD_ANY`, `P_PWILD_NAME`, + `P_PWILD_EXT`). `f_fparse` is identical but calls `p_leave` on error instead of returning it; + `p_fparseasync` adds a trailing `WORD *stat`. [PLIB pp.123–124] +- `INT p_chdir(TEXT *src, TEXT *outp, INT mode, TEXT *subdir);` — parse `src` and change its + directory per `mode`: `P_CD_ROOT`, `P_CD_PARENT`, or `P_CD_SUBDIR` (append zero-terminated + `subdir`). Reserve `P_FNAMESIZE` at `outp`. Async form `p_chdirasync`. [PLIB p.125] + +### 1.3 `p_open` — modes, formats, and access flags + +Channel-based file services all go through `p_open`: + +``` +INT p_open(VOID **ppfcb, TEXT *name, UINT mode); +``` + +On success returns 0 and writes the channel handle to `*ppfcb` (not written on failure). `name` is +parsed with a `NULL` related name. [PLIB p.142] + +`mode` is a bitwise-OR of exactly **one open mode**, exactly **one format**, and any combination of +**access flags**. + +#### Open modes (choose exactly one) [PLIB pp.142–143] + +| Mode | Behaviour | +|--------------|-----------| +| `P_FOPEN` | Open an existing file. `E_FILE_NXIST` if it does not exist. Normally used for read access. | +| `P_FCREATE` | Create a file that must **not** already exist. `E_FILE_EXIST` if it does. Requires `P_FUPDATE` for write. | +| `P_FREPLACE` | If the file exists, open and truncate to zero length; otherwise create it. Requires `P_FUPDATE` for write. | +| `P_FAPPEND` | Same as `P_FOPEN` but initial position is at end of file so the next write appends. `P_FRANDOM` not needed; `P_FUPDATE` needed for write. | +| `P_FUNIQUE` | Create a unique file, using the passed path as the related path; the unique name is written back to `name` (reserve `P_FNAMESIZE` bytes). `P_FUPDATE` not needed. | + +#### Formats (choose exactly one) [PLIB pp.142, 146–147; §list p.121] + +| Format | Meaning | +|-------------------|---------| +| `P_FSTREAM` | Flat binary file. | +| `P_FSTREAM_TEXT` | Flat binary file, but declares the file is text; on `REM::` it makes the remote side present/parse CRLF-terminated records. Locally usually identical to `P_FSTREAM`; no penalty, potential gain on remote. Prefer this for self-processed text files. | +| `P_FTEXT` | Record-oriented text file. Implemented as a client-side layer (the `TXT:` device) over `P_FSTREAM_TEXT`; data is buffered in that layer, so flushing is needed. | +| `P_FDIR` | Open a directory-listing channel (list files/subdirectories). | +| `P_FDEVICE` | Open a device-listing channel (list devices of a node). | +| `P_FNODE` | Open a node-listing channel (list file systems). | +| `P_FFORMAT` | Open a device-format channel (`LOC::` only at time of writing; OR in `P_FLOWDENSITY` for dual-density low-density format). | + +#### Access flags (combine as needed) [PLIB p.143] + +| Flag | Meaning | +|-------------|---------| +| `P_FUPDATE` | Write access as well as read. Writing without it → `E_FILE_RDONLY`. | +| `P_FRANDOM` | Random access required; needed to use `p_seek`. Without it, `p_seek` → `E_FILE_INV`. Do not specify unless you will seek (the FS may optimise sequential-only access). | +| `P_FSHARE` | Do not block the file from being re-opened for **read** access. Without it, a later open → `E_FILE_LOCKED`. **Cannot** be combined with `P_FUPDATE` (shared *write* is unsupported). | + +**Sharing rules:** any number of processes may open the same file for reading only, provided all +readers specify `P_FSHARE`. Multiple writers are never allowed. Once open for reading it may be +re-opened for reading but not writing; once open for writing it may not be re-opened at all. +[PLIB p.140] + +**Examples** [PLIB p.140]: +```c +p_open(&fcb, "fred.dat", P_FSTREAM | P_FSHARE); /* read only */ +p_open(&fcb, "fred.dat", P_FSTREAM | P_FUPDATE | P_FREPLACE | P_FRANDOM);/* writable */ +``` + +Selected `p_open` errors: `E_GEN_NOMEMORY`, `E_GEN_ARG` (illegal flag combination), +`E_FILE_DEVICE`, `E_FILE_NOTREADY`, `E_FILE_EXIST`, `E_FILE_NXIST`, `E_FILE_ACCESS`, +`E_FILE_DIRFULL`, `E_FILE_PROTECT`, `E_FILE_FULL`, `E_FILE_LOCKED`, `E_FILE_DIR`, plus any +`p_fparse` error. [PLIB pp.143–144] + +### 1.4 Reading and writing streams + +``` +INT p_read (VOID *pfcb, VOID *buf, UINT len); +INT p_write(VOID *pfcb, VOID *buf, UINT len); +INT p_close(VOID *pfcb); +``` + +- `p_read` reads `len` bytes (or the bytes remaining before EOF, whichever is smaller) from the + current position; returns the number of bytes read. At EOF, reads 0 bytes and returns + `E_FILE_EOF`. `len` must not exceed `P_FMAXSSIZE` (**16K**). Most efficient in multiples of + `P_FBLKSIZE` (**512**) on 512-byte boundaries. Other errors: `E_FILE_ABORT`, `E_FILE_READ`. + [PLIB p.144] +- `p_write` writes `len` bytes at the current position; increments position by bytes written; + returns 0 on success. `len` ≤ `P_FMAXSSIZE`. On Flash you may overwrite a single byte (`len==1`) + if the new byte only clears bits (then the modification date is not changed). Errors: + `E_FILE_FULL`, `E_FILE_RDONLY`, `E_FILE_ABORT`, `E_FILE_WRITE`. [PLIB p.144] +- `p_close(NULL)` is harmless (returns 0). Close may perform a final buffered write and a date + update, so it can return `p_write`/`p_fdate` errors — but it always closes the channel. To + handle failures cleanly, `p_iow(pfcb, P_FFLUSH)` first. `LOC::` on SIBO does not buffer written + data; EPOC-on-PC does. [PLIB p.143] + +**`p_iow` control operations** on a binary/text channel [PLIB pp.145–146]: +- `INT p_iow(VOID *pfcb, P_FFLUSH);` — flush buffered data and write the modification date. +- `INT p_iow(VOID *pfcb, P_FSETEOF, ULONG *peof);` — set the logical EOF to `*peof` (requires + `P_FUPDATE`). Extending pre-allocates storage on block devices; truncating also reduces the + current position if needed. +- `INT p_iow(VOID *pfcb, P_FCANCEL);` — cancel pending async requests (not actively supported; a + no-op even when a request is pending — simulate a cancel instead). + +### 1.5 Seeking + +**Binary / stream:** +``` +INT p_seek(VOID *pfcb, INT sense, LONG *ppos); +``` +`sense` is one of [PLIB p.144]: + +| `sense` | Meaning | +|-----------|---------| +| `P_FABS` | set position to `*ppos` | +| `P_FEND` | set position to `*ppos` relative to end-of-file | +| `P_FCUR` | set position to `*ppos` relative to current position | + +On success the new position is written back to `*ppos` and 0 returned. `E_FILE_INV` if the channel +was not opened with `P_FRANDOM`. A negative resulting position clamps to the start; beyond EOF +clamps to EOF (use `P_FSETEOF` to extend). Setting `*ppos=0` with `P_FCUR` senses the current +position; with `P_FEND` senses the file length. `f_seek` is the `p_leave`-on-error variant. On +`P_FSTREAM_TEXT` remote channels, only "seek to 0 (`P_FABS`)" and "seek to end (`P_FEND`, rel 0)" +are guaranteed. [PLIB pp.144–145] + +**Record-oriented text (`P_FTEXT`):** `p_seek` sets the **read** position only (writes always go to +EOF), and `sense` is different [PLIB pp.149]: + +| `sense` | Meaning | +|--------------|---------| +| `P_FREWIND` | position to the first record (`ppos` ignored) | +| `P_FRSENSE` | get the position of the last record read/written | +| `P_FRSET` | set the record position previously got with `P_FRSENSE` | + +Requires `P_FRANDOM` (else `E_FILE_INV`). Some remote systems may not support `P_FRSET`/`P_FRSENSE`. + +### 1.6 Text records (`P_FTEXT`) + +Convention: records are terminated by CRLF (CR = 13, LF = 10); a file may optionally end with SUB +(26). On read, the parser also accepts CR, LF, or LFCR as a terminator and treats a SUB as EOF; on +write it appends CRLF only (no SUB). Record content must not exceed `P_FMAXRSIZE` (**256**) bytes +and must not contain CR, LF, or SUB. [PLIB pp.146–148] + +- `p_read` returns the record length (bytes written to `buf`), positioning to the next record. If + `len` < record length, the first `len` bytes are read and `E_FILE_RECORD` returned (still + advances). Returns 0 on a zero-length record; `E_FILE_EOF` past the last record. [PLIB p.148] +- `p_write` writes a record of `len` bytes (0 … `P_FMAXRSIZE`); always appended at EOF; must not + contain delimiters. Errors incl. `E_FILE_RECORD` if oversize. [PLIB p.149] + +### 1.7 Directory, device, and node listing + +All three use the open → repeated `p_iow(..., P_FREAD, ...)` until `E_FILE_EOF` → close pattern. + +- **Nodes:** `p_open(&ncb, "FIL:" or NULL, P_FNODE)`; each `p_iow(ncb, P_FREAD, buf, pinfo)` + writes the next node name (`buf` capacity `P_FSYSNAMESIZE+1`, i.e. 6). Optional `pinfo` + (`P_NINFO*`) gets the same info as `p_ninfo`. [PLIB p.127] +- **Devices:** `p_open(&dcb, node-name, P_FDEVICE)`; `E_GEN_FSYS` if the node is bad, + `E_GEN_NSUP` if the node has no devices (e.g. `ROM::`). Each `p_iow(P_FREAD)` writes the next + device name (`buf` capacity `P_FNAMESIZE`); trailing arg `NULL`. [PLIB p.128] +- **Files:** `p_open(&dcb, name, P_FDIR)`; `name` is parsed with a wildcard related name (`*.*`), + so a name of `""` lists the current directory. Each `p_iow(P_FREAD, buf, pinfo)` writes the next + matching file name (excluding node/device/dir; `buf` capacity `P_FNAMESIZE`). Optional + `pinfo` (`P_INFO*`) carries per-file info. A root directory of a PC-based device may return a + volume-name entry with `P_FAVOLUME` set. [PLIB pp.133–134] + +`P_INFO` (from `p_file.h`) [PLIB p.134]: +```c +typedef struct { + UWORD version; + UWORD status; /* status bits */ + ULONG size; /* size of the file in bytes (end-of-file position) */ + ULONG modst; /* system time of last modification (secs since 1970-01-01) */ + UBYTE spare[4]; +} P_INFO; +``` +`status` bit fields: `P_FAWRITE` (not read-only), `P_FAMOD` (modified), `P_FAHIDDEN`, +`P_FASYSTEM`, `P_FADIR` (directory file), `P_FAVOLUME` (volume-name directory), `P_FATEXT` +(recognised text file; `LOC::` cannot recognise text files). [PLIB p.134] + +### 1.8 File / device / node information and non-channel operations + +- `INT p_finfo(TEXT *name, P_INFO *pinfo);` — info on one file/directory (same fields as a `P_FDIR` + read). Good atomic existence check (`E_FILE_NXIST` if absent). Async `p_finfoasync`. [PLIB p.135] +- `INT p_testpth(TEXT *dname);` — returns 0 if the directory component exists. [PLIB p.135] +- `INT p_ninfo(TEXT *node, P_NINFO *pninfo);` — node info. `P_NINFO { UWORD version; UWORD type; + UWORD formattable; UBYTE spare[26]; }`. `type` is `P_FSYSTYPE_FLAT` or `P_FSYSTYPE_HIER`. + [PLIB pp.127–128] +- `INT p_dinfo(TEXT *dname, P_DINFO *pdinfo);` — device + mounted-medium info. + `P_DINFO { UWORD version; UWORD mediatype; UWORD removable; ULONG size; ULONG free; + UBYTE name[P_VOLUMENAME]; WORD batterystate; UBYTE spare[16]; }`. Low byte of `mediatype`: + `P_FMEDIA_UNKNOWN|FLOPPY|HARDDISK|RAM|FLASH|ROM|WRITEPROTECTED`; high byte flags incl. + `P_FMEDIA_COMPRESSIBLE` (worth compressing out deleted records — **not** set for Flash), + `P_FMEDIA_DYNAMIC`, `P_FMEDIA_INTERNAL`, `P_FMEDIA_DUAL_DENSITY`, `P_FMEDIA_FORMATTABLE`. + [PLIB pp.129–130] +- Other non-channel calls (each with an `…async` twin): `p_rename`, `p_delete` (directory must be + empty), `p_mkdir` (creates intermediate dirs), `p_sfstat` (set attributes; with `P_FAVOLUME` set + in the mask, sets/deletes the volume label), `p_fdate` (set modification date, ≥ 1980-01-01). + [PLIB pp.136–139] +- Formatting: open with `P_FFORMAT` then repeatedly `p_read` (first read yields a `UWORD` total + count, subsequent reads step the format, `E_FILE_EOF` when done). Aborting early corrupts the + medium. [PLIB pp.131–132] + +--- + +## Part 2 — The DBF Database API + +### 2.1 The model + +A **database file (DBF)** is a binary file of **typed, variable-length records**. Used by MC Diary, +Series 3 Database, and OPL data files. DBFs are **Flash-friendly**: records can be appended, +deleted, or replaced in place on a Flash SSD without rewriting the whole file. [PLIB p.195] + +Key model properties [PLIB pp.195–197, 204–207]: + +- **Record type 0 = deleted.** Deleting a record only overwrites its 4-bit type field with zero + (which merely clears bits — legal on Flash). Therefore **deleting does not shrink the file.** +- **Append-only; updates = erase + append.** Updating a record deletes the original and appends + the modified version, so an update **always moves the record to the end of the file**. +- **Reclaiming space:** `DbfCompress` (only on a *compressible* medium — not Flash) or copy the + file record-by-record with `DbfCopyFile` (deleted records are not copied). +- **Sparse index** (optional): one 4-byte address per **sixteenth** record, held in a *separate + segment* (`DBF$nnnn.INX`, `nnnn` a 4-hex-digit number from the channel) so it does not consume + the app's data space. Enables fast random access and fast backward scans; adjusted on + append/delete so it always points to every 16th record. +- **Read-ahead buffer:** each file-server read fills a caller-supplied buffer (typically 4K), + usually pulling in many records; a read for a record already in the buffer just locates it. + **Most DBF services may overwrite the buffer.** The next read after the buffer is disturbed + re-reads the whole buffer (a performance hit), so any direct modification of the buffer that is + *not* done via a DBF service must be followed by `DbfTrash`. Services guaranteed **not** to + alter the buffer: `DbfFlush`, `DbfVersion`, `DbfAppend`, `DbfSense`, `DbfCount`. [PLIB p.196] + +**Record limit:** max **65534** records *of any one visible type*, numbered 0…65533. Since only one +type is visible per open, a file may hold more in aggregate. A file with more than the max (of the +visible type) is logically truncated to the max on open. [PLIB p.197] + +**End-of-file record.** Reading past the last record returns `E_FILE_EOF`; the current record +number (per `DbfSense`) then becomes *last record + 1* — the fictitious **end-of-file record**. +Reading before the first record (`DbfBackRead`/`DbfFindRead` backwards) also gives `E_FILE_EOF` +with current record number 0. If the file has no records, `DbfSense` always returns 0. Services +that operate on the current record (`DbfEraseRead`, `DbfUpdate`, …) do nothing and return +`E_FILE_EOF` when positioned on the EOF record. [PLIB p.197] + +### 2.2 The file header + +DBFs begin with a **22-byte** standard header [PLIB p.196; SysSvc §20]: + +| Byte offset | Contents | +|-------------|----------| +| 0–15 | Zero-terminated file signature (all 16 bytes are verified — pad with trailing zeros). | +| 16, 17 | Version of DBF software used to produce the file. | +| 18, 19 | Offset from start of file to the first record. | +| 20, 21 | Minimum version of DBF software required. | + +The first-record offset allows an **extended header** (application-specific data after the standard +header). If none, the value is **22**. The first record is always the type-2 field information +record. Version-number format: see `DbfVersion`. [PLIB p.196; SysSvc §20] + +### 2.3 Records and the record header word + +In memory, a record is a `DbfRecord` (`p_dbf.h`) [PLIB p.196]: +```c +typedef struct { + UWORD header; /* record header word */ + UBYTE data[2]; /* data to be written... (variable length in practice) */ +} DbfRecord; +``` +The **header word**: top **4 bits** = record **type** (0–15); low **12 bits** = record **length**. +Maximum record length is **4094** bytes (one less than the theoretical 0xFFF = 4095, so a 4094-byte +record plus its 2-byte header fits a 4096-byte buffer). [PLIB p.196; SysSvc §20] + +**Record types** [PLIB pp.196–197; SysSvc §20]: + +| Type | Meaning | +|-------|---------| +| 0 | Deleted record — ignored by all DBF services; never copied by `DbfCopyFile`. | +| 1 | Standard data record (fields per the field information record). Usually the only visible type. | +| 2 | **Field information record (FIR)** — must exist and be the first record; later type-2 records ignored. | +| 3 | Descriptive record — optional, file-wide app data (see `DbfDescRecordRead/Write`). | +| 4–7 | App-specific: copied to a **new** file, **not** appended to an existing file, by `DbfCopyFile`. | +| 8–13 | App-specific: **both** copied to a new file and appended to an existing file (merged) by `DbfCopyFile`. | +| 14 | Reserved for voice records. | +| 15 | Reserved for internal use — do not use. | + +### 2.4 The Field Information Record (FIR) and field types + +The FIR (type 2) holds up to **32 bytes**, one per field, giving each field's type [PLIB p.197; +SysSvc §20]: + +| Byte value | Field type | +|------------|------------| +| 0 | Word | +| 1 | Long | +| 2 | Double | +| 3 | String | +| 4–255 | Reserved | + +So a data record has at most 32 fields — **with one exception:** a record that contains **only +string fields** is not bound by the 32-field limit and may hold any number of fields, subject to +the 4094-byte record cap. Records may contain **fewer** fields than the FIR lists, provided only +*trailing* fields are omitted; records with **more** fields than the FIR are assumed to have the +extra ones as string fields. Only `DbfFindRead`/`DbfFindReadField` assume records match the FIR. +[PLIB p.197] + +### 2.5 String fields (leading byte count) and continuation sub-fields + +A string field is **leading byte-counted text**, so a normal string field holds at most **255** +characters. Longer strings use **continuation sub-fields**: the first 254 characters plus a +terminating byte `0x14`, stored in a string field whose count byte is **255**; the `0x14` + +count-255 signals that the immediately following string field continues the text. This chains +indefinitely, subject only to the 4094-byte record limit. [PLIB p.197] + +### 2.6 The structs + +`DbfHeader` (`p_dbf.h`) — passed to open [PLIB p.199]: +```c +typedef struct { + UBYTE fileType[DbfHeaderNameSize]; /* 16-byte file signature */ + UWORD createVersion; /* software version used to create the file */ + UWORD dataStart; /* offset in file of first record (22 if no ext header) */ + UWORD needVersion; /* minimum software version needed to handle this file */ + UWORD firHeader; /* header word for the field information record */ + UBYTE fir[DbfMaxFirLength]; /* the field information record bytes */ +} DbfHeader; +``` +- When **creating/replacing**, pre-fill *all* elements (header + FIR). There is no gap between the + header and FIR in the struct even when an extended header is used; the *file* leaves a gap of + `dataStart − 22` bytes for it. +- When **opening existing**, pre-fill only `fileType` (the signature); the rest is filled from the + file. All 16 signature bytes are verified — mismatch → `E_FILE_INVALID`. +- `DbfHeaderNameSize` = 16 (implied by "16-byte file signature"). `DbfMaxFirLength` is the FIR + capacity; the FIR maximum length is 32. *(Exact numeric value of `DbfMaxFirLength` not stated + verbatim in the source beyond the max-32 rule — verify in `p_dbf.h`.)* + +`DbfOpenArgs` (`p_dbf.h`) — for `DbfQuickOpen` [PLIB p.200]: +```c +typedef struct { + VOID **pFcb; + UBYTE *fName; + UINT mode; + DbfHeader *pHead; +} DbfOpenArgs; +``` + +`DbfRecord` — see §2.3. + +### 2.7 The DBF functions + +Every DBF function calls `p_panic` if `pFcb` is not a valid DBF channel from `DbfOpen`/ +`DbfQuickOpen`; that clause is omitted per-entry below. Unless noted "may be used on a DBF opened +without an index", the *without-index* behaviour is called out where the manual specifies it. + +#### Opening / closing / flushing + +- `INT DbfOpen(INT *pstate, VOID **pFcb, TEXT *fName, UINT mode, DbfHeader *pHead, UBYTE *pbuffer, UINT len, UINT type);` + Open a DBF channel. `mode` = exactly one of `P_FOPEN`/`P_FCREATE`/`P_FREPLACE`/`P_FAPPEND`/ + `P_FUNIQUE`, optionally OR'd with `P_FUPDATE` and/or `P_FSHARE` (other required stream flags are + supplied automatically; `P_FOPEN` and `P_FAPPEND` are treated identically). `pbuffer`/`len` is + the caller's read-ahead buffer: **len must be 512…16384** (else `E_FILE_RECORD`), and must be at + least as large as the largest record (4096 is guaranteed sufficient). Only records of type `type` + are visible (normally 1; may be 4–14; results undefined for 0/2/3/>14). `*pstate` selects the + open strategy [PLIB pp.198–199]: + - `DbfStateDisabled` — open with a full sparse index; returns only when the index is fully built + (may take a while). + - `DbfStateOpenNoIndex` — open **without** an index; returns fast, but some services are then + disallowed (see each function). + - `DbfStateStart` — open with a sparse index incrementally: call `DbfOpen` repeatedly, feeding + back `*pstate` each time, until `*pstate` becomes `DbfStateStart` again *(as printed — the + source states the loop terminates when the written-back value is `DbfStateStart`; this reads + like an OCR/spec inconsistency, likely intended to be a distinct "finished" state such as + `DbfStateDisabled`/`DbfStateEnd`. **Flag: verify the terminating state in `p_dbf.h`.**)*. + + After a successful open the current record is **0**, so `DbfNextRead` reads record 1 and + `DbfEraseRead` erases record 0; use `DbfFirstRead` to read record 0. Errors: those of + `p_open(P_FSTREAM)`, `p_seek`, `p_read`, plus `E_FILE_INVALID` (bad signature / bad FIR), + `E_FILE_RECORD` (bad buffer length). [PLIB p.199] + +- `INT DbfQuickOpen(INT *pstate, DbfOpenArgs *pargs, UBYTE *pbuffer, UINT len, UINT type);` + As `DbfOpen` but `pFcb`/`fName`/`mode`/`pHead` are bundled in `DbfOpenArgs`. **Preferred** over + `DbfOpen` (more efficient, shorter code); `DbfOpen` retained for compatibility. [PLIB p.200] + +- `INT DbfClose(VOID *pFcb);` — close the file (returns as `p_close`); channel is closed even on + error. May be used without an index. [PLIB p.200] + +- `INT DbfFlush(VOID *pFcb);` — flush all buffers so modified data is written (returns as + `p_write`). Does not alter the buffer. May be used without an index. [PLIB p.200] + +- `VOID DbfTrash(VOID *pFcb);` — tell the DBF layer the read-ahead buffer was overwritten by the + caller and can no longer be relied on. May be used without an index. [PLIB p.200] + +- `INT DbfCopyDown(VOID *pFcb, UINT offset);` — copy the record at `offset` in the buffer to the + start of the buffer (and flag the buffer invalid, so no `DbfTrash` needed); returns the record + length. `offset` must be one previously returned by a read service and the caller must not have + written to the buffer since. May be used without an index. [PLIB p.201] + +#### Whole-file operations + +- `INT DbfCompress(UINT *pstate, VOID *pFcb);` — reclaim space from deleted records **if** the + medium is compressible (otherwise a no-op that still returns 0). After a real compress the + current record is the EOF record. `*pstate` = `DbfStateDisabled` (blocking) or `DbfStateStart` + (incremental loop, terminating as written back). **Should not** be used without an index. + [PLIB p.201] + +- `INT DbfCopyFile(UINT *pstate, VOID *pFcb, TEXT *pTargetName, UINT targetMode, UINT type, INT dir);` + Copy non-deleted records between the current file and `pTargetName`. `targetMode` = same options + as open (target always opened without an index; `P_FUNIQUE` writes back the unique name). + `type` selects a single type (usually 1) or `DbfRecordTypeAll` for all types. `dir` is + `DbfCopyFromHandle` (current → target; target new file with `P_FCREATE`/`P_FREPLACE`/`P_FUNIQUE`, + or append with `P_FOPEN`/`P_FAPPEND`) or `DbfCopyToHandle` (target → current; target sensibly + `P_FOPEN`). Copying to a **new** file always copies the FIR (type 2) and the file header + (incl. extended header) regardless of `type`; **appending** never copies types 2–7. Appending + requires matching signatures and compatible FIRs (identical, or both string-only) else + `E_FILE_INVALID`. `*pstate` supports `DbfStateDisabled`, `DbfStateStart` (est. calls = + file size / buffer size + 2), and `DbfStateCopyAbort` (abort an in-progress `DbfStateStart` + copy). May be used without an index. **Warning:** appending can exceed 65534 records with no + error. [PLIB pp.201–202] + +- `INT DbfFileSize(VOID *pFcb, ULONG *pSize);` — write the open file's size to `*pSize`. May be + used without an index. [PLIB p.202] + +- `UINT DbfVersion(VOID);` — DBF software version as hex `xyyF`: `x` = major (4 bits), `yy` = minor + (8 bits), `F` = release type A/B/F (Alpha/Beta/Final, 4 bits). E.g. `0x110F` → 1.10F. Only the + major number gates whether a file can be handled. May be used without an index. [PLIB p.203] + +#### Extended header and descriptive record + +- `INT DbfExtHeaderRead(UINT cont, VOID *pFcb, VOID *buf, UINT len);` — read up to `len` bytes of + the extended header; returns bytes read. `cont`: 0 = initial read (resets to start of ext + header); 1 = continue; use 0 if reading the whole thing in one call. `E_FILE_EOF` at end. May be + used without an index. [PLIB p.202] +- `INT DbfExtHeaderWrite(UINT cont, VOID *pFcb, VOID *buf, UINT len);` — symmetric writer. May be + used without an index. [PLIB pp.202–203] +- `INT DbfDescRecordRead(VOID *pFcb);` — read the descriptive record to offset 0 of the read-ahead + buffer; returns its length, or `E_FILE_EOF` if none, or `E_FILE_INVALID` if the file was opened + without an index. Sub-records use the same word-header format as main records; ignore (do not + delete) unrecognised sub-record types. [PLIB p.203] +- `INT DbfDescRecordWrite(VOID *pFcb, UINT len);` — write a descriptive record from a `DbfRecord` + at buffer offset 0 (content starts at offset 2; the leading 2 bytes form the header and are not + counted in `len`). Any existing descriptive record is erased first (max one per file); `len==0` + just erases it. `E_FILE_INVALID` if opened without an index. [PLIB p.203] + +#### Reading records + +All read services return the **record data length** (excluding the 2-byte header) and write the +buffer **offset of the DbfRecord** (including its header) to `*pOffset`. The read record becomes +the current record. On `E_FILE_EOF`, `*pOffset` is invalid. + +- `INT DbfAbsRead(VOID *pFcb, UINT recnum, UWORD *pOffset);` — seek to and read record `recnum`. + `E_FILE_EOF` if `recnum` > last record (current → EOF record). May be used without an index. + [PLIB p.204] +- `INT DbfAbsReadSense(VOID *pFcb, UINT recnum, UWORD *pOffset, ULONG *pPos);` — as `DbfAbsRead` + but also writes the file position of the record header to `*pPos`. May be used without an index. + [PLIB p.204] +- `INT DbfNextRead(VOID *pFcb, UWORD *pOffset);` — read the next record. `E_FILE_EOF` if already on + the last record / no records (current → EOF record). May be used without an index. [PLIB p.204] +- `INT DbfBackRead(VOID *pFcb, UWORD *pOffset);` — read the previous record. `E_FILE_EOF` if on the + first record / no records (current → record 0). May be used without an index. [PLIB p.204] +- `INT DbfFirstRead(VOID *pFcb, UWORD *pOffset);` — read the first record. `E_FILE_EOF` if no + records of the current type (current → 0 = EOF record). May be used without an index. [PLIB p.205] +- `INT DbfLastRead(VOID *pFcb, UWORD *pOffset);` — read the last record. `E_FILE_EOF` if none. + **Should not** be used without an index. [PLIB p.205] + +#### Appending / erasing / updating + +- `INT DbfAppend(VOID *pFcb, UINT len);` — append a record of the current type and length `len` + (data only) to EOF and make it current; returns 0. The record must be at the **start** of the + read-ahead buffer as a `DbfRecord` including its 2-byte header; `DbfAppend` builds the type/length + header from those two bytes (not counted in `len`). `E_GEN_OVER` if already 65534 records of the + type; `E_FILE_RECORD` if total (data + 2) exceeds the buffer. Does not alter the buffer. + **Should not** be used without an index. [PLIB p.205] +- `INT DbfEraseRead(INT *pstate, VOID *pFcb, UWORD *pOffset);` — erase the current record and read + the following one (becomes current); returns its length or negative. Two `E_FILE_EOF` cases: + (a) already on the EOF record / no records → does nothing; (b) current was the last record → it + is erased and current becomes the EOF record. Distinguish via `DbfSense`+`DbfCount`, or by + comparing `DbfCount` before/after. `*pstate` = `DbfStateDisabled` / `DbfStateStart` (incremental). + **Should not** be used without an index. [PLIB p.206] +- `INT DbfUpdate(INT *pstate, VOID *pFcb, UINT len);` — erase the current record and append a new + one of length `len` from the read-ahead buffer, making it current. The new record must be a + `DbfRecord` at the start of the buffer (leading 2 bytes = header, not counted in `len`). The old + record is **not** erased until the new one is successfully appended. `E_FILE_EOF` (no-op) if no + records of the current type or if on the EOF record. `*pstate` = `DbfStateDisabled` / + `DbfStateStart`. **Should not** be used without an index. [PLIB p.206] + *(Note: the manual's prose here says "DbfAppend uses these two bytes" — clearly referring to the + same header-construction mechanism.)* + +#### Sensing / counting + +- `UINT DbfSense(VOID *pFcb);` — return the current record number (the EOF record number — 0 if + empty, else count+1 — if a preceding call returned `E_FILE_EOF`). Does not alter the buffer. May + be used without an index. [PLIB p.209] +- `UINT DbfCount(VOID *pFcb);` — return the number of records of the currently visible type without + changing the current record. Does not alter the buffer. **Should not** be used without an index. + [PLIB p.209] + +#### Finding by content + +- `INT DbfFindRead(UINT *pstate, VOID *pFcb, VOID *pBuffer, UINT len, UINT findMode, UINT nStrings, UWORD *pOffset);` + Match wildcard text at `pBuffer` of length `len` (≤ 255) against the first `nStrings` **string + fields** of each record, starting at the current record; returns the matching record's data + length or a negative error. Equivalent to `DbfFindReadField` with `startStr = 0`. `nStrings == + DbfFindAllStrings` searches all string fields to end-of-record. The FIR is used to type the first + 32 fields; fields beyond that are assumed strings. On match, that record becomes current and its + buffer offset is written to `*pOffset`; on no match, `E_FILE_EOF` and current becomes the first + record (backward search) or the EOF record (forward search). May be used without an index + **except** `DbfFindLast` (unpredictable). Panics if `findMode` is malformed. [PLIB pp.208–209] + +- `DbfFindReadField(...)` — same as `DbfFindRead` but with an extra `startStr` argument: match + starts at string field number `startStr` (0 = first string field). **Available only in EPOC + ≥ 3.18.** + **OCR FLAG:** the signature is **garbled/truncated** in the source. The printed fragment reads: + `… UINT nStrings, UWORD *pOffset, UINT startStr);` (line lacks the return type, `pstate`, `pFcb`, + `pBuffer`, `len`, `findMode`). By analogy with `DbfFindRead` the full signature is almost + certainly: + ```c + /* RECONSTRUCTED — verify against p_dbf.h before use */ + INT DbfFindReadField(UINT *pstate, VOID *pFcb, VOID *pBuffer, UINT len, + UINT findMode, UINT nStrings, UWORD *pOffset, UINT startStr); + ``` + [PLIB pp.207–208] + +**`findMode`** is an OR of three parts [PLIB pp.207–208]: +1. **Max match length** in any one string field (0…255; 255 = no truncation) occupies the low part + of `findMode`. +2. **Direction / start**: `DbfFindForwards`, `DbfFindBackwards`, `DbfFindFirst`, `DbfFindLast`. +3. **Case**: `DbfFindCaseIndependent` or `DbfFindCaseDependent`. + +**Finding across continuation sub-fields** (only needed if strings may exceed 255 chars): OR +`0x1400` into `len` (whose base value cannot exceed 255); set the length component of `findMode` to +255 and OR in `0x4000`; only case-independent matching is allowed (OR `DbfFindCaseIndependent`); +then OR in the direction flag as usual. [PLIB p.208] + +> **No per-field getter/setter exists.** The DBF API deals in whole records only. There is no +> function to read or write an individual Word/Long/Double/String field. The application must +> **pack and unpack field bytes by hand** into/out of the `DbfRecord.data` area, following the FIR +> field order and the string leading-byte-count convention. The find services are the only ones +> that interpret field structure, and only for string matching. + +### 2.8 Error names (DBF) + +`E_FILE_EOF` (past end / before start), `E_FILE_INVALID` (bad signature / FIR / no-index misuse), +`E_FILE_RECORD` (bad buffer length or record too big for buffer), `E_GEN_OVER` (65534-record limit +hit on append). `DbfClose`/`DbfFlush` propagate `p_close`/`p_write` errors; most read services +propagate `p_seek`/`p_read` errors. [PLIB pp.199, 205–206] The System Services manual names the +same conditions `EofErr`, `InvalidFileErr`, and the panics `PanicDbf1` (bad handle) / `PanicDbf2` +(bad offset). [SysSvc §20] + +--- + +## Part 3 — Worked recipe: create schema → append → read back + +Assembled strictly from the manual's prose (PLIB ch. 14). Field packing is done by hand because +there is no per-field API. Illustrative — variable names/details are the author's; the API calls, +struct layout, and offsets are per the manual. + +**Schema for the example:** two fields — field 0 = Long (FIR byte `1`), field 1 = String (FIR byte +`3`). + +### Step 1 — Create the file with header + FIR + +Pre-fill a `DbfHeader` completely (creating/replacing requires the full header **and** FIR; +`p.199`). The FIR is itself a type-2 record, so `firHeader` is a record header word: top 4 bits = +type 2, low 12 bits = FIR length (here 2). Provide a read-ahead buffer of 4096 (guaranteed +sufficient; `p.199`) and open with visible `type = 1`. + +```c +#include + +UBYTE buf[4096]; /* read-ahead buffer, 512..16384; >= largest record */ +VOID *fcb; +INT state; +DbfHeader h; +DbfOpenArgs args; + +/* --- header --- */ +p_bfill(&h.fileType[0], DbfHeaderNameSize, 0); /* zero the 16-byte signature */ +p_scpy(&h.fileType[0], "MYAPPDatabase"); /* pad remainder stays 0 (p.196) */ +h.createVersion = DbfVersion(); /* p.203 */ +h.dataStart = 22; /* no extended header (p.196) */ +h.needVersion = DbfVersion(); +/* --- FIR (type 2, length 2): field0=Long(1), field1=String(3) --- */ +h.firHeader = (2 << 12) | 2; /* top 4 bits type, low 12 len */ +h.fir[0] = 1; /* Long (p.197) */ +h.fir[1] = 3; /* String (p.197) */ + +args.pFcb = &fcb; +args.fName = (UBYTE *)"MYDATA.DBF"; +args.mode = P_FCREATE | P_FUPDATE; /* create new, writable (Files ch)*/ +args.pHead = &h; + +state = DbfStateDisabled; /* build index, blocking (p.199) */ +if (DbfQuickOpen(&state, &args, &buf[0], sizeof(buf), 1) < 0) + /* handle error (p_fparse / p_open / E_FILE_* errors) */; +``` + +### Step 2 — Pack a record by hand and append it + +A record to be appended must sit at the **start of the buffer** as a `DbfRecord`: 2-byte header +then data; `DbfAppend` builds the header from those two bytes and `len` is the **data** length only +(`p.205`). Pack the Long field (its byte layout is the app's responsibility), then the String field +as a leading count byte followed by its characters (`p.197`). + +```c +DbfRecord *rec = (DbfRecord *)&buf[0]; +UBYTE *d = &rec->data[0]; /* data starts at buffer offset 2 */ +UINT len = 0; +LONG idNum = 42; +TEXT *nameStr = "Widget"; +UINT nameLen = p_slen(nameStr); /* <= 255 for a plain string field */ + +/* field 0: Long (4 bytes, app-defined byte order) */ +p_bcpy(&d[len], &idNum, sizeof(LONG)); len += sizeof(LONG); +/* field 1: String = leading byte count + chars (p.197) */ +d[len++] = (UBYTE)nameLen; +p_bcpy(&d[len], nameStr, nameLen); len += nameLen; + +/* header bytes are set by DbfAppend from rec->header; type is the current type (1) */ +if (DbfAppend(fcb, len) < 0) /* appends, becomes current record (p.205) */ + /* handle E_GEN_OVER / E_FILE_RECORD / p_write errors */; +``` + +### Step 3 — Read the record back and unpack + +After open the current record is 0 (`p.199`). Reads return the **data length** and write the +`DbfRecord` **offset** (header included) into `*pOffset`. Read the first record, then unpack in FIR +order. (`DbfCopyDown` first if you intend to edit in the buffer — `p.201`/`SysSvc §20`.) + +```c +UWORD off; +INT rlen = DbfFirstRead(fcb, &off); /* read record 0 (p.205) */ +if (rlen >= 0) { + DbfRecord *r = (DbfRecord *)&buf[off]; + UBYTE *p = &r->data[0]; + LONG gotId; + UBYTE slen; + TEXT gotName[256]; + + p_bcpy(&gotId, &p[0], sizeof(LONG)); /* field 0: Long */ + p += sizeof(LONG); + slen = *p++; /* field 1: count byte, then chars (p.197) */ + p_bcpy(&gotName[0], p, slen); + gotName[slen] = 0; + /* gotId == 42, gotName == "Widget" */ +} + +DbfFlush(fcb); /* ensure written data is on the medium (p.200) */ +DbfClose(fcb); /* p.200 */ +``` + +Notes tied to the model: +- To **update** the record, pack a new `DbfRecord` at buffer offset 0 and call + `DbfUpdate(&state, fcb, len)`; the record is deleted and re-appended at EOF (`p.206`). +- To **delete**, position on it (e.g. `DbfAbsRead`) and call `DbfEraseRead(&state, fcb, &off)` + (`p.206`); the file does not shrink until `DbfCompress` (non-Flash) or a `DbfCopyFile` rebuild + (`p.195`). +- `DbfFindRead` / `DbfFindReadField` can locate a record by matching a wildcard against its string + fields (`p.207`–209); numeric fields are not searchable by these services.