fs-inventory
v1.0.0

fs-inventory 1.0.0

Schema

Every column of every table fs-inventory writes.

Schema version 3. A scan directory holds manifest.json and a directory of part files per table; DuckDB reads each table's parts with a glob such as entries/*.parquet.

Types are as DuckDB reads them from Parquet. In CSV and JSON Lines, byte columns are base64 text under their text name, and timestamps are ISO-8601 UTC text (or epoch seconds with --time-format epoch).

Integer columns are signed and wide enough for every value, so they work in tools and databases without unsigned types (PostgreSQL, Spark). The exceptions are dev, ino, and rdev: opaque 64-bit identifiers whose top bit can be set, so they're unsigned (UBIGINT). Byte totals and counts above the signed 64-bit maximum (8 EiB) would be capped, not wrapped.

entries#

One row per file, directory, or other entry.

Column Type Null Description
path VARCHAR Full path. Always valid UTF-8: bytes that aren't valid UTF-8 appear as \xNN, and in CSV, control characters are escaped (\n, \t, \xNN).
parent_path VARCHAR Path of the containing directory; empty for /.
name VARCHAR Final path component. For a scan root, the root's own name.
extension VARCHAR yes Lowercased text after the last . of a name that isn't a directory. None for names starting or ending with . or extensions longer than 32 bytes.
path_raw (text: path_b64) BLOB yes Exact path bytes, when path differs from them (every row with --raw-paths always). Base64 in CSV and JSON Lines, as path_b64. Exact bytes for any row: coalesce(path_raw, encode(path)).
name_flags INTEGER Bitmask of name properties that cause trouble; see Name flags.
kind VARCHAR Entry type; see Kinds.
depth INTEGER Levels below the scan root; 0 for the root.
dir_id BIGINT yes For directories, an ID assigned by the scan, unique within it.
parent_id BIGINT dir_id of the containing directory; 0 for a root.
dev UBIGINT Device of the file system holding the entry (st_dev). Not stable across hosts or remounts.
ino UBIGINT Inode number. Entries with the same (dev, ino) are hard links to one file.
nlink BIGINT Number of hard links.
uid BIGINT Owner user ID.
gid BIGINT Owner group ID.
mode INTEGER Full st_mode: file type and permission bits.
size BIGINT yes Logical size in bytes. Null when skipped with --fields.
alloc_bytes BIGINT yes Space allocated on disk (st_blocks × 512). Smaller than size for sparse, compressed, or tiered files. Null when skipped with --fields.
atime TIMESTAMPTZ yes Last access. Null when skipped with --fields.
mtime TIMESTAMPTZ yes Last change to the contents. Copies and restores often preserve it. Null when skipped with --fields.
ctime TIMESTAMPTZ yes Last change to the contents or metadata. Not creation time. Users can't set it, so it's the reliable signal of recent change. Null when skipped with --fields.
btime TIMESTAMPTZ yes Creation (birth) time, when the platform and file system provide it and it isn't skipped with --fields.
rdev UBIGINT yes Device number, for character and block devices.
dir_entries BIGINT yes For directories that were read: number of immediate children. A subdirectory deleted after it was listed but before it was read is counted, though it has no row.
dir_files BIGINT yes For directories that were read: immediate children that aren't directories.
dir_subdirs BIGINT yes For directories that were read: immediate subdirectories.
dir_bytes BIGINT yes For directories that were read: sum of size over immediate children that aren't directories.
dir_alloc_bytes BIGINT yes For directories that were read: sum of alloc_bytes over immediate children that aren't directories.
is_mount_point BOOLEAN True when this directory is on a different device from its parent.
time_flags INTEGER Bitmask of timestamps that couldn't be represented and were clamped: 1 atime, 2 mtime, 4 ctime, 8 btime.
error VARCHAR yes For directories that couldn't be fully read: the failed operation and error, such as open: EACCES. With --detect-cycles, revisit: already visited through another path marks a directory that was reached again and not re-read. With --stall-timeout, stall: ... marks a directory the scan gave up on.
symlink_target VARCHAR yes Where a symlink points, with --symlink-targets, rendered like path; null otherwise. Added in schema version 3, and last so that older CSV readers can ignore it.
symlink_target_raw (text: symlink_target_b64) BLOB yes Exact target bytes, when symlink_target differs from them (every symlink with --raw-paths always). Base64 in CSV and JSON Lines, as symlink_target_b64.

errors#

One row per failed operation. Present only if there were any.

Column Type Null Description
path VARCHAR Path of the entry the operation was on, rendered like entries.path.
path_raw (text: path_b64) BLOB yes Exact path bytes, when path differs from them. Base64 in CSV and JSON Lines, as path_b64.
op VARCHAR The failed operation: open, readdir, stat, verify (the directory was replaced during the scan), stall (no progress for --stall-timeout, so the scan gave up on the directory), readlink (reading a symlink's target, with --symlink-targets), or internal.
errno INTEGER yes The OS error number, when there is one.
errno_name VARCHAR yes Symbolic name of errno, such as EACCES.
message VARCHAR Human-readable description.
time TIMESTAMPTZ When the error happened.

users#

With --resolve-ids: one row per user ID in entries.uid, with its name. Written when the scan finishes.

Column Type Null Description
uid BIGINT A user ID that owns at least one entry.
name VARCHAR yes The user's name, as the scanning host looked it up when the scan finished; null when no account has that ID (such as a deleted user).

groups#

With --resolve-ids: one row per group ID in entries.gid, with its name. Written when the scan finishes.

Column Type Null Description
gid BIGINT A group ID that owns at least one entry.
name VARCHAR yes The group's name, as the scanning host looked it up when the scan finished; null when no group has that ID.

rollup#

Written by fs-inventory rollup: one row per directory with totals over its whole subtree. Parquet only.

Column Type Null Description
dir_id BIGINT The directory's dir_id in entries.
parent_id BIGINT dir_id of the containing directory; 0 for a root.
depth INTEGER Levels below the scan root; 0 for the root.
path VARCHAR Full path, rendered like entries.path.
path_raw (text: path_b64) BLOB yes Exact path bytes, when path differs from them.
entries BIGINT Entries anywhere below this directory, not counting the directory itself.
files BIGINT Entries below this directory that aren't directories.
dirs BIGINT Directories below this directory.
bytes BIGINT Sum of size over entries below this directory that aren't directories.
alloc_bytes BIGINT Sum of alloc_bytes over entries below this directory that aren't directories.
oldest_mtime TIMESTAMPTZ yes Oldest mtime below this directory, over entries that aren't directories. Null when there are none.
newest_mtime TIMESTAMPTZ yes Newest mtime below this directory, over entries that aren't directories.
newest_ctime TIMESTAMPTZ yes Newest ctime below this directory, over entries that aren't directories.
newest_atime TIMESTAMPTZ yes Newest atime below this directory, over entries that aren't directories.
incomplete_dirs BIGINT Directories in this subtree, including this one, that weren't read completely: errors, mount points not entered, or --max-depth. When it isn't 0, the totals are lower bounds.

rollup_ages#

Written by fs-inventory rollup: file counts and bytes by age for each directory down to --age-depth, measured from the scan's start time. Parquet only.

Column Type Null Description
dir_id BIGINT The directory's dir_id; join with rollup or entries for its path.
time VARCHAR Which timestamp the age is measured from: mtime, ctime, or atime.
bucket INTEGER Age bucket, in order: 0 is future (newer than the scan), then increasing age.
label VARCHAR The bucket's label, such as <30d, 1y-2y, or >=10y, from --age-buckets.
files BIGINT Entries below the directory in this bucket that aren't directories. Only buckets with files have rows.
bytes BIGINT Sum of size over those entries.

Name flags#

Bits of name_flags. Decode them in DuckDB with fs_name_flags(name_flags) from sql/fs_inventory.sql.

Bit Name Meaning
1 invalid_utf8 Not valid UTF-8: a legacy encoding (Latin-1, Shift-JIS, ...) or corruption
2 c0_control Contains an ASCII control character, such as a newline or tab
4 c1_control Contains a C1 control character (U+0080 to U+009F)
8 del Contains DEL (U+007F)
16 bidi_control Contains a bidirectional formatting character that can disguise how a name displays
32 not_nfc Valid UTF-8 but not in Unicode Normalization Form C (common from macOS)
64 leading_trailing_space Starts or ends with a space
128 leading_dash Starts with -, which tools can mistake for an option
256 trailing_dot Ends with ., which Windows strips
512 windows_reserved Contains a character Windows forbids (<>:"|?*\) or is a device name such as CON
1024 long_name Longer than 255 bytes
2048 long_path The full path is at least 4,096 bytes

Kinds#

Values of kind.

Kind Meaning
file Regular file
dir Directory
symlink Symbolic link (never followed)
fifo Named pipe
socket Unix domain socket
char Character device
block Block device
unknown Any other type