vfs

package module
v0.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 8, 2026 License: BSD-3-Clause Imports: 31 Imported by: 1

README

vfs - extensions to pebble/vfs

This is a fork and extension of the vfs mock file system test affordance from Pebble by Cockroach Labs.

https://github.com/cockroachdb/pebble/

The original lacked the Truncate method necessary to test write-ahead-logs that are truncated to a specific size on recovery.

LICENSE: see the LICENSE files herein.

(parts are MIT licensed, parts are Apache 2 licensed, and the top level is BSD 3-clause licensed)

Extensions beyond pebble/vfs

This fork adds several methods to support dual-mode testing (real FS and in-memory FS; via dev created build tags).

File interface additions
  • Truncate(size int64) error -- Truncate a file to a given size. Required for testing write-ahead-log recovery, where a WAL must be truncated to a specific size after a simulated crash.

  • Name() string -- Return the name/path of an open file handle.

FS interface additions
  • ReadDir(dirname string) ([]os.DirEntry, error) -- Read a directory and return sorted os.DirEntry entries. Complements the existing List() (which returns only name strings) by providing type and metadata information for each entry.

  • WalkDir(path string, fn fs.WalkDirFunc) error -- Recursive directory traversal following the io/fs.WalkDirFunc convention. Walks the tree in lexicographic order, supports fs.SkipDir. Implemented for both defaultFS (real disk) and MemFS (in-memory tree).

  • IsReal() bool -- Returns true for the real OS filesystem (defaultFS or vfs.Default), false for MemFS. Lets test setup code branch on the filesystem type without type-asserting on unexported vfs types.

  • MountReadOnlyRealDir(fromRealDir, mountPointInsideDir string) error -- Mount a real OS directory inside a MemFS at a chosen path, read-only. This allows tests running under a MemFS to access large test assets (e.g. reference databases, golden files) from the real filesystem without copying them into memory. Key properties:

    • Read-only: Open, Stat, List, ReadDir, and WalkDir fall through to the real directory. Write operations (Create, MkdirAll, Rename, etc.) targeting the mount point return an error.
    • Disjoint paths: The mount point must not already exist in the MemFS overlay. This prevents ambiguous ownership of paths.
    • Multiple mounts: Several real directories can be mounted at different MemFS paths on the same MemFS instance.
    • Zero overhead when unused: A plain NewMem() with no mounts has no extra allocations or method-dispatch overhead.
    • Propagated on CrashClone(): Mount configuration is carried forward so crash-recovery tests retain access to mounted assets.

    Only supported on MemFS; calling on defaultFS or wrapper filesystems returns an error.

  • MemFS.Save method to durably persist a MemFS to actual, real disk file. The on-disk format is greenpack encoding followed by s2 compression.

  • MemFS.Load method to reify the real file generated by Save back into an MemFS in memory. Useful to resume from a saved time point of interest, such as that generated by NewCrashableMem() and MemFS.CrashClone() during fsync durability/recovery-path testing.


Author: Jason E. Aten, Ph.D.

Copyright (C) 2026, Jason E. Aten, Ph.D. All rights reserved.

LICENSE: BSD 3-clause for the new aggregate; same as pebble. See the LICENSE files herein.

Documentation

Index

Constants

View Source
const FALLOC_FL_INSERT_RANGE = 32

fortunately, this is the same number on linux and darwin. const FALLOC_FL_INSERT_RANGE = unix.FALLOC_FL_INSERT_RANGE

View Source
const FALLOC_FL_KEEP_SIZE = unix.FALLOC_FL_KEEP_SIZE // = 1

commannd 1 what seaweedfs does to pre-allocate space that is not used yet. Not sure--does this also result in a sparse file on linux, but maybe not on Mac??

View Source
const InvalidFd uintptr = ^(uintptr(0))

InvalidFd is a special value returned by File.Fd() when the file is not backed by an OS descriptor. Note: the special value is consistent with what os.File implementation returns on a nil receiver.

View Source
const PunchBelowBytes = 64 << 20

PunchBelowBytes gives the threshold for hole punching to preserve sparsity of small files. This is set to 64MB to reflect that Apple's Filesystem (APFS) will not reliably make a sparse file that is smaller than 32MB or so (depending on configuration) and so for files < PunchBelowBytes, we will come back after copying the file and punch out holes that should be there.

Variables

View Source
var ErrFileTooLarge = fmt.Errorf("extent requested was too large.")

allocated probably zero in this case, especially since we asked for "all-or-nothing"

View Source
var ErrShortAlloc = fmt.Errorf("smaller extent than requested was allocated.")
View Source
var ErrUnsupported = errors.New("pebble: not supported")

ErrUnsupported may be returned a FS when it does not support an operation.

Functions

func Clone

func Clone(srcFS, dstFS FS, srcPath, dstPath string, opts ...CloneOption) (bool, error)

Clone recursively copies a directory structure from srcFS to dstFS. srcPath specifies the path in srcFS to copy from and must be compatible with the srcFS path format. dstDir is the target directory in dstFS and must be compatible with the dstFS path format. Returns (true,nil) on a successful copy, (false,nil) if srcPath does not exist, and (false,err) if an error occurred.

func Copy

func Copy(fs FS, oldname, newname string) error

Copy copies the contents of oldname to newname. If newname exists, it will be overwritten.

func CopyAcrossFS

func CopyAcrossFS(srcFS FS, oldname string, dstFS FS, newname string) error

CopyAcrossFS copies the contents of oldname on srcFS to newname dstFS. If newname exists, it will be overwritten.

func GoroNumber added in v0.1.6

func GoroNumber() int

GoroNumber returns the calling goroutine's number.

func IsNoSpaceError

func IsNoSpaceError(err error) bool

IsNoSpaceError returns true if the given error indicates that the disk is out of space.

func LimitedCopy

func LimitedCopy(fs FS, oldname, newname string, maxBytes int64) error

LimitedCopy copies up to maxBytes from oldname to newname. If newname exists, it will be overwritten.

func LinkOrCopy

func LinkOrCopy(fs FS, oldname, newname string) error

LinkOrCopy creates newname as a hard link to the oldname file. If creating the hard link fails, LinkOrCopy falls back to copying the file (which may also fail if oldname doesn't exist or newname already exists).

Types

type CloneOption

type CloneOption func(*cloneOpts)

A CloneOption configures the behavior of Clone.

var CloneSync CloneOption = func(o *cloneOpts) { o.sync = true }

CloneSync configures Clone to sync files and directories.

var CloneTryLink CloneOption = func(o *cloneOpts) { o.tryLink = true }

CloneTryLink configures Clone to link files to the destination if the source and destination filesystems are the same. If the source and destination filesystems are not the same or the filesystem does not support linking, then Clone falls back to copying.

func CloneSkip

func CloneSkip(fn func(string) bool) CloneOption

CloneSkip configures Clone to skip files for which the provided function returns true when passed the file's path.

type CrashCloneCfg

type CrashCloneCfg struct {
	// UnsyncedDataPercent is the probability that a data block or directory entry
	// that was not synced will be part of the clone. If 0, the clone will contain
	// exactly the data that was last synced. If 100, the clone will be identical
	// to the current filesystem.
	UnsyncedDataPercent int
	// RNG must be set if UnsyncedDataPercent > 0.
	RNG *rand.Rand
}

CrashCloneCfg configures a CrashClone call. The zero value corresponds to the crash clone containing exactly the data that was last synced.

type DeviceID

type DeviceID struct {
	// contains filtered or unexported fields
}

A DeviceID uniquely identifies a block device on which filesystem data is persisted.

func (DeviceID) String

func (d DeviceID) String() string

String returns the string representation of the device ID.

type DiskSlowInfo

type DiskSlowInfo struct {
	// Path of file being written to.
	Path string
	// Operation being performed on the file.
	OpType OpType
	// Size of write in bytes, if the write is sized.
	WriteSize int
	// Duration that has elapsed since this disk operation started.
	Duration time.Duration
}

DiskSlowInfo captures info about detected slow operations on the vfs.

func (DiskSlowInfo) SafeFormat

func (i DiskSlowInfo) SafeFormat(w redact.SafePrinter, _ rune)

SafeFormat implements redact.SafeFormatter.

func (DiskSlowInfo) String

func (i DiskSlowInfo) String() string

type DiskUsage

type DiskUsage struct {
	// Total disk space available to the current process in bytes.
	AvailBytes uint64 `zid:"0"`

	// Total disk space in bytes.
	TotalBytes uint64 `zid:"1"`

	// Used disk space in bytes.
	UsedBytes uint64 `zid:"2"`
}

DiskUsage summarizes disk space usage on a filesystem.

func (*DiskUsage) DecodeMsg added in v0.3.0

func (z *DiskUsage) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable We treat empty fields as if we read a Nil from the wire.

func (*DiskUsage) EncodeMsg added in v0.3.0

func (z *DiskUsage) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (*DiskUsage) Gstring added in v0.3.0

func (z *DiskUsage) Gstring() (r string)

func (*DiskUsage) MarshalMsg added in v0.3.0

func (z *DiskUsage) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (*DiskUsage) Msgsize added in v0.3.0

func (z *DiskUsage) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*DiskUsage) UnmarshalMsg added in v0.3.0

func (z *DiskUsage) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

func (*DiskUsage) UnmarshalMsgWithCfg added in v0.3.0

func (z *DiskUsage) UnmarshalMsgWithCfg(bts []byte, cfg *msgp.RuntimeConfig) (o []byte, err error)

type DiskWriteCategory

type DiskWriteCategory string

DiskWriteCategory is a user-understandable string used to identify and aggregate stats for disk writes. The prefix "pebble-" is reserved for internal Pebble categories.

Some examples include, pebble-wal, pebble-memtable-flush, pebble-manifest and in the Cockroach context includes, sql-row-spill, range-snapshot, crdb-log.

const WriteCategoryUnspecified DiskWriteCategory = "unspecified"

WriteCategoryUnspecified denotes a disk write without a significant category.

type DiskWriteStatsAggregate

type DiskWriteStatsAggregate struct {
	Category     DiskWriteCategory
	BytesWritten uint64
}

DiskWriteStatsAggregate is an aggregate of the bytes written to disk for a given category.

type DiskWriteStatsCollector

type DiskWriteStatsCollector struct {
	// contains filtered or unexported fields
}

DiskWriteStatsCollector collects and aggregates disk write metrics per category.

func NewDiskWriteStatsCollector

func NewDiskWriteStatsCollector() *DiskWriteStatsCollector

NewDiskWriteStatsCollector instantiates a new DiskWriteStatsCollector.

func (*DiskWriteStatsCollector) CreateStat

func (d *DiskWriteStatsCollector) CreateStat(category DiskWriteCategory) *atomic.Uint64

CreateStat inserts a new category to the statsMap if it doesn't already exist, otherwise it returns a pointer to the current stats.

func (*DiskWriteStatsCollector) GetStats

GetStats returns the aggregated metrics for all categories.

type FS

type FS interface {
	// Create creates the named file for reading and writing. If a file
	// already exists at the provided name, it's removed first ensuring the
	// resulting file descriptor points to a new inode.
	Create(name string, category DiskWriteCategory) (File, error)

	// Link creates newname as a hard link to the oldname file.
	Link(oldname, newname string) error

	// Open opens the named file for reading. openOptions provides
	Open(name string, opts ...OpenOption) (File, error)

	// OpenReadWrite opens the named file for reading and writing. If the file
	// does not exist, it is created.
	OpenReadWrite(name string, category DiskWriteCategory, opts ...OpenOption) (File, error)

	// OpenDir opens the named directory for syncing.
	OpenDir(name string) (File, error)

	// Remove removes the named file or directory.
	Remove(name string) error

	// Remove removes the named file or directory and any children it
	// contains. It removes everything it can but returns the first error it
	// encounters.
	RemoveAll(name string) error

	// Rename renames a file. It overwrites the file at newname if one exists,
	// the same as os.Rename.
	Rename(oldname, newname string) error

	// ReuseForWrite attempts to reuse the file with oldname by renaming it to newname and opening
	// it for writing without truncation. It is acceptable for the implementation to choose not
	// to reuse oldname, and simply create the file with newname -- in this case the implementation
	// should delete oldname. If the caller calls this function with an oldname that does not exist,
	// the implementation may return an error.
	ReuseForWrite(oldname, newname string, category DiskWriteCategory) (File, error)

	// MkdirAll creates a directory and all necessary parents. The permission
	// bits perm have the same semantics as in os.MkdirAll. If the directory
	// already exists, MkdirAll does nothing and returns nil.
	MkdirAll(dir string, perm os.FileMode) error

	// Lock locks the given file, creating the file if necessary, and
	// truncating the file if it already exists. The lock is an exclusive lock
	// (a write lock), but locked files should neither be read from nor written
	// to. Such files should have zero size and only exist to co-ordinate
	// ownership across processes.
	//
	// A nil Closer is returned if an error occurred. Otherwise, close that
	// Closer to release the lock.
	//
	// On Linux and OSX, a lock has the same semantics as fcntl(2)'s advisory
	// locks. In particular, closing any other file descriptor for the same
	// file will release the lock prematurely.
	//
	// Attempting to lock a file that is already locked by the current process
	// returns an error and leaves the existing lock untouched.
	//
	// Lock is not yet implemented on other operating systems, and calling it
	// will return an error.
	Lock(name string) (io.Closer, error)

	// List returns a listing of the given directory. The names returned are
	// relative to dir.
	List(dir string) ([]string, error)

	// Stat returns an FileInfo describing the named file.
	Stat(name string) (FileInfo, error)

	// PathBase returns the last element of path. Trailing path separators are
	// removed before extracting the last element. If the path is empty, PathBase
	// returns ".".  If the path consists entirely of separators, PathBase returns a
	// single separator.
	PathBase(path string) string

	// PathJoin joins any number of path elements into a single path, adding a
	// separator if necessary.
	PathJoin(elem ...string) string

	// PathDir returns all but the last element of path, typically the path's directory.
	PathDir(path string) string

	// GetDiskUsage returns disk space statistics for the filesystem where
	// path is any file or directory within that filesystem.
	GetDiskUsage(path string) (DiskUsage, error)

	// Unwrap is implemented by "wrapping" filesystems (those that add some
	// functionality on top of an underlying FS); it returns the wrapped FS.
	// It is used by vfs.Root.
	//
	// Returns nil if this is not a wrapping filesystem.
	Unwrap() FS

	// ReadDir reads the named directory, returning
	// all its directory entries sorted by filename.
	ReadDir(dirname string) ([]os.DirEntry, error)

	IsReal() bool

	WalkDir(path string, f iofs.WalkDirFunc) error

	// MountReadOnlyReadDir mounts fromRealDir inside the MemFS at mountPointInsideDir.
	// This is useful to avoid copying large test assets. Real directories can be
	// read-only mounted inside MemFS with this call. Attempts to write to
	// the read-only mount will return errors. The mount point cannot already
	// exist inside the target MemFS.
	MountReadOnlyRealDir(fromRealDir string, mountPointInsideDir string) error
}

FS is a namespace for files.

The names are filepath names: they may be / separated or \ separated, depending on the underlying operating system.

var Default FS = defaultFS{}

Default is a FS implementation backed by the underlying operating system's file system.

func NewSyncingFS

func NewSyncingFS(fs FS, syncOpts SyncingFileOptions) FS

NewSyncingFS wraps a vfs.FS with one that wraps newly created files with vfs.NewSyncingFile.

func OnDiskFull

func OnDiskFull(fs FS, fn func()) FS

OnDiskFull wraps the provided FS with an FS that examines returned errors, looking for ENOSPC errors. It invokes the provided callback when the underlying filesystem returns an error signifying the storage is out of disk space.

All new writes to the filesystem are blocked while the callback executes, so care must be taken to avoid expensive work from within the callback.

Once the callback completes, any write-oriented operations that encountered ENOSPC are retried exactly once. Once the callback completes, it will not be invoked again until a new operation that began after the callback returned encounters an ENOSPC error.

OnDiskFull may be used to automatically manage a ballast file, which is removed from the filesystem from within the callback. Note that if managing a ballast, the caller should maintain a reference to the inner FS and remove the ballast on the unwrapped FS.

func Root

func Root(fs FS) FS

Root returns the base FS implementation, unwrapping all nested FSs that expose an Unwrap method.

func WithDiskHealthChecks

func WithDiskHealthChecks(
	innerFS FS,
	diskSlowThreshold time.Duration,
	statsCollector *DiskWriteStatsCollector,
	onSlowDisk func(info DiskSlowInfo),
) (FS, io.Closer)

WithDiskHealthChecks wraps an FS and ensures that all write-oriented operations on the FS are wrapped with disk health detection checks and aggregated. Disk operations that are observed to take longer than diskSlowThreshold trigger an onSlowDisk call.

A threshold of zero disables disk-health checking.

func WithLogging

func WithLogging(fs FS, logFn LogFn) FS

WithLogging wraps an FS and logs filesystem modification operations to the given logFn.

type File

type File interface {
	io.Closer
	io.Reader
	io.ReaderAt
	// Unlike the specification for io.Writer.Write(), the vfs.File.Write()
	// method *is* allowed to modify the slice passed in, whether temporarily
	// or permanently. Callers of Write() need to take this into account.
	io.Writer
	// WriteAt() is only supported for files that were opened with FS.OpenReadWrite.
	io.WriterAt

	// Preallocate optionally preallocates storage for `length` at `offset`
	// within the file. Implementations may choose to do nothing.
	Preallocate(offset, length int64) error
	Stat() (FileInfo, error)
	Sync() error

	// SyncTo requests that a prefix of the file's data be synced to stable
	// storage. The caller passes provides a `length`, indicating how many bytes
	// to sync from the beginning of the file. SyncTo is a no-op for
	// directories, and therefore always returns false.
	//
	// SyncTo returns a fullSync return value, indicating one of two possible
	// outcomes.
	//
	// If fullSync is false, the first `length` bytes of the file was queued to
	// be synced to stable storage. The syncing of the file prefix may happen
	// asynchronously. No persistence guarantee is provided.
	//
	// If fullSync is true, the entirety of the file's contents were
	// synchronously synced to stable storage, and a persistence guarantee is
	// provided. In this outcome, any modified metadata for the file is not
	// guaranteed to be synced unless that metadata is needed in order to allow
	// a subsequent data retrieval to be correctly handled.
	SyncTo(length int64) (fullSync bool, err error)

	// SyncData requires that all written data be persisted. File metadata is
	// not required to be synced. Unsophisticated implementations may call Sync.
	SyncData() error

	// Prefetch signals the OS (on supported platforms) to fetch the next length
	// bytes in file (as returned by os.File.Fd()) after offset into cache. Any
	// subsequent reads in that range will not issue disk IO.
	Prefetch(offset int64, length int64) error

	// Fd returns the raw file descriptor when a File is backed by an *os.File.
	// It can be used for specific functionality like Prefetch.
	// Returns InvalidFd if not supported.
	Fd() uintptr

	Truncate(size int64) error

	Name() string
}

File is a readable, writable sequence of bytes.

Typically, it will be an *os.File, but test code may choose to substitute memory-backed implementations.

Write-oriented operations (Write, Sync) must be called sequentially: At most 1 call to Write or Sync may be executed at any given time.

func NewMemFile

func NewMemFile(data []byte) File

NewMemFile returns a memory-backed File implementation. The memory-backed file takes ownership of data.

func NewSyncingFile

func NewSyncingFile(f File, opts SyncingFileOptions) File

NewSyncingFile wraps a writable file and ensures that data is synced periodically as it is written. The syncing does not provide persistency guarantees for these periodic syncs, but is used to avoid latency spikes if the OS automatically decides to write out a large chunk of dirty filesystem buffers. The underlying file is fully synced upon close.

type FileInfo

type FileInfo interface {
	os.FileInfo
	// DeviceID returns the ID of the device on which the file resides.
	DeviceID() DeviceID
}

FileInfo describes a file.

type LogFn

type LogFn func(fmt string, args ...interface{})

LogFn is a function that is used to capture a log when WithLogging is used.

type MemFS

type MemFS struct {
	// contains filtered or unexported fields
}

MemFS implements FS.

func NewCrashableMem

func NewCrashableMem() *MemFS

NewCrashableMem returns a memory-backed FS implementation that supports the CrashClone() method. This method can be used to obtain a copy of the FS after a simulated crash, where only data that was last synced is guaranteed to be there (with no guarantees one way or the other about more recently written data).

Note: when CrashClone() is not necessary, NewMem() is faster and should be preferred.

Expected usage:

	fs := NewCrashableMem()
	db := Open(..., &Options{FS: fs})
	// Do and commit various operations.
	...
	// Make a clone of the FS after a simulated crash.
 crashedFS := fs.CrashClone(CrashCloneCfg{Probability: 50, RNG: rand.New(rand.NewSource(0))})

	// This will finish any ongoing background flushes, compactions but none of these writes will
	// affect crashedFS.
	db.Close()

	// Open the DB against the crash clone.
	db := Open(..., &Options{FS: crashedFS})

func NewMem

func NewMem() *MemFS

NewMem returns a new memory-backed FS implementation.

func (*MemFS) CrashClone

func (y *MemFS) CrashClone(cfg CrashCloneCfg) *MemFS

CrashClone creates a new filesystem that reflects a possible state of this filesystem after a crash at this moment. The new filesystem will contain all data that was synced, and some fraction of the data that was not synced. The latter is controlled by CrashCloneCfg.

func (*MemFS) Create

func (y *MemFS) Create(fullname string, category DiskWriteCategory) (File, error)

Create implements FS.Create.

func (*MemFS) GetDiskUsage

func (y *MemFS) GetDiskUsage(string) (DiskUsage, error)

GetDiskUsage implements FS.GetDiskUsage.

func (*MemFS) IsReal added in v0.1.9

func (y *MemFS) IsReal() bool
func (y *MemFS) Link(oldname, newname string) error

Link implements FS.Link.

func (*MemFS) List

func (y *MemFS) List(dirname string) ([]string, error)

List implements FS.List.

func (*MemFS) Load added in v0.3.0

func (m *MemFS) Load(path string) error

Load is the inverse of Save. It restores s2 compressed, greenpack encoded data from real disk into m. Anything in m to start with is immediately wiped to zero, and then the restore from path proceeds.

Note that Save is reading from the real filesystem on real disk, and filling in a "fake" file system in memory (MemFS).

func (*MemFS) Lock

func (y *MemFS) Lock(fullname string) (io.Closer, error)

Lock implements FS.Lock.

func (*MemFS) MkdirAll

func (y *MemFS) MkdirAll(dirname string, perm os.FileMode) error

MkdirAll implements FS.MkdirAll.

func (*MemFS) MountReadOnlyRealDir added in v0.2.0

func (y *MemFS) MountReadOnlyRealDir(fromRealDir string, mountPointInsideDir string) error

MountReadOnlyRealDir mounts fromRealDir (a real OS directory) inside the MemFS at mountPointInsideDir. The mounted directory is visible through the MemFS for read operations (Open, Stat, List, ReadDir, WalkDir). Write operations to mounted paths return an error. The mount point cannot already exist inside the MemFS.

func (*MemFS) Open

func (y *MemFS) Open(fullname string, opts ...OpenOption) (File, error)

Open implements FS.Open.

func (*MemFS) OpenDir

func (y *MemFS) OpenDir(fullname string) (File, error)

OpenDir implements FS.OpenDir.

func (*MemFS) OpenReadWrite

func (y *MemFS) OpenReadWrite(
	fullname string, category DiskWriteCategory, opts ...OpenOption,
) (File, error)

OpenReadWrite implements FS.OpenReadWrite.

func (*MemFS) PathBase

func (*MemFS) PathBase(p string) string

PathBase implements FS.PathBase.

func (*MemFS) PathDir

func (*MemFS) PathDir(p string) string

PathDir implements FS.PathDir.

func (*MemFS) PathJoin

func (*MemFS) PathJoin(elem ...string) string

PathJoin implements FS.PathJoin.

func (*MemFS) ReadDir added in v0.1.3

func (y *MemFS) ReadDir(dirname string) ([]os.DirEntry, error)

ReadDir implements FS.ReadDir.

Inside MemFS, you could technically lock the internal mutex and write a custom iterator to dig through the unexported memNode tree. However, I strongly advise against this. Pebble's crash-simulation (StrictFS) relies heavily on intercepted state logic. If you bypass the public methods, you risk reading "un-synced" files during a simulated crash.

Instead, because MemFS operates entirely in RAM without system call overhead, composing y.List and y.Stat is the safest approach. It executes in nanoseconds and perfectly preserves Pebble's crash-simulation integrity.

func (*MemFS) Remove

func (y *MemFS) Remove(fullname string) error

Remove implements FS.Remove.

func (*MemFS) RemoveAll

func (y *MemFS) RemoveAll(fullname string) error

RemoveAll implements FS.RemoveAll.

func (*MemFS) Rename

func (y *MemFS) Rename(oldname, newname string) error

Rename implements FS.Rename.

func (*MemFS) ReuseForWrite

func (y *MemFS) ReuseForWrite(oldname, newname string, category DiskWriteCategory) (File, error)

ReuseForWrite implements FS.ReuseForWrite.

func (*MemFS) Save added in v0.3.0

func (y *MemFS) Save(path string) error

Save saves the MemFS y to disk under path. The recommend naming convention for path is to end in .green.s2, since the data is greenpack encoded and then s2 compressed. We leave that up to the caller's discretion, and write to whatever path we are given.

Note that Save is writing from a "fake" file system in memory (MemFS), to the real filesystem on disk at path.

func (*MemFS) Stat

func (y *MemFS) Stat(name string) (FileInfo, error)

Stat implements FS.Stat.

func (*MemFS) String

func (y *MemFS) String() string

String dumps the contents of the MemFS.

func (*MemFS) TestingSetDiskUsage

func (y *MemFS) TestingSetDiskUsage(usage DiskUsage)

TestingSetDiskUsage sets the disk usage that will be reported on subsequent GetDiskUsage calls; used for testing. If usage is empty, GetDiskUsage will return ErrUnsupported (which is also the default when TestingSetDiskUsage is not called).

func (*MemFS) UnsafeGetFileDataBuffer

func (y *MemFS) UnsafeGetFileDataBuffer(fullname string) ([]byte, error)

UnsafeGetFileDataBuffer returns the buffer holding the data for a file. Must not be used while concurrent updates are happening to the file. The buffer cannot be modified while concurrent file reads are happening.

func (*MemFS) Unwrap

func (y *MemFS) Unwrap() FS

Unwrap implements FS.Unwrap.

func (*MemFS) UseWindowsSemantics

func (y *MemFS) UseWindowsSemantics(windowsSemantics bool)

UseWindowsSemantics configures whether the MemFS implements Windows-style semantics, in particular with respect to whether any of an open file's links may be removed. Windows semantics default to off.

func (*MemFS) WalkDir added in v0.1.10

func (y *MemFS) WalkDir(root string, fn iofs.WalkDirFunc) error

type OpType

type OpType uint8

OpType is the type of IO operation being monitored by a diskHealthCheckingFile.

const (
	OpTypeUnknown OpType = iota
	OpTypeWrite
	OpTypeSync
	OpTypeSyncData
	OpTypeSyncTo
	OpTypeCreate
	OpTypeLink
	OpTypeMkdirAll
	OpTypePreallocate
	OpTypeRemove
	OpTypeRemoveAll
	OpTypeRename
	OpTypeReuseForWrite
)

The following OpTypes is limited to the subset of file system operations that a diskHealthCheckingFile supports (namely writes and syncs).

func (OpType) String

func (o OpType) String() string

String implements fmt.Stringer.

type OpenOption

type OpenOption interface {
	// Apply is called on the file handle after it's opened.
	Apply(File)
}

OpenOption provide an interface to do work on file handles in the Open() call.

var RandomReadsOption OpenOption = &randomReadsOption{}

RandomReadsOption is an OpenOption that optimizes opened file handle for random reads, by calling fadvise() with POSIX_FADV_RANDOM on Linux systems to disable readahead.

var SequentialReadsOption OpenOption = &sequentialReadsOption{}

SequentialReadsOption is an OpenOption that optimizes opened file handle for sequential reads, by calling fadvise() with POSIX_FADV_SEQUENTIAL on Linux systems to enable readahead.

type SerzMemFS added in v0.3.0

type SerzMemFS struct {
	Root *SerzMemNode `zid:"0"`

	LockedFiles map[string]bool `zid:"1"`

	Crashable bool `zid:"2"`

	WindowsSemantics bool      `zid:"3"`
	Usage            DiskUsage `zid:"4"`

	Mounts map[string]string `zid:"5"`
}

SerzMemFS is the serialized to disk version of a MemFS.

func (*SerzMemFS) DecodeMsg added in v0.3.0

func (z *SerzMemFS) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable We treat empty fields as if we read a Nil from the wire.

func (*SerzMemFS) EncodeMsg added in v0.3.0

func (z *SerzMemFS) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (*SerzMemFS) Gstring added in v0.3.0

func (z *SerzMemFS) Gstring() (r string)

func (*SerzMemFS) MarshalMsg added in v0.3.0

func (z *SerzMemFS) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (*SerzMemFS) Msgsize added in v0.3.0

func (z *SerzMemFS) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*SerzMemFS) UnmarshalMsg added in v0.3.0

func (z *SerzMemFS) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

func (*SerzMemFS) UnmarshalMsgWithCfg added in v0.3.0

func (z *SerzMemFS) UnmarshalMsgWithCfg(bts []byte, cfg *msgp.RuntimeConfig) (o []byte, err error)

type SerzMemNode added in v0.3.0

type SerzMemNode struct {
	IsDir bool `zid:"0"`

	Data       []byte    `zid:"1"`
	SyncedData []byte    `zid:"2"`
	ModTime    time.Time `zid:"3"`

	Children       map[string]*SerzMemNode `zid:"4"`
	SyncedChildren map[string]*SerzMemNode `zid:"5"`
}

SerzMemNode is the serialized version of memNode.

func (*SerzMemNode) DecodeMsg added in v0.3.0

func (z *SerzMemNode) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable We treat empty fields as if we read a Nil from the wire.

func (*SerzMemNode) EncodeMsg added in v0.3.0

func (z *SerzMemNode) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (*SerzMemNode) Gstring added in v0.3.0

func (z *SerzMemNode) Gstring() (r string)

func (*SerzMemNode) MarshalMsg added in v0.3.0

func (z *SerzMemNode) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (*SerzMemNode) Msgsize added in v0.3.0

func (z *SerzMemNode) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*SerzMemNode) UnmarshalMsg added in v0.3.0

func (z *SerzMemNode) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

func (*SerzMemNode) UnmarshalMsgWithCfg added in v0.3.0

func (z *SerzMemNode) UnmarshalMsgWithCfg(bts []byte, cfg *msgp.RuntimeConfig) (o []byte, err error)

type SyncingFileOptions

type SyncingFileOptions struct {
	// NoSyncOnClose elides the automatic Sync during Close if it's not possible
	// to sync the remainder of the file in a non-blocking way.
	NoSyncOnClose   bool
	BytesPerSync    int
	PreallocateSize int
}

SyncingFileOptions holds the options for a syncingFile.

Directories

Path Synopsis
Package vfstest provides facilities for interacting with or faking filesystems during tests and benchmarks.
Package vfstest provides facilities for interacting with or faking filesystems during tests and benchmarks.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL