dag

package
v1.3.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 4, 2025 License: MIT Imports: 18 Imported by: 3

Documentation

Index

Constants

View Source 
const DefaultBatchSize = 4 * 1024 * 1024 // 4MB default batch size
View Source 
const DefaultChunkSize = 2048 * 1024 // 2048 * 1024 bytes = 2 megabytes

Variables

View Source 
var BatchSize = DefaultBatchSize
View Source 
var ChunkSize = DefaultChunkSize

Functions

func DisableBatching added in v1.3.7 

func DisableBatching()

func DisableChunking added in v1.2.4 

func DisableChunking()

func GenerateDummyDirectory 

func GenerateDummyDirectory(path string, minItems, maxItems, minDepth, maxDepth int)

func GenerateLargeDummyDirectory added in v1.3.7 

func GenerateLargeDummyDirectory(path string, minItems, maxItems, minDepth, maxDepth int, fileSize int)

func GenerateMerkleProofTriggerDirectory added in v1.3.7 

func GenerateMerkleProofTriggerDirectory(path string, numChildrenPerDir int, fileSize int)

GenerateMerkleProofTriggerDirectory creates a directory structure specifically designed to trigger the batched transmission Merkle proof bug. It creates directories with many children that will be batched together, requiring Merkle proof generation.

func GenerateMerkleProofTriggerDirectoryOutOfOrder added in v1.3.7 

func GenerateMerkleProofTriggerDirectoryOutOfOrder(path string, numChildrenPerDir int, fileSize int)

GenerateMerkleProofTriggerDirectoryOutOfOrder creates a directory structure specifically designed to trigger the batched transmission Merkle proof bug. It creates directories with many children that will be batched together, requiring Merkle proof generation. This version uses specific filenames that will be out of alphabetical order.

func GetHash 

func GetHash(hash string) string

func GetLabel 

func GetLabel(hash string) string

func HasLabel 

func HasLabel(hash string) bool

func SetBatchSize added in v1.3.7 

func SetBatchSize(size int)

func SetChunkSize 

func SetChunkSize(size int)

func SetDefaultBatchSize added in v1.3.7 

func SetDefaultBatchSize()

func SetDefaultChunkSize added in v1.2.4 

func SetDefaultChunkSize()

Types

type BatchedTransmissionPacket added in v1.3.7 

type BatchedTransmissionPacket struct {
	Leaves        []*DagLeaf
	Relationships map[string]string
	Proofs        map[string]*ClassicTreeBranch
}

func BatchedTransmissionPacketFromCBOR added in v1.3.7 

func BatchedTransmissionPacketFromCBOR(data []byte) (*BatchedTransmissionPacket, error)

BatchedTransmissionPacketFromCBOR deserializes a BatchedTransmissionPacket from CBOR format

func BatchedTransmissionPacketFromJSON added in v1.3.7 

func BatchedTransmissionPacketFromJSON(data []byte) (*BatchedTransmissionPacket, error)

BatchedTransmissionPacketFromJSON deserializes a BatchedTransmissionPacket from JSON format

func BatchedTransmissionPacketFromSerializable added in v1.3.7 

func BatchedTransmissionPacketFromSerializable(s *SerializableBatchedTransmissionPacket) *BatchedTransmissionPacket

BatchedTransmissionPacketFromSerializable reconstructs a BatchedTransmissionPacket from its serializable form

func (*BatchedTransmissionPacket) GetRootLeaf added in v1.3.7 

func (packet *BatchedTransmissionPacket) GetRootLeaf() *DagLeaf

GetRootLeaf returns the root leaf from the batch.

func (*BatchedTransmissionPacket) ToCBOR added in v1.3.7 

func (packet *BatchedTransmissionPacket) ToCBOR() ([]byte, error)

ToCBOR serializes a BatchedTransmissionPacket to CBOR format

func (*BatchedTransmissionPacket) ToJSON added in v1.3.7 

func (packet *BatchedTransmissionPacket) ToJSON() ([]byte, error)

ToJSON serializes a BatchedTransmissionPacket to JSON format

func (*BatchedTransmissionPacket) ToSerializable added in v1.3.7 

func (packet *BatchedTransmissionPacket) ToSerializable() *SerializableBatchedTransmissionPacket

ToSerializable converts a BatchedTransmissionPacket to its serializable form

type ClassicTreeBranch 

type ClassicTreeBranch struct {
	Leaf  string
	Proof *merkletree.Proof
}

type Dag 

type Dag struct {
	Root  string
	Leafs map[string]*DagLeaf
}

func CreateDag 

func CreateDag(path string, timestampRoot bool) (*Dag, error)

func CreateDagAdvanced 

func CreateDagAdvanced(path string, additionalData map[string]string) (*Dag, error)

func CreateDagCustom added in v1.2.3 

func CreateDagCustom(path string, rootAdditionalData map[string]string, processor LeafProcessor) (*Dag, error)

CreateDagCustom creates a DAG with custom metadata for each leaf

func FromCBOR 

func FromCBOR(data []byte) (*Dag, error)

func FromJSON 

func FromJSON(data []byte) (*Dag, error)

func FromSerializable 

func FromSerializable(s *SerializableDag) *Dag

FromSerializable reconstructs a Dag from its serializable form

func ReadDag 

func ReadDag(path string) (*Dag, error)

func (*Dag) ApplyAndVerifyBatchedTransmissionPacket added in v1.3.7 

func (d *Dag) ApplyAndVerifyBatchedTransmissionPacket(packet *BatchedTransmissionPacket) error

ApplyAndVerifyBatchedTransmissionPacket verifies then applies a batched transmission packet

func (*Dag) ApplyAndVerifyTransmissionPacket added in v1.2.5 

func (d *Dag) ApplyAndVerifyTransmissionPacket(packet *TransmissionPacket) error

ApplyAndVerifyTransmissionPacket verifies then applies a transmission packet

func (*Dag) ApplyBatchedTransmissionPacket added in v1.3.7 

func (d *Dag) ApplyBatchedTransmissionPacket(packet *BatchedTransmissionPacket)

ApplyBatchedTransmissionPacket applies a batched transmission packet to the DAG

func (*Dag) ApplyTransmissionPacket added in v1.1.0 

func (d *Dag) ApplyTransmissionPacket(packet *TransmissionPacket)

ApplyTransmissionPacket applies a transmission packet to the DAG

func (*Dag) CreateDirectory 

func (dag *Dag) CreateDirectory(path string) error

func (*Dag) GetBatchedLeafSequence added in v1.3.7 

func (d *Dag) GetBatchedLeafSequence() []*BatchedTransmissionPacket

GetBatchedLeafSequence returns an ordered sequence of batched transmission packets Each batch contains multiple related leaves that share common branches for efficient transmission

func (*Dag) GetContentFromLeaf 

func (dag *Dag) GetContentFromLeaf(leaf *DagLeaf) ([]byte, error)

func (*Dag) GetLeafSequence added in v1.1.0 

func (d *Dag) GetLeafSequence() []*TransmissionPacket

GetLeafSequence returns an ordered sequence of leaves for transmission Each packet contains a leaf, its parent hash, and any proofs needed for verification

func (*Dag) GetPartial 

func (d *Dag) GetPartial(start, end int) (*Dag, error)

GetPartial returns a new DAG containing only the requested leaves and their verification paths

func (*Dag) IsPartial 

func (d *Dag) IsPartial() bool

IsPartial returns true if this DAG is a partial DAG (has fewer leaves than the total count)

func (*Dag) IterateDag 

func (d *Dag) IterateDag(processLeaf func(leaf *DagLeaf, parent *DagLeaf) error) error

func (*Dag) RemoveAllContent added in v1.2.4 

func (d *Dag) RemoveAllContent()

func (*Dag) ToCBOR 

func (dag *Dag) ToCBOR() ([]byte, error)

func (*Dag) ToJSON 

func (dag *Dag) ToJSON() ([]byte, error)

func (*Dag) ToSerializable 

func (dag *Dag) ToSerializable() *SerializableDag

ToSerializable converts a Dag to its serializable form

func (*Dag) Verify 

func (d *Dag) Verify() error

Verify checks the integrity of the DAG, automatically choosing between full and partial verification

func (*Dag) VerifyBatchedTransmissionPacket added in v1.3.7 

func (d *Dag) VerifyBatchedTransmissionPacket(packet *BatchedTransmissionPacket) error

VerifyBatchedTransmissionPacket verifies a batched transmission packet independently

func (*Dag) VerifyTransmissionPacket added in v1.2.5 

func (d *Dag) VerifyTransmissionPacket(packet *TransmissionPacket) error

VerifyTransmissionPacket verifies a transmission packet independently

type DagBranch 

type DagBranch struct {
	Leaf         *DagLeaf
	Path         []*DagLeaf
	MerkleProofs map[string]*ClassicTreeBranch
}

type DagBuilder 

type DagBuilder struct {
	Leafs map[string]*DagLeaf
}

func CreateDagBuilder 

func CreateDagBuilder() *DagBuilder

func (*DagBuilder) AddLeaf 

func (b *DagBuilder) AddLeaf(leaf *DagLeaf, parentLeaf *DagLeaf) error

func (*DagBuilder) BuildDag 

func (b *DagBuilder) BuildDag(root string) *Dag

func (*DagBuilder) GetLatestLabel 

func (b *DagBuilder) GetLatestLabel() string

func (*DagBuilder) GetNextAvailableLabel 

func (b *DagBuilder) GetNextAvailableLabel() string

type DagLeaf 

type DagLeaf struct {
	Hash              string
	ItemName          string
	Type              LeafType
	ContentHash       []byte
	Content           []byte
	ClassicMerkleRoot []byte
	CurrentLinkCount  int
	LatestLabel       string
	LeafCount         int
	Links             map[string]string
	ParentHash        string
	AdditionalData    map[string]string
	MerkleTree        *merkletree.MerkleTree
	LeafMap           map[string]merkletree.DataBlock
	Proofs            map[string]*ClassicTreeBranch
}

func CreateDummyLeaf 

func CreateDummyLeaf(name string) (*DagLeaf, error)

func FindRandomChild 

func FindRandomChild(leaf *DagLeaf, leafs map[string]*DagLeaf) *DagLeaf
func (leaf *DagLeaf) AddLink(hash string)

func (*DagLeaf) Clone 

func (leaf *DagLeaf) Clone() *DagLeaf

func (*DagLeaf) CreateDirectoryLeaf 

func (leaf *DagLeaf) CreateDirectoryLeaf(path string, dag *Dag) error

func (*DagLeaf) GetBranch 

func (leaf *DagLeaf) GetBranch(key string) (*ClassicTreeBranch, error)

func (*DagLeaf) GetIndexForKey 

func (leaf *DagLeaf) GetIndexForKey(key string) (int, bool)
func (leaf *DagLeaf) HasLink(hash string) bool

func (*DagLeaf) SetLabel 

func (leaf *DagLeaf) SetLabel(label string)

func (*DagLeaf) ToSerializable 

func (leaf *DagLeaf) ToSerializable() *SerializableDagLeaf

ToSerializable converts a DagLeaf to its serializable form

func (*DagLeaf) VerifyBranch 

func (leaf *DagLeaf) VerifyBranch(branch *ClassicTreeBranch) error

func (*DagLeaf) VerifyLeaf 

func (leaf *DagLeaf) VerifyLeaf() error

func (*DagLeaf) VerifyRootLeaf 

func (leaf *DagLeaf) VerifyRootLeaf() error

type DagLeafBuilder 

type DagLeafBuilder struct {
	ItemName string
	Label    int64
	LeafType LeafType
	Data     []byte
	Links    map[string]string
}

func CreateDagLeafBuilder 

func CreateDagLeafBuilder(name string) *DagLeafBuilder
func (b *DagLeafBuilder) AddLink(label string, hash string)

func (*DagLeafBuilder) BuildLeaf 

func (b *DagLeafBuilder) BuildLeaf(additionalData map[string]string) (*DagLeaf, error)

func (*DagLeafBuilder) BuildRootLeaf 

func (b *DagLeafBuilder) BuildRootLeaf(dag *DagBuilder, additionalData map[string]string) (*DagLeaf, error)

func (*DagLeafBuilder) SetData 

func (b *DagLeafBuilder) SetData(data []byte)

func (*DagLeafBuilder) SetType 

func (b *DagLeafBuilder) SetType(leafType LeafType)

type LeafProcessor added in v1.2.3 

type LeafProcessor func(path string, relPath string, entry fs.DirEntry, isRoot bool, leafType LeafType) map[string]string

LeafProcessor is a function that generates metadata for a leaf path: The full path to the file/directory relPath: The relative path within the DAG entry: The file/directory entry information isRoot: Whether this is the root leaf leafType: The type of leaf (file, directory, chunk) Returns additional metadata to be added to the leaf

type LeafType 

type LeafType string
const (
	FileLeafType      LeafType = "file"
	ChunkLeafType     LeafType = "chunk"
	DirectoryLeafType LeafType = "directory"
)

type SerializableBatchedTransmissionPacket added in v1.3.7 

type SerializableBatchedTransmissionPacket struct {
	Leaves        []*SerializableDagLeaf
	Relationships map[string]string
	Proofs        map[string]*ClassicTreeBranch `json:"proofs,omitempty" cbor:"proofs,omitempty"`
}

SerializableBatchedTransmissionPacket is a minimal version of BatchedTransmissionPacket for efficient serialization

type SerializableDag 

type SerializableDag struct {
	Root  string
	Leafs map[string]*SerializableDagLeaf
}

SerializableDag is a minimal version of Dag for efficient serialization

type SerializableDagLeaf 

type SerializableDagLeaf struct {
	Hash              string
	ItemName          string
	Type              LeafType
	ContentHash       []byte
	Content           []byte
	ClassicMerkleRoot []byte
	CurrentLinkCount  int
	LatestLabel       string
	LeafCount         int
	Links             map[string]string
	AdditionalData    map[string]string
	StoredProofs      map[string]*ClassicTreeBranch `json:"stored_proofs,omitempty" cbor:"stored_proofs,omitempty"`
}

SerializableDagLeaf is a minimal version of DagLeaf for efficient serialization

type SerializableTransmissionPacket added in v1.1.0 

type SerializableTransmissionPacket struct {
	Leaf       *SerializableDagLeaf
	ParentHash string
	Proofs     map[string]*ClassicTreeBranch `json:"proofs,omitempty" cbor:"proofs,omitempty"`
}

SerializableTransmissionPacket is a minimal version of TransmissionPacket for efficient serialization

type TransmissionPacket added in v1.1.0 

type TransmissionPacket struct {
	Leaf       *DagLeaf
	ParentHash string
	Proofs     map[string]*ClassicTreeBranch
}

func TransmissionPacketFromCBOR added in v1.1.0 

func TransmissionPacketFromCBOR(data []byte) (*TransmissionPacket, error)

TransmissionPacketFromCBOR deserializes a TransmissionPacket from CBOR format

func TransmissionPacketFromJSON added in v1.1.0 

func TransmissionPacketFromJSON(data []byte) (*TransmissionPacket, error)

TransmissionPacketFromJSON deserializes a TransmissionPacket from JSON format

func TransmissionPacketFromSerializable added in v1.1.0 

func TransmissionPacketFromSerializable(s *SerializableTransmissionPacket) *TransmissionPacket

TransmissionPacketFromSerializable reconstructs a TransmissionPacket from its serializable form

func (*TransmissionPacket) ToCBOR added in v1.1.0 

func (packet *TransmissionPacket) ToCBOR() ([]byte, error)

ToCBOR serializes a TransmissionPacket to CBOR format

func (*TransmissionPacket) ToJSON added in v1.1.0 

func (packet *TransmissionPacket) ToJSON() ([]byte, error)

ToJSON serializes a TransmissionPacket to JSON format

func (*TransmissionPacket) ToSerializable added in v1.1.0 

func (packet *TransmissionPacket) ToSerializable() *SerializableTransmissionPacket

ToSerializable converts a TransmissionPacket to its serializable form

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.