encryptdb

package module
v0.0.0-...-379a0d7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 29, 2024 License: Apache-2.0 Imports: 3 Imported by: 0

README

EncryptDB

EncryptDB is a decentralized database engine designed in Go, providing secure and scalable data storage solutions. As part of the Encrypt Distributed Ledger (EDL) organization, EncryptDB is built to support distributed systems and blockchain projects by ensuring data integrity, high availability, and enhanced security.

⚠️ Warning

This project is still in active development and is subject to changes. Future versions may introduce breaking changes, new features, or improvements. We recommend keeping your implementation flexible and checking for updates regularly to stay informed about the latest changes and upgrades.

Table of Contents

  1. Overview
  2. Features
  3. Installation
  4. Quick Start
  5. Usage
  6. Configuration
  7. Contributing
  8. License

Overview

EncryptDB is engineered to offer a robust decentralized database solution that integrates seamlessly with blockchain and distributed applications. It is written in Go to leverage the language's concurrency model and strong standard library, aiming to provide a high-performance, reliable, and secure data storage engine.

Features

  • Decentralized Architecture: Supports distributed data storage across multiple nodes, ensuring high availability and fault tolerance.
  • Secure Storage: Uses advanced cryptographic techniques to protect data integrity and confidentiality.
  • Scalable: Designed to handle large datasets and high throughput, optimizing both read and write operations.
  • Transaction Support: Provides ACID compliance for transactional operations, ensuring data consistency.
  • Lightweight and Fast: Minimal dependencies and optimized for performance in resource-constrained environments.
  • Easy Integration: Seamlessly integrates with existing Go applications and distributed systems.

Installation

To install EncryptDB, you'll need to have Go installed on your machine. You can install the library using go get:

go get github.com/EncrypteDL/EncryptDB

Quick Start

Here's a simple example to get you started with EncryptDB:

package main

import (
    "fmt"
    "log"
    "github.com/EncrypteDL/EncryptDB"
)

func main() {
    // Initialize the database
    db, err := EncryptDB.Open("path/to/db")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // Insert a record
    err = db.Put("key", "value")
    if err != nil {
        log.Fatal(err)
    }

    // Retrieve a record
    value, err := db.Get("key")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("Value:", value)
}

Usage

Inserting JSON Data

EncryptDB allows you to store and manage JSON data efficiently. Here’s how you can insert a JSON object into the database:

Example JSON Object
{
  "Date": "2024-08-23T20:49:39.779374+01:00",
  "FileFormat": "tar.gz",
  "FilePath": "/var/folders/_0/h7kj277n3bvbdh1x9ymfdh1w0000gn/T/TestNewTarGzBackupoutpath_directory973048178/002/001.tar.gz",
  "Stores": ["yeet"],
  "Checksum": {
    "Type": "sha256",
    "Value": "61b64d243a90ba8a946268f0f81481666641e1afddd5820f6e9ec1be1ffe445f"
  },
  "Size": 0
}
Storing JSON in EncryptDB

To store a JSON object, you can serialize it to a string and then use the Put method:

package main

import (
    "encoding/json"
    "log"
    "github.com/EncrypteDL/EncryptDB"
)

func main() {
    db, err := EncryptDB.Open("path/to/db")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // JSON object to be stored
    jsonData := map[string]interface{}{
        "Date": "2024-08-23T20:49:39.779374+01:00",
        "FileFormat": "tar.gz",
        "FilePath": "/var/folders/_0/h7kj277n3bvbdh1x9ymfdh1w0000gn/T/TestNewTarGzBackupoutpath_directory973048178/002/001.tar.gz",
        "Stores": []string{"yeet"},
        "Checksum": map[string]string{
            "Type":  "sha256",
            "Value": "61b64d243a90ba8a946268f0f81481666641e1afddd5820f6e9ec1be1ffe445f",
        },
        "Size": 0,
    }

    // Serialize JSON data to a string
    jsonString, err := json.Marshal(jsonData)
    if err != nil {
        log.Fatal(err)
    }

    // Store JSON string in EncryptDB
    err = db.Put("backup:001", string(jsonString))
    if err != nil {
        log.Fatal(err)
    }

    log.Println("JSON data has been stored successfully.")
}
Opening a Database

To open a database, use the Open method, specifying the path to your database file:

db, err := EncryptDB.Open("path/to/db")
if err != nil {
    // handle error
}
defer db.Close()
Inserting and Retrieving Data

Use Put to insert data and Get to retrieve it:

err := db.Put("username", "zakaria")
if err != nil {
    // handle error
}

value, err := db.Get("username")
if err != nil {
    // handle error
}
fmt.Println("Username:", value)

Configuration

EncryptDB can be configured via a configuration file or environment variables. Configuration options include:

  • StoragePath: Path where the database files will be stored.
  • EncryptionKey: Key used for encrypting data at rest.
  • ReplicationFactor: Number of copies of the data to be replicated across nodes.

Contributing

We welcome contributions to EncryptDB! To contribute:

  1. Fork the repository.
  2. Create a new branch with a descriptive name.
  3. Make your changes.
  4. Submit a pull request detailing your changes.

Please ensure that your code adheres to the project's coding standards and includes appropriate tests.

License

EncryptDB is released under the MIT License. See the LICENSE file for more details.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrNotStore = errors.New("provided Filer does not implement Store")

Functions

func IsStore

func IsStore(filer Filer) bool

Types

type Filer

type Filer interface {

	// Backend returns the underlying key/value store.
	Backend() any

	// Has should return true if the given key has an associated value.
	Has(key []byte) bool
	// Get should retrieve the byte slice corresponding to the given key, and any associated errors upon failure.
	Get(key []byte) ([]byte, error)
	// Put should insert the value data in a way that is associated and can be retrieved by the given key data.
	Put(key []byte, value []byte) error
	// Delete should delete the key and the value associated with the given key, and return an error upon failure.
	Delete(key []byte) error
	// Close should safely end any Filer operations of the given dataStore and close any relevant handlers.
	Close() error
	// Sync should take any volatile data and solidify it somehow if relevant. (ram to disk in most cases)
	Sync() error

	Keys() [][]byte
	Len() int
}

Filer is is a way to implement any generic key/value store. These functions should be plug and play with most of the popular key/value store golang libraries.

NOTE: Many key/value golang libraries will already implement this interface already. This exists for more potential granular control in the case that they don't. Otherwise you'd have to build a wrapper around an existing key/value store to satisfy an overencompassing interface.

type Keeper

type Keeper interface {
	// Path should return the base path where all stores should be stored under. (likely as subdirectories)
	Path() string

	// Init should initialize our Filer at the given path, to be referenced and called by dataStore.
	Init(name string, options ...any) error
	// With provides access to the given dataStore by providing a pointer to the related Filer.
	With(name string) Filer
	// WithNew should initialize a new Filer at the given path and return a pointer to it.
	WithNew(name string, options ...any) Filer

	// Destroy should remove the Filer by the given name.
	// It is up to the implementation to decide if the data should be removed or not.
	Destroy(name string) error

	Discover() ([]string, error)

	AllStores() map[string]Filer

	// BackupAll should create a backup of all [Filer] instances in the [Keeper].
	BackupAll(archivePath string) (models.Backup, error)

	// RestoreAll should restore all [Filer] instances from the given archive.
	RestoreAll(archivePath string) error

	Meta() models.Metadata

	Close(name string) error

	CloseAll() error
	SyncAll() error
	SyncAndCloseAll() error
}

Keeper will be in charge of the more meta operations involving Filers. This includes operations like initialization, syncing to disk if applicable, and backing up.

  • When opening a folder of Filers, it should be able to discover and initialize all of them.
  • Additionally, it should be able to confirm the type of the underlying key/value store.

type KeeperCreator

type KeeperCreator func(path string, opt ...any) (Keeper, error)

type Searcher

type Searcher interface {
	// PrefixScan must retrieve all keys in the datastore and stream them to the given channel.
	PrefixScan(prefix string) (<-chan keyvalue.KeyValue, chan error)
	// Search must be able to search through the value contents of our database and stream the results to the given channel.
	Search(query string) (<-chan keyvalue.KeyValue, chan error)
	// ValueExists searches for an exact match of the given value and returns the key that contains it.
	ValueExists(value []byte) (key []byte, ok bool)
}

Searcher must be able to search through our datastore(s) with strings.

type Store

type Store interface {
	Filer
	Searcher
}

Store is an implementation of a Filer and a Searcher.

func ToStore

func ToStore(filer Filer) (Store, error)

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL