README
¶
Gloo Framework
A Go framework for building composable Unix-style command-line tools.
What is Gloo?
Gloo lets you build commands that work like traditional Unix tools - they read from stdin, write to stdout, and can be piped together. Commands are type-safe, composable, and easy to test.
Quick Start
Installing
go get github.com/gloo-foo/framework
Using Commands
Commands are used just like Unix tools:
import (
"github.com/gloo-foo/framework"
"github.com/yupsh/grep"
)
func main() {
// Simple usage
gloo.MustRun(grep.Grep("error", "logfile.txt"))
// With flags
gloo.MustRun(grep.Grep("ERROR", "log.txt", grep.IgnoreCase))
// From stdin
gloo.MustRun(grep.Grep("pattern"))
}
Piping Commands
Commands can be piped together:
// cat file.txt | grep "error" | sort
pipeline := gloo.Pipe(
cat.Cat("file.txt"),
grep.Grep("error"),
sort.Sort(),
)
gloo.MustRun(pipeline)
Building Commands
See framework-examples for complete examples of building your own commands.
Basic Command Structure
package mycommand
import gloo "github.com/gloo-foo/framework"
type command struct {
inputs gloo.Inputs[gloo.File, Flags]
}
func MyCommand(params ...any) gloo.Command {
inputs := gloo.Initialize[gloo.File, Flags](params...)
return command{inputs: inputs}
}
func (c command) Executor() gloo.CommandExecutor {
// Implementation details...
}
Custom Types
Commands can work with structured data:
type LogEntry struct {
Level string
Message string
}
// Commands can process strongly-typed data
inputs := gloo.Initialize[LogEntry, Flags](params...)
API Reference
See API_REFERENCE.md for complete API documentation.
Core Types
Command- Interface for all commandsCommandExecutor- Function that executes a commandInputs[T, O]- Parsed parameters with positionals and flagsFile- Type for file path parameters
Main Functions
gloo.Run(cmd)- Execute a commandgloo.MustRun(cmd)- Execute a command, panic on errorgloo.Pipe(cmd1, cmd2, ...)- Chain commands togethergloo.Initialize[T, O](params...)- Parse command parameters
Examples
See framework-examples for complete working examples:
- Custom struct parameters with flags
- Strongly-typed positional arguments
- Building pipeable commands
Testing
Commands are easy to test:
func TestMyCommand(t *testing.T) {
input := strings.NewReader("test input")
output := &bytes.Buffer{}
cmd := MyCommand("args")
err := cmd.Executor()(context.Background(), input, output, os.Stderr)
// Assert on output.String()
}
License
See LICENSE file.
Documentation
¶
Overview ¶
Package gloo provides a framework for building Unix-style command-line tools in Go.
For Command Users ¶
If you're using gloo.foo commands, you'll primarily interact with:
- Command interface: Represents any executable command
- Run: Execute a command with os.Stdin/Stdout/Stderr
- MustRun: Execute a command and panic on error (useful for examples/tests)
Example:
cmd := cat.Cat("file.txt")
gloo.Run(cmd)
For Command Developers ¶
If you're building gloo.foo commands, see:
- types.go: Core types (File, Inputs, Switch)
- initialize.go: Parameter parsing with Initialize()
- helpers.go: Helper patterns for common command implementations
Index ¶
- func Must(err error)
- func MustRun(cmd Command)
- func MustRunChannel[T any](cmd ChannelCommand[T])
- func ReaderToChannelParsed[T any](ctx context.Context, r io.Reader, out chan<- Row[T]) error
- func ReaderToChannelWithParser[T any](ctx context.Context, r io.Reader, out chan<- Row[T], ...) error
- func Run(cmd Command) error
- func RunChannel[T any](cmd ChannelCommand[T]) error
- func RunChannelWithContext[T any](ctx context.Context, cmd ChannelCommand[T]) error
- func RunWithContext(ctx context.Context, cmd Command) error
- type ChannelCommand
- type ChannelExecutor
- type Command
- type CommandExecutor
- type File
- type Inputs
- func (inputs Inputs[T, O]) Close() error
- func (inputs Inputs[T, O]) Reader(stdin io.Reader) io.Reader
- func (inputs Inputs[T, O]) Readers() []io.Reader
- func (inputs Inputs[T, O]) ToChannelBytes(ctx context.Context, stdin io.Reader, out chan<- Row[[]byte]) error
- func (inputs Inputs[T, O]) ToChannelString(ctx context.Context, stdin io.Reader, out chan<- Row[string]) error
- func (inputs Inputs[T, O]) Wrap(executor CommandExecutor) CommandExecutor
- func (inputs Inputs[T, O]) WrapChannelBytes(executor ChannelExecutor[[]byte]) CommandExecutor
- func (inputs Inputs[T, O]) WrapChannelString(executor ChannelExecutor[string]) CommandExecutor
- type Row
- type RowParser
- type Switch
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Must ¶
func Must(err error)
Must panics if the provided error is non-nil. This is useful for examples and tests where you want to fail fast on errors.
Example:
result, err := someOperation() gloo.Must(err) // proceed with result
func MustRun ¶
func MustRun(cmd Command)
MustRun runs a command and panics if it returns an error. This is useful for examples and tests where you want to fail fast.
Example:
func ExampleCat() {
gloo.MustRun(cat.Cat("file.txt"))
}
func MustRunChannel ¶ added in v0.0.4
func MustRunChannel[T any](cmd ChannelCommand[T])
MustRunChannel runs a channel command and panics if it returns an error.
func ReaderToChannelParsed ¶ added in v0.0.4
ReaderToChannelParsed converts an io.Reader to a channel of Row[T] for custom types that implement RowParser or encoding.TextUnmarshaler.
This enables automatic parsing of custom types from text input.
Example:
type LogEntry struct {
Level string
Message string
}
func (e *LogEntry) ParseRow(line string) error {
parts := strings.SplitN(line, " ", 2)
if len(parts) != 2 {
return fmt.Errorf("invalid log format")
}
e.Level = parts[0]
e.Message = parts[1]
return nil
}
ch := make(chan Row[LogEntry], 100)
go ReaderToChannelParsed(ctx, reader, ch)
for row := range ch {
if row.Err != nil {
log.Printf("Parse error: %v", row.Err)
continue
}
fmt.Printf("Level: %s, Message: %s\n", row.Data.Level, row.Data.Message)
}
func ReaderToChannelWithParser ¶ added in v0.0.4
func ReaderToChannelWithParser[T any](ctx context.Context, r io.Reader, out chan<- Row[T], parse func(string) (T, error)) error
ReaderToChannelWithParser converts an io.Reader to a channel of Row[T] using a custom parser function. This is useful when you want to provide a parsing function without implementing an interface.
Example:
type LogEntry struct {
Level string
Message string
}
parser := func(line string) (LogEntry, error) {
parts := strings.SplitN(line, " ", 2)
if len(parts) != 2 {
return LogEntry{}, fmt.Errorf("invalid format")
}
return LogEntry{Level: parts[0], Message: parts[1]}, nil
}
ch := make(chan Row[LogEntry], 100)
go ReaderToChannelWithParser(ctx, reader, ch, parser)
func Run ¶
Run executes a command with the standard os.Stdin, os.Stdout, and os.Stderr streams. This is the primary way to run commands in production code.
Example:
cmd := cat.Cat("file.txt")
if err := gloo.Run(cmd); err != nil {
log.Fatal(err)
}
func RunChannel ¶ added in v0.0.4
func RunChannel[T any](cmd ChannelCommand[T]) error
RunChannel executes a ChannelCommand with the standard os.Stdin, os.Stdout, and os.Stderr streams. This is the primary way to run channel-based commands.
Example:
cmd := ChannelLineTransform(func(line string) (string, bool) {
return strings.ToUpper(line), true
})
if err := gloo.RunChannel(cmd); err != nil {
log.Fatal(err)
}
func RunChannelWithContext ¶ added in v0.0.4
func RunChannelWithContext[T any](ctx context.Context, cmd ChannelCommand[T]) error
RunChannelWithContext executes a ChannelCommand with a custom context.
Example:
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := gloo.RunChannelWithContext(ctx, cmd); err != nil {
log.Fatal(err)
}
func RunWithContext ¶
RunWithContext executes a command with a custom context and the standard os.Stdin, os.Stdout, and os.Stderr streams. Use this when you need to pass cancellation signals, deadlines, or context values to the command.
Example:
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cmd := grep.Grep("pattern", "largefile.txt")
if err := gloo.RunWithContext(ctx, cmd); err != nil {
log.Fatal(err)
}
Types ¶
type ChannelCommand ¶ added in v0.0.4
type ChannelCommand[T any] interface { ChannelExecutor() ChannelExecutor[T] }
ChannelCommand represents a channel-based executable command. This is the channel equivalent of the Command interface.
typ Parameters:
- T: The type of data flowing through the channels
type ChannelExecutor ¶ added in v0.0.4
ChannelExecutor is the function signature for channel-based command execution. It receives context, input channel, and output channel.
typ Parameters:
- T: The type of data flowing through the channels
The executor should:
- Read from the input channel until it's closed
- Process each Row and send results to the output channel
- Return an error if processing fails
- NOT close the output channel (the framework handles that)
func Batch ¶ added in v0.0.4
func Batch[T any](size int, fn func([]T) ([]T, error)) ChannelExecutor[T]
Batch groups rows into batches of a specified size and processes them together. This is useful for operations that are more efficient when processing multiple items at once.
Example:
batchInsert := Batch[string](100, func(batch []string) ([]string, error) {
// Insert batch into database
db.InsertMany(batch)
return batch, nil
})
func ParallelMap ¶ added in v0.0.4
func ParallelMap[T any](workers int, fn func(T) (T, bool, error)) ChannelExecutor[T]
ParallelMap applies a function to each row in parallel using multiple goroutines. This is useful for CPU-intensive operations that can benefit from parallelism.
⚠️ THREAD SAFETY WARNING: Your function receives the input data directly. DO NOT mutate the input. Always return a NEW value instead of modifying the input.
SAFE:
result := process(input) // Read input, create new output return result, true, nil
UNSAFE:
input.Field = "modified" // Mutating input - DON'T DO THIS! return input, true, nil
Treat the input parameter as READ-ONLY. Any mutation may cause undefined behavior or data races.
Example:
func ExpensiveOperation(line string) string {
// Some CPU-intensive work on line
return result // Return NEW string
}
parallelTransform := ParallelMap(4, func(line string) (string, bool, error) {
return ExpensiveOperation(line), true, nil // OK: strings immutable
})
func Pipe ¶ added in v0.0.4
func Pipe[T any](first ChannelExecutor[T], rest ...ChannelExecutor[T]) ChannelExecutor[T]
Pipe connects multiple channel executors into a pipeline. The output of each executor becomes the input of the next.
Example:
grep := ChannelLineTransform(func(line string) (string, bool) {
return line, strings.Contains(line, "ERROR")
})
sort := ChannelAccumulateAndProcess(func(lines []string) []string {
sort.Strings(lines)
return lines
})
pipeline := Pipe(grep.ChannelExecutor(), sort.ChannelExecutor())
type Command ¶
type Command interface {
Executor() CommandExecutor
}
Command represents an executable command. all gloo.foo commands implement this interface.
func Pipeline ¶ added in v0.0.4
func Pipeline[T any](executor ChannelExecutor[T]) Command
Pipeline creates a Command from a channel executor pipeline. This allows you to build complex pipelines using channels and run them with the standard Run function.
Example:
pipeline := gloo.Pipeline(
gloo.ChannelLineTransform(func(line string) (string, bool) {
return strings.ToUpper(line), true
}),
)
gloo.Run(pipeline)
type CommandExecutor ¶
CommandExecutor is the function signature for executing a command. It receives context, input/output streams, and returns an error.
type File ¶
type File string
File represents a file path that should be opened for reading. When used as positional type with Initialize, the framework automatically opens the files. If no files are provided, stdin is used.
Example:
inputs := gloo.Initialize[gloo.File, flags](params...) // Framework automatically opens files and handles stdin
type Inputs ¶
type Inputs[T any, O any] struct { Positional []T // Parsed positional arguments Flags O // Parsed flags Ambiguous []any // Arguments that couldn't be parsed // contains filtered or unexported fields }
Inputs holds parsed command parameters: positional arguments, flags, and I/O streams. This is the primary type command developers work with.
typ Parameters:
- T: typ of positional arguments (e.g., gloo.File, string, custom types)
- O: typ of flags struct
Example:
type command gloo.Inputs[gloo.File, myFlags]
func (c command) Executor() gloo.CommandExecutor {
inputs := gloo.Inputs[gloo.File, myFlags](c)
return inputs.Wrap(...)
}
func Initialize ¶
Initialize parses parameters into Inputs and automatically handles file opening based on the positional type T:
- gloo.File: Automatically opens files for reading (or uses stdin if no files)
- io.Reader: Wraps readers for stdin
- Other types: Just parses (commands define their own types as needed)
Example:
func Cat(parameters ...any) gloo.Command {
inputs := gloo.Initialize[gloo.File, flags](parameters...)
return command(inputs)
}
Custom types:
type DirPath string // Command-defined type inputs := gloo.Initialize[DirPath, flags](parameters...)
func (Inputs[T, O]) Close ¶
Close closes all opened file handles. Call this when done with the inputs.
Example:
inputs := gloo.Initialize[gloo.File, flags](params...) defer inputs.Close()
func (Inputs[T, O]) Reader ¶
Reader returns a single io.Reader that combines all stdin sources. If multiple files were opened, they're concatenated. If none, returns the stdin parameter. This is useful for commands that want to treat multiple files as one stream.
Example:
func (c command) Executor() gloo.CommandExecutor {
return func(ctx context.Context, stdin io.Reader, stdout, stderr io.Writer) error {
input := gloo.Inputs[gloo.File, flags](c).Reader(stdin)
scanner := bufio.NewScanner(input)
// ... process combined stream
}
}
func (Inputs[T, O]) Readers ¶
Readers returns the opened readers for commands that need direct access to io.Reader. This allows commands to work with readers without caring about file paths.
Example:
for _, r := range inputs.Readers() {
scanner := bufio.NewScanner(r)
// ... process each file separately
}
func (Inputs[T, O]) ToChannelBytes ¶ added in v0.0.4
func (inputs Inputs[T, O]) ToChannelBytes(ctx context.Context, stdin io.Reader, out chan<- Row[[]byte]) error
ToChannelBytes converts the input readers to a channel of Row[[]byte].
func (Inputs[T, O]) ToChannelString ¶ added in v0.0.4
func (inputs Inputs[T, O]) ToChannelString(ctx context.Context, stdin io.Reader, out chan<- Row[string]) error
ToChannelString converts the input readers to a channel of Row[string]. This is useful for commands that want to use channels internally.
Example:
func (c command) Executor() gloo.CommandExecutor {
inputs := gloo.Inputs[gloo.File, flags](c)
return func(ctx context.Context, stdin io.Reader, stdout, stderr io.Writer) error {
ch := make(chan Row[string], 100)
go inputs.ToChannelString(ctx, stdin, ch)
for row := range ch {
fmt.Fprintln(stdout, row.Data)
}
return nil
}
}
func (Inputs[T, O]) Wrap ¶
func (inputs Inputs[T, O]) Wrap(executor CommandExecutor) CommandExecutor
Wrap wraps a CommandExecutor to automatically use the correct input source. This allows commands to use helper functions without worrying about stdin vs files. The framework automatically routes input based on how the command was initialized.
Example:
func (c command) Executor() gloo.CommandExecutor {
inputs := gloo.Inputs[gloo.File, flags](c)
return inputs.Wrap(
gloo.AccumulateAndProcess(func(lines []string) []string {
return c.process(lines)
}).Executor(),
)
}
func (Inputs[T, O]) WrapChannelBytes ¶ added in v0.0.4
func (inputs Inputs[T, O]) WrapChannelBytes(executor ChannelExecutor[[]byte]) CommandExecutor
WrapChannelBytes wraps a ChannelExecutor[[]byte] to automatically convert from io streams.
func (Inputs[T, O]) WrapChannelString ¶ added in v0.0.4
func (inputs Inputs[T, O]) WrapChannelString(executor ChannelExecutor[string]) CommandExecutor
WrapChannelString wraps a ChannelExecutor[string] to automatically convert from io streams. This allows channel-based commands to work with the standard io-based framework.
Example:
func (c command) Executor() gloo.CommandExecutor {
inputs := gloo.Inputs[gloo.File, flags](c)
return inputs.WrapChannelString(
gloo.ChannelLineTransform(func(line string) (string, bool) {
return c.process(line), true
}).ChannelExecutor(),
)
}
type Row ¶ added in v0.0.4
type Row[T any] struct { Data T // The actual data Err error // Optional error associated with this row }
Row represents a single unit of data flowing through a channel. It can contain strings (for line-oriented processing), []byte, or any user-defined type.
typ Parameters:
- T: The type of data being passed (string, []byte, or custom types)
⚠️ THREAD SAFETY: Row[T] is passed by value through channels, but if T contains pointers, slices, or maps, multiple goroutines may share references to the same underlying data.
- In single-consumer pipelines: Safe to mutate Data (you own it)
- With fan-out or parallel processing: Treat Data as READ-ONLY or copy explicitly
- Commands should be immutable after construction
See THREADING.md for detailed threading model and best practices.
Example with strings (always safe - strings are immutable):
type command struct { pattern string }
func (c command) ChannelExecutor() ChannelExecutor[string] {
return func(ctx context.Context, in <-chan Row[string], out chan<- Row[string]) error {
for row := range in {
if strings.Contains(row.Data, c.pattern) {
out <- row // Safe: strings immutable
}
}
return nil
}
}
Example with custom types:
type LogEntry struct {
Timestamp time.Time
Level string
Message string
}
func FilterByLevel(level string) ChannelExecutor[LogEntry] {
return func(ctx context.Context, in <-chan Row[LogEntry], out chan<- Row[LogEntry]) error {
for row := range in {
if row.Data.Level == level {
out <- row // Safe in linear pipeline
}
}
return nil
}
}
type RowParser ¶ added in v0.0.4
RowParser is an interface for types that can parse themselves from a text line. Custom types can implement this to enable automatic conversion from io.Reader.
Example:
type LogEntry struct {
Level string
Message string
}
func (e *LogEntry) ParseRow(line string) error {
parts := strings.SplitN(line, " ", 2)
if len(parts) != 2 {
return fmt.Errorf("invalid format")
}
e.Level = parts[0]
e.Message = parts[1]
return nil
}
Now LogEntry can be used with ReaderToChannelParsed:
ch := make(chan Row[LogEntry], 100) go ReaderToChannelParsed(ctx, reader, ch)
type Switch ¶
type Switch[T any] interface { Configure(*T) }
Switch is the interface for flag types. all flag types should implement this interface to configure the flags struct.
Example:
type ignoreCase bool
const (
CaseSensitive ignoreCase = false
CaseInsensitive ignoreCase = true
)
func (f ignoreCase) Configure(flags *flags) {
flags.ignoreCase = f
}
Source Files
Directories