parser

High-Performance HTTP Template Parser

A Go module that provides a high-performance parser based on text/template for HTTP requests. This module is designed to handle non-rewindable HTTP input streams efficiently while providing template caching and automatic recompilation on file changes.

Performance: Capable of processing 40,000+ requests per second with template caching, optimized for high-throughput web applications and microservices.

Features

• Re-readable HTTP Requests: Handles non-rewindable HTTP input streams by buffering them in memory
• Template Caching: High-performance template caching with LRU eviction
• File Watching: Automatic template recompilation when template files change
• Flexible Template Loading: Supports file system and memory-based template loaders
• Rich Request Data: Extracts headers, query parameters, form data, and body content
• Custom Function Maps: Support for custom template functions
• Thread-Safe: Concurrent access safe across goroutines
• Benchmarked Performance: Optimized for high-throughput scenarios

Installation

go get github.com/fabricates/parser

Quick Start

package main

import (
    "bytes"
    "fmt"
    "net/http"
    "strings"
    
    "github.com/fabricates/parser"
)

func main() {
    // Simple configuration (uses MemoryLoader by default)
    config := parser.Config{
        MaxCacheSize: 100,
        FuncMap:      parser.DefaultFuncMap(),
    }
    
    // Create parser
    p, err := parser.NewParser(config)
    if err != nil {
        panic(err)
    }
    defer p.Close()
    
    // Add template dynamically
    err = p.UpdateTemplate("greeting", "Hello {{.Request.Method}} from {{.Request.URL.Path}}!", "v1")
    if err != nil {
        panic(err)
    }
    
    // Create HTTP request
    req, _ := http.NewRequest("GET", "http://example.com/api/users", nil)
    
    // Parse template
    var output bytes.Buffer
    err = p.Parse("greeting", req, &output)
    if err != nil {
        panic(err)
    }
    
    fmt.Print(output.String()) // Output: Hello GET from /api/users!
}

Alternative: Explicit MemoryLoader

func main() {
    // Create a memory-based template loader explicitly
    loader := parser.NewMemoryLoader()
    loader.AddTemplate("greeting", "Hello {{.Request.Method}} from {{.Request.URL.Path}}!")
    
    // Create parser configuration
    config := parser.Config{
        TemplateLoader: loader,
        MaxCacheSize:   100,
        WatchFiles:     false,
        FuncMap:        parser.DefaultFuncMap(),
    }
    
    // Create parser
    p, err := parser.NewParser(config)
    if err != nil {
        panic(err)
    }
    defer p.Close()
    
    // Create HTTP request
    req, _ := http.NewRequest("GET", "http://example.com/api/users", nil)
    
    // Parse template
    var output bytes.Buffer
    err = p.Parse("greeting", req, &output)
    if err != nil {
        panic(err)
    }
    
    fmt.Print(output.String()) // Output: Hello GET from /api/users!
}

Core Components

Parser Interface

The main interface provides methods for template parsing and management:

type Parser interface {
    Parse(templateName string, request *http.Request, output io.Writer) error
    ParseWith(templateName string, request *http.Request, data interface{}, output io.Writer) error
    UpdateTemplate(name string, content string, hash string) error
    GetCacheStats() CacheStats
    Close() error
}

Template Loaders

File System Loader

Load templates from the file system with optional file watching:

loader := parser.NewFileSystemLoader("/path/to/templates", ".tmpl", true)

config := parser.Config{
    TemplateLoader: loader,
    WatchFiles:     true, // Enable automatic reloading
    MaxCacheSize:   50,
}

p, err := parser.NewParser(config)

Memory Loader

For testing or when templates are embedded:

loader := parser.NewMemoryLoader()
loader.AddTemplate("welcome", "Welcome {{.Custom.username}}!")

config := parser.Config{
    TemplateLoader: loader,
    MaxCacheSize:   10,
}

p, err := parser.NewParser(config)

Request Data Structure

Templates have access to structured request data:

type RequestData struct {
    Request *http.Request           // Original HTTP request
    Headers map[string][]string     // HTTP headers
    Query   map[string][]string     // Query parameters
    Form    map[string][]string     // Form data (for POST requests)
    Body    string                  // Request body as string
    Custom  interface{}             // Custom data passed to ParseWith
}

Dynamic Template Updates

You can dynamically add or update templates at runtime using the UpdateTemplate method:

// Add a new template
templateContent := "Hello {{.Request.Method}} from {{.Request.URL.Path}}!"
err := parser.UpdateTemplate("greeting", templateContent, "hash123")
if err != nil {
    log.Fatalf("Failed to update template: %v", err)
}

// Later, update the same template with new content
newContent := "Updated: {{.Request.Method}} {{.Request.URL.Path}}"
err = parser.UpdateTemplate("greeting", newContent, "hash456")
if err != nil {
    log.Fatalf("Failed to update template: %v", err)
}

// Use the updated template
var output bytes.Buffer
err = parser.Parse("greeting", request, &output)

Template Examples

Basic Request Information

Method: {{.Request.Method}}
URL: {{.Request.URL.Path}}
User-Agent: {{index .Headers "User-Agent" 0}}

Query Parameters

{{if .Query.name}}
Hello {{index .Query "name" 0}}!
{{end}}

Form Data

{{if .Form.username}}
Username: {{index .Form "username" 0}}
{{end}}

Request Body

{{if .Body}}
Received: {{.Body}}
{{end}}

Custom Data

customData := map[string]interface{}{
    "user_id": 123,
    "role":    "admin",
}

err := parser.ParseWith("template", request, customData, output)

User ID: {{.Custom.user_id}}
Role: {{.Custom.role}}

Built-in Template Functions

The parser includes useful template functions:

• upper: Convert string to uppercase
• lower: Convert string to lowercase
• title: Convert string to title case
• trim: Remove leading/trailing whitespace
• default: Provide default value for empty/nil values
• header: Get request header value
• query: Get query parameter value
• form: Get form field value

Example usage:

Name: {{.Custom.name | upper | default "Anonymous"}}
Content-Type: {{header .Request "Content-Type"}}

File Watching

When WatchFiles is enabled, the parser automatically detects template file changes and recompiles them:

config := parser.Config{
    TemplateLoader: parser.NewFileSystemLoader("./templates", ".tmpl", true),
    WatchFiles:     true,
    MaxCacheSize:   100,
}

p, err := parser.NewParser(config)
// Templates will be automatically reloaded when files change

Performance Optimizations

Template Caching

Templates are cached after compilation with LRU eviction:

config := parser.Config{
    TemplateLoader: loader,
    MaxCacheSize:   100, // Cache up to 100 templates
}

// Get cache statistics
stats := p.GetCacheStats()
fmt.Printf("Cache: %d/%d, Hits: %d\n", stats.Size, stats.MaxSize, stats.HitCount)

Re-readable Requests

HTTP request bodies are automatically buffered to allow multiple reads:

// The parser handles this automatically
rereadableReq, err := parser.NewRereadableRequest(originalRequest)
rereadableReq.Reset() // Reset body for re-reading

Error Handling

The module defines specific error types:

var (
    ErrTemplateNotFound = errors.New("template not found")
    ErrWatcherClosed    = errors.New("file watcher is closed")
    ErrInvalidConfig    = errors.New("invalid configuration")
    ErrParserClosed     = errors.New("parser is closed")
)

Testing

Run the test suite:

go test -v

Run benchmarks:

go test -bench=.

Example benchmark results:

BenchmarkParserParse-2           193795    6008 ns/op
BenchmarkRequestExtraction-2     314965    3690 ns/op

Examples

See the /examples directory for complete usage examples:

• examples/basic/: Basic usage with different request types
• More examples coming soon!

Configuration

type Config struct {
    TemplateLoader TemplateLoader    // How to load templates (defaults to MemoryLoader if nil)
    WatchFiles     bool              // Enable file watching (FileSystemLoader only)
    MaxCacheSize   int               // Template cache size (0 = unlimited)
    FuncMap        template.FuncMap  // Custom template functions
}

Default Behavior

If no TemplateLoader is specified in the config, the parser will automatically use a MemoryLoader by default. This allows you to create a parser with minimal configuration:

// Simple config with default MemoryLoader
config := parser.Config{
    MaxCacheSize: 100,
}

p, err := parser.NewParser(config)
if err != nil {
    panic(err)
}

// Add templates dynamically using UpdateTemplate
err = p.UpdateTemplate("greeting", "Hello {{.Request.Method}}!", "hash123")

Performance

The parser is designed for high performance with several optimizations:

Benchmark Results

Performance characteristics on a typical server (Intel Xeon E5-2680 v2 @ 2.80GHz):

Cache Performance

Template caching provides significant performance benefits:

Recommendation: Use cache size 10-50 for most applications.

Body Size Impact

Request body size affects memory usage and performance:

Concurrent Performance

The parser maintains good performance under concurrent load:

• Concurrent Parsing: ~40,000 ops/sec with multiple goroutines
• Thread-safe: No performance degradation with concurrent access
• Lock-free reads: Template cache uses efficient concurrent access patterns

Optimization Tips

1. Use appropriate cache size: 10-50 templates for most applications
2. Minimize template complexity: Simpler templates execute faster
3. Reuse parser instances: Creating parsers has overhead
4. Pre-load templates: Use UpdateTemplate to warm the cache
5. Monitor cache hit rates: Use GetCacheStats() to optimize cache size

Thread Safety

All components are designed to be thread-safe and can be used concurrently across multiple goroutines.

Contributing

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.