requests

Go Requests

English | 简体中文

Go Requests is a modern, type-safe HTTP client library for Go, inspired by Python's Requests library. It leverages Go 1.18+ generics to provide compile-time type safety and a fluent API for building HTTP requests.

Features

• Type-Safe Generic Methods: GetJSON[T], PostJSON[T], etc. return Result[T] for automatic response parsing
• Fluent Request Builder: Chain methods to construct complex requests with NewGet, NewPost, etc.
• Unified Result Type: Result[T] wraps both parsed data and response metadata
• Middleware Support: Add cross-cutting concerns like logging, authentication, retry
• Session Management: Connection pooling, cookie persistence, default headers
• Pluggable Codecs: JSON, XML built-in with custom codec support
• Retry Mechanism: Configurable retry policies with exponential backoff
• Request/Response Hooks: Observe lifecycle events for logging and metrics
• File Upload: Multipart uploads with progress tracking
• HTTP/2 Support: Automatic HTTP/2 with fallback to HTTP/1.1
• Connection Pool Optimization: Configurable pool sizes and idle timeouts
• High Performance: Object pooling, zero-copy optimizations, close to net/http performance

Install

go get github.com/sunerpy/requests

Quick Start

Simple GET Request

import "github.com/sunerpy/requests"

// Basic GET request
resp, err := requests.Get("https://api.github.com")
if err != nil {
    log.Fatal(err)
}
fmt.Println("Status:", resp.StatusCode)
fmt.Println("Body:", resp.Text())

// GET with query parameters using options
resp, err := requests.Get("https://api.github.com/search/repos",
    requests.WithQuery("q", "golang"),
    requests.WithQuery("sort", "stars"),
)

Generic JSON Response with Result[T]

type User struct {
    ID   int    `json:"id"`
    Name string `json:"name"`
}

// GetJSON returns Result[T] which wraps both data and response metadata
result, err := requests.GetJSON[User]("https://api.example.com/user/1")
if err != nil {
    log.Fatal(err)
}

// Access parsed data
fmt.Printf("User: %+v\n", result.Data())

// Access response metadata
fmt.Printf("Status: %d\n", result.StatusCode())
fmt.Printf("Success: %v\n", result.IsSuccess())
fmt.Printf("Headers: %v\n", result.Headers())

POST with JSON Body

type CreateUser struct {
    Name  string `json:"name"`
    Email string `json:"email"`
}

type UserResponse struct {
    ID int `json:"id"`
}

// PostJSON returns Result[T]
result, err := requests.PostJSON[UserResponse](
    "https://api.example.com/users",
    CreateUser{Name: "John", Email: "[email protected]"},
)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("Created user ID: %d\n", result.Data().ID)
fmt.Printf("Status: %d\n", result.StatusCode())

Using Request Builder

// Use convenience constructors for common methods
req, err := requests.NewPost("https://api.example.com/users").
    WithHeader("X-Custom-Header", "value").
    WithQuery("version", "v2").
    WithJSON(map[string]string{"name": "John"}).
    WithTimeout(10 * time.Second).
    Build()

// Or use NewRequestBuilder for any method
req, err := requests.NewRequestBuilder(requests.MethodPatch, "https://api.example.com/users/1").
    WithJSON(map[string]string{"name": "Jane"}).
    Build()

// Execute with DoJSON for automatic JSON parsing
result, err := requests.DoJSON[UserResponse](
    requests.NewPost("https://api.example.com/users").
        WithJSON(CreateUser{Name: "John", Email: "[email protected]"}),
)

Using Request Options

// Add headers, query params, auth, etc. using variadic options
resp, err := requests.Get("https://api.example.com/data",
    requests.WithHeader("X-Custom-Header", "value"),
    requests.WithQuery("page", "1"),
    requests.WithBearerToken("my-token"),
    requests.WithTimeout(10 * time.Second),
)

Advanced Usage

Session with Defaults

session := requests.NewSession().
    WithBaseURL("https://api.example.com").
    WithTimeout(30 * time.Second).
    WithHeader("Authorization", "Bearer token").
    WithHTTP2(true).
    WithMaxIdleConns(100)

defer session.Close()

// Build request using the new Builder API
req, _ := requests.NewGet("/users").Build()

// Execute with session
resp, err := session.Do(req)

Using Context for Timeout and Cancellation

import "context"

// Create a context with timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

session := requests.NewSession()
defer session.Close()

req, _ := requests.NewGet("https://api.example.com/data").Build()

// Execute with context - respects timeout and cancellation
resp, err := session.DoWithContext(ctx, req)
if err != nil {
    if err == context.DeadlineExceeded {
        log.Println("Request timed out")
    } else if err == context.Canceled {
        log.Println("Request was cancelled")
    }
    return
}
fmt.Println("Response:", resp.Text())

Context Cancellation Example

ctx, cancel := context.WithCancel(context.Background())

// Cancel the request after 2 seconds
go func() {
    time.Sleep(2 * time.Second)
    cancel()
}()

session := requests.NewSession()
defer session.Close()

req, _ := requests.NewGet("https://api.example.com/slow-endpoint").Build()
resp, err := session.DoWithContext(ctx, req)
if err != nil {
    // Handle cancellation
    log.Printf("Request cancelled: %v", err)
}

Session with Retry Policy

session := requests.NewSession().
    WithBaseURL("https://api.example.com").
    WithTimeout(30 * time.Second).
    WithRetry(requests.RetryPolicy{
        MaxAttempts:     3,
        InitialInterval: 100 * time.Millisecond,
        MaxInterval:     5 * time.Second,
        Multiplier:      2.0,
        RetryIf: func(resp *requests.Response, err error) bool {
            if err != nil {
                return true // Retry on network errors
            }
            // Retry on 5xx errors and 429 Too Many Requests
            return resp.StatusCode >= 500 || resp.StatusCode == 429
        },
    })

defer session.Close()

req, _ := requests.NewGet("/users").Build()
resp, err := session.Do(req) // Will automatically retry on failure

Session with Middleware

// Create a logging middleware
loggingMiddleware := requests.MiddlewareFunc(func(req *requests.Request, next requests.Handler) (*requests.Response, error) {
    start := time.Now()
    log.Printf("Request: %s %s", req.Method, req.URL)
    
    resp, err := next(req)
    
    duration := time.Since(start)
    if resp != nil {
        log.Printf("Response: %d in %v", resp.StatusCode, duration)
    }
    return resp, err
})

// Create session with middleware
session := requests.NewSession().
    WithBaseURL("https://api.example.com").
    WithMiddleware(loggingMiddleware)

defer session.Close()

req, _ := requests.NewGet("/users").Build()
resp, err := session.Do(req) // Middleware will log request/response

Middleware

// Logging middleware
loggingMiddleware := requests.MiddlewareFunc(func(req *requests.Request, next requests.Handler) (*requests.Response, error) {
    log.Printf("Request: %s %s", req.Method, req.URL)
    resp, err := next(req)
    if resp != nil {
        log.Printf("Response: %d", resp.StatusCode)
    }
    return resp, err
})

chain := requests.NewMiddlewareChain(loggingMiddleware)

Retry Policy

policy := requests.ExponentialRetryPolicy(3, 100*time.Millisecond, 10*time.Second)

executor := requests.NewRetryExecutor(policy)

resp, err := executor.Execute(nil, func() (*requests.Response, error) {
    return requests.Get("https://api.example.com/data")
})

Request/Response Hooks

hooks := requests.NewHooks()

// Log all requests
hooks.OnRequest(func(req *requests.Request) {
    log.Printf("Sending: %s %s", req.Method, req.URL)
})

// Log all responses with duration
hooks.OnResponse(func(req *requests.Request, resp *requests.Response, duration time.Duration) {
    log.Printf("Received: %d in %v", resp.StatusCode, duration)
})

// Log errors
hooks.OnError(func(req *requests.Request, err error, duration time.Duration) {
    log.Printf("Error: %v", err)
})

File Upload

import "github.com/sunerpy/requests/url"

upload, err := url.NewFileUpload("file", "/path/to/file.pdf")
if err != nil {
    log.Fatal(err)
}

upload.WithProgress(func(uploaded, total int64) {
    fmt.Printf("Progress: %d/%d (%.1f%%)\n", uploaded, total, float64(uploaded)/float64(total)*100)
})

Custom Codec

import "github.com/sunerpy/requests/codec"

// Register custom codec
type YAMLCodec struct{}

func (c *YAMLCodec) Encode(v any) ([]byte, error) { /* ... */ }
func (c *YAMLCodec) Decode(data []byte, v any) error { /* ... */ }
func (c *YAMLCodec) ContentType() string { return "application/yaml" }

codec.Register("application/yaml", &YAMLCodec{})

API Reference

Package requests

Session Methods

Session Configuration Methods

Request Builder

Result[T] Methods

Request Options

Migration Guide

From v1.x to v2.x

1. Result[T] Type: Generic methods now return (Result[T], error) instead of (T, *Response, error)

   // Old
   user, resp, err := requests.GetJSONWithResponse[User](url)
   
   // New
   result, err := requests.GetJSON[User](url)
   user := result.Data()
   statusCode := result.StatusCode()
   

2. Request Builder: Use NewGet, NewPost, etc. instead of NewRequest(method, url, params, body)

   // Old
   req, err := requests.NewRequest(requests.MethodGet, url, params, nil)
   
   // New
   req, err := requests.NewGet(url).WithQuery("key", "value").Build()
   

3. Removed Methods: *WithResponse methods are removed, use Result[T] instead

4. Single Import: Import only github.com/sunerpy/requests for most use cases

Contributing

Contributions are welcome! Please:

1. Fork this repository
2. Create a feature branch
3. Make your changes with tests
4. Submit a pull request

Performance

The library is optimized for high performance with:

• Object Pooling: RequestBuilder, RequestConfig, FastValues pools to reduce allocations
• Header Ownership Transfer: Avoids unnecessary cloning operations
• Response Buffer Pool: Tiered buffer pools (4KB/32KB) for response reading
• Close to net/http: Only ~6 more allocations than raw net/http

BenchmarkBasicRequests/NetHTTP_Get     40μs, 69 allocs, 6.3KB
BenchmarkBasicRequests/Requests_Get    44μs, 75 allocs, 6.3KB  (only +6 allocs)

For performance-critical code, use FastValues for URL parameter building:

import "github.com/sunerpy/requests/url"

// Lock-free FastValues for single-threaded scenarios
fv := url.AcquireFastValues()
defer url.ReleaseFastValues(fv)
fv.Add("page", "1")
fv.Add("limit", "10")

See Performance Optimization Guide for details.

Documentation

For detailed documentation, see:

• Getting Started
• API Reference
• Performance Optimization
• Migration Guide
• Examples

Project Structure

requests/
├── internal/           # Internal packages (not for external use)
│   ├── client/         # HTTP client core implementation
│   ├── models/         # Internal data models
│   └── utils/          # Internal utilities
├── codec/              # Encoder/decoder implementations
├── url/                # URL utilities and file upload
├── docs/               # Documentation
│   ├── guides/         # User guides
│   └── examples/       # Code examples
├── test/               # Benchmark tests
├── example/            # Example application
├── session.go          # Session management
├── methods.go          # HTTP method functions
├── types.go            # Type exports
└── Makefile            # Build automation

License

MIT License - see LICENSE for details.