README
¶
Cloud Spanner Engine for sqlc
This package provides Cloud Spanner (GoogleSQL) support for sqlc using the memefish parser.
Implementation Overview
The Spanner engine implementation consists of several key components:
- Parser (
parse.go): Handles SQL statement parsing using memefish and metadata extraction - AST Converter (
convert.go): Converts memefish AST to sqlc's internal AST format - Standard Library (
stdlib.go): Defines Spanner's built-in functions and types - Reserved Words (
reserved.go): Handles Spanner SQL reserved keywords
Key Features
Complete Support
- All DML operations (SELECT, INSERT, UPDATE, DELETE)
- THEN RETURN clause (Spanner's equivalent of RETURNING)
- Common Table Expressions (WITH clause)
- All JOIN types (INNER, LEFT, RIGHT, FULL, CROSS)
- Comprehensive subquery support (scalar, EXISTS, IN, ARRAY, table)
- Full operator support (arithmetic, comparison, logical, bitwise)
- CASE expressions and conditional logic
- Type casting and conversions
- Aggregate functions
- TABLESAMPLE for random sampling
- Parameter support with @ syntax
Partial Support
- SELECT AS STRUCT/VALUE (detected but not fully transformed)
- SELECT * EXCEPT/REPLACE (detected but not implemented)
- DotStar (table.*) syntax (parsed but not expanded)
- DDL operations (basic CREATE/DROP TABLE only)
Architecture Decisions
- Parser Choice: Uses memefish (Cloud Spanner SQL parser) instead of ZetaSQL to avoid CGO dependencies
- AST Conversion: Direct conversion from memefish AST to sqlc's PostgreSQL-based AST
- Function Names: Preserved as-is for case-insensitive catalog lookup
- List Initialization: All walked Lists must be initialized with empty Items arrays
Testing
Run tests with:
go test ./internal/engine/spanner/...
Test with Spanner emulator:
# Start emulator
gcloud emulators spanner start
# Set environment
export SPANNER_EMULATOR_HOST=localhost:9010
# Run tests
go test ./internal/engine/spanner/...
Known Limitations
- DotStar Expansion: table.* generates interface{} instead of expanding columns
- STRUCT Type Inference: Limited with column references
- DDL Support: Limited to basic table operations
- EXCEPT/REPLACE: Not implemented for column filtering
See FEATURES.md for detailed feature status.
Contributing
When adding new features:
- Check how other engines (PostgreSQL, SQLite) handle similar features
- Add tests in internal/endtoend/testdata/spanner_features/
- Update FEATURES.md with implementation status
- Follow the patterns established in convert.go for AST conversion
Documentation
¶
Overview ¶
Package spanner implements AST conversion from memefish (Spanner SQL parser) to sqlc's internal AST.
Key architectural decisions:
List initialization: All sqlcast.List fields that are walked by the compiler must be initialized with empty Items arrays, not nil. The compiler's Walk function expects to iterate over these lists and will panic on nil. Fields that are conditionally accessed (like WhereClause) can be nil.
Star (*) handling: Spanner's Star nodes must be wrapped in ColumnRef to match PostgreSQL's AST structure. This wrapping is critical for the compiler's hasStarRef() and column expansion logic to work correctly. The pattern is: ResTarget -> ColumnRef -> A_Star.
THEN RETURN conversion: Spanner's THEN RETURN clause is converted to PostgreSQL's RETURNING clause, maintaining the same AST structure patterns (especially for Star nodes).
Function names: Spanner supports namespaced functions (e.g., NET.IPV4_TO_INT64, SAFE.DIVIDE). All path components are joined with dots to preserve the full function name for resolution.
Package spanner provides the Cloud Spanner SQL parser for sqlc.
Named Parameters Design Decision ¶
Unlike PostgreSQL, MySQL, and SQLite engines which convert named parameters (e.g., @name or sqlc.arg('name')) to positional parameters in the SQL query, the Spanner engine preserves named parameters (@name) in the generated SQL.
This is because:
- Cloud Spanner natively supports named parameters with @ syntax
- The go-sql-spanner driver can bind positional arguments to named parameters
- Other drivers (lib/pq, go-sql-driver/mysql) don't support sql.Named()
For consistency with other engines and to minimize changes to the common code generation logic, sqlc currently generates code using positional arguments even for Spanner. This avoids the need for engine-specific parameter handling in the shared codegen layer, relying instead on the go-sql-spanner driver's ability to automatically bind positional arguments to named parameters.
This is a pragmatic trade-off for the initial implementation. Future versions may consider more native support for named parameters using sql.Named() once the common codegen layer can better accommodate engine-specific differences.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewCatalog ¶
func NewIdentifier ¶
Types ¶
type Parameter ¶
Parameter represents a query parameter with its name and position
func ExtractParameters ¶
ExtractParameters extracts all @param style parameters from an AST node Uses ast.Preorder for cleaner, more idiomatic implementation
type Parser ¶
type Parser struct{}
func (*Parser) CommentSyntax ¶
func (p *Parser) CommentSyntax() source.CommentSyntax
CommentSyntax returns the comment syntax supported by Spanner
func (*Parser) IsReservedKeyword ¶
IsReservedKeyword checks if a string is a reserved keyword in Spanner SQL. It uses memefish's built-in IsKeyword function which perfectly matches the official Cloud Spanner reserved keywords list.
This implementation is maintenance-free as it automatically stays in sync with memefish's keyword definitions, which are based on the official Spanner SQL specification.
Source Files