Releases

🎯 Simplified pgvector Support

This release simplifies pgvector integration by removing the pgvector-python library dependency and using native JSON parsing instead. This results in zero external dependencies, better Pydantic compatibility, and simpler code.

✨ What's Changed

Removed Dependencies

• Removed pgvector-python dependency - No longer required for vector support
• Removed numpy dependency - Vector data returned as native Python list[float]

Simplified Implementation

• PostgreSQL returns vectors as strings like '[0.1,0.2,0.3]'
• Automatically parsed to list[float] using built-in JSON parsing
• Removed 28 lines of adapter registration complexity
• No async adapter registration needed

Performance

• Negligible overhead: <0.1ms per vector for JSON parsing
• For 1000 vectors: ~10-100ms difference vs adapter (1-10% overhead)
• Completely acceptable for typical LLM embedding use cases

🚀 Key Benefits

✅ Zero External Dependencies - No pgvector-python, no numpy required ✅ Simpler Code - Removed async adapter registration logic ✅ Perfect Pydantic Compatibility - Returns list[float], not numpy.ndarray ✅ No Breaking Changes - Same public API, drop-in upgrade ✅ All Tests Passing - 227 tests with zero regressions

📦 Installation

pip install psycopg-toolkit==0.3.1

💻 Usage Example

from pydantic import BaseModel
from psycopg_toolkit import BaseRepository
from uuid import UUID

class DocumentEmbedding(BaseModel):
    id: UUID
    document_id: UUID
    embedding: list[float]  # Auto-detected as vector field
    metadata: dict[str, Any] | None = None

class EmbeddingRepository(BaseRepository[DocumentEmbedding, UUID]):
    def __init__(self, db_connection):
        super().__init__(
            db_connection=db_connection,
            table_name="embeddings",
            model_class=DocumentEmbedding,
            primary_key="id",
            auto_detect_vector=True,  # Automatically detects list[float]
        )

# Usage - works seamlessly with PostgreSQL pgvector extension
embedding_vector = [0.1] * 384  # 384-dimensional vector
doc = DocumentEmbedding(
    id=uuid4(),
    document_id=uuid4(),
    embedding=embedding_vector,
    metadata={"model": "all-MiniLM-L6-v2"}
)

created = await repo.create(doc)
assert isinstance(created.embedding, list)  # Returns list[float]!

🔧 Technical Details

Before (v0.3.0):

• Required pgvector-python>=0.4.1 dependency
• Used register_vector_async() adapter registration
• Returned vectors as numpy.ndarray
• Required numpy as transitive dependency

After (v0.3.1):

• No external dependencies needed
• Simple JSON parsing with JSONHandler.deserialize()
• Returns vectors as native Python list[float]
• Direct Pydantic compatibility

📝 Full Changelog

Commits:

• 82ad82a - refactor: Remove pgvector-python dependency, use JSON parsing for vectors
• Updated documentation in .claude/CLAUDE.md
• Removed adapter registration from BaseRepository
• Added vector field parsing in _postprocess_data()

Files Changed: 4 files (+37, -152 lines)

• pyproject.toml - Removed pgvector dependency
• src/psycopg_toolkit/repositories/base.py - Simplified vector handling
• .claude/CLAUDE.md - Updated documentation
• uv.lock - Dependency updates

🧪 Testing

All 227 tests passing:

• 12 vector field unit tests
• 4 pgvector integration tests
• 211 other tests (JSONB, arrays, dates, transactions, etc.)
• Zero regressions

⚠️ Breaking Changes

None - This is a drop-in replacement for v0.3.0. The public API remains identical.

---

Full Changelog: https://github.com/descoped/psycopg-toolkit/compare/v0.3.0...v0.3.1

🎉 Native pgvector Support

This release adds native support for PostgreSQL's pgvector extension with automatic vector field detection, following the same pattern as our existing JSONB support.

✨ New Features

pgvector Integration

• Automatic vector field detection - Auto-detects list[float] type hints as pgvector columns
• Lazy async adapter registration - Uses register_vector_async() for proper async handling
• Graceful degradation - Works seamlessly even if pgvector package is not installed
• Per-connection registration - Properly handles connection pooling
• Type-safe operations - Full type safety with list[float] <-> PostgreSQL vector

Repository Parameters

• auto_detect_vector (default: True) - Automatically detect vector fields from type hints
• vector_fields - Explicitly specify vector field names (overrides auto-detection)
• Vector fields automatically excluded from JSON processing

Usage Example

from pydantic import BaseModel
from uuid import UUID

class DocumentEmbedding(BaseModel):
    id: UUID
    embedding: list[float]  # Auto-detected as vector!
    metadata: dict[str, Any]

class EmbeddingRepository(BaseRepository[DocumentEmbedding, UUID]):
    def __init__(self, db_connection):
        super().__init__(
            db_connection=db_connection,
            table_name="embeddings",
            model_class=DocumentEmbedding,
            primary_key="id",
            # auto_detect_vector=True is default
        )

# Just works - no manual parsing needed!
doc = DocumentEmbedding(
    id=uuid4(),
    embedding=[0.1] * 384,
    metadata={"model": "all-MiniLM-L6-v2"}
)
created = await repo.create(doc)
assert isinstance(created.embedding, list)  # ✅ Returns as list[float]

🧪 Testing Improvements

New Tests

• 12 unit tests - Vector field detection and type inspection
• 4 integration tests - Full CRUD operations with pgvector
• 227 total tests - All passing with zero regressions

Test Organization

• Reorganized tests into proper subdirectories:
  • tests/unit/ - Unit tests (mocked, no database)
  • tests/integration/ - Integration tests (real PostgreSQL)
  • tests/performance/ - Performance benchmarks
• Updated testcontainers to use pgvector/pgvector:pg17
• Added float32 precision handling (pgvector uses float32, Python uses float64)

📦 Dependencies

• Optional dependency: pgvector>=0.4.1 in vector dependency group
• Install with: pip install psycopg-toolkit[vector]
• Graceful degradation if not installed

📚 Documentation

• ✅ Updated CLAUDE.md with pgvector support details
• ✅ Updated README.md with pgvector feature
• ✅ Added comprehensive usage examples
• ✅ Documented vector field configuration options
• ✅ Added pgvector support components section

🛠️ Code Quality

• Fixed all ruff linting issues (SIM102, B905, SIM222)
• Applied consistent code formatting
• Added comprehensive async testing guide to README
• Improved test organization and structure

📊 Statistics

+681 lines added
-388 lines removed
18 files changed

Core implementation:
- src/psycopg_toolkit/repositories/base.py (+72 lines)
- src/psycopg_toolkit/utils/type_inspector.py (+102 lines)

New tests:
- tests/integration/test_pgvector.py (119 lines, 4 tests)
- tests/unit/test_type_inspector.py (+174 lines, 12 tests)

🔧 Technical Details

Type Detection (TypeInspector)

• detect_vector_fields() - Detects list[float] fields
• Supports Optional[list[float]] and list[float] | None
• Handles legacy typing.List[float] syntax
• Excludes vector fields from JSON processing

Adapter Registration (BaseRepository)

• Lazy registration on first database operation
• Uses async variant register_vector_async()
• Per-connection registration (not global)
• Logs helpful warnings if pgvector not installed

Float Precision

• pgvector uses float32 (4 bytes per dimension)
• Python uses float64 (8 bytes)
• Tests use approximate comparison with tolerance of 0.001

🔄 Migration Guide

No breaking changes! Existing code continues to work without modifications.

To use pgvector support:

1. Install pgvector: pip install pgvector>=0.4.1
2. Enable pgvector extension in PostgreSQL: CREATE EXTENSION vector;
3. Use list[float] type hints in your Pydantic models
4. Vector fields are automatically detected and handled!

🙏 Credits

Feature implemented following the AI instruction specification and integrated following the existing JSONB adapter pattern for consistency.

---

Full Changelog: https://github.com/descoped/psycopg-toolkit/compare/v0.2.2...v0.3.0

Release Notes - v0.2.2

🕒 Enhanced Date and Timestamp Support

This release significantly improves how psycopg-toolkit handles PostgreSQL date and timestamp fields, making it easier to work with temporal data across your applications.

✨ New Features

• Full TIMESTAMP Support: The date_fields parameter now properly handles PostgreSQL TIMESTAMP and TIMESTAMPTZ columns in addition to DATE columns
• Unified Date/DateTime Conversion: Both datetime.date and datetime.datetime objects are now seamlessly converted to ISO format strings for Pydantic models
• Comprehensive Field Coverage: You can now include all temporal fields (DATE, TIMESTAMP, TIMESTAMPTZ) in the date_fields parameter for consistent handling

🐛 Bug Fixes

• Fixed Duplicate Processing: Removed duplicate date field processing logic in _postprocess_data method that could cause performance issues
• Improved Type Handling: Better detection and conversion of both date and datetime objects from PostgreSQL

📚 Documentation Improvements

• Enhanced Examples: Updated array_and_date_fields.py example to demonstrate proper TIMESTAMP usage
• Clearer Guidelines: Added comprehensive documentation explaining when and how to use date_fields for all temporal column types
• Common Pitfalls Section: New documentation section highlighting typical mistakes and their solutions
• Expected Warnings Note: Added clarification about expected "Failed to deserialize JSON field" warnings when using auto_detect_json=True

🧪 Testing

• New Test Coverage: Added specific tests for datetime field conversion to ensure reliability
• Expanded Test Suite: Enhanced existing date field tests to cover more edge cases

💡 Usage Updates

Before v0.2.2:

date_fields={"birthdate"}  # Only DATE columns

After v0.2.2:

date_fields={"birthdate", "created_at", "updated_at", "last_login"}  # ALL temporal fields

⚠️ Important Notes

• Migration Recommended: If you're using timestamp fields, update your date_fields configuration to include all temporal columns for consistent behavior
• No Breaking Changes: Existing code will continue to work, but you may want to expand your date_fields configuration to take advantage of the improved timestamp support

🔧 Technical Details

• Enhanced _preprocess_data and _postprocess_data methods for better temporal field handling
• Improved logging to distinguish between date and datetime conversions
• Better error handling for edge cases in temporal data processing

Release Notes - v0.2.1

🚀 New Features

PostgreSQL Array Field Support

The BaseRepository now supports native PostgreSQL array columns with the new array_fields parameter. This allows you to preserve list fields as PostgreSQL arrays (e.g., TEXT[], INTEGER[]) instead of converting them to JSONB.

class OAuthClient(BaseModel):
    id: UUID
    redirect_uris: list[str]      # PostgreSQL TEXT[] array
    grant_types: list[str]        # PostgreSQL TEXT[] array  
    metadata: dict[str, Any]      # JSONB field

class ClientRepository(BaseRepository[OAuthClient, UUID]):
    def __init__(self, db_connection):
        super().__init__(
            db_connection=db_connection,
            table_name="oauth_clients",
            model_class=OAuthClient,
            primary_key="id",
            auto_detect_json=True,  # Detects metadata as JSON
            array_fields={"redirect_uris", "grant_types"}  # Keep as arrays
        )

Automatic Date Field Conversion

The new date_fields parameter enables automatic conversion between PostgreSQL DATE/TIMESTAMP types and ISO format strings in Pydantic models.

class User(BaseModel):
    id: UUID
    username: str
    birthdate: str | None  # Expects ISO date string
    created_at: str        # Expects ISO datetime string

class UserRepository(BaseRepository[User, UUID]):
    def __init__(self, db_connection):
        super().__init__(
            db_connection=db_connection,
            table_name="users",
            model_class=User,
            primary_key="id",
            date_fields={"birthdate", "created_at"}  # Auto-convert dates
        )

🔧 Improvements

• Array fields take precedence over JSON auto-detection
• Date fields handle both date objects and strings transparently
• Better handling of mixed field types in complex models

🐛 Bug Fixes

• Fixed psycopg adapter check to prevent array handling conflicts
• psycopg adapters now disabled by default to ensure correct array field processing

📚 Documentation

• Added comprehensive examples in examples/array_and_date_fields.py
• Updated docs/jsonb_support.md with array and date field configuration
• Added test coverage for array and date field functionality

💔 Breaking Changes

• psycopg JSON adapters are now disabled by default to prevent conflicts with array field handling. This ensures that PostgreSQL arrays are preserved correctly when using array_fields.

Release Notes - v0.2.0

🚀 New Features

Comprehensive PostgreSQL JSONB Support

This release introduces robust, production-ready support for PostgreSQL's JSONB data type with seamless Pydantic model integration.

Automatic JSON Field Detection

• Automatically detects dict, list, and complex types (including Optional and Union variants) as JSONB columns
• Uses the new TypeInspector utility for intelligent type analysis
• Supports model inheritance and nested type detection
• Can be disabled for manual control when needed

Dual Processing Modes

1. psycopg Native Adapters (Default) - High-performance JSON handling at the driver level
2. Custom JSON Processing - Fallback mechanism with enhanced type support for UUID, datetime, date, time, Decimal, set, and frozenset

Flexible Configuration

• New BaseRepository parameters: json_fields, auto_detect_json, and strict_json_processing
• Database-level control via enable_json_adapters setting
• Support for both strict and lenient JSON processing modes

Enhanced Error Handling

• New specialized exceptions: JSONProcessingError, JSONSerializationError, JSONDeserializationError
• Comprehensive error messages for debugging
• Protection against JSON injection and malformed data

Real-World Examples

• Production-ready JSONB usage patterns
• Complex nested structure handling
• PostgreSQL JSONB operators and querying techniques
• Transaction support with JSONB fields
• Performance optimization guidance

🔧 Development Tooling Modernization

Migration to Hatch and uv

• Replaced Poetry with Hatch for building and uv for dependency management
• Updated pyproject.toml to follow PEP 621 standard
• Modern dependency resolution with uv.lock

Enhanced CI/CD

• New GitHub Actions workflows:
  • build-test.yml - Comprehensive CI testing
  • release.yml - Automated PyPI releases
  • benchmark.yml - Performance regression testing
• Integrated Ruff for fast, comprehensive linting and formatting
• Automated dependency updates via Renovate

📊 Performance Testing

• Comprehensive JSONB performance benchmarks
• Memory usage analysis tools
• Comparison metrics for JSONB vs traditional columns
• Bulk operation performance testing

📚 Documentation

• Dedicated jsonb_support.md documentation
• Multiple real-world JSONB examples
• Migration guides for existing projects
• Best practices for production use

🏗️ Architecture Improvements

• Introduced factory pattern to resolve circular dependencies
• Better separation of concerns between Database and TransactionManager
• Improved type hints with TYPE_CHECKING imports

🔒 Security Enhancements

• SQL injection prevention using SQL identifiers for table names
• JSON injection protection in custom processing mode

🧪 Testing Infrastructure

• Reorganized test structure for better organization
• Added comprehensive test utilities for JSONB schema management
• Edge case testing for circular references, Unicode, and special numeric values

🚀 Quick Start with JSONB

from psycopg_toolkit import BaseRepository
from pydantic import BaseModel

class UserProfile(BaseModel):
    id: int
    name: str
    settings: dict  # Automatically stored as JSONB
    tags: list[str]  # Automatically stored as JSONB

class UserRepository(BaseRepository[UserProfile, int]):
    def __init__(self, db_connection):
        super().__init__(
            db_connection=db_connection,
            table_name="user_profiles",
            model_class=UserProfile,
            primary_key="id"
        )

# That's it! Your dict and list fields are now JSONB columns

🔄 Migration Notes

While this release includes some breaking changes, sensible defaults minimize disruption:

• Python 3.11+ is now required (previously 3.9+)
• JSON adapters are enabled by default for better performance
• Fields typed as dict or list are automatically treated as JSONB

For existing projects wanting to maintain previous behavior:

# Disable automatic JSON handling
settings = DatabaseSettings(..., enable_json_adapters=False)
repo = BaseRepository(..., auto_detect_json=False)

See the full migration guide in the documentation for detailed instructions.

Changes:

• Update README with comprehensive examples and usage guides
• Improve docstrings and code comments in core components
• Add advanced transaction and schema management examples
• Enhance error handling documentation
• Refactor and expand test coverage

Changes:

• Simplified Database and TransactionManager classes
• Added comprehensive error handling
• Improved connection pool management
• Updated project dependencies
• Enhanced test suite with test containers

Initial release of Psycopg Toolkit providing async-first PostgreSQL database management for Python.

Features:

Database:

• Async connection pooling via psycopg-pool
• Exponential backoff retry with configurable attempts
• Runtime pool optimization and health checks
• Automatic cleanup on shutdown

Transactions:

• Automatic rollback on errors
• Nested transaction support via savepoints
• Connection state management
• Integrated repository support

Repository Pattern:

• Generic type-safe CRUD operations
• Pydantic model integration
• Parameterized query building
• Bulk operation support

Requires Python 3.11+ and PostgreSQL 13+.