validate_partial Alias Conversion Fix

Issue Summary

The validate_partial method in all generated model classes had a critical bug where it was not converting Python field names to Firestore aliases. This caused Firestore batch updates to create new fields with Python names instead of updating the existing aliased fields.

The Problem

Before (Buggy Code)

@classmethod
def validate_partial(cls, data: dict) -> dict[str, Any]:
    return {
        k: TypeAdapter(cls.model_fields[k].annotation).validate_python(v)
        for k, v in data.items()
        if k in cls.model_fields
    }

Bug: The method returned the same keys that were passed in.

Example of Incorrect Behavior

# Input with Python field names
updates = InviteModel.validate_partial({
    'processed': True,
    'reminder_sent': True
})

# Output: {'processed': True, 'reminder_sent': True}  ❌ WRONG
# Should be: {'INT_P': True, 'INT_R_S': True}  ✅ CORRECT

Impact on Firestore

When used with batch.update() (as in invite_db.py):

updates = InviteModel.validate_partial({'processed': True, 'reminder_sent': True})
batch.update(doc_ref, updates)

This would create NEW fields in Firestore:

  • ❌ Created: processed (new field with Python name)
  • ❌ Created: reminder_sent (new field with Python name)

Instead of updating the existing aliased fields:

  • ✅ Should update: INT_P (existing field)
  • ✅ Should update: INT_R_S (existing field)

The Solution

After (Fixed Code)

@classmethod
def validate_partial(cls, data: "Unpack[InviteDict]") -> dict[str, Any]:
    """
    Uses Unpack to give you autocomplete and static warnings
    if you pass an invalid key or type in your code.

    Returns a dict with Firestore field names (aliases) for use with batch.update().
    """
    result = {}
    for k, v in data.items():
        if k in cls.model_fields:
            field_info = cls.model_fields[k]
            validated_value = TypeAdapter(field_info.annotation).validate_python(v)
            # Use alias if defined, otherwise use field name
            output_key = field_info.alias if field_info.alias else k
            result[output_key] = validated_value
    return result

Key Changes

  1. Extracts field info: field_info = cls.model_fields[k]
  2. Uses alias if defined: output_key = field_info.alias if field_info.alias else k
  3. Returns aliased keys: result[output_key] = validated_value

Example of Correct Behavior

# Input with Python field names
updates = InviteModel.validate_partial({
    'processed': True,
    'reminder_sent': True,
    'user_created': False
})

# Output: {'INT_P': True, 'INT_R_S': True, 'INT_U_C': False}  ✅ CORRECT

Now when passed to batch.update(), it correctly updates the aliased fields in Firestore!

Files Modified

1. Template File (Generator)

2. Generated Model Files (All Regenerated)

All models in functions/flipdare/generated/model/ were regenerated:

  • archived_model.py
  • chat_comment_model.py
  • chat_model.py
  • content_model.py
  • dare_model.py
  • friend_model.py
  • group_member_model.py
  • group_model.py
  • invite_model.py
  • liked_model.py
  • notification_model.py
  • pledge_model.py
  • pledge_stats_model.py
  • user_model.py

3. Regeneration Command

python .\code_generation\codegen_models.py

Field Name Mappings (InviteModel Example)

Python Field NameFirestore AliasDescription
versionVERSIONVersion number
processedINT_PProcessing status
error_countINT_EError counter
user_createdINT_U_CUser created flag
email_sentINT_E_SEmail sent flag
friends_createdINT_F_CFriends created flag
indexedINT_IIndexing status
reminder_sentINT_R_SReminder sent flag

Usage Patterns

✅ Correct Usage (Now Fixed)

# Creating partial updates for Firestore
updates = InviteModel.validate_partial({
    'processed': True,
    'reminder_sent': True
})

# Updates now contain aliases: {'INT_P': True, 'INT_R_S': True}
batch.update(doc_ref, updates)

✅ Creating Models Still Works

# InviteDict uses Python field names
data: InviteDict = {
    'from_uid': 'user123',
    'from_email': 'user@example.com',
    'to_email': 'friend@example.com',
    'processed': True
}

# Model creation with Python names still works
invite = InviteModel(**data)

✅ Serialization with Aliases

invite = InviteModel(...)
json_data = invite.as_json()  # Uses by_alias=True, returns aliased keys

Testing

To verify the fix:

  1. Check output of validate_partial:

    updates = InviteModel.validate_partial({'processed': True})
    assert 'INT_P' in updates  # Should have alias
    assert 'processed' not in updates  # Should NOT have Python name
    
  2. Verify Firestore updates:

    • Check that existing INT_P, INT_R_S, etc. fields are updated
    • Check that no new processed, reminder_sent fields are created

Historical Context

The old commented-out implementation in app_base_model.py DID support aliases and preserved the original key (whether Python name or alias). However, the generated model implementation didn’t follow this pattern and only accepted Python names, returning them as-is without conversion.

All generated models with internal fields using aliases were affected:

  • InviteModel (INT_P, INT_R_S, INT_U_C, INT_E_S, INT_F_C, INT_I)
  • UserModel (various internal fields)
  • DareModel, GroupModel, FriendModel, etc.

Conclusion

This fix ensures that validate_partial correctly converts Python field names to Firestore aliases, preventing the creation of duplicate fields with Python names in the Firestore database. All generated models have been updated to include this fix.

Trigger and Job Processing Refactor

Summary

Refactored trigger data handling to use proper type annotations with PersistedWrapper types, clarifying that all trigger data comes from Firestore databases and includes document IDs.

Changes Made

1. trigger_data.py - Type Clarification

Before:

TModel = TypeVar("TModel", bound=BaseWrapper)

After:

# TModel is bound to BaseWrapper (the wrapper class, e.g. UserWrapper, DareWrapper)
# All instances returned are PersistedWrapper[TModel] since they come from Firestore with doc_id  
TModel = TypeVar("TModel", bound=BaseWrapper)

Key Improvements:

  • Added comprehensive docstrings to TriggerData and UpdateTriggerData classes
  • Clarified that TModel is the wrapper class (UserWrapper, DareWrapper, etc.)
  • Documented that all methods return PersistedWrapper[TModel] instances
  • Explained VERSION field tracking for user vs backend updates

2. app_triggers.py - Fixed Model Class Usage

Before (INCORRECT):

def tr_user_new(event: NewEventType) -> None:
    _new(PersistedWrapper[UserWrapper], AppJobType.TR_USER, event)

def _new(model_class: type[PersistedWrapper], job_type: AppJobType, event: NewEventType) -> None:
    get_job_admin().trigger_job(job_type, TriggerData(model_class, job_type.label, event))

After (CORRECT):

def tr_user_new(event: NewEventType) -> None:
    _new(UserWrapper, AppJobType.TR_USER, event)

TModel = TypeVar("TModel", bound=BaseWrapper)

def _new(model_class: type[TModel], job_type: AppJobType, event: NewEventType) -> None:
    get_job_admin().trigger_job(job_type, TriggerData(model_class, job_type.label, event))

Why This Matters:

  • PersistedWrapper[UserWrapper] is a generic alias, not an instantiable type
  • The correct type to pass is the wrapper class itself: UserWrapper
  • TriggerData._create_model() internally creates PersistedWrapper[TModel] from the wrapper class
  • Fixed for all trigger types: User, Invite, Friend, Dare, Group, GroupMember, Issue, PaymentIssue, Flag, Pledge

3. job_admin.py - Enhanced Documentation

Added comprehensive docstring:

def trigger_job(self, job_type: AppJobType, validator: TriggerData) -> None:
    """
    Creates a job in the database from trigger data.
    
    Flow:
    1. Firestore trigger fires (e.g., user document created/updated)
    2. TriggerData validates and creates PersistedWrapper[TModel] from the event
    3. This method stores the model as a dict in the jobs collection
    4. AppScheduler later processes these jobs, fetching fresh data from Firestore
    
    Args:
        job_type: The type of job (e.g., TR_USER, TR_DARE)
        validator: TriggerData containing PersistedWrapper[TModel] from the trigger event
    """

4. app_scheduler.py - Process Flow Documentation

Added comprehensive docstring:

def _process_trigger(self, job_type: AppJobType) -> None:
    """
    Process trigger jobs from the database.
    
    Flow:
    1. Fetches AppJobModel instances (PersistedWrapper[AppJobModel]) from jobs collection
    2. Each job contains obj_id pointing to the actual document
    3. Trigger handler (e.g., trigger_user) receives the job
    4. Handler fetches fresh PersistedWrapper[TModel] from Firestore using obj_id
    5. Handler calls processor with the fresh model data
    
    This ensures handlers always work with current data, not stale trigger snapshots.
    """

Type Flow Diagram

┌─────────────────────────────────────────────────────────────────┐
│ 1. Firestore Trigger Fires                                      │
│    Document: user/abc123 created/updated                        │
└────────────────────┬────────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────────────┐
│ 2. app_triggers.py                                               │
│    _new(UserWrapper, AppJobType.TR_USER, event)                 │
│                                                                  │
│    TModel = UserWrapper (the wrapper class)                     │
└────────────────────┬────────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────────────┐
│ 3. TriggerData(UserWrapper, "tr_user_new", event)               │
│    - Extracts doc_id from event.params["doc_id"]                │
│    - Calls UserWrapper.from_dict(data)                          │
│    - Returns PersistedWrapper[UserWrapper] with doc_id set      │
│                                                                  │
│    validator.model → PersistedWrapper[UserWrapper]              │
└────────────────────┬────────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────────────┐
│ 4. job_admin.trigger_job()                                       │
│    - Calls validator.model.as_dict()                            │
│    - Stores dict in jobs collection                             │
│                                                                  │
│    Job document: {obj_id: "abc123", model_data: {...}}         │
└────────────────────┬────────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────────────┐
│ 5. app_scheduler._process_trigger()                             │
│    - Fetches job: PersistedWrapper[AppJobModel]                │
│    - Gets obj_id = "abc123"                                     │
│    - Calls trigger handler                                       │
└────────────────────┬────────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────────────┐
│ 6. user_admin.trigger_user(job: AppJobWrapper)                  │
│    - Calls user_bridge.get(job.obj_id)                         │
│    - Gets fresh PersistedWrapper[UserWrapper] from Firestore   │
│    - Calls user_processor.process_user(user, is_update=...)    │
└────────────────────┬────────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────────────┐
│ 7. user_processor.process_user()                                │
│    - Receives PersistedWrapper[UserWrapper]                    │
│    - Processes with StepProcessor                               │
│    - Updates model state fields                                 │
└─────────────────────────────────────────────────────────────────┘

Key Concepts

PersistedWrapper[TModel]

  • Represents a wrapper instance that has been persisted to Firestore
  • Always has a doc_id set (never None)
  • Created from Firestore documents via Wrapper.from_dict(data)
  • Used throughout the codebase for database-sourced models

BaseWrapper (Wrapper Classes)

  • The base class for all wrappers: UserWrapper, DareWrapper, GroupWrapper, etc.
  • These are the types passed to TriggerData
  • NOT instances - just the class itself

Type Parameters

TModel = TypeVar("TModel", bound=BaseWrapper)

# TModel is the wrapper class (e.g., UserWrapper)
# Methods return PersistedWrapper[TModel] instances
class TriggerData(Generic[TModel]):
    def __init__(self, model_class: type[TModel], ...):
        # model_class is UserWrapper, DareWrapper, etc.
        ...
    
    @property
    def model(self) -> PersistedWrapper[TModel] | None:
        # Returns PersistedWrapper[UserWrapper] if model_class was UserWrapper
        return self._instance

Version Field Tracking

The VERSION field (integer) is used to determine the source of Firestore updates:

Backend/Server Updates (VERSION unchanged)

  • Triggered by Cloud Functions
  • Triggered by admin operations
  • Triggered by scheduled jobs
  • UpdateTriggerData.version_changed() returns False

User/Client Updates (VERSION increments)

  • Triggered by mobile app
  • Triggered by web app
  • Triggered by direct user actions
  • UpdateTriggerData.version_changed() returns True

This allows triggers to differentiate between user-initiated changes and backend processing, preventing infinite trigger loops.

Benefits of This Refactor

  1. Type Safety: Correct type annotations prevent runtime errors
  2. Clarity: Clear documentation explains the data flow
  3. Maintainability: Future developers understand why PersistedWrapper is used
  4. Consistency: All triggers follow the same pattern
  5. Documentation: Comprehensive docstrings explain the architecture

Testing Considerations

When testing triggers:

# CORRECT: Pass the wrapper class
validator = TriggerData(UserWrapper, "test_trigger", event)

# INCORRECT: Don't pass PersistedWrapper[UserWrapper]
validator = TriggerData(PersistedWrapper[UserWrapper], "test_trigger", event)  # ❌

# validator.model returns PersistedWrapper[UserWrapper]
user = validator.model
assert user is not None
assert user.doc_id == "abc123"  # doc_id is populated from event params

Files Modified

  1. functions/flipdare/job/trigger_data.py

    • Added type comments and comprehensive docstrings
    • Clarified TModel type parameter usage
  2. functions/flipdare/app_triggers.py

    • Fixed all trigger functions to pass wrapper class instead of PersistedWrapper
    • Updated helper functions (_new, _upd, _sub) with correct generic types
    • Changed 13 trigger functions (user, invite, friend, dare, group, etc.)
  3. functions/flipdare/job/job_admin.py

    • Enhanced trigger_job docstring with data flow explanation
  4. functions/flipdare/admin/core/app_scheduler.py

    • Enhanced _process_trigger docstring with processing flow

Validation

All files compile without errors:

  • ✅ trigger_data.py - No errors
  • ✅ app_triggers.py - No errors
  • ✅ job_admin.py - No errors
  • ✅ app_scheduler.py - No errors

Future Considerations

  • Consider adding integration tests that verify the full trigger → job → processor flow
  • Consider adding type assertions in critical paths to catch type mismatches early
  • Consider adding validation that PersistedWrapper instances always have doc_id set