LCO DMA - Technical Architecture¶

Comprehensive technical overview of the LCO Data Management Application - a construction cost estimation and project controls platform.

System Architecture¶

High-Level Overview¶

flowchart TD
    subgraph Browser["Client Browser — React SPA (localhost:5173)"]
        SPA["MSAL Provider → Zustand Stores → React Router → Pages"]
    end

    subgraph Backend["FastAPI Backend (localhost:8000)"]
        Auth["Auth Layer<br/>(Entra ID + API Keys)"]
        API["API Layer<br/>(Routers)"]
        Service["Service Layer<br/>(Business Logic)"]
        Repo["Repository Layer<br/>BaseRepository → Domain Repositories (CRUD + Batch Ops)"]
        Auth --- API --- Service
        Auth --> Repo
        API --> Repo
        Service --> Repo
    end

    Browser -->|"HTTP/REST + Bearer JWT"| Backend

    Repo --> Cosmos["Cosmos DB<br/>(NoSQL)"]
    Repo --> Bus["Service Bus<br/>(Async Msgs)"]
    Repo --> Blob["Blob Storage<br/>(Files)"]

Production Architecture (Azure)¶

Component	Service	Purpose
Frontend	Azure Static Web Apps	React SPA hosting, global CDN
Backend API	Azure Web App	FastAPI with Gunicorn/Uvicorn
Import Processor	Azure Functions (Flex)	Async MTO import processing
Database	Azure Cosmos DB (Serverless)	Document storage
Message Queue	Azure Service Bus	Async task queuing
File Storage	Azure Blob Storage	Import file staging
Auth	Microsoft Entra ID	User authentication

Frontend Architecture¶

Technology Stack¶

Technology	Version	Purpose
React	18.3.1	UI framework
TypeScript	5.8.3	Type safety
Vite	6.3.6	Build tool + HMR
React Router	6.30.1	Client-side routing
Zustand	5.0.8	Global state management
TanStack Table	8.21.3	Data tables
TanStack Virtual	3.13.12	List virtualization
Axios	1.11.0	HTTP client
MSAL React	3.0.23	Azure AD authentication
Tailwind CSS	3.4.17	Utility CSS
shadcn/ui	(copied)	Component library
Radix UI	Multiple	Accessible primitives

Directory Structure¶

frontend/src/
├── auth/                 # MSAL configuration
│   ├── msalConfig.ts     # Auth settings
│   └── msalInstance.ts   # Singleton MSAL client
├── components/           # React components
│   ├── ui/               # shadcn/ui primitives (42 components)
│   ├── table/            # DataTable system
│   ├── auth/             # AuthRequired, AuthOverlay, LoginButton
│   ├── estimation-tabs/  # Lazy-loaded estimation tabs
│   ├── import/           # File import UI
│   └── sidebar/          # Navigation sidebar
├── hooks/                # Custom React hooks
│   ├── auth/             # useAuth
│   ├── foundation/       # useDebouncedValue, usePagination, etc.
│   ├── factories/        # createInfiniteResourceHook
│   ├── infinite/         # useInfiniteMTOs, useInfiniteEquipmentTags
│   ├── resources/        # useEquipmentTags, useCOAKpis
│   ├── crews/            # useServiceCrews, useCrewMembers
│   ├── mto/              # useMTOManagement, useMTOCalculations
│   ├── table/            # useTableViewState
│   └── ui/               # useDialogManager, useToast
├── pages/                # Route-level components (39 pages)
├── services/api/         # API client layer (31 modules)
│   ├── config.ts         # Axios + MSAL interceptor
│   └── [domain].ts       # Domain-specific API calls
├── stores/               # Zustand stores
│   ├── authStore.ts      # Authentication state
│   └── projectStore.ts   # Project context
├── types/                # TypeScript interfaces (24 files)
├── lib/                  # Utilities (cn for classnames)
├── config/               # Configuration
├── constants/            # Static data
└── utils/                # Domain utilities

State Management¶

The application uses a layered state approach:

Layer	Technology	Purpose
Global State	Zustand	Auth state, project context
Server State	Custom hooks	API data fetching/caching
UI State	useState	Component-local state
URL State	React Router	Navigation, deep linking

Zustand Stores:

authStore - Authentication status, user, roles, session management
Persists: overlayDismissed, wasAuthenticated (localStorage)
Helpers: canEdit() checks for Contributor role
projectStore - Active project/service context
Avoids prop drilling for nested components
Auto-derives query params for API calls

Hook Factory Pattern¶

The codebase uses a factory pattern for consistent data fetching:

// createInfiniteResourceHook creates hooks with:
// - Pagination (offset or continuation token)
// - Item deduplication by ID
// - Request abortion on filter change
// - Consistent API: items, loadMore, refetch, hasNextPage
const useInfiniteMTOs = createInfiniteResourceHook<MTO, MTOFilters>({
  fetchPage: mtoApi.getMTOsByProject,
  getId: (item) => item.id,
  errorMessage: 'Failed to load MTOs',
});

DataTable System¶

Location: frontend/src/components/table/

A sophisticated reusable table built on TanStack Table with: - Column visibility and drag-and-drop ordering (dnd-kit) - Query builder filters (AND/OR groups, multiple operators) - Client-side or server-side sorting - Row selection with bulk actions - Editable cells with validation - Manual or automatic pagination

Routing Structure¶

Route	Page	Description
`/`	HomePage	Dashboard
`/projects`	ProjectsPage	Project list
`/projects/:projectId`	ProjectDetailPage	Project details
`/projects/:projectId/services/:serviceId`	EstimationServicePage	Main estimation workspace
`/projects/:projectId/services/:serviceId/crews/:serviceCrewId/edit`	EditServiceCrewPage	Crew editing
`/projects/:projectId/analysis`	ProjectAnalysisPage	Analysis dashboard
`/reference-tables/*`	Various	Universal data tables
`/settings/api-keys`	ApiKeysPage	API key management

Authentication Flow¶

sequenceDiagram
    participant User
    participant useAuth
    participant MSAL
    participant Microsoft
    participant Zustand
    participant Axios as Axios Interceptor
    participant Backend

    User->>useAuth: 1. Clicks Sign In → useAuth.login()
    useAuth->>MSAL: MSAL popup
    MSAL->>Microsoft: 2. Authenticate
    Microsoft-->>MSAL: Returns tokens → MSAL caches
    useAuth->>Zustand: 3. Syncs → Sets user, roles, status
    Axios->>MSAL: 4. API requests → acquireTokenSilent()
    Axios->>Backend: Request with token
    Backend-->>Axios: 5. Backend validates JWT → Returns data

Session expiry: Toast notification prompts re-authentication.

Backend Architecture¶

Technology Stack¶

Technology	Version	Purpose
FastAPI	0.115+	Web framework
Python	3.12	Runtime
Pydantic	2.5+	Validation/serialization
Azure Cosmos SDK	4.5+	Database client
fastapi-azure-auth	6+	JWT validation
bcrypt	4+	API key hashing
Azure Service Bus SDK	7+	Message queuing
Azure Blob SDK	12+	File storage

Layered Architecture¶

flowchart TD
    API["<b>API Layer</b><br/>backend/app/api/v1/routers/ (37 routers)<br/>- HTTP handling, validation, dependency injection<br/>- Auth: require_auth, require_reader_role,<br/>require_contributor_role"]
    Schema["<b>Schema Layer</b><br/>backend/app/schemas/ (20+ files)<br/>- Pydantic models: Base, Create, Update, Response<br/>- Field aliases: snake_case ↔ camelCase<br/>- Computed fields via model_post_init"]
    Repo["<b>Repository Layer</b><br/>backend/app/repositories/ (23 files)<br/>- BaseRepository: CRUD, batch ops, pagination<br/>- Domain repositories: business logic<br/>- Batch processing with checkpointing"]
    Service["<b>Service Layer</b><br/>backend/app/services/<br/>- exporters/: Excel export<br/>- parsers/: Excel read/write<br/>- storage/: Azure Blob Storage<br/>- messaging/: Azure Service Bus<br/>- tasks/: Background task tracking"]
    DB["<b>Database Layer</b><br/>backend/app/db/cosmos.py<br/>- Singleton CosmosDB client<br/>- Container configuration (24 containers)"]

    API --> Schema --> Repo --> Service --> DB

Key Routers¶

Router	Prefix	Description
mto.py	/mto	MTO CRUD, search, bulk ops, import/export
projects.py	/projects	Project CRUD with details
crews.py	/crews	Crew templates, members, trades
service_crews.py	/service-crews	Service crew assignments
workflows.py	/workflows	Scheduling workflow CRUD
scurve.py	/scurve	S-Curve data endpoints
schedules.py	/schedules	Schedule management
schedule.py	/schedule	Activity endpoints (P6 data)
wbs.py	/wbs	WBS containers and items
estimation_comparison.py	/estimation-comparison	Cross-estimation analysis
api_keys.py	/api-keys	API key management
(26 more)	various	COA, equipment, material, packages, etc.

Repository Pattern¶

BaseRepository provides: - create(), get_by_id(), update(), upsert(), delete() - query(), query_with_pagination() - _execute_batch_create/upsert/delete() - 100-item batches with retry - Partition key abstraction via get_partition_key()

Domain Repositories extend with: - Business-specific queries - Relationship management - Computed field population

Batch Operations with Checkpointing¶

Large imports (1000+ items) use a resilient pattern:

flowchart TD
    Split["1. Split into 100-item batches"]
    Batch["2. For each batch:<br/>a. Execute with 3x retry for throttling (429)<br/>b. Save checkpoint to Cosmos DB<br/>c. 0.5s delay for rate limiting"]
    Fail["3. On failure:<br/>a. Service Bus retries up to 10x<br/>b. Resume from last checkpoint<br/>c. Final failure: rollback all items"]

    Split --> Batch
    Batch -->|on failure| Fail
    Fail -->|resume from checkpoint| Batch

Authentication System¶

Dual Authentication: 1. API Key (X-API-Key header) - for programmatic access - Format: lco_ + 32 random characters - Only bcrypt hash stored (prefix as partition key) - Expiration and revocation support

Azure Entra ID JWT - for interactive users
Validated via fastapi-azure-auth
Roles extracted from token claims

Role-Based Access: - require_reader_role - Reader OR Contributor - require_contributor_role - Contributor only (write operations)

Database Design¶

Azure Cosmos DB Configuration¶

Setting	Value
API	Core (SQL)
Consistency	Session
Throughput	Serverless (auto-scaling)
Containers	24

Container Configuration¶

Container	Partition Key	Purpose
projects	/clientId	Projects by client
clients	/clientId	Client organizations
services	/projectId	Services by project
material_takeoff	/projectId	MTO items by project
service_crews	/serviceCrewId	Service crew assignments
crews	/crewId	Crew templates
crew_members	/locationKey	Rate cards by location
crew_trades	/tradeCode	Trade definitions
equipment	/equipmentCode	Equipment catalog
equipment-tags	/projectId	Project equipment
material	/materialCode	Material catalog
project_material_costs	/projectId	Project material pricing
wbs	/wbsCode	Work breakdown structure
subcontractors	/projectId	Project subcontractors
tasks	/projectId	Background task tracking
api_keys	/keyPrefix	API key storage

Type Discriminator Pattern¶

All documents include a type field for filtering:

SELECT * FROM c WHERE c.type = 'project' AND c.clientId = @clientId

This enables multiple entity types in shared containers.

Schema Patterns¶

Base → Create → Update → Full Document:

class ProjectBase(BaseSchema):       # Shared fields
class ProjectCreate(ProjectBase):    # Required for creation
class ProjectUpdate(BaseSchema):     # All optional for updates
class Project(ProjectBase, BaseDocument):  # Full document
class ProjectDetailed(Project):      # With related data

Alias Convention: - Python: snake_case (e.g., project_id) - JSON/API: camelCase (e.g., projectId) - Cosmos DB: camelCase (stored as-is)

Business Domain¶

Core Entities¶

erDiagram
    Client ||--o{ Project : "1:N"
    Project ||--o{ Service : "1:N"
    Service ||--o{ ServiceCrew : "1:N"
    Service ||--o{ MaterialTakeoff : "1:N (MTO)"
    Project ||--o{ WBS : "1:N (Work Breakdown Structure)"
    Project ||--o{ Subcontractor : "1:N"
    Project ||--o{ EquipmentTag : "1:N"

Service Types¶

Type	Purpose
estimation	Cost estimation services
loan-monitoring	Financial oversight for lenders
project-controls	Budget/timeline tracking
claims	Construction claims management

MTO Disciplines (9 Types)¶

Discipline	Complexity	Key Fields
piping	Highest (140+ fields)	Pipe, fittings, flanges, valves, welding
electrical_equipment	High	Transformers, switchgear, motors
mechanical_equipment	High	Pumps, vessels, compressors
bulk_electrical	Medium	Cables, conduits, trays
instrumentation	Medium	Sensors, analyzers, control valves
civil	Medium	Excavation, earthwork
concrete	Medium	Foundations, structures
structural	Medium	Steel frames, beams
architectural	Low	Finishes, partitions

Cost Calculation Chain¶

totalManhours = quantity × manhourPerUnit × laborProdFactor
laborUnitCost = manhourPerUnit × laborProdFactor × hourlyCost
totalLaborCost = quantity × laborUnitCost

If subcontractorCostAllInRate > 0:
    totalEstimationUnitPrice = subcontractorCostAllInRate
Else:
    totalEstimationUnitPrice = laborUnitCost + materialUnitPrice + equipmentUnitPrice

totalCost = quantity × totalEstimationUnitPrice

Async Import Workflow¶

flowchart TD
    Upload["User Upload"] --> FastAPI
    FastAPI --> Blob["Blob Storage"]
    Blob --> Task["Task Record (queued)"]
    Task --> Bus["Service Bus Message"]
    Bus --> Func["Azure Function"]
    Func --> Parse["Parse Excel by Discipline"]
    Parse --> BatchProc["Batch Process (100/batch)"]
    BatchProc --> Save["Save with Checkpointing"]
    Save --> Status["Update Task Status"]
    Status --> Result["Task Complete/Failed"]
    Result --> Poll["Frontend Polls"]

API Reference¶

Endpoint Categories¶

Category	Base Path	Endpoints
Projects	/api/v1/projects	6
Services	/api/v1/projects/{pid}/services	4
MTO	/api/v1/mto	15+
Crews	/api/v1/crews	12
Service Crews	/api/v1/service-crews	5
Equipment	/api/v1/equipment	4
WBS	/api/v1/wbs	4
API Keys	/api/v1/api-keys	3

Common Response Format¶

{
  "data": [...],
  "total": 100,
  "skip": 0,
  "limit": 20,
  "hasMore": true
}

Interactive Documentation¶

Swagger UI: /api/docs
ReDoc: /api/redoc
OpenAPI Spec: /api/openapi.json

Development¶

Commands¶

Task	Command
Frontend dev	`npm run dev:frontend` (port 5173)
Backend dev	`npm run dev:backend` (port 8000)
Frontend lint	`cd frontend && npm run lint`
Frontend types	`cd frontend && npx tsc -b --noEmit`
Backend lint	`cd backend && ruff check . && ruff format --check .`
Backend types	`cd backend && mypy app/`
Backend tests	`cd backend && pytest`
Install all	`npm run install:all`

Environment Variables¶

Frontend (frontend/.env.local):

VITE_API_URL=http://localhost:8000/api/v1
VITE_AZURE_CLIENT_ID=<frontend-app-id>
VITE_AZURE_TENANT_ID=<tenant-id>
VITE_AZURE_API_SCOPE=api://<backend-app-id>/user_impersonation

Backend (backend/.env):

COSMOS_ENDPOINT=https://<account>.documents.azure.com:443/
COSMOS_KEY=<primary-key>
DATABASE_NAME=lco-construction
AZURE_AUTH_ENABLED=true
AZURE_CLIENT_ID=<backend-app-id>
AZURE_TENANT_ID=<tenant-id>
API_KEYS_ENABLED=true

CI/CD Pipelines¶

Workflow	Trigger	Actions
ci-frontend-lint	Push/PR with frontend changes	ESLint, TypeScript
ci-backend-quality	Push/PR with backend changes	Ruff, mypy, pytest
cd-uat / cd-prod	Push to main	Build → Deploy frontend + API
cd-sandbox-uat-from-main	Push to main	Build + deploy sandbox UAT API + worker

Unified Estimation-Item Model¶

Single-Document Semantics¶

The Estimation Master (EM) tab and the MTO discipline tabs are two views over one document per row in the EstimationItems Cosmos container. There is no sync logic between EM and MTO — both surfaces read from and write to the same document.

Container: EstimationItems
Partition key: /projectId
Canonical writer: app.repositories.estimation_item.EstimationItemRepository.upsert(item: dict) (inherited from BaseRepository)
Canonical reader: app.repositories.unified_estimation_item_adapter.UnifiedEstimationItemAdapter (delegates to EstimationItemRepository)

Write Surfaces¶

Two router endpoints write to the container:

Single-item PATCH — PATCH /estimation-items/{service_id}/items/{item_id}?projectId=… (backend/app/api/v1/routers/estimation_items.py, update_item). Used by the MTO discipline tab via useMTOCellEdit. Accepts an open-ended dict[str, Any] and merges into the existing document via _resolve_and_enrich_item → repo.upsert().
Batch save — POST /estimation-items/{service_id}/batch-save?projectId=… (batch_save). Used by the EM tab via useEstimationMasterCellEdit → estimationItemsApi.batchSave(). Iterates and upserts a list of items, applying PROTECTED_FIELDS filter per row.

Open-by-Default Field Policy¶

Both endpoints filter writes against a PROTECTED_FIELDS frozenset (identity, partition, system metadata, calculated totals). Every other field — including description1–description5 and lcoDescription — passes through unchanged. Adding a new editable column generally requires no router or repository change, only a schema update.

Concurrency¶

Writes are last-writer-wins. upsert() is a Cosmos point-write with no ETag check. Two concurrent edits to the same item will resolve to the later write; there is no optimistic-concurrency enforcement at the application layer. Single-cell editing in the UI minimizes the practical contention window, but multi-row batch saves can race against single PATCHes.

No Cross-Container Sync¶

There is no shadow MTO container, no sync job, and no eventing. The legacy MTORepository is deprecated; UnifiedEstimationItemAdapter is the authoritative read path for both EM and MTO surfaces.

Design Decisions¶

Why This Architecture?¶

Decision	Rationale
FastAPI	Async support, auto OpenAPI docs, Pydantic integration
Cosmos DB	Serverless pricing, flexible schema, Azure integration
Zustand	Lightweight, no boilerplate, good DevTools
Repository pattern	Testability, separation of concerns
Hook factories	Consistent data fetching, code reuse
shadcn/ui	Full ownership, Radix accessibility, Tailwind integration

Known Limitations¶

No React Query/SWR - manual data fetching state
No WebSocket - requires polling for live updates
No offline support - requires internet connection
Cross-partition queries - some queries span partitions

Quality Targets¶

Metric	Target
Component size	< 500 lines
useState per component	< 8
Props per component	< 6
Build/lint	Pass
Type coverage	100%

Quick Start: docs/QUICK_START.md
API Reference: docs/API_REFERENCE.md
Authentication: docs/auth/AUTH_IMPLEMENTATION.md
Database: docs/database/INDEX.md
Frontend: docs/frontend/ARCHITECTURE.md
Backend: docs/backend/ARCHITECTURE.md

LCO DMA - Technical Architecture¶

System Architecture¶

High-Level Overview¶

Production Architecture (Azure)¶

Frontend Architecture¶

Technology Stack¶

Directory Structure¶

State Management¶

Hook Factory Pattern¶

DataTable System¶

Routing Structure¶

Authentication Flow¶

Backend Architecture¶

Technology Stack¶

Layered Architecture¶

Key Routers¶

Repository Pattern¶

Batch Operations with Checkpointing¶

Authentication System¶

Database Design¶

Azure Cosmos DB Configuration¶

Container Configuration¶

Type Discriminator Pattern¶

Schema Patterns¶

Business Domain¶

Core Entities¶

Service Types¶

MTO Disciplines (9 Types)¶

Cost Calculation Chain¶

Async Import Workflow¶

API Reference¶

Endpoint Categories¶

Common Response Format¶

Interactive Documentation¶

Development¶

Commands¶

Environment Variables¶

CI/CD Pipelines¶

Unified Estimation-Item Model¶

Single-Document Semantics¶

Write Surfaces¶

Open-by-Default Field Policy¶

Concurrency¶

No Cross-Container Sync¶

Design Decisions¶

Why This Architecture?¶

Known Limitations¶

Quality Targets¶

Related Documentation¶