/Python & Data Analytics/Phase 3

Phase 3Days 25-36

Files, Errors, OOP & Reproducible Workflows

Make analytical programs survive imperfect inputs and handoffs

Phase Goal

Design file workflows with explicit errors, configuration, domain objects, logs, and reproducible environments.

Progress

Notes

Open the written lectures for this course before checking off the phase topics.

Projects

Archive Condition Survey Pipeline

Day 25: Text, CSV & JSON Boundaries

Build the first-principles mental model for Text versus bytes and encoding; name the question it answers and the unit of analysisPredict and trace a tiny example of Text versus bytes and encoding before inspecting the resultApply Text versus bytes and encoding to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Text versus bytes and encoding; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for CSV dialects, quoting, and delimiters; name the question it answers and the unit of analysisPredict and trace a tiny example of CSV dialects, quoting, and delimiters before inspecting the resultApply CSV dialects, quoting, and delimiters to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind CSV dialects, quoting, and delimiters; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for JSON values, nesting, and schema expectations; name the question it answers and the unit of analysisPredict and trace a tiny example of JSON values, nesting, and schema expectations before inspecting the resultApply JSON values, nesting, and schema expectations to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind JSON values, nesting, and schema expectations; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Round trips and lossy conversion; name the question it answers and the unit of analysisPredict and trace a tiny example of Round trips and lossy conversion before inspecting the resultApply Round trips and lossy conversion to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Round trips and lossy conversion; diagnose the visible failure and perform the corrective check

Retrieve one recent mechanism and one older prerequisite before consulting notesComplete a guided example, then reconstruct its critical path from a blank artifactSolve a changed task that requires choosing among already-taught toolsRecord the decision changed, failure tested, help used, and adaptive 1/3/7/14/30-day review cue

Day 26: Paths & Safe File Operations

Build the first-principles mental model for pathlib joining and resolution; name the question it answers and the unit of analysisPredict and trace a tiny example of pathlib joining and resolution before inspecting the resultApply pathlib joining and resolution to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind pathlib joining and resolution; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Directory iteration and explicit glob selection; name the question it answers and the unit of analysisPredict and trace a tiny example of Directory iteration and explicit glob selection before inspecting the resultApply Directory iteration and explicit glob selection to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Directory iteration and explicit glob selection; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Temporary outputs and atomic replace; name the question it answers and the unit of analysisPredict and trace a tiny example of Temporary outputs and atomic replace before inspecting the resultApply Temporary outputs and atomic replace to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Temporary outputs and atomic replace; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Overwrite, race, and traversal risks; name the question it answers and the unit of analysisPredict and trace a tiny example of Overwrite, race, and traversal risks before inspecting the resultApply Overwrite, race, and traversal risks to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Overwrite, race, and traversal risks; diagnose the visible failure and perform the corrective check

Day 27: Exceptions as Contracts

Build the first-principles mental model for Traceback reading; name the question it answers and the unit of analysisPredict and trace a tiny example of Traceback reading before inspecting the resultApply Traceback reading to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Traceback reading; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for try/except/else/finally flow; name the question it answers and the unit of analysisPredict and trace a tiny example of try/except/else/finally flow before inspecting the resultApply try/except/else/finally flow to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind try/except/else/finally flow; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Specific exceptions and preserved context; name the question it answers and the unit of analysisPredict and trace a tiny example of Specific exceptions and preserved context before inspecting the resultApply Specific exceptions and preserved context to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Specific exceptions and preserved context; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for raise and failures not to swallow; name the question it answers and the unit of analysisPredict and trace a tiny example of raise and failures not to swallow before inspecting the resultApply raise and failures not to swallow to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind raise and failures not to swallow; diagnose the visible failure and perform the corrective check

Day 28: Logging & Diagnostic Evidence

Build the first-principles mental model for Log levels and audiences; name the question it answers and the unit of analysisPredict and trace a tiny example of Log levels and audiences before inspecting the resultApply Log levels and audiences to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Log levels and audiences; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for File, row, rule, and run context; name the question it answers and the unit of analysisPredict and trace a tiny example of File, row, rule, and run context before inspecting the resultApply File, row, rule, and run context to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind File, row, rule, and run context; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Handlers and formatting; name the question it answers and the unit of analysisPredict and trace a tiny example of Handlers and formatting before inspecting the resultApply Handlers and formatting to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Handlers and formatting; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Secrets, personal data, and noisy logs; name the question it answers and the unit of analysisPredict and trace a tiny example of Secrets, personal data, and noisy logs before inspecting the resultApply Secrets, personal data, and noisy logs to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Secrets, personal data, and noisy logs; diagnose the visible failure and perform the corrective check

Day 29: Configuration & Environments

Build the first-principles mental model for Configuration precedence; name the question it answers and the unit of analysisPredict and trace a tiny example of Configuration precedence before inspecting the resultApply Configuration precedence to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Configuration precedence; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Environment variables and secrets boundaries; name the question it answers and the unit of analysisPredict and trace a tiny example of Environment variables and secrets boundaries before inspecting the resultApply Environment variables and secrets boundaries to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Environment variables and secrets boundaries; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for venv, pip, requirements, and pinning; name the question it answers and the unit of analysisPredict and trace a tiny example of venv, pip, requirements, and pinning before inspecting the resultApply venv, pip, requirements, and pinning to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind venv, pip, requirements, and pinning; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Version drift and setup verification; name the question it answers and the unit of analysisPredict and trace a tiny example of Version drift and setup verification before inspecting the resultApply Version drift and setup verification to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Version drift and setup verification; diagnose the visible failure and perform the corrective check

Day 30: Classes for Domain Rules

Build the first-principles mental model for State, behavior, identity, and class cost; name the question it answers and the unit of analysisPredict and trace a tiny example of State, behavior, identity, and class cost before inspecting the resultApply State, behavior, identity, and class cost to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind State, behavior, identity, and class cost; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for __init__, attributes, methods, and invariants; name the question it answers and the unit of analysisPredict and trace a tiny example of __init__, attributes, methods, and invariants before inspecting the resultApply __init__, attributes, methods, and invariants to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind __init__, attributes, methods, and invariants; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Class versus dictionary versus function; name the question it answers and the unit of analysisPredict and trace a tiny example of Class versus dictionary versus function before inspecting the resultApply Class versus dictionary versus function to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Class versus dictionary versus function; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for God objects and hidden mutation; name the question it answers and the unit of analysisPredict and trace a tiny example of God objects and hidden mutation before inspecting the resultApply God objects and hidden mutation to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind God objects and hidden mutation; diagnose the visible failure and perform the corrective check

Day 31: Dataclasses & Validated Records

Build the first-principles mental model for Generated record methods; name the question it answers and the unit of analysisPredict and trace a tiny example of Generated record methods before inspecting the resultApply Generated record methods to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Generated record methods; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Hints, defaults, factories, and frozen state; name the question it answers and the unit of analysisPredict and trace a tiny example of Hints, defaults, factories, and frozen state before inspecting the resultApply Hints, defaults, factories, and frozen state to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Hints, defaults, factories, and frozen state; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Post-init validation and derived properties; name the question it answers and the unit of analysisPredict and trace a tiny example of Post-init validation and derived properties before inspecting the resultApply Post-init validation and derived properties to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Post-init validation and derived properties; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Mutable fields and scattered validation; name the question it answers and the unit of analysisPredict and trace a tiny example of Mutable fields and scattered validation before inspecting the resultApply Mutable fields and scattered validation to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Mutable fields and scattered validation; diagnose the visible failure and perform the corrective check

Day 32: Composition, Inheritance & Protocols

Build the first-principles mental model for Composition for has-a relationships; name the question it answers and the unit of analysisPredict and trace a tiny example of Composition for has-a relationships before inspecting the resultApply Composition for has-a relationships to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Composition for has-a relationships; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Inheritance and substitutability; name the question it answers and the unit of analysisPredict and trace a tiny example of Inheritance and substitutability before inspecting the resultApply Inheritance and substitutability to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Inheritance and substitutability; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Duck typing and behavioral contracts; name the question it answers and the unit of analysisPredict and trace a tiny example of Duck typing and behavioral contracts before inspecting the resultApply Duck typing and behavioral contracts to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Duck typing and behavioral contracts; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Fragile hierarchies and concrete-type checks; name the question it answers and the unit of analysisPredict and trace a tiny example of Fragile hierarchies and concrete-type checks before inspecting the resultApply Fragile hierarchies and concrete-type checks to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Fragile hierarchies and concrete-type checks; diagnose the visible failure and perform the corrective check

Day 33: Context Managers & Resource Safety

Build the first-principles mental model for Acquire-use-release lifecycle; name the question it answers and the unit of analysisPredict and trace a tiny example of Acquire-use-release lifecycle before inspecting the resultApply Acquire-use-release lifecycle to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Acquire-use-release lifecycle; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for with for managed resources; name the question it answers and the unit of analysisPredict and trace a tiny example of with for managed resources before inspecting the resultApply with for managed resources to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind with for managed resources; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Custom context managers; name the question it answers and the unit of analysisPredict and trace a tiny example of Custom context managers before inspecting the resultApply Custom context managers to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Custom context managers; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Leaks, partial writes, and masked exceptions; name the question it answers and the unit of analysisPredict and trace a tiny example of Leaks, partial writes, and masked exceptions before inspecting the resultApply Leaks, partial writes, and masked exceptions to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Leaks, partial writes, and masked exceptions; diagnose the visible failure and perform the corrective check

Day 34: Regex for Bounded Extraction

Build the first-principles mental model for Literals, classes, quantifiers, groups, and anchors; name the question it answers and the unit of analysisPredict and trace a tiny example of Literals, classes, quantifiers, groups, and anchors before inspecting the resultApply Literals, classes, quantifiers, groups, and anchors to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Literals, classes, quantifiers, groups, and anchors; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Raw strings and compiled patterns; name the question it answers and the unit of analysisPredict and trace a tiny example of Raw strings and compiled patterns before inspecting the resultApply Raw strings and compiled patterns to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Raw strings and compiled patterns; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Named extraction and fullmatch validation; name the question it answers and the unit of analysisPredict and trace a tiny example of Named extraction and fullmatch validation before inspecting the resultApply Named extraction and fullmatch validation to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Named extraction and fullmatch validation; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Complexity, readability, and string-method alternatives; name the question it answers and the unit of analysisPredict and trace a tiny example of Complexity, readability, and string-method alternatives before inspecting the resultApply Complexity, readability, and string-method alternatives to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Complexity, readability, and string-method alternatives; diagnose the visible failure and perform the corrective check

Day 35: Auditable Pipeline Stages

Build the first-principles mental model for Explicit stage dependencies; name the question it answers and the unit of analysisPredict and trace a tiny example of Explicit stage dependencies before inspecting the resultApply Explicit stage dependencies to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Explicit stage dependencies; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Manifests, checksums, and run IDs; name the question it answers and the unit of analysisPredict and trace a tiny example of Manifests, checksums, and run IDs before inspecting the resultApply Manifests, checksums, and run IDs to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Manifests, checksums, and run IDs; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Safe retries and preserved failed inputs; name the question it answers and the unit of analysisPredict and trace a tiny example of Safe retries and preserved failed inputs before inspecting the resultApply Safe retries and preserved failed inputs to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Safe retries and preserved failed inputs; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Partial success, duplicates, and recovery; name the question it answers and the unit of analysisPredict and trace a tiny example of Partial success, duplicates, and recovery before inspecting the resultApply Partial success, duplicates, and recovery to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Partial success, duplicates, and recovery; diagnose the visible failure and perform the corrective check

Day 36: Phase Project — Archive Survey Pipeline

Build the first-principles mental model for Condition schema and controlled vocabulary; name the question it answers and the unit of analysisPredict and trace a tiny example of Condition schema and controlled vocabulary before inspecting the resultApply Condition schema and controlled vocabulary to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Condition schema and controlled vocabulary; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Validated ingestion and rejection quarantine; name the question it answers and the unit of analysisPredict and trace a tiny example of Validated ingestion and rejection quarantine before inspecting the resultApply Validated ingestion and rejection quarantine to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Validated ingestion and rejection quarantine; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Tested prioritization rules; name the question it answers and the unit of analysisPredict and trace a tiny example of Tested prioritization rules before inspecting the resultApply Tested prioritization rules to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Tested prioritization rules; diagnose the visible failure and perform the corrective check

Build the first-principles mental model for Inspection brief, audit trail, and handoff drill; name the question it answers and the unit of analysisPredict and trace a tiny example of Inspection brief, audit trail, and handoff drill before inspecting the resultApply Inspection brief, audit trail, and handoff drill to a realistic analyst decision while stating assumptions, units, and boundariesBreak one assumption behind Inspection brief, audit trail, and handoff drill; diagnose the visible failure and perform the corrective check

Deliver Archive Condition Survey Pipeline for a regional archive conservator; support the decision: which collections should receive the next preservation inspection slots?Reconstruct the critical workflow without the example open and disclose help usedFind and repair a planted data, logic, or communication failure with evidenceRespond to a changed requirement without being told which technique to useState limitations, non-goals, uncertainty, and what evidence would change the recommendation

Capstone

Archive Condition Survey Pipeline

Build for a regional archive conservator. The result must support this decision: which collections should receive the next preservation inspection slots? Include reproducible inputs, validation evidence, one tested failure, a changed requirement, and an explicit non-goal.

Reproducible source-to-output workflow
Documented quality checks and assumptions
Decision-ready output for a named user
Independent reconstruction and transfer evidence

Phase Complete!

After this phase, you'll be able to:

Text, CSV & JSON Boundaries
Paths & Safe File Operations
Exceptions as Contracts
Logging & Diagnostic Evidence
Configuration & Environments
Classes for Domain Rules
Dataclasses & Validated Records
Composition, Inheritance & Protocols

Independently deliver Archive Condition Survey Pipeline, diagnose a planted failure, explain the decision logic, and transfer the workflow to a changed requirement.

Back to all phases

Phase Goal

Day 25: Text, CSV & JSON Boundaries

Text versus bytes and encoding

CSV dialects, quoting, and delimiters

JSON values, nesting, and schema expectations

Round trips and lossy conversion

Practice, retrieval, and analyst judgment

Day 26: Paths & Safe File Operations

pathlib joining and resolution

Directory iteration and explicit glob selection

Temporary outputs and atomic replace

Overwrite, race, and traversal risks

Practice, retrieval, and analyst judgment

Day 27: Exceptions as Contracts

Traceback reading

try/except/else/finally flow

Specific exceptions and preserved context

raise and failures not to swallow

Practice, retrieval, and analyst judgment

Day 28: Logging & Diagnostic Evidence

Log levels and audiences

File, row, rule, and run context

Handlers and formatting

Secrets, personal data, and noisy logs

Practice, retrieval, and analyst judgment

Day 29: Configuration & Environments

Configuration precedence

Environment variables and secrets boundaries

venv, pip, requirements, and pinning

Version drift and setup verification

Practice, retrieval, and analyst judgment

Day 30: Classes for Domain Rules

State, behavior, identity, and class cost

__init__, attributes, methods, and invariants

Class versus dictionary versus function

God objects and hidden mutation

Practice, retrieval, and analyst judgment

Day 31: Dataclasses & Validated Records

Generated record methods

Hints, defaults, factories, and frozen state

Post-init validation and derived properties

Mutable fields and scattered validation

Practice, retrieval, and analyst judgment

Day 32: Composition, Inheritance & Protocols

Composition for has-a relationships

Inheritance and substitutability

Duck typing and behavioral contracts

Fragile hierarchies and concrete-type checks

Practice, retrieval, and analyst judgment

Day 33: Context Managers & Resource Safety

Acquire-use-release lifecycle

with for managed resources

Custom context managers

Leaks, partial writes, and masked exceptions

Practice, retrieval, and analyst judgment

Day 34: Regex for Bounded Extraction

Literals, classes, quantifiers, groups, and anchors

Raw strings and compiled patterns

Named extraction and fullmatch validation

Complexity, readability, and string-method alternatives

Practice, retrieval, and analyst judgment

Day 35: Auditable Pipeline Stages

Explicit stage dependencies

Manifests, checksums, and run IDs

Safe retries and preserved failed inputs

Partial success, duplicates, and recovery

Practice, retrieval, and analyst judgment

Day 36: Phase Project — Archive Survey Pipeline

Condition schema and controlled vocabulary

Validated ingestion and rejection quarantine

Tested prioritization rules

Inspection brief, audit trail, and handoff drill

Phase gate: independent delivery and transfer

Archive Condition Survey Pipeline

Phase Complete!

init, attributes, methods, and invariants