Building Software

Engineering Fundamentals for the Agent Era

Contents Section 8, Security

Sensitive Data and Secrets

Mistakes to catch in review

  1. API keys or passwords hardcoded in source code or committed to the repository.

  2. Personal data sent to a third-party analytics service or model provider that has no need for it.

  3. Tokens or personal data placed in URLs, where they end up in server logs, browser history and referrer headers.

  4. Test fixtures and screenshots built from real customer data.

Knowing which data is sensitive and which is not, where it travels, and how to keep the sensitive set small.

Topics

Data Classification
Sorting data into sensitivity classes so handling rules follow the class instead of case-by-case judgment.
Personal and Regulated Data
Personal information and health, financial and children's data, and the laws that govern them.
Secrets Management
Keeping credentials out of code and logs, in a secrets store, injected at run time and rotated on a schedule.
Data Minimization and Retention
Collecting less, keeping it for a shorter time, and deleting it on schedule.
Where Sensitive Data Leaks
Logs, URLs, error messages, analytics, crash reports, backups and the prompts sent to models.

You understand it when you can

  • Classify every field in a schema as public, internal, confidential or restricted, and justify each choice.
  • Trace one sensitive field through a system and list every place it lands, including logs, backups, caches and third parties.
  • Explain how a secret should be stored, injected into a running service and rotated.

Drill

An agent integrated error tracking by sending each exception's full context, including request headers and form fields, to a third-party service, and put password-reset tokens in the query string of the reset link. Find every place sensitive data now lands outside your control.

Start here

Watch

Read

Data Privacy: A Runbook for Engineers

Nishant Bhajaria, 2022.

An engineer's guide to classifying data by privacy risk, cataloging where it lives, controlling what is shared with third parties, and building deletion into the architecture.

Practical Data Privacy: Enhancing Privacy and Security in Data

Katharine Jarmul, 2023.

Covers data governance, pseudonymization and anonymization, and building privacy into data pipelines, so you can minimize what sensitive data flows downstream.

Primary sources