Versioning and Lock Files¶

Version numbers mean something (sometimes). Lock files ensure reproducibility (when you use them). This chapter explains how versioning works, why lock files matter, and how to avoid the common pitfalls.

Semantic Versioning: The Promise¶

Most modern packages follow semantic versioning (semver): MAJOR.MINOR.PATCH

The promise: if you're using version 1.2.3, you can safely update to 1.2.4 (patch) or 1.3.0 (minor). Only 2.0.0 (major) requires attention.

This enables automated updates. If the contract holds, you can auto-update patches without fear.

Semantic Versioning: The Reality¶

The promise is aspirational. Reality is messier.

"Minor" Releases That Break Everything¶

# Worked in 1.2.0
result = library.process(data)

# 1.3.0 "minor" release
# Oops, the function signature changed
result = library.process(data, new_required_param)  # Breaks

Some maintainers increment MINOR when they should increment MAJOR. Some don't consider edge cases as "breaking." Some make genuine mistakes.

Pre-1.0 Means Anything Goes¶

Projects with version 0.x.y aren't bound by semver rules:

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

That 0.9.0 package might break everything in 0.10.0. The version number doesn't protect you.

Fast-Moving Ecosystems¶

In some ecosystems, breaking changes are constant:

• Frontend frameworks with major releases every year
• ML libraries with rapid API evolution
• Early-stage languages with shifting best practices

Semver provides guidance, but you still need to test.

Version Specifiers¶

When you declare dependencies, you specify which versions are acceptable.

The Specifiers¶

The npm Footguns¶

npm defaults are particularly surprising:

{
  "dependencies": {
    "lodash": "^4.17.0"
  }
}

The ^ means "4.17.0 or higher, but less than 5.0.0." This allows 4.18.0, 4.19.0, and so on—any of which could potentially break your code.

When you run npm install, you might get different versions on different days. That's... not great for reproducibility.

It stopped being only a reproducibility problem a while ago. A version range is a standing instruction to install code that does not exist yet, from whoever controls that package name at the moment you run it. When North Korean state actors social-engineered the Axios maintainer and published a malicious release, ^ did the rest — see Axios. When a maintainer account takeover turned npm's caching layer into a self-propagating worm, 2,234 poisoned versions went out in a single morning and every range pointing at them resolved on the next install — see keyv / ChainDrop.

^ means "automatically install whatever gets published next." Most days that's a bug fix.

Safe Defaults¶

For production code, prefer tighter constraints:

# pip - exact versions
pip install pandas==2.0.3

# npm - exact versions
npm install --save-exact lodash@4.17.21

Trade the convenience of automatic updates for the stability of known versions.

The Specifier Nobody Taught You: Age¶

Every constraint above answers which version. None of them answers how old, and that turns out to be the axis that matters most against a hostile publish.

Malicious versions get caught. The catching takes hours; the propagation takes minutes. A build that refuses to install anything published in the last few days sits out that window entirely — not because it evaluated the package, but because it waited long enough for someone else to.

# .npmrc — npm 11.10.0+, unit is days
min-release-age=3

Every major package manager now ships some form of this. See Time as a Control for the cross-manager table and the honest limits — most importantly that it delays every update, including the security fix you actually wanted.

Lock Files Are Not Optional¶

Lock files record the exact versions that were actually installed—not the ranges you specified, but the specific versions that were resolved.

What Lock Files Capture¶

A lock file includes:

• Exact versions — lodash@4.17.21, not ^4.17.0
• Transitive dependencies — Everything, not just direct deps
• Integrity hashes — Checksums to verify downloads
• Resolution context — Which registry, what platform

Why This Matters¶

Without a lock file:

# Monday
$ pip install pandas>=2.0
# Resolves to pandas==2.0.3

# Tuesday (new release happened)
$ pip install pandas>=2.0
# Resolves to pandas==2.1.0 (different behavior, possible bugs)

With a lock file:

# Monday
$ pip install -r requirements.txt
# Installs pandas==2.0.3, creates lock

# Tuesday
$ pip install -r requirements.txt  # (using lock)
# Still installs pandas==2.0.3 (reproducible)

Lock Files by Ecosystem¶

The table gives you the filename. What it can't give you is each ecosystem's temperament — why go.sum verifies against a public checksum database while requirements.txt verifies nothing, or why Cargo got this right on the first release and Python is still arguing about it. Those differences decide what your lock file actually buys you. The appendices take them one at a time: Python, Node and npm, Go, Rust.

Python's Lock File Problem¶

Python's ecosystem is fragmented. pip doesn't have a native lock file—requirements.txt is typically used, but it's weak:

# requirements.txt (not a real lock file)
pandas>=2.0  # Version range, not locked

# requirements.txt (better, but manual)
pandas==2.0.3  # Pinned, but transitive deps not captured

# requirements.txt (pip freeze output)
numpy==1.24.3
pandas==2.0.3
python-dateutil==2.8.2
pytz==2023.3
six==1.16.0
tzdata==2023.3
# Better! But no hashes, manual updates are error-prone

For proper lock files in Python, use tools:

pip-tools:

# requirements.in (what you want)
pandas>=2.0

# pip-compile generates requirements.txt (locked)
pip-compile requirements.in

# requirements.txt includes exact versions with hashes

Poetry:

# pyproject.toml (what you want)
[tool.poetry.dependencies]
pandas = "^2.0"

# poetry.lock (auto-generated, includes everything)

uv:

# pyproject.toml
[project]
dependencies = ["pandas>=2.0"]

# uv.lock (auto-generated)
uv lock

The Cardinal Rules¶

1. Lock Files Go in Version Control¶

# YES
git add package-lock.json
git commit -m "Update dependencies"

# NO - never ignore lock files
echo "package-lock.json" >> .gitignore

The lock file is part of your project. It defines what your project actually uses.

2. CI Installs from Lock File¶

Your CI should reproduce exactly what you tested locally:

# npm - use ci, not install
npm ci  # Installs exactly what's in lock file

# pip-tools
pip install -r requirements.txt  # The compiled file

# Poetry
poetry install  # Uses poetry.lock

npm install might update the lock file. npm ci uses it as-is.

That distinction is the whole defense. During the keyv worm, a CI pipeline running npm ci against a lock file committed before the compromise never resolved a poisoned version — there was nothing to resolve. The pipelines that got hit were the ones running npm install, which is to say the ones that asked the registry a fresh question on a bad morning.

The LiteLLM compromise is the cleanest controlled experiment on this the ecosystem has produced. Same package, same day, same attacker, two populations:

The malicious versions existed for forty minutes. Whether that mattered to you came down entirely to whether your install read versions from a lock or asked the registry fresh.

3. Update Lock Files Intentionally¶

Don't let CI regenerate lock files. Updates should be deliberate:

# Update a specific package
npm update lodash
git diff package-lock.json  # Review what changed
git add package-lock.json
git commit -m "Update lodash to 4.17.21 for security fix"

4. Review Lock File Changes in PRs¶

When lock files change, ask:

• What versions changed?
• Were new dependencies added?
• Were any removed?
• Was this intentional?

Lock file changes can hide surprises—new transitive dependencies, unexpected version jumps, or even supply chain attacks.

That last one isn't hypothetical, and it's worth knowing what it looks like in a diff. The keyv worm reached victims through packages most of them had never heard of, one to three hops down; the only visible artifact in a normal workflow was a lock file gaining entries nobody chose. A reviewer skimming for "did our direct dependencies change?" would have passed it. Read the transitive lines too — they're where this class of attack lives.

Vendoring: The Nuclear Option¶

Vendoring means copying dependencies directly into your project:

my-project/
├── src/
├── vendor/
│   ├── lodash/
│   ├── axios/
│   └── ... (all dependencies)
└── package.json

When to Vendor¶

• High-security environments — Air-gapped networks, compliance requirements
• Reproducibility-critical — Research that must be reproducible indefinitely
• Registry distrust — Can't rely on npm/PyPI being available
• Audit requirements — Need to review every line of dependency code

The Tradeoffs¶

Vendoring is powerful but expensive. For most projects, lock files provide sufficient reproducibility without the maintenance burden.

Modern Alternatives¶

Instead of vendoring, consider:

• Private registry mirrors — Cache approved packages internally
• Artifact storage — Store built artifacts alongside code
• Reproducible builds — Verify you can rebuild from source

Quick Reference¶

Python Lock File Options¶

npm/Node Lock File Commands¶

# Install from lock file (CI)
npm ci

# Update lock file
npm update

# Check for issues
npm audit

Lock File Checklist¶

• Lock file exists
• Lock file is in version control
• CI uses lock file (not resolving fresh)
• Lock file changes are reviewed
• Update process is documented