Let's try: pre-commit before you commit
Make sure the change is clean and ready to push to Git
Git is a must for developers. It is easy to store our source code, track, and review, but have you been ensure that you pushed clean code and aligned with your team’s standards to the repo?
Introduce “pre-commit”
pre-commit is a tool to automatically run scripts to lint, check, validate, and much more for our source code. Once setup it lets us know before committing unclean code. That’s why it’s called pre-commit.
The concept is to have a configuration with desired hooks. Each hook will trigger a script to check our code, and all hooks in the configuration must be run before we commit the code to repo. If any hook fails, we can see and fix it then commit and push again.
flowchart
subgraph local["git:local repo"]
pc["pre-commit"]
subgraph conf["config file"]
hr1["hook repo 1"] --> h1["hook 1"] --> chk1["check format"]
hr1 --> h2["hook 2"] --> chk2["check syntax"]
hr2["hook repo 2"] --> h3["hook 3"] --> chk3["check paths"]
hr2 --> h4["hook 1"] --> chk4["check files"]
end
end
subgraph remote["git:remote repo"]
repo
end
developer --"will commit"--> pc --> conf
conf --"all passed ✔︎"--> committed --"push"--> remote
Simple right?
Here is the webpage of pre-commit.
Setup
We can setup pre-commit with just 1-2-3 like this.
1. Install pre-commit
We can install pre-commit in many ways. I prefer installing it via homebrew, and there is pip way as well.
1
2
3
4
5
6
7
8
9
# install via homebrew
brew install pre-commit
# install via pip
pip install pre-commit
# verify pre-commit
pre-commit --version
pre-commit -V
2. Install pre-commit hooks
After installing pre-commit, we have to install its hooks into our Git local repo.
1
2
3
4
pre-commit install
# output should be:
# pre-commit installed at .git/hooks/pre-commit
3. Create a config file
Last, tell pre-commit what to do. It needs a configuration file named “.pre-commit-config.yaml”. We can create an empty file then add contents ourselves or create from sample like this.
1
2
3
4
5
# create an empty config file
touch .pre-commit-config.yaml
# create a config file from sample
pre-commit sample-config > .pre-commit-config.yaml
This is the sample configuration from pre-commit sample-config.
1
2
3
4
5
6
7
8
9
10
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
We can add yaml schema from schema store like this to validate the configurations (old blog: yaml).
1
2
3
4
5
# yaml-language-server: $schema=https://www.schemastore.org/pre-commit-config.json
repos:
- repo: ...
hooks:
- ...
Run it
Let’s say I left a trailing space in the Python file and I use the sample configuration from pre-commit sample-config.
When I try to commit it, the error should be shown like this.
pre-commitcan only execute on staged files. We have togit add <file>or it may be skipped.
However, we can run it manually without committing first. So the commands are here.
1
2
3
4
5
6
7
8
9
# stage files
git add .
# pre-commit on changed files
pre-commit run
# pre-commit on all files (recommended)
pre-commit run --all-files
pre-commit run -a
Syntax
Here is the basic syntax of .pre-commit-config.yaml file. The completed documentation can be found at the link at the top of this blog.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
repos:
- repo: <url | local | meta> # [required] url of the hook repo or local or meta (for debugging)
rev: <version> # version of the hook repo
hooks:
- id: <id> # [required] unique id for the hook
name: <name> # name of the hook
entry: <entry command> # entry command to run the hook
language: <language> # language of the hook, e.g. system, python, nodejs
files: <filepath regex> # regex to match files
types: # hook types, e.g. [python, text, yaml, json]
- "<type>"
pass_filenames: <bool> # whether to pass filenames to the hook
args: # arguments to pass to the hook
- "<argument 1>"
- "<argument 2>"
additional_dependencies: # additional dependencies to install
- "<dependency 1>"
- "<dependency 2>"
Integrated with Github Actions
We can add pre-commit step into Github Actions (old blog: Github Actions) like this.
1
2
3
4
5
6
7
8
9
10
11
12
13
name: <name>
on:
jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: 3.x
- uses: pre-commit/actions@v3.0.1
At the last line #13 is the step we execute pre-commit. There are many ways to run in Github Actions:
- use pre-commit/actions like above.
- use pre-commit.ci
- call
pre-commit run -adirectly in the step.
Choose the one you like.
Hook types
We usually set repo to be Github repo or local while meta is for debugging purposes.
Repo hooks
We just add the config based on the community hooks and it’s ready to work for us.
This is the simple template for repo hooks.
1
2
3
4
5
repos:
- repo: <hook repo url>
rev: <branch>
hooks:
- id: <hook id>
For example, I want to check for any vulnerabilities in my Python code. I can use these:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
repos:
- repo: https://github.com/mxab/pre-commit-trivy.git
rev: v0.15.0
hooks:
- id: trivyfs-docker
args:
- --skip-dirs
- ./tests
- . # last arg indicates the path/file to scan
- id: trivyconfig-docker
args:
- --skip-dirs
- ./tests
- . # last arg indicates the path/file to scan
The example above is Trivy, the security scanner tool and the hook repo is from community.
Local hooks
local is great and flexible when we want to run our own scripts. However, we need to have those tools in our environment such as we want to run unittest or pytest so we have to make sure that we have it installed in the local machine or the virtual environment.
1
2
3
4
5
6
7
8
9
10
repos:
- repo: local
hooks:
- id: <hook id>
name: <hook name>
entry: <entry command>
language: <language e.g. system, python, nodejs>
types: [<hook type>]
pass_filenames: <true | false>
additional_dependencies: [<dependency 1>, <dependency 2>]
For example, I want to test with unittest (old blog: unittest) and also pytest (old blog: pytest) on my Python code so I would add two local hooks with the relevant entry. Like this.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
repos:
- repo: local
hooks:
- id: unit-test
name: unit-test
entry: python3 -m unittest discover
language: python
types: [python]
pass_filenames: false
- id: pytest
name: pytest
entry: python3 -m pytest
language: python
types: [python]
additional_dependencies:
- "pytest"
There I can run check if everything is good.
1
2
git add .
pre-commit run -a
Or let it show everything that executes and logs.
1
2
git add .
pre-commit run -a --verbose
Command list
Here is the compilation of command I use frequently.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# install
homebrew install pre-commit
pip install pre-commit
# install hooks
pre-commit install
# create sample config file
pre-commit sample-config > .pre-commit-config.yaml
# run manually
git add .
pre-commit run
# run all files manually
pre-commit run -a
pre-commit run --all-files
# run all files manually with verbose
pre-commit run -a --verbose
# clean cache from the run
pre-commit clean
# update `rev` in `.pre-commit-config.yaml` to the latest
pre-commit autoupdate
Interesting hook sites
These links are pre-commit hook repos I think they’re useful for developing our checks before deployment in various situations.
- pre-commit-hooks: basic hooks for Python,
- pre-commit-trivy: hooks for scanning vulnerabilities, secrets, and misconfigurations.
- Collection of git hooks for Terraform to be used with pre-commit framework
- Github topic: pre-commit
- Github topic: precommit
Repo
I created a repo for pre-commit samples here. This repo also includes .pre-commit-config.yaml in some applications.
