ci: Configuration for circle-ci¶
Description¶
Motivation¶
Running all checks via make locally is great but having this done externally and automatically is better. For external pull-requests but not only.
And it allows to run jobs that are long to run locally and where a failure is rare, but that are run “just in case”, like building the project and running tests on the build.
Rationale¶
I chose to use circle-ci because the ease of how we can define dependencies.
All jobs simply call make commands (after a bit of preparation)
Info¶
- Hash
fcd2779c4a1fc545cb105ab3b71a43fc70af8fdc
- Date
2019-06-03 13:55:00 +0200
- Parents
docs(git): Explain that git-flow will be used [a0569dde] — 2019-06-03 13:54:59 +0200
- Children
ci: Make the CI do all checks for every commits [20753915] — 2019-06-03 13:55:00 +0200
- Branches
- Tags
(No tags)
Changes¶
.circleci/config.yml¶
- Type
Added
- Stats
+268 -0
@@ -0,0 +1,268 @@
+version: 2
+
+references:
+
+ # docker container for python only jobs
+ python_only_config: &python_only_config
+ working_directory: ~/isshub
+ docker:
+ - image: circleci/python:3.7.3
+
+ # build steps to save/restore the directory used by pip to cache downloaded packages
+ save_pip_cache: &save_pip_cache
+ save_cache:
+ key: v1-pip-cache-{{ .Branch }}-{{ .Revision }}
+ paths:
+ - ~/.cache/pip
+ restore_pip_cache: &restore_pip_cache
+ restore_cache:
+ keys:
+ - v1-pip-cache-{{ .Branch }}-{{ .Revision }}
+ - v1-pip-cache-{{ .Branch }}
+ - v1-pip-cache
+
+ # shortcut to attach the workspace before each job
+ attach_workspace: &attach_workspace
+ attach_workspace:
+ at: "~/"
+
+
+# jobs definition: they are used in ``workflows``
+jobs:
+
+ # get the code from git and save the repo to pass it to the next job
+ checkout_code:
+ <<: *python_only_config
+ steps:
+ - checkout
+ - persist_to_workspace:
+ root: "~/"
+ paths:
+ - isshub
+
+ # install the project code and dependencies and save the venv and pip cache
+ install_code:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - *restore_pip_cache
+ - run:
+ name: Install code
+ command: |
+ python -m venv ~/venv
+ source ~/venv/bin/activate
+ pip install --upgrade pip
+ make dev
+ - *save_pip_cache
+ - persist_to_workspace:
+ root: "~/"
+ paths:
+ - venv
+ - isshub
+
+ check_commit:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Check commit message
+ command: |
+ source ~/venv/bin/activate
+ make check-commit
+
+ # 4 next jobs are linters: mypy, black, flake8 and pylint
+ # they all use the workspace
+ linter_mypy:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Run "mypy" linter
+ command: |
+ source ~/venv/bin/activate
+ make mypy
+
+ linter_isort:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Run "isort" linter
+ command: |
+ source ~/venv/bin/activate
+ make check-isort
+
+ linter_black:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Run "black" linter
+ command: |
+ source ~/venv/bin/activate
+ make check-black
+
+ linter_flake8:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Run "flake8" linter
+ command: |
+ source ~/venv/bin/activate
+ make flake8
+
+ linter_pylint:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Run "pylint" linter
+ command: |
+ source ~/venv/bin/activate
+ make pylint
+
+ # run the tests
+ run_tests:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Run tests
+ command: |
+ source ~/venv/bin/activate
+ make test
+
+ # build the documentation
+ build_doc:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Auth with github
+ # github changes their keys sometimes and we run into this issue:
+ # https://circleci.com/gh/Isshub-io/isshub/118
+ # so this should fix that here
+ command: |
+ mkdir -p ~/.ssh/
+ echo -e "Host github.com\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n" > ~/.ssh/config
+ chmod 600 ~/.ssh/config
+ ssh-keyscan -Ht rsa github.com >> ~/.ssh/known_hosts
+ - run:
+ name: Build documentation
+ command: |
+ source ~/venv/bin/activate
+ make doc
+
+ # will build the python package, using the tag as the base version, suffixed with info from git describe if not a tag
+ build_python_package:
+ <<: *python_only_config
+ steps:
+ - *attach_workspace
+ - run:
+ name: Build package
+ command: |
+ sed -i -e "s/^version = .*$/version = $(git describe --tags)/" setup.cfg
+ source ~/venv/bin/activate
+ make dist
+ - store_artifacts:
+ path: dist/
+ - save_cache:
+ key: v1-isshub-dist-{{ .Revision }}
+ paths:
+ - ~/isshub/dist
+
+ # will test that the python wheel package is installable and works
+ test_python_package_whl:
+ <<: *python_only_config
+ steps:
+ - checkout
+ - *restore_pip_cache
+ - restore_cache:
+ keys:
+ - v1-isshub-dist-{{ .Revision }}
+ - run:
+ name: Check package
+ command: |
+ ls -la
+ mv dist _dist
+ rm -r isshub
+ make full-clean
+ python -m venv ~/venv
+ source ~/venv/bin/activate
+ pip install --upgrade pip
+ pip install $(ls -tr _dist/*.whl | tail -n 1)[tests]
+ make tests
+
+ # will test that the python tar.gz package is installable and works
+ test_python_package_targz:
+ <<: *python_only_config
+ steps:
+ - checkout
+ - *restore_pip_cache
+ - restore_cache:
+ keys:
+ - v1-isshub-dist-{{ .Revision }}
+ - run:
+ name: Check package
+ command: |
+ ls -la
+ python -m venv ~/venv
+ source ~/venv/bin/activate
+ pip install --upgrade pip
+ mv dist _dist
+ rm -r isshub
+ make full-clean
+ python -m venv ~/venv
+ source ~/venv/bin/activate
+ pip install --upgrade pip
+ pip install $(ls -tr _dist/*.tar.gz | tail -n 1)[tests]
+ make tests
+
+workflows:
+ version: 2
+
+ isshub:
+ jobs:
+ - checkout_code
+ - install_code:
+ requires:
+ - checkout_code
+ - check_commit:
+ requires:
+ - install_code
+ - linter_mypy:
+ requires:
+ - install_code
+ - linter_isort:
+ requires:
+ - install_code
+ - linter_black:
+ requires:
+ - install_code
+ - linter_flake8:
+ requires:
+ - install_code
+ - linter_pylint:
+ requires:
+ - install_code
+ - run_tests:
+ requires:
+ - install_code
+ - build_doc:
+ requires:
+ - install_code
+ - build_python_package:
+ requires:
+ - linter_mypy
+ - linter_isort
+ - linter_black
+ - linter_flake8
+ - linter_pylint
+ - run_tests
+ - test_python_package_whl:
+ requires:
+ - build_python_package
+ - test_python_package_targz:
+ requires:
+ - build_python_package
README.rst¶
- Type
Modified
- Stats
+10 -0
@@ -296,6 +296,16 @@ The commits are so important to me that they will be available in the "Internals
Also, I'm a fan of `git-flow <https://nvie.com/posts/a-successful-git-branching-model/>`_ and it (the `avh edition <https://github.com/petervanderdoes/gitflow-avh>`_) will be used in this project, but adding the username in the branch name (easier to filter on if many users), so for example my own development branches will be prefixed by `features/twidi/`. (If branches from other people in pull requests do not follow this pattern, it will be done on my side)
+Continuous integration
+======================
+
+All of these tools will be run via continuous integration, on `CircleCi <https://circleci.com/>`_.
+
+One status will be posted on Github for each job for every pull requests.
+
+To access the list of jobs: https://circleci.com/gh/Isshub-io/isshub
+
+
''''''
Coding
''''''