charlie880624/STDC
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add pypi deployment (#11)

Browse Source 
* add pypi deployment

* remove useless jobs

* fixed typo and cleanup

* delete build.tx

* add change log

Co-authored-by: Jiarui XU <xvjiarui0826@gmail.com>
This commit is contained in:
Kai Chen 2020-07-13 20:54:32 +08:00 committed by GitHub
parent b975d3b72a
commit 550e02b7cb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 91 additions and 126 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/CONTRIBUTING.md vendored
View File

@ -27,7 +27,7 @@ We use the following tools for linting and formatting:
- [yapf](https://github.com/google/yapf): formatter - [yapf](https://github.com/google/yapf): formatter
- [isort](https://github.com/timothycrosley/isort): sort imports - [isort](https://github.com/timothycrosley/isort): sort imports
Style configurations of yapf and isort can be found in [.style.yapf](../.style.yapf) and [.isort.cfg](../.isort.cfg). Style configurations of yapf and isort can be found in [setup.cfg](../setup.cfg) and [.isort.cfg](../.isort.cfg).
We use [pre-commit hook](https://pre-commit.com/) that checks and formats for `flake8`, `yapf`, `isort`, `trailing whitespaces`, We use [pre-commit hook](https://pre-commit.com/) that checks and formats for `flake8`, `yapf`, `isort`, `trailing whitespaces`,
fixes `end-of-files`, sorts `requirments.txt` automatically on every commit. fixes `end-of-files`, sorts `requirments.txt` automatically on every commit.

96
.github/workflows/build.yml vendored
View File

@ -1,12 +1,8 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
name: build name: build
on: [push, pull_request] on: [push, pull_request]
jobs: jobs:
lint: lint:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
@ -53,49 +49,49 @@ jobs:
python-version: 3.7 python-version: 3.7
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2 uses: actions/setup-python@v2
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install CUDA - name: Install CUDA
if: ${{matrix.torch == '1.5.0+cu101'}} if: ${{matrix.torch == '1.5.0+cu101'}}
run: | run: |
export INSTALLER=cuda-repo-${UBUNTU_VERSION}_${CUDA}_amd64.deb export INSTALLER=cuda-repo-${UBUNTU_VERSION}_${CUDA}_amd64.deb
wget http://developer.download.nvidia.com/compute/cuda/repos/${UBUNTU_VERSION}/x86_64/${INSTALLER} wget http://developer.download.nvidia.com/compute/cuda/repos/${UBUNTU_VERSION}/x86_64/${INSTALLER}
sudo dpkg -i ${INSTALLER} sudo dpkg -i ${INSTALLER}
wget https://developer.download.nvidia.com/compute/cuda/repos/${UBUNTU_VERSION}/x86_64/7fa2af80.pub wget https://developer.download.nvidia.com/compute/cuda/repos/${UBUNTU_VERSION}/x86_64/7fa2af80.pub
sudo apt-key add 7fa2af80.pub sudo apt-key add 7fa2af80.pub
sudo apt update -qq sudo apt update -qq
sudo apt install -y cuda-${CUDA_SHORT/./-} cuda-cufft-dev-${CUDA_SHORT/./-} sudo apt install -y cuda-${CUDA_SHORT/./-} cuda-cufft-dev-${CUDA_SHORT/./-}
sudo apt clean sudo apt clean
export CUDA_HOME=/usr/local/cuda-${CUDA_SHORT} export CUDA_HOME=/usr/local/cuda-${CUDA_SHORT}
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${CUDA_HOME}/include:${LD_LIBRARY_PATH} export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${CUDA_HOME}/include:${LD_LIBRARY_PATH}
export PATH=${CUDA_HOME}/bin:${PATH} export PATH=${CUDA_HOME}/bin:${PATH}
sudo apt-get install -y ninja-build sudo apt-get install -y ninja-build
- name: Install Pillow - name: Install Pillow
if: ${{matrix.torchvision == '0.4.2+cpu'}} if: ${{matrix.torchvision == '0.4.2+cpu'}}
run: pip install Pillow==6.2.2 run: pip install Pillow==6.2.2
- name: Install PyTorch - name: Install PyTorch
run: pip install torch==${{matrix.torch}} torchvision==${{matrix.torchvision}} -f https://download.pytorch.org/whl/torch_stable.html run: pip install torch==${{matrix.torch}} torchvision==${{matrix.torchvision}} -f https://download.pytorch.org/whl/torch_stable.html
- name: Install mmseg dependencies - name: Install mmseg dependencies
run: | run: |
pip install mmcv-full==latest+torch${{matrix.torch}} -f https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html pip install mmcv-full==latest+torch${{matrix.torch}} -f https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html
pip install -r requirements.txt pip install -r requirements.txt
- name: Build and install - name: Build and install
run: rm -rf .eggs && pip install -e . run: rm -rf .eggs && pip install -e .
- name: Run unittests and generate coverage report - name: Run unittests and generate coverage report
run: | run: |
coverage run --branch --source mmseg -m pytest tests/ coverage run --branch --source mmseg -m pytest tests/
coverage xml coverage xml
coverage report -m --omit="mmseg/utils/*","mmseg/apis/*" coverage report -m
# Only upload coverage report for python3.7 && pytorch1.5 # Only upload coverage report for python3.7 && pytorch1.5
- name: Upload coverage to Codecov - name: Upload coverage to Codecov
if: ${{matrix.torch == '1.5.0+cu101' && matrix.python-version == '3.7'}} if: ${{matrix.torch == '1.5.0+cu101' && matrix.python-version == '3.7'}}
uses: codecov/codecov-action@v1.0.10 uses: codecov/codecov-action@v1.0.10
with: with:
file: ./coverage.xml file: ./coverage.xml
flags: unittests flags: unittests
env_vars: OS,PYTHON env_vars: OS,PYTHON
name: codecov-umbrella name: codecov-umbrella
fail_ci_if_error: false fail_ci_if_error: false

20
.github/workflows/deploy.yml vendored Normal file
View File

@ -0,0 +1,20 @@
name: deploy
on: push
jobs:
build-n-publish:
runs-on: ubuntu-latest
if: startsWith(github.event.ref, 'refs/tags')
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.7
uses: actions/setup-python@v2
with:
python-version: 3.7
- name: Build MMSegmentation
run: python setup.py sdist bdist_wheel
- name: Publish distribution to PyPI
run: |
pip install twine
twine upload dist/* -u __token__ -p ${{ secrets.pypi_password }}

4
.style.yapf
View File

@ -1,4 +0,0 @@
[style]
BASED_ON_STYLE = pep8
BLANK_LINE_BEFORE_NESTED_CLASS_OR_DEF = true
SPLIT_BEFORE_EXPRESSION_AFTER_OPENING_PAREN = true

4
README.md
View File

@ -41,6 +41,10 @@ The master branch works with **PyTorch 1.3 to 1.5**.
This project is released under the [Apache 2.0 license](LICENSE). This project is released under the [Apache 2.0 license](LICENSE).
## Changelog
v0.5.0 was released in 10/7/2020.
## Benchmark and model zoo ## Benchmark and model zoo
Results and models are available in the [model zoo](docs/model_zoo.md). Results and models are available in the [model zoo](docs/model_zoo.md).

42
docs/install.md
View File

@ -18,41 +18,31 @@ conda activate open-mmlab
b. Install PyTorch and torchvision following the [official instructions](https://pytorch.org/). b. Install PyTorch and torchvision following the [official instructions](https://pytorch.org/).
Here we use PyTorch 1.5.0 and CUDA 10.1. Here we use PyTorch 1.5.0 and CUDA 10.1.
You may also switch to other version by specifying version number. You may also switch to other version by specifying the version number.
```shell ```shell
conda install pytorch=1.5.0 torchvision cudatoolkit=10.1 -c pytorch conda install pytorch=1.5.0 torchvision cudatoolkit=10.1 -c pytorch
``` ```
c. Clone the mmsegmentation repository. c. Install [MMCV](https://mmcv.readthedocs.io/en/latest/) following the [official instructions](https://mmcv.readthedocs.io/en/latest/#installation).
Either `mmcv` or `mmcv-full` is compatible with MMSegmentation, but for methods like CCNet and PSANet, CUDA ops in `mmcv-full` is required
The pre-build mmcv-full (with PyTorch 1.5 and CUDA 10.1) can be installed by running: (other available versions could be found [here](https://mmcv.readthedocs.io/en/latest/#install-with-pip))
```shell ```shell
git clone http://github.com/open-mmlab/mmsegmentation
cd mmsegmentation
```
d. Install [MMCV](https://mmcv.readthedocs.io/en/latest/).
Either *mmcv* or *mmcv-full* is compatible with MMSegmentation, but for methods like CCNet and PSANet, CUDA ops in *mmcv-full* is required
The pre-build *mmcv-full* could be installed by running: (available versions could be found [here](https://mmcv.readthedocs.io/en/latest/#install-with-pip))
```
pip install mmcv-full==latest+torch1.5.0+cu101 -f https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html pip install mmcv-full==latest+torch1.5.0+cu101 -f https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html
``` ```
Optionally, you could also install lite version by running: d. Install MMSegmentation.
```
pip install mmcv
```
or build full version from source:
```
pip install mmcv-full
```
e. Install build requirements and then install MMSegmentation.
```shell ```shell
pip install -r requirements/build.txt # or "pip install -r requirements.txt" for everything. pip install mmseg # install the latest release
pip install -e . # or "python setup.py develop" ```
or
```shell
pip install git+https://github.com/open-mmlab/mmsegmentation.git # install the master branch
``` ```
Note: Note:
@ -68,7 +58,6 @@ you can install it before installing MMCV.
4. Some dependencies are optional. Simply running `pip install -e .` will only install the minimum runtime requirements. 4. Some dependencies are optional. Simply running `pip install -e .` will only install the minimum runtime requirements.
To use optional dependencies like `cityscapessripts` either install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -e .[optional]`). Valid keys for the extras field are: `all`, `tests`, `build`, and `optional`. To use optional dependencies like `cityscapessripts` either install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -e .[optional]`). Valid keys for the extras field are: `all`, `tests`, `build`, and `optional`.
### A from-scratch setup script ### A from-scratch setup script
Here is a full script for setting up mmsegmentation with conda and link the dataset path (supposing that your dataset path is $DATA_ROOT). Here is a full script for setting up mmsegmentation with conda and link the dataset path (supposing that your dataset path is $DATA_ROOT).
@ -78,11 +67,8 @@ conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab conda activate open-mmlab
conda install pytorch=1.5.0 torchvision cudatoolkit=10.1 -c pytorch conda install pytorch=1.5.0 torchvision cudatoolkit=10.1 -c pytorch
git clone http://github.com/open-mmlab/mmsegmentation
cd mmsegmentation
pip install mmcv-full==latest+torch1.5.0+cu101 -f https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html pip install mmcv-full==latest+torch1.5.0+cu101 -f https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html
pip install -r requirements/build.txt pip install git+https://github.com/open-mmlab/mmsegmentation.git
pip install -e .
mkdir data mkdir data
ln -s $DATA_ROOT data ln -s $DATA_ROOT data

8
docs/model_zoo.md
View File

@ -35,7 +35,7 @@ Please refer to [PSPNet](https://github.com/open-mmlab/mmsegmentation/blob/maste
### DeepLabV3 ### DeepLabV3
Please refer to [DeepLabV3](https://github.com/open-mmlab/mmsegmentatio/blob/master/configs/deeplabv3) for details. Please refer to [DeepLabV3](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/deeplabv3) for details.
### PSANet ### PSANet
@ -43,7 +43,7 @@ Please refer to [PSANet](https://github.com/open-mmlab/mmsegmentation/blob/maste
### DeepLabV3+ ### DeepLabV3+
Please refer to [DeepLabV3+](https://github.com/open-mmlab/mmsegmentatio/blob/master/configs/deeplabv3plus) for details. Please refer to [DeepLabV3+](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/deeplabv3plus) for details.
### UPerNet ### UPerNet
@ -51,11 +51,11 @@ Please refer to [UPerNet](https://github.com/open-mmlab/mmsegmentation/blob/mast
### NonLocal Net ### NonLocal Net
Please refer to [NonLocal Net](https://github.com/open-mmlab/mmsegmentatio/blob/master/configs/nlnet) for details. Please refer to [NonLocal Net](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/nlnet) for details.
### EncNet ### EncNet
Please refer to [NonLocal Net](https://github.com/open-mmlab/mmsegmentatio/blob/master/configs/encnet) for details. Please refer to [NonLocal Net](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/encnet) for details.
### CCNet ### CCNet

4
mmseg/__init__.py
View File

@ -1,3 +1,3 @@
from .version import __version__, short_version from .version import __version__, short_version, version_info
__all__ = ['__version__', 'short_version'] __all__ = ['__version__', 'short_version', 'version_info']

1
requirements.txt
View File

@ -1,4 +1,3 @@
-r requirements/build.txt
-r requirements/optional.txt -r requirements/optional.txt
-r requirements/runtime.txt -r requirements/runtime.txt
-r requirements/tests.txt -r requirements/tests.txt

2
requirements/build.txt
View File

@ -1,2 +0,0 @@
# These must be installed before building mmsegmentation
numpy

1
requirements/tests.txt
View File

@ -1,4 +1,3 @@
asynctest
codecov codecov
flake8 flake8
interrogate interrogate

33
setup.py
View File

@ -1,13 +1,8 @@
#!/usr/bin/env python
import os import os
import subprocess import subprocess
import time import time
from setuptools import find_packages, setup from setuptools import find_packages, setup
import torch
from mmcv.utils.parrots_wrapper import (BuildExtension, CppExtension,
CUDAExtension)
def readme(): def readme():
with open('README.md', encoding='utf-8') as f: with open('README.md', encoding='utf-8') as f:
@ -85,32 +80,6 @@ def get_version():
return locals()['__version__'] return locals()['__version__']
def make_cuda_ext(name, module, sources, sources_cuda=[]):
define_macros = []
extra_compile_args = {'cxx': []}
if torch.cuda.is_available() or os.getenv('FORCE_CUDA', '0') == '1':
define_macros += [('WITH_CUDA', None)]
extension = CUDAExtension
extra_compile_args['nvcc'] = [
'-D__CUDA_NO_HALF_OPERATORS__',
'-D__CUDA_NO_HALF_CONVERSIONS__',
'-D__CUDA_NO_HALF2_OPERATORS__',
]
sources += sources_cuda
else:
print('Compiling {} without CUDA'.format(name))
extension = CppExtension
# raise EnvironmentError('CUDA is required to compile MMSegmentation!')
return extension(
name='{}.{}'.format(module, name),
sources=[os.path.join(*module.split('.'), p) for p in sources],
define_macros=define_macros,
extra_compile_args=extra_compile_args)
def parse_requirements(fname='requirements.txt', with_version=True): def parse_requirements(fname='requirements.txt', with_version=True):
"""Parse the package dependencies listed in a requirements file but strips """Parse the package dependencies listed in a requirements file but strips
specific versioning information. specific versioning information.
@ -199,7 +168,6 @@ if __name__ == '__main__':
keywords='computer vision, semantic segmentation', keywords='computer vision, semantic segmentation',
url='http://github.com/open-mmlab/mmsegmentation', url='http://github.com/open-mmlab/mmsegmentation',
packages=find_packages(exclude=('configs', 'tools', 'demo')), packages=find_packages(exclude=('configs', 'tools', 'demo')),
package_data={'mmseg.ops': ['*/*.so']},
classifiers=[ classifiers=[
'Development Status :: 4 - Beta', 'Development Status :: 4 - Beta',
'License :: OSI Approved :: Apache Software License', 'License :: OSI Approved :: Apache Software License',
@ -219,5 +187,4 @@ if __name__ == '__main__':
'optional': parse_requirements('requirements/optional.txt'), 'optional': parse_requirements('requirements/optional.txt'),
}, },
ext_modules=[], ext_modules=[],
cmdclass={'build_ext': BuildExtension},
zip_safe=False) zip_safe=False)