1 Windows上安装Detectron2

按以下流程安装

git clone https://github.com/facebookresearch/detectron2.git
conda create --name detectron2 python=3.9
conda activate detectron2
conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=11.8 -c pytorch -c nvidia
cd detectron2
python setup.py build --force develop

Detectron2编译需要C++编译器，所以在Windows上需要安装Visual Studio，这里建议安装Visual Studio 2019或者2022。

2 安装的坑

2.1 unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2019 (inclusive) are supported!

解决方法：找到以下cuda头文件，

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include\crt\host_config.h

找到该文件的151行，找到以下代码块

#if defined(_WIN32)

#if _MSC_VER < 1910 || _MSC_VER >= 1940

#error -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.

#elif _MSC_VER >= 1910 && _MSC_VER < 1910

#pragma message("support for this version of Microsoft Visual Studio has been deprecated! Only the versions between 2017 and 2022 (inclusive) are supported!")

#endif /* (_MSC_VER < 1910 || _MSC_VER >= 1940) || (_MSC_VER >= 1910 && _MSC_VER < 1910) */

#endif /* _WIN32 */
#endif  /* !__NV_NO_HOST_COMPILER_CHECK */

修改_MSC_VER < 1910 || _MSC_VER >= 1940 这一行，修改为

#if _MSC_VER < 1910 || _MSC_VER >= 2000  /*before:1940*/

修改完保存该文件，重新使用

python setup.py build --force develop

进行编译。

2.2 nms_rotated_cuda.cu

报错信息

nms_rotated_cuda.cu
D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(14): error: name must be a namespace name

D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(68): error: identifier "single_box_iou_rotated" is undefined
          detected during instantiation of "void nms_rotated_cuda_kernel(int, double, const T *, unsigned long long *) [with T=double]"
(105): here

D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(68): error: type name is not allowed
          detected during instantiation of "void nms_rotated_cuda_kernel(int, double, const T *, unsigned long long *) [with T=double]"
(105): here

D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(68): warning: expression has no effect
          detected during instantiation of "void nms_rotated_cuda_kernel(int, double, const T *, unsigned long long *) [with T=double]"
(105): here

D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(58): warning: variable "cur_box" was set but never used
          detected during instantiation of "void nms_rotated_cuda_kernel(int, double, const T *, unsigned long long *) [with T=double]"
(105): here

D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(68): error: identifier "single_box_iou_rotated" is undefined
          detected during instantiation of "void nms_rotated_cuda_kernel(int, double, const T *, unsigned long long *) [with T=float]"
(105): here

D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(68): error: type name is not allowed
          detected during instantiation of "void nms_rotated_cuda_kernel(int, double, const T *, unsigned long long *) [with T=float]"
(105): here

D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(68): warning: expression has no effect
          detected during instantiation of "void nms_rotated_cuda_kernel(int, double, const T *, unsigned long long *) [with T=float]"
(105): here

D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu(58): warning: variable "cur_box" was set but never used
          detected during instantiation of "void nms_rotated_cuda_kernel(int, double, const T *, unsigned long long *) [with T=float]"
(105): here

5 errors detected in the compilation of "D:/mega_work/python/detectron2-main/detectron2-main/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu".
error: command 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\bin\\nvcc.exe' failed with exit code 1

解决方法，在detectron2项目仓库找到以下文件：detectron2/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu，修改上面的包含文件内容，原文件内容为：

// Copyright (c) Facebook, Inc. and its affiliates.
#include <ATen/ATen.h>
#include <ATen/cuda/CUDAContext.h>
#include <c10/cuda/CUDAGuard.h>
#include <ATen/cuda/CUDAApplyUtils.cuh>
#ifdef WITH_CUDA
#include "../box_iou_rotated/box_iou_rotated_utils.h"
#endif
// TODO avoid this when pytorch supports "same directory" hipification
#ifdef WITH_HIP
#include "box_iou_rotated/box_iou_rotated_utils.h"
#endif

using namespace detectron2;

namespace {
int const threadsPerBlock = sizeof(unsigned long long) * 8;
}

修改为：

// Copyright (c) Facebook, Inc. and its affiliates.
#include <ATen/ATen.h>
#include <ATen/cuda/CUDAContext.h>
#include <c10/cuda/CUDAGuard.h>
#include <ATen/cuda/CUDAApplyUtils.cuh>
/*#ifdef WITH_CUDA
#include "../box_iou_rotated/box_iou_rotated_utils.h"
#endif
// TODO avoid this when pytorch supports "same directory" hipification
#ifdef WITH_HIP
#include "box_iou_rotated/box_iou_rotated_utils.h"
#endif
*/
#include "box_iou_rotated/box_iou_rotated_utils.h"

using namespace detectron2;

namespace {
int const threadsPerBlock = sizeof(unsigned long long) * 8;
}

修改完保存该文件，重新使用

python setup.py build --force develop

进行编译。