在Debian下搭建OpenWiFi开发环境

OpenWiFi是个不错的东西(除了许可选择比较激进外^@^),还是要十分感谢原作者开源这样一个项目让大家学习的。为了学习方便,我搞个最便宜的开发板AntSDR AD9363版本先测一下。本篇记录一下在Debian中的OpenWiFi开发环境搭建过程(作为CentOS的忠实用户,要全面转到Debian下了)。

本次用的服务器其实是我之前搭建TensorFlow的一个环境,使用的是debian-10.11.0版本。安装Debian10可以参考另一篇文章的配置Debian部分: Debian 10安装TensorFlow | OopsDump 。我看了下其他人的文章,有用Windows和Linux结合的,我感觉有点麻烦,就选择All in Linux了。反正Vivado和Matlab都有Linux版本,完全不影响使用。

安装必要的程序:

# apt-get install vim build-essential python3.7-dev python3-pip git
# apt-get install libncurses5-dev device-tree-compiler libssl-dev

在用户Home下创建一个工作目录:(以后工程都放在该目录下)

$ mkdir ~/workspace
$ cd ~/workspace

提示,命令中以#开始表示以root用户执行,以$开始表示以普通用户执行。

安装Vivado 2018.3

先安装个依赖:(不然安装会卡在结束位置)

# apt-get install libtinfo5

下载Xilinx Vivado 2018.3。然后解压文件:

$ tar zxvf Xilinx_Vivado_SDK_2018.3_1207_2324.tar.gz

然后,使用图形界面运行Vivado安装文件:

$ su -
# ./xsetup

如果图形界面无法显示,是加入用户的x11认证到root:

# xauth add $(xauth -f ~<username>/.Xauthority list|tail -1)
# xauth list
# export DISPLAY=:<n为上面显示的最后一列的:后值>
# ./xsetup

在安装时,注意以下几个选项:
1. 在安装Edition选择时,HL Design和HL System都可以,因为涉及到WebPack License问题,所以我选了HL Design。
2. 支持芯片选能用到的就行,可以减少大小。
3. 安装位置,建议改为/opt/Xilinx。

安装之后,会弹出许可导入界面,我们使用XC7Z020用WebPack不需要许可,可以直接关闭。

然后试一下打开Vivado:

$ . /opt/Xilinx/SDK/2018.3/settings64.sh 
$ vivado

****** Vivado v2018.3 (64-bit)
  **** SW Build 2405991 on Thu Dec  6 23:36:41 MST 2018
  **** IP Build 2404404 on Fri Dec  7 01:43:56 MST 2018
    ** Copyright 1986-2018 Xilinx, Inc. All Rights Reserved.

start_gui

打开DocNAV会遇到libpng12找不到的问题,可以手动下载并安装这个:
https://blog.lazy-evaluation.net/data/libpng12-0_1.2.50-2+deb8u3dzu1_amd64.deb

# dpkg -i libpng12-0_1.2.50-2+deb8u3dzu1_amd64.deb
# apt-get install libcanberra-gtk-module

将默认文本编辑器改为VSCode(关于VSCode的安装,参考上一篇Debian中安装环境的文章)。进入Vivado后,通过菜单Tools->Settings打开配置。在配置中,他选择“Text Editor”页面,在Current Editor中选择最后的“Custom Editor”,配置参数为:

code -g [file name]:[line number]

安装Jtag驱动程序:

# cd /opt/Xilinx/Vivado/2018.3/data/xicom/cable_drivers/lin64/install_script/install_drivers
# ./install_drivers

安装其他环境

提示:本章中的内容为非必要操作,可以选择有用的安装。

普通用户权限可使用USB串口

串口设备默认需要Root权限打开,为了方便使用USB转串口,更改默认权限:

# vi /etc/udev/rules.d/70-ttyusb.rules
(加入下面信息)
KERNEL=="ttyUSB[0-9]*", MODE="0666"

安装免费版本的ModelSim 10.6d

免费版ModelSim for Intel FPGA Starter Edition可以10000行的Verilog,验证一些小的模块够用了。由于ModelSim 10.6d为32位程序,需要安装一下必要的环境:(为了安装速度快,也可以下载32位的iso系统,并加入源列表)

# dpkg --add-architecture i386
# apt-get update
# apt-get install lib32z1 libx11-6:i386 libxext6:i386 libxft2:i386

在Intel FPGA的网站下载18.1版本的ModelSim(对应10.6d版本): Intel FPGAs 下载中心

Intel ModelSim 18.1(10.6d)下载

使用Root用户安装时,选择Starter版本,建议安装到/opt目录。(注意:Root用户安装前要先修改DISPLAY环境变量,参考之前Vivado安装)

安装完成后,进入modelsim安装目录下的 /opt/intelFPGA_pro/18.1/modelsim_ase/bin 运行./vsim 出现  Error: Cannot find ”./../linux_rh60/vsim”。需要修改 /opt/intelFPGA_pro/18.1/modelsim_ase/vco文件:

 case $utype in
   2.4.[7-9]*)      vco="linux" ;;
   2.4.[1-9][0-9]*) vco="linux" ;;
   2.[5-9]*)        vco="linux" ;;
   2.[1-9][0-9]*)   vco="linux" ;;
   3.[0-9]*)        vco="linux" ;;
   *)               vco="linux_rh60" ;;
中linux_rh60改为linux。

在一些应用调用时会出现错误,需要再创建一个软链:

# cd /opt/intelFPGA_pro/18.1/modelsim_ase
# ln -s linuxaloem/ linuxpe

联合仿真库编译:

以Root权限打开Vivado,菜单中选择Tools -> Compile Simulation Libraries…。按下面图片填入后,点Compile。

使用ModelSim 10.6d创建Vivado的仿真库

编译完成后,确认errors为0即可。

检查Vivado的ModelSim用库编译结果

在菜单中选择Tools -> Settings…,再选择3rd Party Simulators页,填入下面信息:

配置Vivado的仿真设置

对于工程,需要配置使用的仿真工具:

安装IVerilog

这里再安装一下Verilog开源工具环境:

# apt-get install iverilog gtkwave

获取代码

由于众所周知的原因,使用Git clone GitHUB的代码比较慢,可以搭个梯子,并配置socks5的代理:(改为你实际的代理端口,也可以手动更改~/.gitconfig文件)

$ git config --global http.proxy 'scoks5://127.0.0.1:1080'
$ git config --global https.proxy 'scoks5://127.0.0.1:1080'

创建目录~/workspace/openwifi,将openwifi的代码都放在此目录中。

$ mkdir ~/workspace/openwifi

使用Git clone出最新SW工程:git clone https://github.com/open-sdr/openwifi.git。
Clone完后, 进入目录再更新Linux的submodule:git submodule update –init –recursive。

再使用Git clone出最新HW工程:git clone https://github.com/open-sdr/openwifi-hw.git。
Clone完后,进入目录再更新ADI HDL和OpenOFDM的submodule:git submodule update –init –recursive。

下载ADI提供的镜像(后续OpenWiFi版本可能有变化,以OpenWiFi最新的Readme.md为准):https://swdownloads.analog.com/cse/2019_R1-2020_02_04.img.xz

$ cd ~/workspace/openwifi
$ git clone --recursive https://github.com/open-sdr/openwifi.git
$ git clone --recursive https://github.com/open-sdr/openwifi-hw.git
$ wget https://swdownloads.analog.com/cse/2019_R1-2020_02_04.img.xz
$ xz -d 2019_R1-2020_02_04.img.xz
$ cd openwifi/adi-linux
$ git pull origin 2019_R1:2019_R1
$ cd ../adi-linux-64
$ git pull origin 2019_R1:2019_R1

编译系统

先介绍一下Vivado安装后,和编译程序相关的环境。这里的环境没有使用PetaLinux,而是使用了原始的方式。

环境中,需要包含进必要的环境变量,可以使用:

$ source /opt/Xilinx/Vivado/2018.3/settings64.sh

使用OpenWiFi官方的方法更新系统

先准备一个不小于8G的SDCard,这里AntSDR用的是TF卡。

直接写入SDCard

先要卸载自动挂载的SDCard设备,这里使用的是(/dev/mmcblk0p1),可能与你的不同,替换为你系统中的设备(如果SDCard存在多个分区,需要分别)。然后,使用dd命令将img文件写入SDCard中。最后,使用sync命令进行同步后,方可拔出设备。

$ su -
# umount /dev/mmcblk0p1
# dd if=2019_R1-2020_02_04.img of=/dev/mmcblk0 bs=4194304
# sync

挂载img到系统

也可以将img文件直接挂载到系统中。先使用fdisk查看一下img的分区信息。

# fdisk -lu 2019_R1-2020_02_04.img
Disk 2019_R1-2020_02_04.img: 7.4 GiB, 7948206080 bytes, 15523840 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00096174

Device                  Boot   Start      End  Sectors  Size Id Type
2019_R1-2020_02_04.img1         8192  2105343  2097152    1G  b W95 FAT32
2019_R1-2020_02_04.img2      2105344 15501311 13395968  6.4G 83 Linux
2019_R1-2020_02_04.img3         4096     6143     2048    1M a2 unknown

Partition table entries are not in disk order.

为了同时编辑不同分区文件,这里使用losetup将img分割为不同的loop文件。

# losetup -P -f --show 2019_R1-2020_02_04.img
# losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE                                                DIO LOG-SEC
/dev/loop1         0      0         1  0 /opt/debian_dvd/firmware-10.11.0-amd64-DVD-1.iso           0     512
/dev/loop4         0      0         0  0 /home/oopsdump/workspace/openwifi/2019_R1-2020_02_04.img   0     512
/dev/loop2         0      0         1  0 /opt/debian_dvd/debian-10.11.0-amd64-DVD-3.iso             0     512
/dev/loop0         0      0         1  0 /opt/debian_dvd/debian-10.11.0-amd64-DVD-2.iso             0     512
/dev/loop3         0      0         1  0 /opt/debian_dvd/debian-10.11.0-amd64-DVD-1.iso             0     512
# ls /dev/loop4*
/dev/loop4  /dev/loop4p1  /dev/loop4p2	/dev/loop4p3

将需要的分区进行挂载:

# mkdir -p /mnt/adimg/BOOT /mnt/adimg/rootfs
# mount /dev/loop4p1 /mnt/adimg/BOOT
# mount /dev/loop4p2 /mnt/adimg/rootfs

显示目录:

# ls /mnt/adimg/BOOT
README.txt				       zynq-zc702-adv7511-ad9361-fmcomms2-3
socfpga_arria10_socdk_ad9172_fmc	       zynq-zc702-adv7511-ad9361-fmcomms5
socfpga_arria10_socdk_adrv9009		       zynq-zc702-adv7511-ad9364-fmcomms4
socfpga_arria10_socdk_adrv9371		       zynq-zc706-adv7511
socfpga_arria10_socdk_daq2		       zynq-zc706-adv7511-ad6676-fmc
socfpga_arria10_socdk_fmclidar1		       zynq-zc706-adv7511-ad9172-fmc-ebz
socfpga_cyclone5_sockit_arradio		       zynq-zc706-adv7511-ad9265-fmc-125ebz
uEnv.txt				       zynq-zc706-adv7511-ad9361-fmcomms2-3
VERSION					       zynq-zc706-adv7511-ad9361-fmcomms5
zynq-adrv9361-z7035-bob			       zynq-zc706-adv7511-ad9361-fmcomms5-ext-lo-adf5355
zynq-adrv9361-z7035-bob-cmos		       zynq-zc706-adv7511-ad9364-fmcomms4
zynq-adrv9361-z7035-fmc			       zynq-zc706-adv7511-ad9434-fmc-500ebz
zynq-adrv9361-z7035-packrf		       zynq-zc706-adv7511-ad9625-fmcadc2
zynq-adrv9364-z7020-bob			       zynq-zc706-adv7511-ad9625-fmcadc3
zynq-adrv9364-z7020-bob-cmos		       zynq-zc706-adv7511-ad9739a-fmc
zynq-adrv9364-z7020-packrf		       zynq-zc706-adv7511-adrv9008-1
zynq-common				       zynq-zc706-adv7511-adrv9008-2
zynqmp-adrv9009-zu11eg-revb-adrv2crr-fmc-revb  zynq-zc706-adv7511-adrv9009
zynqmp-common				       zynq-zc706-adv7511-adrv9371
zynqmp-zcu102-rev10-ad9172-fmc-ebz-mode4       zynq-zc706-adv7511-adrv9375
zynqmp-zcu102-rev10-ad9361-fmcomms2-3	       zynq-zc706-adv7511-fmcdaq2
zynqmp-zcu102-rev10-ad9361-fmcomms5	       zynq-zc706-adv7511-fmcdaq3-revC
zynqmp-zcu102-rev10-ad9364-fmcomms4	       zynq-zc706-adv7511-fmcjesdadc1
zynqmp-zcu102-rev10-adrv9008-1		       zynq-zc706-adv7511-fmclidar1
zynqmp-zcu102-rev10-adrv9008-2		       zynq-zc706-adv7511-fmcomms11
zynqmp-zcu102-rev10-adrv9009		       zynq-zed-adv7511
zynqmp-zcu102-rev10-adrv9371		       zynq-zed-adv7511-ad9361-fmcomms2-3
zynqmp-zcu102-rev10-adrv9375		       zynq-zed-adv7511-ad9364-fmcomms4
zynqmp-zcu102-rev10-fmcdaq2		       zynq-zed-adv7511-ad9467-fmc-250ebz
zynqmp-zcu102-rev10-fmcdaq3		       zynq-zed-adv7511-cn0363
zynqmp-zcu102-rev10-fmclidar1		       zynq-zed-imageon
zynq-zc702-adv7511

# ls /mnt/adimg/rootfs
bin  boot  dev	etc  home  lib	lost+found  media  mnt	opt  proc  root  run  sbin  share  srv	sys  tmp  usr  var

编译OpenWiFi文件

对于挂载SDCard的方式,改为实际目录。

使用OpenWiFi的Readme中提供的方法编译:

$ export SDCARD_DIR=/mnt/adimg/
$ export XILINX_DIR=/opt/Xilinx/
$ export OPENWIFI_HW_DIR=/home/oopsdump/workspace/openwifi/openwifi-hw/
$ export BOARD_NAME=antsdr
$ cd openwifi/user_space
$ ./update_sdcard.sh $OPENWIFI_HW_DIR $XILINX_DIR $BOARD_NAME $SDCARD_DIR

更新的文件主要在:rootfs/lib/modules/,rootfs/root/openwifi/,BOOT/BOOT.BIN,BOOT/devicetree.dtb,BOOT/openwifi/

生成FPGA的bit

配置Vivado工程

【TODO】

获取Xilinx的IP评估使用许可

获取Xilinx Viterbi Decoder的Evaluation License:

【TODO】

本文参考了:

https://blog.lazy-evaluation.net/posts/linux/vivado-2018-3-buster.html
https://blog.csdn.net/long_fly/article/details/78643258

Debian 10安装TensorFlow

由于CentOS8即将停止维护,所以将系统更换为了Debian,并将TensorFlow在Debian 10上重新安装一下。这里使用的是 Debian 10.11.0版本,机器显卡使用的是1080Ti。如果在CentOS8中安装,可以参考我之前写的步骤: CentOS8安装TensorFlow | OopsDump

先安装Debian 10.11.0,就不按步骤具体说明了。主要注意几点:
1. 安装时,最好选择英文。
2. 分区时,要把默认的系统分区分大一些,或者分出一个 分区 用于挂载/opt目录。
3. 建议安装Gnome桌面环境。

配置Debian

下面的操作都要使用root权限。有一点需要提醒的是,Debian中切换到root用户,要使用“su -”而不能直接用“su”, 不然$PATH中不会包含sbin目录。

关闭自动待机,在界面程序中Settings->Power->Automatic suspend = off。

远程连接的情况,还可以完全禁止休眠和睡眠:(重新开启将mask改为unmask)

# systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

安装完成后,进入系统,配置软件源为本地DVD文件,因为这样可以最快安装,不联网的机器不用担心漏洞问题,也减少自动升级造成的麻烦。先新建个目录 /opt/debian_dvd/ ,将DVD1-3及firmware DVD保存到/opt/debian_dvd/目录下。再在/media/目录创建几个子目录用于开机挂CD,我用了/mdeia/debian_dvd1,/mdeia/debian_dvd2,/mdeia/debian_dvd3,/mdeia/debian_dvd4。然后修改/etc/fstab,增加开机挂CD文件。

(...)
/opt/debian_dvd/debian-10.11.0-amd64-DVD-1.iso /media/debian_dvd1 iso9660 defaults 0 0
/opt/debian_dvd/debian-10.11.0-amd64-DVD-2.iso /media/debian_dvd2 iso9660 defaults 0 0
/opt/debian_dvd/debian-10.11.0-amd64-DVD-3.iso /media/debian_dvd3 iso9660 defaults 0 0
/opt/debian_dvd/firmware-10.11.0-amd64-DVD-1.iso /media/debian_dvd4 iso9660 defaults 0 0

修改/etc/apt/sources.list文件,在其它非注释行首加#改为注释,增加指定DVD为软件源:

deb [trusted=yes] file:/media/debian_dvd1 buster contrib main
deb [trusted=yes] file:/media/debian_dvd2 buster contrib main
deb [trusted=yes] file:/media/debian_dvd3 buster contrib main
deb [trusted=yes] file:/media/debian_dvd4 buster contrib main non-free

deb http://deb.debian.org/debian/ buster main contrib
deb-src http://deb.debian.org/debian/ buster main contrib

如果不想重启机器,可以手动挂载:

# mount -o loop /opt/debian_dvd/debian-10.11.0-amd64-DVD-1.iso /media/debian_dvd1  (其他类似)

更新软件源信息:apt-get update

下面就可以安装一些常用的包了:

# apt-get install vim curl build-essential git python3.7-dev python3-pip linux-headers-$(uname -r) libglvnd-dev dkms pkg-config

将当前用户加入sudo:

# vim /etc/sudoers
(加入一行当前用户)
oopsdump ALL=(ALL) ALL

对于远程的童鞋,要关闭vim的鼠标支持,不能无法用putty上右键粘贴。修改/usr/share/vim/vim81/defaults.vim,将mouse=a该为mouse-=a。

配置远程桌面

安装XRDP远程桌面:

# apt-get install xrdp 
# systemctl enable xrdp

如果远程登录后是桌面空白,可以尝试:

# apt-get purge xserver-xorg-legacy

在使用的用户下:(无需root)

# echo gnome-session > ~/.xsession

如果在远程登录后,遇到提示“Authentication is required to create a color profile”,需要输入管理员密码。可以增加下面文件: /etc/polkit-1/localauthority.conf.d/02-allow-colord-oopsdump-com.conf,内容如下:(注意:将下面的{group}替换为你需要允许的用户组, {group} 可以使用video组)

polkit.addRule(function(action, subject) {
 if ((action.id == "org.freedesktop.color-manager.create-device" ||
 action.id == "org.freedesktop.color-manager.create-profile" ||
 action.id == "org.freedesktop.color-manager.delete-device" ||
 action.id == "org.freedesktop.color-manager.delete-profile" ||
 action.id == "org.freedesktop.color-manager.modify-device" ||
 action.id == "org.freedesktop.color-manager.modify-profile") &&
 subject.isInGroup("{group}")) {
 return polkit.Result.YES;
 }
 });

要注意的是,如果用户在图形界面没有Logout,远程是无法登录的。可以用SSH执行下面命令,强制用户注销:(DISPLAY值根据实际情况修改,可以ps -ef|grep Xorg查看带“:”的值)

env DISPLAY=:0.0 gnome-session-quit --logout

配置中文环境支持:

# dpkg-reconfigure locales
**选上zh_CN.UTF-8
**默认语言仍为en_US.UTF-8

安装中文输入支持:

# apt-get install fcitx fcitx-googlepinyin
# fcitx-configtool 
**在弹出界面点左下+号
**去掉Only Show Current Language的勾
**选中最下的Google Pinyin,然后OK
**然后就可以Ctrl + Space切换输入法

安装VSCode用来编码:

# curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
# install -o root -g root -m 644 microsoft.gpg /usr/share/keyrings/microsoft-archive-keyring.gpg
# sh -c 'echo "deb [arch=amd64,arm64,armhf signed-by=/usr/share/keyrings/microsoft-archive-keyring.gpg] https://packages.microsoft.com/repos/vscode stable main" > /etc/apt/sources.list.d/vscode.list'
# apt-get install apt-transport-https
# apt-get update
# apt-get install code

安装Nvidia闭源驱动

Nvidia的显卡驱动可以在官方网站下载:https://www.nvidia.com/Download/index.aspx?lang=en-us。选择GeForce类型, GeForce 10系列,GTX 1080Ti。

也可用我的下载地址直接下载:https://us.download.nvidia.cn/XFree86/Linux-x86_64/470.57.02/NVIDIA-Linux-x86_64-470.57.02.run

先关闭原开源的Nouveau 驱动:(原因可以参考早前写的在CentOS8下安装Nviadia驱动的过程。)

# vim /etc/modprobe.d/nouveau
文件中输入:
blacklist nouveau
options nouveau modeset=0
# depmod -ae
# vim /etc/default/grub
文件尾增加:
GRUB_CMDLINE_LINUX_DEFAULT="quiet rd.driver.blacklist=nouveau nouveau.modeset=0"
# update-grub2
# update-initramfs -u

关闭系统GUI,并重启:(使用GUI仍然会自动加载nouveau)

# systemctl set-default multi-user.target
# systemctl reboot

先停止Nouveau的驱动:rmmod nouveau

安装Nvidia的闭源驱动:./NVIDIA-Linux-x86_64-470.57.02.run

恢复图形界面并重启:

# systemctl set-default graphical.target
# systemctl reboot

我发现仍然会加载Nouveau的驱动,但Nvidia的驱动已经可以加载了。这时,可以验证一下:

# nvidia-smi
Thu Nov  1 09:08:34 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:05:00.0  On |                  N/A |
| 21%   38C    P8    11W / 250W |     65MiB / 11176MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1040      G   /usr/lib/xorg/Xorg                 39MiB |
|    0   N/A  N/A      1070      G   /usr/bin/gnome-shell               22MiB |
+-----------------------------------------------------------------------------+

安装CUDA和Cudnn

CUDA 11.4 安装按官网步骤提示即可。 也可用我下载地址: https://developer.download.nvidia.com/compute/cuda/11.4.2/local_installers/cuda_11.4.2_470.57.02_linux.run

安装CUDA 11.4:./cuda_11.4.2_470.57.02_linux.run
安装时,可以不再安装驱动:(因为上面安装的一样版本的驱动)

lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x CUDA Installer                                                               x
x - [ ] Driver                                                                 x
x      [ ] 470.57.02                                                           x
x + [X] CUDA Toolkit 11.4                                                      x
x   [X] CUDA Samples 11.4                                                      x
x   [X] CUDA Demo Suite 11.4                                                   x
x   [X] CUDA Documentation 11.4                                                x
x   Options                                                                    x
x   Install                                                                    x
x                                                                              x
x                                                                              x
x Up/Down: Move | Left/Right: Expand | 'Enter': Select | 'A': Advanced options x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

验证是否安装成功:

# cd /usr/local/cuda/samples/1_Utilities/deviceQuery
# make
.....
# ./deviceQuery
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "NVIDIA GeForce GTX 1080 Ti"
  CUDA Driver Version / Runtime Version          11.4 / 11.4
  CUDA Capability Major/Minor version number:    6.1
  Total amount of global memory:                 11176 MBytes (11719409664 bytes)
  (028) Multiprocessors, (128) CUDA Cores/MP:    3584 CUDA Cores
  GPU Max Clock rate:                            1582 MHz (1.58 GHz)
  Memory Clock rate:                             5505 Mhz
  Memory Bus Width:                              352-bit
  L2 Cache Size:                                 2883584 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total shared memory per multiprocessor:        98304 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 2 copy engine(s)
  Run time limit on kernels:                     Yes
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Device supports Managed Memory:                Yes
  Device supports Compute Preemption:            Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 5 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 11.4, CUDA Runtime Version = 11.4, NumDevs = 1
Result = PASS

# make clean
...

需要手动下载Cudnn来安装: https://developer.nvidia.com/rdp/cudnn-download

下载Cudnn需要注册,如果不想注册,可以使用我下载用的地址: https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.2.4/11.4_20210831/cudnn-11.4-linux-x64-v8.2.4.15.tgz

安装方法:

# cd /usr/local/
# tar -xzvf <your save directory>cudnn-11.4-linux-x64-v8.2.4.15.tgz
# sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*

安装虚拟运行环境

安装virtualenv,来进行TensorFlow与CentOS本身包的隔离:

# pip3 install --upgrade virtualenv

创建一个公用虚拟环境:

# mkdir -p /var/venvs/
# virtualenv --system-site-packages  /var/venvs/tensorflow

将下面的Diff信息修改到/var/venvs/tensorflow/bin/activate:

        unset _OLD_VIRTUAL_PYTHONHOME
    fi
+   if ! [ -z "${_OLD_VIRTUAL_LIB:+_}" ] ; then
+       LD_LIBRARY_PATH="$_OLD_VIRTUAL_LIBPATH_OPPSDUMP_COM"
+       export LD_LIBRARY_PATH
+       unset _OLD_VIRTUAL_LIBPATH_OPPSDUMP_COM
+   fi

    # This should detect bash and zsh, which have a hash command that must

  _OLD_VIRTUAL_PATH="$PATH"
- PATH="$VIRTUAL_ENV/bin:$PATH"
+ PATH="$VIRTUAL_ENV/bin:/usr/local/cuda/bin:$PATH"
  export PATH
+ _OLD_VIRTUAL_LIBPATH_OPPSDUMP_COM="$LD_LIBRARY_PATH"
+ LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"
+ export LD_LIBRARY_PATH
+ export CUDADIR=/usr/local/cuda

# unset PYTHONHOME if set

当需要使用TensorFlow时,先进行:

# source /var/venvs/tensorflow/bin/activate

安装TensorFlow

安装TensorFlow:

**CPU版本
# pip3 install --upgrade tensorflow
**GPU版本
# pip3 install --upgrade tensorflow-gpu
**旧版本
# pip3 install tensorflow=={package_version}
**其他包PyTorch
# pip3 install --upgrade torch8
**如果安装tensorflow时出现h5py安装问题,可以使用下面方法:
# env H5PY_SETUP_REQUIRES=0 pip3 install -U --no-build-isolation h5py==3.1.0

测试命令:

$ python3 -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

如果遇到问题:

/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
(...)

降低numpy版本:

# pip3 uninstall numpy
# pip3 install numpy==1.16.4

再次运行可以成功了。