In order to study the coding of RDMA, I need build the corresponding environment. Due to the lack of hardware, I found Soft-RoCE would be the first choice after some research work.
Struggle Against RXE
Internet shows that the RXE need kernel support and user space codes.
Firstly, we git clone the [rxe-dev] to compile and install the new kernel to support RoCE. Then restart with the new kernel. Finally, compile the [librxe-dev] to get the utilities of Soft-Roce.
When I switched to the compiled new kernel, but it failed to start the system. When I configured the [librxe-dev], it showed “configure: error: <infiniband/driver.h> not found. librxe requires libibverbs”.
This page also shows the same issue, and several people also hangs on this error.
MosesAlexander’s comment of “I just found that the rxe functionality is all in rdma-core now.” gives me sudden inspiration. The rdma-core has already contained all the RXE utilities, and it looks like that there’s no maintenance for the two repositories [rxe-dev] and [librxe-dev].
Just “yum -y install libibverbs libibverbs-devel libibverbs-utils librdmacm librdmacm-devel librdmacm-utils” will be OK !
Notice: I blog this post at 2018/09/03, based on CentOS 7 (3.10.0-862.el7.x86_64), whose kernel support RDMA-related technologies originally.
Run some commands to verify:
[root@localhost ~]# rxe_cfg start Name Link Driver Speed NMTU IPv4_addr RDEV RMTU ens33 yes e1000 virbr0 no bridge virbr0-nic no tun [root@localhost ~]# rxe_cfg add ens33 [root@localhost ~]# rxe_cfg status Name Link Driver Speed NMTU IPv4_addr RDEV RMTU ens33 yes e1000 rxe0 1024 (3) virbr0 no bridge virbr0-nic no tun [root@localhost ~]# ibv_devices device node GUID ------ ---------------- rxe0 020c29fffe495c4d
Also, you could run example codes from the-geek-in-the-corner.