pip&apt安装加速

更新时间:

参考本节,全局替换 pip、apt 和 Conda 源为趋动云镜像源,加速开发环境中的依赖包和工具安装。

趋动云镜像源:https://pypi.virtaicloud.com/repository/pypi/simple 为趋动云官方为用户提供的镜像源,在趋动云平台提供的开发环境中拉取该源的包时,其拉取速度优于其他国内镜像源。

# 背景说明

在开发环境中安装工具/依赖包时,默认使用官方镜像源进行安装,官方镜像源为国外源,会出现不稳定的情况(即安装时下载包的速度很慢)。
为提高环境准备效率,平台为用户提供专属的 pip 源和 apt 源(即趋动云镜像源),在平台开发环境中使用该源安装,会比使用国内其他镜像源更快。比如,使用不同的源安装 diango 时,包的下载速度如下所示:

清华源:12.8 MB/s 趋动源:52.9 MB/s 官方源:38.0 KB/s

# apt换源

  1. 开发环境的网页终端中执行如下命令,替换全局 apt 源。
    cp /etc/apt/sources.list /etc/apt/sources.listBak && \
    sed -i "s#http://archive.ubuntu.com/ubuntu/#https://mirrors.virtaicloud.com/repository/ubuntu/#g" /etc/apt/sources.list && \
    sed -i "s#http://security.ubuntu.com/ubuntu/#https://mirrors.virtaicloud.com/repository/security/#g" /etc/apt/sources.list
    
  2. 执行如下命令更新。
    apt-get update
    
    更新完成后,再使用 apt-get install 安装工具/包,则会直接从趋动云镜像源拉取安装包并安装。

# pip换源

本部分仅介绍如何将默认 “官方 pip 源” 全局替换成 “趋动源”,关于 pip 源的详细及其他替换可参考 pip镜像源

  1. 执行如下命令设置全局 pip 源。

    pip config set global.index-url https://pypi.virtaicloud.com/repository/pypi/simple
    
  2. 执行如下命令验证是否替换成功。

    pip3 config get global.index-url
    

    若返回如下信息,则全局镜像源替换成功。

    https://pypi.virtaicloud.com/repository/pypi/simple
    

    后续使用 pip install <packagename> 时,会默认从趋动源拉取安装包并安装。

# Conda换源

参考如下描述,全局替换 Conda 下载源为趋动源,后续 conda install <package_name> 时,会默认从趋动源拉取安装包并安装。

# 普通源替换

  1. 执行如下命令删除 ~/.condarc 文件。
    rm -rf ~/.condarc
    
  2. 执行如下命令全局设置 Conda 源为趋动源。
    conda config --add default_channels https://conda.virtaicloud.com/repository/anaconda/main
    conda config --add default_channels https://conda.virtaicloud.com/repository/anaconda/r
    conda config --add default_channels https://conda.virtaicloud.com/repository/anaconda/msys2
    

# 特殊包源替换

表格中的包为特殊包,需从特殊源下载,执行表中对应命令可设置该特殊包的 Conda 下载源为趋动源。

包名 换源命令
MindSpore conda config --set custom_channels.MindSpore https://conda.virtaicloud.com/repository/cloud
Paddle conda config --set custom_channels.Paddle https://conda.virtaicloud.com/repository/cloud
auto conda config --set custom_channels.auto https://conda.virtaicloud.com/repository/cloud
biobakery conda config --set custom_channels.biobakery https://conda.virtaicloud.com/repository/cloud
bioconda conda config --set custom_channels.bioconda https://conda.virtaicloud.com/repository/cloud
c4aarch64 conda config --set custom_channels.c4aarch64 https://conda.virtaicloud.com/repository/cloud
caffe2 conda config --set custom_channels.caffe2 https://conda.virtaicloud.com/repository/cloud
conda-forge conda config --set custom_channels.conda-forge https://conda.virtaicloud.com/repository/cloud
deepmodeling conda config --set custom_channels.deepmodeling https://conda.virtaicloud.com/repository/cloud
dglteam conda config --set custom_channels.dglteam https://conda.virtaicloud.com/repository/cloud
fastai conda config --set custom_channels.fastai https://conda.virtaicloud.com/repository/cloud
fermi conda config --set custom_channels.fermi https://conda.virtaicloud.com/repository/cloud
idaholab conda config --set custom_channels.idaholab https://conda.virtaicloud.com/repository/cloud
intel conda config --set custom_channels.intel https://conda.virtaicloud.com/repository/cloud
matsci conda config --set custom_channels.matsci https://conda.virtaicloud.com/repository/cloud
menpo conda config --set custom_channels.menpo https://conda.virtaicloud.com/repository/cloud
mordred-descriptor conda config --set custom_channels.mordred-descriptor https://conda.virtaicloud.com/repository/cloud
msys2 conda config --set custom_channels.msys2 https://conda.virtaicloud.com/repository/cloud
numba conda config --set custom_channels.numba https://conda.virtaicloud.com/repository/cloud
ohmeta conda config --set custom_channels.ohmeta https://conda.virtaicloud.com/repository/cloud
omnia conda config --set custom_channels.omnia https://conda.virtaicloud.com/repository/cloud
peterjc123 conda config --set custom_channels.peterjc123 https://conda.virtaicloud.com/repository/cloud
plotly conda config --set custom_channels.plotly https://conda.virtaicloud.com/repository/cloud
psi4 conda config --set custom_channels.psi4 https://conda.virtaicloud.com/repository/cloud
pytorch conda config --set custom_channels.pytorch https://conda.virtaicloud.com/repository/cloud
pytorch-lts conda config --set custom_channels.pytorch-lts https://conda.virtaicloud.com/repository/cloud
pytorch-test conda config --set custom_channels.pytorch-test https://conda.virtaicloud.com/repository/cloud
pytorch3d conda config --set custom_channels.pytorch3d https://conda.virtaicloud.com/repository/cloud
pyviz conda config --set custom_channels.pyviz https://conda.virtaicloud.com/repository/cloud
qiime2 conda config --set custom_channels.qiime2 https://conda.virtaicloud.com/repository/cloud
rapidsai conda config --set custom_channels.rapidsai https://conda.virtaicloud.com/repository/cloud
rdkit conda config --set custom_channels.rdkit https://conda.virtaicloud.com/repository/cloud
simpleitk conda config --set custom_channels.simpleitk https://conda.virtaicloud.com/repository/cloud
stackless conda config --set custom_channels.stackless https://conda.virtaicloud.com/repository/cloud
ursky conda config --set custom_channels.ursky https://conda.virtaicloud.com/repository/cloud

表中 conda config --set custom_channels.<packageName> https://conda.virtaicloud.com/repository/cloud 命令实际会将 包名 与相应 下载源 写入 ~/.condarc/custom_channels 文件,您也可通过新建与修改该文件来设置特殊包的下载源。

custom_channels 文件格式如下:

custom_channels:  
  ursky:https://conda.virtaicloud.com/repository/cloud
  stackless:https://conda.virtaicloud.com/repository/cloud
  pytorch:https://conda.virtaicloud.com/repository/cloud