Singularity - 入门简介

发表于 | 分类于 ， ，
Singularity can be used with Docker images. This feature was included because developers use and really like using Docker and scientists have already put much resources into creating Docker images. Thus, one of our early goals was to support Docker.

基本概念

Singularity 是基于Docker已经进行应用阶段开发的。针对和Docker的官方说明如下

  • You don’t need Docker installed
  • You can shell into a Singularity-ized Docker image
  • You can run a Docker image instantly as a Singularity image
  • You can pull a Docker image (without sudo)
  • You can build images with bases from assembled Docker layers that include environment, guts, and labels

Singularity 安装

可以直接使用conda进行安装，这里最好使用root账户安装，不然后续使用依然可能存在权限问题。

1
conda create -n  singularity  singularity

Singularity 环境配置

1
2
# 配置singularity 下载镜像时的存储目录
export SINGULARITY_CACHEDIR=/ifstj2/B2C_RD_H1/Personal/liubo/singularity

基于docker镜像构建singature镜像

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# 方式1：从docker uri开始
singularity pull docker://godlovedc/lolcow # 下载pre-built image
singularity build mylolcow_latest.sif docker://godlovedc/lolcow # 下载后再build成镜像


# 方式2：从本地缓存的docker image开始
$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
godlovedc/lolcow    latest              577c1fe8e6d8        16 months ago       241MB

$ sudo singularity build lolcow_from_docker_cache.sif docker-daemon://godlovedc/lolcow:latest
# 和方式1 的区别
# 从原来的docker变成了docker-daemon；
# 这里需要加sudo，因为docker程序运行时需要sudo权限；
# docker的镜像后面必须要加TAG标签，可以通过sudo docker images查看TAG标签。

# 方式3：从镜像tar文件开始
$ sudo docker images
REPOSITORY           TAG              IMAGE ID       CREATED        SIZE  
hello-world          latest           d1165f221234   5 months ago   13.3kB

$ sudo docker save d1165f221234 -o hello_world.tar

$ singularity build hello_world.sif docker-archive://hello_world.tar
INFO:    Starting build...
Getting image source signatures
Copying blob f22b99068db9 done
Copying config 1189c90721 done
Writing manifest to image destination
Storing signatures
2021/08/15 16:46:15  info unpack layer: sha256:a99912efe9c767b280c869d2e734bef3c92d29d45d4e3beefbeb5a1924ed7445
INFO:    Creating SIF file...
INFO:    Build complete: hello_world.sif
# 和方式1的主要区别
# 原本的docker://换成了docker-archive://，方式2里面是docker-daemon://；
# 这里不需要加sudo了，只需要当前用户有hello_world.tar文件的可访问权限即可；
# 这里用的是tar文件，singularity也可以处理tar.gz文件。

运行Singularity 容器

1
2
3
4
5
# 使用exec直接运行任务
share/app/singularity/3.8.1/bin/singularity exec ensembl-vep_release_108.2.sif vep

# 运行容器后进入容器交互式执行任务
share/app/singularity/3.8.1/bin/singularity shell ensembl-vep_release_108.2.sif

挂在目录( -B)

1
/share/app/singularity/3.8.1/bin/singularity exec -B /ifstj2/B2C_RD_H1/Research_and_Development/ -B /ifstj2/B2C_RD_H1/Personal/liubo  ensembl-vep_release_108.2.sif  vep --fork 4  --cache  --dir_cache /ifstj2/B2C_RD_H1/Research_and_Development/03.database/vep_cache_dir   --assembly GRCh37  --offline  --refseq  --exclude_predicted  --fasta /ifstj2/B2C_RD_H1/Research_and_Development/03.database/Hg19/hg19.fa  -i /ifstj2/B2C_RD_H1/Personal/liubo/singularity/22S03467478.indel.final.vcf  -o /ifstj2/B2C_RD_H1/Personal/liubo/singularity/test/22S03467478.indel.vcfanno.tab  --format vcf  --vcf_info_field ANN  --force_overwrite  --tab  --verbose  --no_escape  --everything  --shift_3prime 1  --dir_plugins  /ifstj2/B2C_RD_H1/Research_and_Development/03.database/VEP_plugins/VEP_plugins-release-108  --plugin TSSDistance  --plugin TERT  --custom /ifstj2/B2C_RD_H1/Personal/liubo/0.Pipeline/aio.NewAnnotation/subpipeline/submodel_annotation_vep/Database/human_maked.gene.gff.gz,,gff  --custom /ifstj2/B2C_RD_H1/Research_and_Development/03.database/vep_inhouse_data/clinvar.vcf.gz,ClinVar,vcf,exact,0,CLNSIG,CLNREVSTAT,CLNDN

使用root身份运行

-------------本文结束感谢您的阅读-------------