Hardware : Différence entre versions
De ClustersSophia
(→CPU clusters) |
(→SuperMicro 4029 node) |
||
| Ligne 180 : | Ligne 180 : | ||
* CPU-GPU connection with a PCIe gen3 16x interface | * CPU-GPU connection with a PCIe gen3 16x interface | ||
* GPU-GPU connection with NVLink 2.0 (25GB/s per direction per link, 1 or 2 links per card pair) | * GPU-GPU connection with NVLink 2.0 (25GB/s per direction per link, 1 or 2 links per card pair) | ||
| − | * CPU : 2x Xeon SP Gold 6126 @ 2.6 GHz | + | * CPU : 2x Xeon SP Gold 6126 @ 2.6 GHz (24 cores) |
* RAM capacity : 384 GB | * RAM capacity : 384 GB | ||
* storage : | * storage : | ||
| Ligne 189 : | Ligne 189 : | ||
* 1x infiniband FDR card | * 1x infiniband FDR card | ||
* hyperthreading active | * hyperthreading active | ||
| − | |||
== Asus ESC8000 GPU node == | == Asus ESC8000 GPU node == | ||
Version du 9 mai 2022 à 18:17
CPU & GPU summary
The current platform includes:
- 16 dual-Xeon SP 10 cores, 9.6GB/core RAM
- 1 quad-Xeon SP 20 cores, 12.8GB/core RAM
- 8 dual-Xeon 8 cores, 16GB/core RAM
- 16 dual-Xeon 10 cores, 9.6GB/core RAM
- 6 quad-Opteron 16 cores, 4GB/core RAM
- 6 quad-Opteron 12 cores, ~5.3GB/core RAM
- 1 dual-EPYC 32 cores, 16GB/core RAM
- 1 dual-EPYC 32 cores, 6GB/core RAM
- 2 dual-EPYC 16 cores, 8GB/core RAM
- 2 dual-EPYC 24 cores, 10GB/core RAM
- 164 Nvidia GPU
The number of CPU cores available is :
- 144 Xeon SP Gold 2.6GHz cores : ~6.7 TFlops AVX-512 (9.2 TFlops peak)
- 320 Xeon SP Silver 2.2GHz cores : ~8.8 TFlops AVX2 (11.2 TFlops peak) / ~6.4 TFlops AVX-512 (9.7 TFlops peak)
- 80 Xeon SP Gold 2.4GHz cores : ~2.4 TFlops AVX2 (3.1 Tflops peak) / ~3.8 Tflops AVX-512 (4.9 Tflops peak)
- 128 Xeon E5 v2 2.6GHz cores : ~2.4 Tflops (2.7 Tflops peak)
- 320 Xeon E5 v2 2.8GHz cores : ~5.9 Tflops (7.2 Tflops peak)
- 384 Opteron 2.3GHz cores : ~2.7 Tflops (3.5 Tflops peak)
- 288 Opteron 2.2GHz cores : ~2 Tflops (2.5 Tflops peak)
- 64 EPYC 2.9GHz cores : ~1.5 Tflops AVX2
- 64 EPYC 2.5GHz cores : ~1.25 Tflops AVX2
- 32 EPYC 2.8GHz cores : ~1.25 Tflops AVX2
- 48 EPYC 2.65GHz cores : ~1.25 Tflops AVX2
The total cumulated CPU computing power is close to ~37.9 TFlops in double precision (44,5 Tflops peak)
The number of GPU cores available is :
- 625088 Streaming Processor Cores
CPU clusters
Dell R7525 nodes (2020)
Dell R7525 dual-EPYC 7502 @ 2.5GHz (64 cores) : 1 node ( nef059 )
- RAM capacity : 384 GB RAM
- storage : system 2x900 GB SATA SSD RAID-1 + local scratch data 7x7.5TB SAS HDD 44 TB RAID-5 + controller H745
- 2x gigabit network ports (one connected)
- infiniband EDR card (connected to FDR switch)
- hyperthreading active
Dell R7525 dual-EPYC 7542 @ 2.9GHz (64 cores) : 1 node ( nef058 )
- RAM capacity : 1024 GB RAM
- storage : system 2x223 GB SATA SSD RAID-1 + local scratch data 5x445GB SAS SSD 1.8 TB RAID-5 + controller H745
- 2x gigabit network ports (one connected)
- infiniband EDR card (connected to FDR switch)
- hyperthreading active
Dell C6420 cluster (2019)
Dell C6420 dual-Xeon Cascade Lake SP Gold 6240 @ 2.60GHz (36 cores) : 4 nodes ( nef054 to nef057 )
- RAM capacity : 384 GB RAM
- storage : system 2x600 GB SATA RAID-1 + local scratch data 960 BB SATA SSD RAID-0 + controller H330
- 1x gigabit network port
- 1x infiniband FDR card
- hyperthreading active
- optimal performance with AVX-512, AVX/AVX2 support
Dell R940 node (2017)
Dell R940 quad-Xeon SP Gold 6148 @ 2.40GHz (80 cores) : 1 node ( nef053 )
- RAM capacity : 1024 GB RAM
- storage : system 2x600 GB SATA RAID-1 + local scratch data 1.92 TB SATA SSD + controller H740P
- 4x gigabit network ports (one connected)
- infiniband EDR card (connected to FDR switch)
- hyperthreading active
- optimal performance with AVX-512, AVX/AVX2 support
Dell C6420 cluster (2017)
Dell C6420 dual-Xeon Skylake SP Silver 4114 @ 2.20GHz (20 cores) : 16 nodes ( nef037 to nef052 )
- RAM capacity : 192 GB RAM
- 1x600GB 10kRPM SAS HardDisk drive
- 1x gigabit network port
- 1x infiniband FDR card
- hyperthreading active
- AVX-512 support, optimal performance with AVX/AVX2
Dell C6220 cluster (2015)
Dell C6220 dual-Xeon E5-2650 v2 @ 2.60GHz (16 cores) : 8 nodes ( nef029 to nef036 )
- RAM capacity : 256 GB RAM
- 1x1TB SATA HardDisk drive
- 2x gigabit network ports (one connected)
- 2x infiniband QDR card (one connected)
- hyperthreading not active
Dell C6220 cluster (2014)
Dell C6220 dual-Xeon E5-2680 v2 @ 2.80GHz (20 cores) : 16 nodes ( nef013 to nef028 )
- RAM capacity : 192 GB RAM
- 1x2TB SATA HardDisk drive
- 2x gigabit network ports (one connected)
- 1x infiniband FDR card (QDR used)
- hyperthreading not active
Dell C6145 cluster (2013)
Dell C6145 quad-Opterons 6376 @ 2.3Ghz (64 cores) : 6 nodes ( nef007 to nef012 )
- RAM capacity : 256 GB RAM (512GB on nef011 and nef012)
- 1x500GB SATA HardDisk drive
- 2x gigabit network ports (one connected)
- 1x infiniband QDR card
- hyperthreading not supported
Dell R815 cluster (2010)
Dell R815 quad-Opterons 6174 @ 2.2Ghz (48 cores) : 6 nodes ( nef001 to nef006 )
- RAM capacity : 256 GB RAM
- 2x600GB SAS HardDisk drive (RAID-0)
- 4x gigabit network ports (one connected)
- 1x infiniband QDR card
- hyperthreading not supported
GPU nodes
SuperMicro 4029 node
SuperMicro 4029GP-TVRT : 1 node (nefgpu41 )
- 4x Nvidia Tesla V100 SXM2 GPUs cards
- 5120 CUDA cores per card
- 640 tensor cores per card
- 32GB of RAM capacity per card
- Simple precision performance peak: 15.7 Tflops per card
- Double precision performance peak: 7.8 Tflops per card
- Tensor performance peak : 125 TFlops per card
- 900 GB/s GPU memory bandwidth with error correction (ECC)
- CPU-GPU connection with a PCIe gen3 16x interface
- GPU-GPU connection with NVLink 2.0 (25GB/s per direction per link, 1 or 2 links per card pair)
- CPU : 2x Xeon SP Gold 6126 @ 2.6 GHz (24 cores)
- RAM capacity : 384 GB
- storage :
- system & /tmp : 240GB SSD SATA RAID-1
- /local/tmp scratch data : 4.8 TB SSD SATA RAID-5
- RAID controller SAS 12Gb/s
- 2x 10Gbps ethernet ports (one connected in 1Gb/s)
- 1x infiniband FDR card
- hyperthreading active
Asus ESC8000 GPU node
Asus ESC8000G4 / Carri HighServer nodes : 1 node (nefgpu40 )
- 8x GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface
- 3584 CUDA cores per card
- 11GB of RAM capacity per card
- Simple precision performance peak: 10.6 Tflops per card
- Double precision performance peak: 0.3 Tflops per card
- 484 GB/s GPU memory bandwidth
- PCIe single-root topology (2 PCIe 96 lane switches)
- topology can be software modified (BIOS & reboot) to dual-root for an experiment campaign
- CPU : 2x Xeon SP Gold 5115 @ 2.4 GHz
- RAM capacity : 256 GB
- storage :
- system & /tmp : RAID-1 2x512 GB SATA SSD
- /local/tmp scratch data : RAID-0 4 TB SATA SSD
- RAID controller SAS 12Gb/s
- 4x gigabit ethernet ports (one connected)
- 1x infiniband FDR card
- hyperthreading active
Dell R7525 GPU nodes
Dell R7525 dual-EPYC 7282 @ 2.8GHz (16 cores) : 2 nodes ( nefgpu52 to nefgpu53 )
Dell R7525 dual-EPYC 7413 @ 2.65GHz (24 cores) : 2 nodes ( nefgpu54 to nefgpu55 )
- Nvidia A40 PCIe GPUs cards
- 10752 CUDA cores per card
- 336 tensor cores per card
- 48GB of RAM capacity per card
- Tensor performance peak : 149.6 TFlops per card
- Simple precision performance peak: 74.8 Tflops per card
- Double precision performance peak: 37.4 Tflops per card
- 696 GB/s GPU memory bandwidth with error correction (ECC)
- 2x gigabit network ports (one connected)
- infiniband EDR card (connected to FDR switch)
- hyperthreading active
- optimal performance with AVX-256, AVX/AVX2 support
| Node name | Funding team | GPU cards | Node CPU | Node RAM | Node storage |
| nefgpu52 | NEO | 3x A40 | 2x AMD EPYC 7282 | 256 GB | system & /tmp : RAID-1 2x 480SSD /local/mixed : RAID-5 5x 480 SSD |
| nefgpu53 | ACENTAURI | 3x A40 | 2x AMD EPYC 7282 | 256 GB | system & /tmp : RAID-1 2x 480SSD /local/mixed : RAID-5 5x 480 SSD |
| nefgpu53 | 3IA | 3x A40 | 2x AMD EPYC 7413 | 512 GB | system & /tmp : RAID-1 2x 480SSD /local/mixed : RAID-5 5x 960 SSD |
| nefgpu54 | 3IA | 3x A40 | 2x AMD EPYC 7413 | 512 GB | system & /tmp : RAID-1 2x 480SSD /local/mixed : RAID-5 5x 960 SSD |
Dell R740 GPU nodes
Dell R740 nodes: dual-Xeon Skylake or CascadeLake SP : 5 nodes ( nefgpu42 to nefgpu46)
- Nvidia Tesla V100 PCIe GPUs cards
- 5120 CUDA cores per card
- 640 tensor cores per card
- 32GB of RAM capacity per card
- Tensor performance peak : 112 TFlops per card
- Simple precision performance peak: 14 Tflops per card
- Double precision performance peak: 7 Tflops per card
- 900 GB/s GPU memory bandwidth with error correction (ECC)
- Nvidia Tesla T4 GPUs cards connected with a PCIe gen3 16x interface
- 2560 CUDA cores per card
- 320 tensor cores per card
- 16GB of RAM capacity per card
- Simple precision performance peak: 8.1 Tflops per card
- Double precision performance peak: 0.3 Tflops per card
- 300 GB/s GPU memory bandwidth with error correction (ECC)
- 2-4x ethernet ports (one connected in gigabit)
- 1x infiniband FDR card
- hyperthreading active
| Node name | Funding team | GPU cards | Node CPU | Node RAM | Node storage |
| nefgpu42 | common | 3x Tesla T4 | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 2x 480GB SSD |
| nefgpu43 | common | 3x Tesla T4 | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 2x 480GB SSD |
| nefgpu44 | common | 3x Tesla T4 | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 2x 480GB SSD |
| nefgpu45 | common | 3x Tesla T4 | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 2x 480GB SSD |
| nefgpu46 | STARS | 3x Tesla V100 | 2x Xeon Silver 4215 | 384 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD |
Dell T640 GPU nodes
Dell T640 nodes: dual-Xeon Skylake or CascadeLake SP : 21 nodes ( nefgpu{24-39} and nefgpu{47-51})
- Nvidia Quadro RTX 6000 GPUs cards connected with a PCIe gen3 16x interface
- 4608 CUDA cores per card
- 576 tensor cores per card
- 24GB of RAM capacity per card
- Tensor performance peak : 130.5 TFlops per card
- Half precision performance peak: 32.6 Tflops per card
- Simple precision performance peak: 16.3 Tflops per card
- Double precision performance peak: 0.5 Tflops per card
- 624 GB/s GPU memory bandwidth with error correction (ECC)
- optional NVLink 2.0 (2x25GB/s per direction) by pair of card
- Nvidia Quadro RTX 8000 GPUs cards connected with a PCIe gen3 16x interface
- 4608 CUDA cores per card
- 576 tensor cores per card
- 48GB of RAM capacity per card
- Tensor performance peak : 130.5 TFlops per card
- Half precision performance peak: 32.6 Tflops per card
- Simple precision performance peak: 16.3 Tflops per card
- Double precision performance peak: 0.5 Tflops per card
- 624 GB/s GPU memory bandwidth with error correction (ECC)
- optional NVLink 2.0 (2x25GB/s per direction) by pair of card
- GeForce RTX 2080 Ti GPUs cards connected with a PCIe gen3 16x interface
- 4352 CUDA cores per card
- 544 tensor cores per card
- 11GB of RAM capacity per card
- Half precision performance peak: 23.5 Tflops per card
- Simple precision performance peak: 11.8 Tflops per card
- Double precision performance peak: 0.4 Tflops per card
- 616 GB/s GPU memory bandwidth
- GeForce Titan RTX GPUs cards connected with a PCIe gen3 16x interface
- 4608 CUDA cores per card
- 576 tensor cores per card
- 24GB of RAM capacity per card
- Half precision performance peak: 24.9 Tflops per card
- Simple precision performance peak: 12.4 Tflops per card
- Double precision performance peak: 0.4 Tflops per card
- 672 GB/s GPU memory bandwidth
- Power limited to 210W to prevent overheat
- GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface
- 3584 CUDA cores per card
- 11GB of RAM capacity per card
- Simple precision performance peak: 10.6 Tflops per card
- Double precision performance peak: 0.3 Tflops per card
- 484 GB/s GPU memory bandwidth
- GeForce GTX Titan X GPUs cards connected with a PCIe gen3 16x interface
- 3072 CUDA cores per card
- 12GB of RAM capacity per card
- Simple precision performance peak: 7.0 Tflops per card
- Double precision performance peak: 0.2 Tflops per card
- 336.5 GB/s GPU memory bandwidth
- 2-4x ethernet ports (one connected in gigabit)
- 1x infiniband FDR card
- hyperthreading active
| Node name | Funding team | GPU cards | Node CPU | Node RAM | Node storage |
| nefgpu24 | EPIONE | 4x GTX 1080 Ti | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : RAID-0 1x 960GB SSD |
| nefgpu25 | ZENITH | 4x GTX 1080 Ti | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-0 4x 480GB SSD /local/read : RAID-0 8x 480GB SSD read intensive |
| nefgpu26 | WIMMICS | 4x GTX 1080 Ti | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : RAID-0 4x 480GB SSD |
| nefgpu27 | ZENITH | 4x RTX 2080 Ti | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-0 2x 400GB SSD /local/read : RAID-0 4x 480GB SSD read intensive |
| nefgpu28 | GRAPHDECO | 4x GTX 1080 Ti | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 400GB SSD |
| nefgpu29 | EPIONE | 2x GTX Titan-X | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 480GB SSD |
| nefgpu30 | common | 4x RTX 2080 Ti | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 4x 480GB SSD |
| nefgpu31 | common | 4x RTX 2080 Ti | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 4x 480GB SSD |
| nefgpu32 | common | 4x Titan RTX | 2x Xeon Silver 4110 | 192 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 4x 480GB SSD |
| nefgpu33 | CHORALE | 4x RTX 2080 Ti | 2x Xeon Silver 4110 | 96 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-0 1x 480GB SSD |
| nefgpu34 | GRAPHDECO | 4x RTX 6000 w/ NVLink by pair |
2x Xeon Gold 6240 | 384 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD |
| nefgpu35 | STARS | 4x RTX 6000 w/ NVLink by pair |
2x Xeon Silver 4215 | 192 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD |
| nefgpu36 | STARS | 4x RTX 6000 w/ NVLink by pair |
2x Xeon Silver 4215 | 192 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD |
| nefgpu37 | GRAPHDECO | 4x RTX 8000 w/ NVLink by pair |
2x Xeon Gold 6240 | 384 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD |
| nefgpu38 | AYANA | 2x RTX 8000 w/ NVLink by pair |
2x Xeon Silver 4214R | 384 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD |
| nefgpu39 | STARS | 4x RTX 8000 w/ NVLink by pair |
2x Xeon Silver 4216 | 384 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 900GB SATA SSD |
| nefgpu47 | I3S/SPARKS | 4x RTX 8000 w/ NVLink by pair |
2x Xeon Gold 6230R | 384 GB | system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 900GB SSD |
| nefgpu48 | I3S/SPARKS | 4x RTX 8000 w/ NVLink by pair |
2x Xeon Gold 6230R | 384 GB | system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 900GB SSD |
| nefgpu49 | I3S/SPARKS | 4x RTX 8000 w/ NVLink by pair |
2x Xeon Gold 6238R | 768 GB | system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 1.8TB SSD |
| nefgpu50 | I3S/SPARKS | 4x RTX 8000 w/ NVLink by pair |
2x Xeon Gold 6230R | 384 GB | system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 900GB SSD |
| nefgpu51 | I3S/SPARKS | 4x RTX 8000 w/ NVLink by pair |
2x Xeon Gold 6230R | 384 GB | system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 900GB SSD |
Dell T630 GPU nodes
Dell T630 nodes: dual-Xeon E5-26xx : 17 nodes ( nefgpu07 to nefgpu23)
- GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface
- 3584 CUDA cores per card
- 11GB of RAM capacity per card
- Simple precision performance peak: 10.6 Tflops per card
- Double precision performance peak: 0.3 Tflops per card
- 484 GB/s GPU memory bandwidth
- GeForce GTX 1080 GPUs cards connected with a PCIe gen3 16x interface
- 2560 CUDA cores per card
- 8GB of RAM capacity per card
- Simple precision performance peak: 8.2 Tflops per card
- Double precision performance peak: 0.3 Tflops per card
- 320 GB/s GPU memory bandwidth
- GeForce GTX Titan X GPUs cards connected with a PCIe gen3 16x interface
- 3072 CUDA cores per card
- 12GB of RAM capacity per card
- Simple precision performance peak: 7.0 Tflops per card
- Double precision performance peak: 0.2 Tflops per card
- 336.5 GB/s GPU memory bandwidth
- 4x gigabit ethernet ports (one connected)
- 1x infiniband FDR card
| Node name | Funding team | GPU cards | Node CPU | Node RAM | Node storage | Hyper threading active ? |
| nefgpu07 | EPIONE | 4x GTX 1080 Ti | 2x E5-2620v3 | 128 GB | system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM | no |
| nefgpu08 | ZENITH | 4x GTX 1080 Ti | 2x E5-2630v3 | 64 GB | system & /tmp : RAID-0 2x 300GB SAS 15kRPM /local/mixed : 1x 800GB SSD /local/read : RAID-0 2x 1.92TB SSD read intensive |
no |
| nefgpu09 | GRAPHDECO | 4x Titan X | 2x E5-2630v4 | 48 GB | system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 400GB SSD |
no |
| nefgpu10 | STARS | 4x Titan X | 2x E5-2630v4 | 128 GB | system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 1.6TB SSD |
no |
| nefgpu11 | STARS | 4x GTX 1080 | 2x E5-2630v4 | 128 GB | system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 1.6TB SSD |
no |
| nefgpu12 | STARS | 4x GTX 1080 | 2x E5-2630v4 | 128 GB | system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 1.6TB SSD |
yes |
| nefgpu13 | GRAPHDECO | 4x GTX 1080 Ti | 2x E5-2650v4 | 64 GB | system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 400GB SSD |
yes |
| nefgpu14 | STARS | 4x GTX 1080 Ti | 2x E5-2620v4 | 128 GB | system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 400GB SSD |
yes |
| nefgpu15 | STARS | 4x GTX 1080 Ti | 2x E5-2620v4 | 128 GB | system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 400GB SSD |
yes |
| nefgpu16 | EPIONE | 4x GTX 1080 Ti | 2x E5-2630v4 | 128 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : 1x 1.6TB SSD |
yes |
| nefgpu17 | ZENITH | 4x GTX 1080 Ti | 2x E5-2630v4 | 64 GB | system & /tmp : RAID-0 2x 600GB SAS 10kRPM /local/mixed : 1x 1.6TB SSD /local/read : RAID-0 2x 1.92TB SSD read intensive |
yes |
| nefgpu18 | common | 4x GTX 1080 Ti | 2x E5-2630v4 | 128 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : 1x 1.6TB SSD |
yes |
| nefgpu19 | common | 4x GTX 1080 Ti | 2x E5-2630v4 | 128 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : 1x 1.6TB SSD |
yes |
| nefgpu20 | common | 4x GTX 1080 Ti | 2x E5-2630v4 | 128 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : 1x 1.6TB SSD |
yes |
| nefgpu21 | STARS | 4x GTX 1080 Ti | 2x E5-2620v4 | 128 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : 1x 480GB SSD |
yes |
| nefgpu22 | STARS | 3x GTX 1080 Ti | 2x E5-2620v4 | 128 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : 1x 480GB SSD |
yes |
| nefgpu23 | TITANE-EPITOME | 4x GTX 1080 Ti | 2x E5-2630v4 | 64 GB | system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : 1x 1.6TB SSD |
yes |
Dell R730 GPU node
Dell R730 nodes: dual-Xeon E5-26xx : 1 node ( nefgpu01)
- Tesla K80 GPU cards connected with a PCIe gen3 16x interface
- 2x Tesla GK210 GPUs per card
- 4992 CUDA cores per card
- 2x 12GB RAM capacity per card with error correction (ECC)
- Simple precision performance peak: 5.61 Tflops per card
- Double precision performance peak: 1.87 Tflops per card
- 2x 240 GB/s GPU memory bandwidth with error correction (ECC)
- 4x gigabit ethernet ports (one connected)
- 1x infiniband QDR card
- hyperthreading not active
| Node name | Funding team | Number of GPU cards | Node CPU | Node RAM | Node storage |
| nefgpu01 | MATHNEURO | 1x K80 | 2x E5-2623v4 | 32 GB | 2x 400GB SSD |
Storage
All nodes have access to common storage :
- common storage : /home
- 31 TiB, available to all users, quotas per user
- 1 Dell PowerEdge R540 server with 2 RAID-10 array 10 x 10TB SAS 7.2 kRPM disks, infiniband FDR, NFS access
- capacity distributed and scalable common storage : /data
- ~610TiB real capacity, 684TB theoretical capacity (06/2019)
- permanent storage : 1TiB quota per team + teams may buy additional quota (please contact cluster administrators)
- scratch storage : variable size (initially ~40TiB), no quota limit, for temporary storage (data may be purged)
- BeeGFS filesystem on multiple hardware :
- 4 Dell PowerEdge R740xd ; 1.92TB metadata : RAID-1 array 2 x 1.92TB SSD mixed use MLC disks (or RAID-10 array 4 x 960GB) ; 2x {60 or 72}TB data : 2 x RAID-6 array 8 x {10 or 12}TB SAS 7.2 kRPM disks
- 2 Dell PowerEdge R730xd ; 800GB metadata : RAID-1 array 2 x 800 GB SSD mixed use MLC disks ; 2 x {36 or 48}TB data: 2 x RAID-6 array 8 x {6 or 8}TB SAS 7.2 kRPM disks
- infiniband FDR
- ~610TiB real capacity, 684TB theoretical capacity (06/2019)
More details about quotas at: Disk space management
