Hardware

CPU & GPU summary

The current platform includes:

• 16 dual-Xeon SP 10 cores, 9.6GB/core RAM
• 1 quad-Xeon SP 20 cores, 12.8GB/core RAM
• 8 dual-Xeon 8 cores, 16GB/core RAM
• 16 dual-Xeon 10 cores, 9.6GB/core RAM
• 6 quad-Opteron 16 cores, 4GB/core RAM
• 6 quad-Opteron 12 cores, ~5.3GB/core RAM
• 1 dual-EPYC 32 cores, 16GB/core RAM
• 1 dual-EPYC 32 cores, 6GB/core RAM
• 2 dual-EPYC 16 cores, 8GB/core RAM
• 2 dual-EPYC 24 cores, 10GB/core RAM
• 164 Nvidia GPU

The number of CPU cores available is :

• 144 Xeon SP Gold 2.6GHz cores : ~6.7 TFlops AVX-512 (9.2 TFlops peak)
• 320 Xeon SP Silver 2.2GHz cores : ~8.8 TFlops AVX2 (11.2 TFlops peak) / ~6.4 TFlops AVX-512 (9.7 TFlops peak)
• 80 Xeon SP Gold 2.4GHz cores : ~2.4 TFlops AVX2 (3.1 Tflops peak) / ~3.8 Tflops AVX-512 (4.9 Tflops peak)
• 128 Xeon E5 v2 2.6GHz cores : ~2.4 Tflops (2.7 Tflops peak)
• 320 Xeon E5 v2 2.8GHz cores : ~5.9 Tflops (7.2 Tflops peak)
• 384 Opteron 2.3GHz cores : ~2.7 Tflops (3.5 Tflops peak)
• 288 Opteron 2.2GHz cores : ~2 Tflops (2.5 Tflops peak)
• 64 EPYC 2.9GHz cores : ~1.5 Tflops AVX2
• 64 EPYC 2.5GHz cores : ~1.25 Tflops AVX2
• 32 EPYC 2.8GHz cores : ~1.25 Tflops AVX2
• 48 EPYC 2.65GHz cores : ~1.25 Tflops AVX2

The total cumulated CPU computing power is close to ~37.9 TFlops in double precision (44,5 Tflops peak)

The number of GPU cores available is :

• 625088 Streaming Processor Cores

CPU clusters

Dell R7525 nodes (2020)

200px

Dell R7525 dual-EPYC 7502 @ 2.5GHz (64 cores) : 1 node ( nef059 )

• RAM capacity : 384 GB RAM
• storage : system 2x900 GB SATA SSD RAID-1 + local scratch data 7x7.5TB SAS HDD 44 TB RAID-5 + controller H745
• 2x gigabit network ports (one connected)
• infiniband EDR card (connected to FDR switch)
• hyperthreading active

Dell R7525 dual-EPYC 7542 @ 2.9GHz (64 cores) : 1 node ( nef058 )

• RAM capacity : 1024 GB RAM
• storage : system 2x223 GB SATA SSD RAID-1 + local scratch data 5x445GB SAS SSD 1.8 TB RAID-5 + controller H745
• 2x gigabit network ports (one connected)
• infiniband EDR card (connected to FDR switch)
• hyperthreading active

Dell C6420 cluster (2019)

Dell C6420 dual-Xeon Cascade Lake SP Gold 6240 @ 2.60GHz (36 cores) : 4 nodes ( nef054 to nef057 )

• RAM capacity : 384 GB RAM
• storage : system 2x600 GB SATA RAID-1 + local scratch data 960 BB SATA SSD RAID-0 + controller H330
• 1x gigabit network port
• 1x infiniband FDR card
• hyperthreading active
• optimal performance with AVX-512, AVX/AVX2 support

Dell R940 node (2017)

Dell R940 quad-Xeon SP Gold 6148 @ 2.40GHz (80 cores) : 1 node ( nef053 )

• RAM capacity : 1024 GB RAM
• storage : system 2x600 GB SATA RAID-1 + local scratch data 1.92 TB SATA SSD + controller H740P
• 4x gigabit network ports (one connected)
• infiniband EDR card (connected to FDR switch)
• hyperthreading active
• optimal performance with AVX-512, AVX/AVX2 support

Dell C6420 cluster (2017)

Dell C6420 dual-Xeon Skylake SP Silver 4114 @ 2.20GHz (20 cores) : 16 nodes ( nef037 to nef052 )

• RAM capacity : 192 GB RAM
• 1x600GB 10kRPM SAS HardDisk drive
• 1x gigabit network port
• 1x infiniband FDR card
• hyperthreading active
• AVX-512 support, optimal performance with AVX/AVX2

Dell C6220 cluster (2015)

Dell C6220 dual-Xeon E5-2650 v2 @ 2.60GHz (16 cores) : 8 nodes ( nef029 to nef036 )

• RAM capacity : 256 GB RAM
• 1x1TB SATA HardDisk drive
• 2x gigabit network ports (one connected)
• 2x infiniband QDR card (one connected)
• hyperthreading not active

Dell C6220 cluster (2014)

Dell C6220 dual-Xeon E5-2680 v2 @ 2.80GHz (20 cores) : 16 nodes ( nef013 to nef028 )

• RAM capacity : 192 GB RAM
• 1x2TB SATA HardDisk drive
• 2x gigabit network ports (one connected)
• 1x infiniband FDR card (QDR used)
• hyperthreading not active

 Dell C6145 cluster (2013)

Dell C6145 quad-Opterons 6376 @ 2.3Ghz (64 cores) : 6 nodes ( nef007 to nef012 )

• RAM capacity : 256 GB RAM (512GB on nef011 and nef012)
• 1x500GB SATA HardDisk drive
• 2x gigabit network ports (one connected)
• 1x infiniband QDR card
• hyperthreading not supported

Dell R815 cluster (2010)

Dell R815 quad-Opterons 6174 @ 2.2Ghz (48 cores) : 6 nodes ( nef001 to nef006 )

• RAM capacity : 256 GB RAM
• 2x600GB SAS HardDisk drive (RAID-0)
• 4x gigabit network ports (one connected)
• 1x infiniband QDR card
• hyperthreading not supported

GPU nodes

HPE DL385 GPU nodes (2023)

HPE DL385 dual-AMD EPYC 7662 @ 2.0GHz (64 cores) : 1 node ( nefgpu56 )

• Nvidia A100 PCIe GPUs cards
  • 6912 CUDA cores per card
  • 432 tensor cores per card
  • 40GB of RAM capacity per card
  • Tensor performance peak : 155.92 TFlops per card
  • Simple precision performance peak: 77.97 Tflops per card
  • Double precision performance peak: 19.49 Tflops per card
  • 1555 GB/s GPU memory bandwidth with error correction (ECC)
• 2x gigabit network ports (one connected)
• infiniband EDR card (connected to FDR switch)
• hyperthreading active

Node details

Node name	Funding team	GPU cards	Node CPU	Node RAM	Node storage
nefgpu56	STARS	3x A100	2x AMD EPYC 7662	1024 GB	} Dell R7525 GPU nodes (2020) Dell R7525 dual-AMD EPYC 7282 @ 2.8GHz (16 cores) : 2 nodes ( nefgpu52 to nefgpu53 ) Dell R7525 dual-AMD EPYC 7413 @ 2.65GHz (24 cores) : 2 nodes ( nefgpu54 to nefgpu55 ) Nvidia A40 PCIe GPUs cards 10752 CUDA cores per card 336 tensor cores per card 48GB of RAM capacity per card Tensor performance peak : 149.6 TFlops per card Simple precision performance peak: 74.8 Tflops per card Double precision performance peak: 37.4 Tflops per card 696 GB/s GPU memory bandwidth with error correction (ECC) 2x gigabit network ports (one connected) infiniband EDR card (connected to FDR switch) hyperthreading active Node details Node name Funding team GPU cards Node CPU Node RAM Node storage nefgpu52 NEO 3x A40 2x AMD EPYC 7282 256 GB system & /tmp : RAID-1 2x 480SSD /local/mixed : RAID-5 5x 480 SSD nefgpu53 ACENTAURI 3x A40 2x AMD EPYC 7282 256 GB system & /tmp : RAID-1 2x 480SSD /local/mixed : RAID-5 5x 480 SSD nefgpu54 3IA 3x A40 2x AMD EPYC 7413 512 GB system & /tmp : RAID-1 2x 480SSD /local/mixed : RAID-5 5x 960 SSD nefgpu55 3IA 3x A40 2x AMD EPYC 7413 512 GB system & /tmp : RAID-1 2x 480SSD /local/mixed : RAID-5 5x 960 SSD SuperMicro 4029 node (2019) SuperMicro 4029GP-TVRT : 1 node (nefgpu41 ) 4x Nvidia Tesla V100 SXM2 GPUs cards 5120 CUDA cores per card 640 tensor cores per card 32GB of RAM capacity per card Simple precision performance peak: 15.7 Tflops per card Double precision performance peak: 7.8 Tflops per card Tensor performance peak : 125 TFlops per card 900 GB/s GPU memory bandwidth with error correction (ECC) CPU-GPU connection with a PCIe gen3 16x interface GPU-GPU connection with NVLink 2.0 (25GB/s per direction per link, 1 or 2 links per card pair) CPU : 2x Xeon SP Gold 6126 @ 2.6 GHz (24 cores) RAM capacity : 384 GB storage : system & /tmp : 240GB SSD SATA RAID-1 /local/tmp scratch data : 4.8 TB SSD SATA RAID-5 RAID controller SAS 12Gb/s 2x 10Gbps ethernet ports (one connected in 1Gb/s) 1x infiniband FDR card hyperthreading active Asus ESC8000 GPU node (2018) Asus ESC8000G4 / Carri HighServer nodes : 1 node (nefgpu40 ) 8x GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface 3584 CUDA cores per card 11GB of RAM capacity per card Simple precision performance peak: 10.6 Tflops per card Double precision performance peak: 0.3 Tflops per card 484 GB/s GPU memory bandwidth PCIe single-root topology (2 PCIe 96 lane switches) topology can be software modified (BIOS & reboot) to dual-root for an experiment campaign CPU : 2x Xeon SP Gold 5115 @ 2.4 GHz RAM capacity : 256 GB storage : system & /tmp : RAID-1 2x512 GB SATA SSD /local/tmp scratch data : RAID-0 4 TB SATA SSD RAID controller SAS 12Gb/s 4x gigabit ethernet ports (one connected) 1x infiniband FDR card hyperthreading active Dell R740 GPU nodes (2019) Dell R740 nodes: dual-Xeon Skylake or CascadeLake SP : 5 nodes ( nefgpu42 to nefgpu46) Nvidia Tesla V100 PCIe GPUs cards 5120 CUDA cores per card 640 tensor cores per card 32GB of RAM capacity per card Tensor performance peak : 112 TFlops per card Simple precision performance peak: 14 Tflops per card Double precision performance peak: 7 Tflops per card 900 GB/s GPU memory bandwidth with error correction (ECC) Nvidia Tesla T4 GPUs cards connected with a PCIe gen3 16x interface 2560 CUDA cores per card 320 tensor cores per card 16GB of RAM capacity per card Simple precision performance peak: 8.1 Tflops per card Double precision performance peak: 0.3 Tflops per card 300 GB/s GPU memory bandwidth with error correction (ECC) 2-4x ethernet ports (one connected in gigabit) 1x infiniband FDR card hyperthreading active Node details Node name Funding team GPU cards Node CPU Node RAM Node storage nefgpu42 common 3x Tesla T4 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 2x 480GB SSD nefgpu43 common 3x Tesla T4 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 2x 480GB SSD nefgpu44 common 3x Tesla T4 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 2x 480GB SSD nefgpu45 common 3x Tesla T4 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 2x 480GB SSD nefgpu46 STARS 3x Tesla V100 2x Xeon Silver 4215 384 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD Dell T640 GPU nodes (2018-2019-2020) Dell T640 nodes: dual-Xeon Skylake or CascadeLake SP : 21 nodes ( nefgpu{24-39} and nefgpu{47-51}) Nvidia RTX A6000 GPUs cards connected with a PCIe gen3 16x interface (Ampere) 10752 CUDA cores per card 336 tensor cores per card 48GB of RAM capacity per card Tensor performance peak : 309.7 TFlops per card Simple precision performance peak: 38.7 Tflops per card 768 GB/s GPU memory bandwidth with error correction (ECC) Nvidia Quadro RTX 6000 GPUs cards connected with a PCIe gen3 16x interface (Turing) 4608 CUDA cores per card 576 tensor cores per card 24GB of RAM capacity per card Tensor performance peak : 130.5 TFlops per card Half precision performance peak: 32.6 Tflops per card Simple precision performance peak: 16.3 Tflops per card Double precision performance peak: 0.5 Tflops per card 624 GB/s GPU memory bandwidth with error correction (ECC) optional NVLink 2.0 (2x25GB/s per direction) by pair of card Nvidia Quadro RTX 8000 GPUs cards connected with a PCIe gen3 16x interface (Turing) 4608 CUDA cores per card 576 tensor cores per card 48GB of RAM capacity per card Tensor performance peak : 130.5 TFlops per card Half precision performance peak: 32.6 Tflops per card Simple precision performance peak: 16.3 Tflops per card Double precision performance peak: 0.5 Tflops per card 672 GB/s GPU memory bandwidth with error correction (ECC) optional NVLink 2.0 (2x25GB/s per direction) by pair of card GeForce RTX 2080 Ti GPUs cards connected with a PCIe gen3 16x interface 4352 CUDA cores per card 544 tensor cores per card 11GB of RAM capacity per card Half precision performance peak: 23.5 Tflops per card Simple precision performance peak: 11.8 Tflops per card Double precision performance peak: 0.4 Tflops per card 616 GB/s GPU memory bandwidth GeForce Titan RTX GPUs cards connected with a PCIe gen3 16x interface 4608 CUDA cores per card 576 tensor cores per card 24GB of RAM capacity per card Half precision performance peak: 24.9 Tflops per card Simple precision performance peak: 12.4 Tflops per card Double precision performance peak: 0.4 Tflops per card 672 GB/s GPU memory bandwidth Power limited to 210W to prevent overheat GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface 3584 CUDA cores per card 11GB of RAM capacity per card Simple precision performance peak: 10.6 Tflops per card Double precision performance peak: 0.3 Tflops per card 484 GB/s GPU memory bandwidth GeForce GTX Titan X GPUs cards connected with a PCIe gen3 16x interface 3072 CUDA cores per card 12GB of RAM capacity per card Simple precision performance peak: 7.0 Tflops per card Double precision performance peak: 0.2 Tflops per card 336.5 GB/s GPU memory bandwidth 2-4x ethernet ports (one connected in gigabit) 1x infiniband FDR card hyperthreading active Node details Node name Funding team GPU cards Node CPU Node RAM Node storage nefgpu24 EPIONE 4x GTX 1080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : RAID-0 1x 960GB SSD nefgpu25 ZENITH 4x GTX 1080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-0 4x 480GB SSD /local/read : RAID-0 8x 480GB SSD read intensive nefgpu26 WIMMICS 4x GTX 1080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : RAID-0 4x 480GB SSD nefgpu27 ZENITH 4x RTX 2080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-0 2x 400GB SSD /local/read : RAID-0 4x 480GB SSD read intensive nefgpu28 GRAPHDECO 4x RTX A6000 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 400GB SSD nefgpu29 EPIONE 2x GTX Titan-X 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 480GB SSD nefgpu30 common 4x RTX 2080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 4x 480GB SSD nefgpu31 common 4x RTX 2080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 4x 480GB SSD nefgpu32 common 4x Titan RTX 2x Xeon Silver 4110 192 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : RAID-0 4x 480GB SSD nefgpu33 CHORALE 4x RTX 2080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-0 1x 480GB SSD nefgpu34 GRAPHDECO 4x RTX 6000 w/ NVLink by pair 2x Xeon Gold 6240 384 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD nefgpu35 STARS 4x RTX 6000 w/ NVLink by pair 2x Xeon Silver 4215 192 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD nefgpu36 STARS 4x RTX 6000 w/ NVLink by pair 2x Xeon Silver 4215 192 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD nefgpu37 GRAPHDECO 4x RTX 8000 w/ NVLink by pair 2x Xeon Gold 6240 384 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD nefgpu38 AYANA 2x RTX 8000 w/ NVLink by pair 2x Xeon Silver 4214R 384 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 960GB SSD nefgpu39 STARS 4x RTX 8000 w/ NVLink by pair 2x Xeon Silver 4216 384 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/mixed : RAID-5 5x 900GB SATA SSD nefgpu47 I3S/SPARKS 4x RTX 8000 w/ NVLink by pair 2x Xeon Gold 6230R 384 GB system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 900GB SSD nefgpu48 I3S/SPARKS 4x RTX 8000 w/ NVLink by pair 2x Xeon Gold 6230R 384 GB system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 900GB SSD nefgpu49 I3S/SPARKS 4x RTX 8000 w/ NVLink by pair 2x Xeon Gold 6238R 768 GB system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 1.8TB SSD nefgpu50 I3S/SPARKS 4x RTX 8000 w/ NVLink by pair 2x Xeon Gold 6230R 384 GB system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 900GB SSD nefgpu51 I3S/SPARKS 4x RTX 8000 w/ NVLink by pair 2x Xeon Gold 6230R 384 GB system & /tmp : RAID-1 2x 450GB SATA SSD /local/mixed : RAID-5 5x 900GB SSD Dell T630 GPU nodes (2016-2017) Dell T630 nodes: dual-Xeon E5-26xx : 17 nodes ( nefgpu07 to nefgpu23) GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface 3584 CUDA cores per card 11GB of RAM capacity per card Simple precision performance peak: 10.6 Tflops per card Double precision performance peak: 0.3 Tflops per card 484 GB/s GPU memory bandwidth GeForce GTX 1080 GPUs cards connected with a PCIe gen3 16x interface 2560 CUDA cores per card 8GB of RAM capacity per card Simple precision performance peak: 8.2 Tflops per card Double precision performance peak: 0.3 Tflops per card 320 GB/s GPU memory bandwidth GeForce GTX Titan X GPUs cards connected with a PCIe gen3 16x interface 3072 CUDA cores per card 12GB of RAM capacity per card Simple precision performance peak: 7.0 Tflops per card Double precision performance peak: 0.2 Tflops per card 336.5 GB/s GPU memory bandwidth 4x gigabit ethernet ports (one connected) 1x infiniband FDR card Node details Node name Funding team GPU cards Node CPU Node RAM Node storage Hyper threading active ? nefgpu07 EPIONE 4x GTX 1080 Ti 2x E5-2620v3 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM no nefgpu08 ZENITH 4x GTX 1080 Ti 2x E5-2630v3 64 GB system & /tmp : RAID-0 2x 300GB SAS 15kRPM /local/mixed : 1x 800GB SSD /local/read : RAID-0 2x 1.92TB SSD read intensive no nefgpu09 GRAPHDECO 4x Titan X 2x E5-2630v4 48 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 400GB SSD no nefgpu10 STARS 4x Titan X 2x E5-2630v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 1.6TB SSD no nefgpu11 STARS 4x GTX 1080 2x E5-2630v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 1.6TB SSD no nefgpu12 STARS 4x GTX 1080 2x E5-2630v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 1.6TB SSD yes nefgpu13 GRAPHDECO 4x GTX 1080 Ti 2x E5-2650v4 64 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 400GB SSD yes nefgpu14 STARS 4x GTX 1080 Ti 2x E5-2620v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 400GB SSD yes nefgpu15 STARS 4x GTX 1080 Ti 2x E5-2620v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM /local : 1x 400GB SSD yes nefgpu16 EPIONE 4x GTX 1080 Ti 2x E5-2630v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : 1x 1.6TB SSD yes nefgpu17 ZENITH 4x GTX 1080 Ti 2x E5-2630v4 64 GB system & /tmp : RAID-0 2x 600GB SAS 10kRPM /local/mixed : 1x 1.6TB SSD /local/read : RAID-0 2x 1.92TB SSD read intensive yes nefgpu18 common 4x GTX 1080 Ti 2x E5-2630v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : 1x 1.6TB SSD yes nefgpu19 common 4x GTX 1080 Ti 2x E5-2630v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : 1x 1.6TB SSD yes nefgpu20 common 4x GTX 1080 Ti 2x E5-2630v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local/tmp : 1x 1.6TB SSD yes nefgpu21 STARS 4x GTX 1080 Ti 2x E5-2620v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : 1x 480GB SSD yes nefgpu22 STARS 3x GTX 1080 Ti 2x E5-2620v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : 1x 480GB SSD yes nefgpu23 TITANE-EPITOME 4x GTX 1080 Ti 2x E5-2630v4 64 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM /local : 1x 1.6TB SSD yes Dell R730 GPU node (2016) Dell R730 nodes: dual-Xeon E5-2623v4 @ 2.6 GHz : 1 node ( nefgpu01) Tesla K80 GPU cards connected with a PCIe gen3 16x interface 2x Tesla GK210 GPUs per card 4992 CUDA cores per card 2x 12GB RAM capacity per card with error correction (ECC) Simple precision performance peak: 5.61 Tflops per card Double precision performance peak: 1.87 Tflops per card 2x 240 GB/s GPU memory bandwidth with error correction (ECC) 4x gigabit ethernet ports (one connected) 1x infiniband QDR card hyperthreading not active Node details Node name Funding team Number of GPU cards Node CPU Node RAM Node storage nefgpu01 MATHNEURO 1x K80 2x E5-2623v4 32 GB 2x 400GB SSD  Storage All nodes have access to common storage : common storage : /home 31 TiB, available to all users, quotas per user 1 Dell PowerEdge R540 server with 2 RAID-10 array 10 x 10TB SAS 7.2 kRPM disks, infiniband FDR, NFS access capacity distributed and scalable common storage : /data ~610TiB real capacity, 684TB theoretical capacity (06/2019) permanent storage : 1TiB quota per team + teams may buy additional quota (please contact cluster administrators) scratch storage : variable size (initially ~40TiB), no quota limit, for temporary storage (data may be purged) BeeGFS filesystem on multiple hardware : 4 Dell PowerEdge R740xd ; 1.92TB metadata : RAID-1 array 2 x 1.92TB SSD mixed use MLC disks (or RAID-10 array 4 x 960GB) ; 2x {60 or 72}TB data : 2 x RAID-6 array 8 x {10 or 12}TB SAS 7.2 kRPM disks 2 Dell PowerEdge R730xd ; 800GB metadata : RAID-1 array 2 x 800 GB SSD mixed use MLC disks ; 2 x {36 or 48}TB data: 2 x RAID-6 array 8 x {6 or 8}TB SAS 7.2 kRPM disks infiniband FDR More details about quotas at: Disk space management