Hardware : Différence entre versions

De ClustersSophia
Aller à : navigation, rechercher
Ligne 13 : Ligne 13 :
 
*    6 quad-Opteron 16 cores, 4GB/core RAM
 
*    6 quad-Opteron 16 cores, 4GB/core RAM
 
*    6 quad-Opteron 12 cores, ~5.3GB/core RAM
 
*    6 quad-Opteron 12 cores, ~5.3GB/core RAM
*    138 Nvidia GPU
+
*    144 Nvidia GPU
  
 
The number of CPU cores available is :
 
The number of CPU cores available is :
Ligne 27 : Ligne 27 :
  
 
The number of GPU cores available is :
 
The number of GPU cores available is :
*    '''467392''' Streaming Processor Cores  '''~1341.8 Tflops peak in single precision''' ( ~75.2 Tflops peak in double precision )
+
*    '''496064''' Streaming Processor Cores  '''~1435.0 Tflops peak in single precision''' ( ~91.2 Tflops peak in double precision )
  
  
Ligne 190 : Ligne 190 :
 
== Dell R740 GPU nodes ==
 
== Dell R740 GPU nodes ==
  
'''Dell R740''' nodes: dual-Xeon Skylake SP : 4 nodes ( '''nefgpu42 to nefgpu45''')  
+
'''Dell R740''' nodes: dual-Xeon Skylake or CascadeLake SP : 5 nodes ( '''nefgpu42 to nefgpu46''')
 +
* Nvidia Tesla V100 PCIe GPUs cards
 +
**        5120 CUDA cores per card
 +
**        640 tensor cores per card
 +
**        32GB of RAM capacity per card
 +
**        Tensor performance peak : 112 TFlops per card
 +
**        Simple precision performance peak: 14 Tflops per card
 +
**        Double precision performance peak: 7 Tflops per card
 +
**        900 GB/s GPU memory bandwidth with error correction (ECC)
 
* Nvidia Tesla T4 GPUs cards connected with a PCIe gen3 16x interface
 
* Nvidia Tesla T4 GPUs cards connected with a PCIe gen3 16x interface
 
**        2560 CUDA cores per card
 
**        2560 CUDA cores per card
Ligne 239 : Ligne 247 :
 
| 96 GB
 
| 96 GB
 
| system & /tmp : RAID-1 2x 600GB SAS 10kRPM<br>/local : RAID-0 2x 480GB SSD
 
| system & /tmp : RAID-1 2x 600GB SAS 10kRPM<br>/local : RAID-0 2x 480GB SSD
 +
|-
 +
| nefgpu46
 +
| STARS
 +
| 2x Tesla V100
 +
| 2x Xeon Silver 4215
 +
| 384 GB
 +
| system & /tmp : RAID-1 2x 600GB SAS 10kRPM<br>/local : RAID-5 5x 960GB SSD
 
|}
 
|}
  
Ligne 249 : Ligne 264 :
 
**        576 tensor cores per card
 
**        576 tensor cores per card
 
**        24GB of RAM capacity per card  
 
**        24GB of RAM capacity per card  
**        Tensor cores mixed FP16/FP32 performance : 130.5 TFlops per card
+
**        Tensor performance peak : 130.5 TFlops per card
 
**        Half precision performance peak: 32.6 Tflops per card
 
**        Half precision performance peak: 32.6 Tflops per card
 
**        Simple precision performance peak: 16.3 Tflops per card
 
**        Simple precision performance peak: 16.3 Tflops per card
Ligne 370 : Ligne 385 :
 
| nefgpu34
 
| nefgpu34
 
| GRAPHDECO
 
| GRAPHDECO
| 4x RTX 6000 w/ NVLink by pair
+
| 4x RTX 6000<br> w/ NVLink by pair
 
| 2x Xeon Gold 6240
 
| 2x Xeon Gold 6240
 
| 384 GB
 
| 384 GB

Version du 22 novembre 2019 à 17:30



CPU & GPU summary

The current platform includes:

  • 16 dual-Xeon SP 10 cores, 9.6GB/core RAM
  • 1 quad-Xeon SP 20 cores, 12.8GB/core RAM
  • 8 dual-Xeon 8 cores, 16GB/core RAM
  • 16 dual-Xeon 10 cores, 9.6GB/core RAM
  • 6 quad-Opteron 16 cores, 4GB/core RAM
  • 6 quad-Opteron 12 cores, ~5.3GB/core RAM
  • 144 Nvidia GPU

The number of CPU cores available is :

  • 144 Xeon SP Gold 2.6GHz cores : ~6.7 TFlops AVX-512 (9.2 TFlops peak)
  • 320 Xeon SP Silver 2.2GHz cores : ~8.8 TFlops AVX2 (11.2 TFlops peak) / ~6.4 TFlops AVX-512 (9.7 TFlops peak)
  • 80 Xeon SP Gold 2.4GHz cores : ~2.4 TFlops AVX2 (3.1 Tflops peak) / ~3.8 Tflops AVX-512 (4.9 Tflops peak)
  • 128 Xeon E5 v2 2.6GHz cores : ~2.4 Tflops (2.7 Tflops peak)
  • 320 Xeon E5 v2 2.8GHz cores : ~5.9 Tflops (7.2 Tflops peak)
  • 384 Opteron 2.3GHz cores : ~2.7 Tflops (3.5 Tflops peak)
  • 288 Opteron 2.2GHz cores : ~2 Tflops (2.5 Tflops peak)

The total cumulated CPU computing power is close to ~35.2 TFlops in double precision (41.1 Tflops peak)


The number of GPU cores available is :

  • 496064 Streaming Processor Cores ~1435.0 Tflops peak in single precision ( ~91.2 Tflops peak in double precision )



CPU clusters

Dell C6420 cluster (2019)

Dell C6420 dual-Xeon Cascade Lake SP Gold 6240 @ 2.60GHz (36 cores) : 4 nodes ( nef054 to nef057 )

C6420.jpg


  • RAM capacity : 384 GB RAM
  • storage : system 2x600 GB SATA RAID-1 + local scratch data 960 BB SATA SSD RAID-0 + controller H330
  • 1x gigabit network port
  • 1x infiniband FDR card
  • hyperthreading active
  • optimal performance with AVX-512, AVX/AVX2 support


Dell C6420 cluster (2017)

Dell C6420 dual-Xeon Skylake SP Silver 4114 @ 2.20GHz (20 cores) : 16 nodes ( nef037 to nef052 )

C6420.jpg


  • RAM capacity : 192 GB RAM
  • 1x600GB 10kRPM SAS HardDisk drive
  • 1x gigabit network port
  • 1x infiniband FDR card
  • hyperthreading active
  • AVX-512 support, optimal performance with AVX/AVX2


Dell R940 node

Dell R940 quad-Xeon SP Gold 6148 @ 2.40GHz (80 cores) : 1 node ( nef053 )

Dellr940.jpg
  • RAM capacity : 1024 GB RAM
  • storage : system 2x600 GB SATA RAID-1 + local scratch data 1.92 TB SATA SSD + controller H740P
  • 4x gigabit network ports (one connected)
  • infiniband EDR card (connected to FDR switch)
  • hyperthreading active
  • optimal performance with AVX-512, AVX/AVX2 support


Dell C6220 cluster (2015)

Dell C6220 dual-Xeon E5-2650 v2 @ 2.60GHz (16 cores) : 8 nodes ( nef029 to nef036 )

C6220-300x191.jpg


  • RAM capacity : 256 GB RAM
  • 1x1TB SATA HardDisk drive
  • 2x gigabit network ports (one connected)
  • 2x infiniband QDR card (one connected)
  • hyperthreading not active


Dell C6220 cluster (2014)

Dell C6220 dual-Xeon E5-2680 v2 @ 2.80GHz (20 cores) : 16 nodes ( nef013 to nef028 )

C6220-300x191.jpg


  • RAM capacity : 192 GB RAM
  • 1x2TB SATA HardDisk drive
  • 2x gigabit network ports (one connected)
  • 1x infiniband FDR card (QDR used)
  • hyperthreading not active


 Dell C6145 cluster

Dell C6145 quad-Opterons 6376 @ 2.3Ghz (64 cores) : 6 nodes ( nef007 to nef012 )

Cluster-c6145.jpg


  • RAM capacity : 256 GB RAM (512GB on nef011 and nef012)
  • 1x500GB SATA HardDisk drive
  • 2x gigabit network ports (one connected)
  • 1x infiniband QDR card
  • hyperthreading not supported


Dell R815 cluster

Dell R815 quad-Opterons 6174 @ 2.2Ghz (48 cores) : 6 nodes ( nef001 to nef006 )

Cluster-dellr815.jpg


  • RAM capacity : 256 GB RAM
  • 2x600GB SAS HardDisk drive (RAID-0)
  • 4x gigabit network ports (one connected)
  • 1x infiniband QDR card
  • hyperthreading not supported


GPU nodes

SuperMicro 4029 node

SuperMicro 4029GP-TVRT : 1 node (nefgpu41 )

SYS-4028GR-TVRT.jpg
  • 4x Nvidia Tesla V100 SXM2 GPUs cards
    • 5120 CUDA cores per card
    • 640 tensor cores per card
    • 32GB of RAM capacity per card
    • Simple precision performance peak: 15.7 Tflops per card
    • Double precision performance peak: 7.8 Tflops per card
    • Tensor performance peak : 125 TFlops per card
    • 900 GB/s GPU memory bandwidth with error correction (ECC)
  • CPU-GPU connection with a PCIe gen3 16x interface
  • GPU-GPU connection with NVLink 2.0 (25GB/s per direction per link, 1 or 2 links per card pair)
  • CPU : 2x Xeon SP Gold 6126 @ 2.6 GHz
  • RAM capacity : 384 GB
  • storage :
    • system & /tmp : 240GB SSD SATA RAID-1
    • /local/tmp scratch data : 4.8 TB SSD SATA RAID-5
    • RAID controller SAS 12Gb/s
  • 2x 10Gbps ethernet ports (one connected in 1Gb/s)
  • 1x infiniband FDR card
  • hyperthreading active


Asus ESC8000 GPU node

Asus ESC8000G4 / Carri HighServer nodes : 1 node (nefgpu40 )

  • 8x GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface
    • 3584 CUDA cores per card
    • 11GB of RAM capacity per card
    • Simple precision performance peak: 10.6 Tflops per card
    • Double precision performance peak: 0.3 Tflops per card
    • 484 GB/s GPU memory bandwidth
  • PCIe single-root topology (2 PCIe 96 lane switches)
    • topology can be software modified (BIOS & reboot) to dual-root for an experiment campaign
  • CPU : 2x Xeon SP Gold 5115 @ 2.4 GHz
  • RAM capacity : 256 GB
  • storage :
    • system & /tmp : RAID-1 2x512 GB SATA SSD
    • /local/tmp scratch data : RAID-0 4 TB SATA SSD
    • RAID controller SAS 12Gb/s
  • 4x gigabit ethernet ports (one connected)
  • 1x infiniband FDR card
  • hyperthreading active


 Dell R740 GPU nodes

Dell R740 nodes: dual-Xeon Skylake or CascadeLake SP : 5 nodes ( nefgpu42 to nefgpu46)

  • Nvidia Tesla V100 PCIe GPUs cards
    • 5120 CUDA cores per card
    • 640 tensor cores per card
    • 32GB of RAM capacity per card
    • Tensor performance peak : 112 TFlops per card
    • Simple precision performance peak: 14 Tflops per card
    • Double precision performance peak: 7 Tflops per card
    • 900 GB/s GPU memory bandwidth with error correction (ECC)
  • Nvidia Tesla T4 GPUs cards connected with a PCIe gen3 16x interface
    • 2560 CUDA cores per card
    • 320 tensor cores per card
    • 16GB of RAM capacity per card
    • Simple precision performance peak: 8.1 Tflops per card
    • Double precision performance peak: 0.3 Tflops per card
    • 300 GB/s GPU memory bandwidth
  • 2-4x ethernet ports (one connected in gigabit)
  • 1x infiniband FDR card
  • hyperthreading active
Node details
Node name Funding team GPU cards Node CPU Node RAM Node storage
nefgpu42 common 3x Tesla T4 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : RAID-0 2x 480GB SSD
nefgpu43 common 3x Tesla T4 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : RAID-0 2x 480GB SSD
nefgpu44 common 3x Tesla T4 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : RAID-0 2x 480GB SSD
nefgpu45 common 3x Tesla T4 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : RAID-0 2x 480GB SSD
nefgpu46 STARS 2x Tesla V100 2x Xeon Silver 4215 384 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : RAID-5 5x 960GB SSD


 Dell T640 GPU nodes

Dell T640 nodes: dual-Xeon Skylake or CascadeLake SP : 11 nodes ( nefgpu24 to nefgpu34)

  • Nvidia Quadro RTX 6000 GPUs cards connected with a PCIe gen3 16x interface
    • 4608 CUDA cores per card
    • 576 tensor cores per card
    • 24GB of RAM capacity per card
    • Tensor performance peak : 130.5 TFlops per card
    • Half precision performance peak: 32.6 Tflops per card
    • Simple precision performance peak: 16.3 Tflops per card
    • Double precision performance peak: 0.5 Tflops per card
    • 624 GB/s GPU memory bandwidth
    • optional NVLink 2.0 (2x25GB/s per direction) by pair of card
  • GeForce RTX 2080 Ti GPUs cards connected with a PCIe gen3 16x interface
    • 4352 CUDA cores per card
    • 544 tensor cores per card
    • 11GB of RAM capacity per card
    • Half precision performance peak: 23.5 Tflops per card
    • Simple precision performance peak: 11.8 Tflops per card
    • Double precision performance peak: 0.4 Tflops per card
    • 616 GB/s GPU memory bandwidth
  • GeForce Titan RTX GPUs cards connected with a PCIe gen3 16x interface
    • 4608 CUDA cores per card
    • 576 tensor cores per card
    • 24GB of RAM capacity per card
    • Half precision performance peak: 24.9 Tflops per card
    • Simple precision performance peak: 12.4 Tflops per card
    • Double precision performance peak: 0.4 Tflops per card
    • 672 GB/s GPU memory bandwidth
    • Power limited to 210W to prevent overheat
  • GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface
    • 3584 CUDA cores per card
    • 11GB of RAM capacity per card
    • Simple precision performance peak: 10.6 Tflops per card
    • Double precision performance peak: 0.3 Tflops per card
    • 484 GB/s GPU memory bandwidth
  • GeForce GTX Titan X GPUs cards connected with a PCIe gen3 16x interface
    • 3072 CUDA cores per card
    • 12GB of RAM capacity per card
    • Simple precision performance peak: 7.0 Tflops per card
    • Double precision performance peak: 0.2 Tflops per card
    • 336.5 GB/s GPU memory bandwidth
  • 2-4x ethernet ports (one connected in gigabit)
  • 1x infiniband FDR card
  • hyperthreading active
Node details
Node name Funding team GPU cards Node CPU Node RAM Node storage
nefgpu24 EPIONE 4x GTX 1080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : RAID-0 1x 960GB SSD
nefgpu25 ZENITH 4x GTX 1080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-0 4x 480GB SSD
/local/read : RAID-0 8x 480GB SSD read intensive
nefgpu26 WIMMICS 4x GTX 1080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : RAID-0 4x 480GB SSD
nefgpu27 ZENITH 4x RTX 2080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-0 2x 400GB SSD
/local/read : RAID-0 4x 480GB SSD read intensive
nefgpu28 GRAPHDECO 4x GTX 1080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-5 5x 400GB SSD
nefgpu29 EPIONE 2x GTX Titan-X 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-5 5x 480GB SSD
nefgpu30 common 4x RTX 2080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-0 4x 480GB SSD
nefgpu31 common 4x RTX 2080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-0 4x 480GB SSD
nefgpu32 common 4x Titan RTX 2x Xeon Silver 4110 192 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-0 4x 480GB SSD
nefgpu33 CHORALE 4x RTX 2080 Ti 2x Xeon Silver 4110 96 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-0 1x 480GB SSD
nefgpu34 GRAPHDECO 4x RTX 6000
w/ NVLink by pair
2x Xeon Gold 6240 384 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/mixed : RAID-5 5x 960GB SSD


 Dell T630 GPU nodes

Dell T630 nodes: dual-Xeon E5-26xx : 17 nodes ( nefgpu07 to nefgpu23)

  • GeForce GTX 1080 Ti GPUs cards connected with a PCIe gen3 16x interface
    • 3584 CUDA cores per card
    • 11GB of RAM capacity per card
    • Simple precision performance peak: 10.6 Tflops per card
    • Double precision performance peak: 0.3 Tflops per card
    • 484 GB/s GPU memory bandwidth
  • GeForce GTX 1080 GPUs cards connected with a PCIe gen3 16x interface
    • 2560 CUDA cores per card
    • 8GB of RAM capacity per card
    • Simple precision performance peak: 8.2 Tflops per card
    • Double precision performance peak: 0.3 Tflops per card
    • 320 GB/s GPU memory bandwidth
  • GeForce GTX Titan X GPUs cards connected with a PCIe gen3 16x interface
    • 3072 CUDA cores per card
    • 12GB of RAM capacity per card
    • Simple precision performance peak: 7.0 Tflops per card
    • Double precision performance peak: 0.2 Tflops per card
    • 336.5 GB/s GPU memory bandwidth
  • 4x gigabit ethernet ports (one connected)
  • 1x infiniband FDR card
Node details
Node name Funding team GPU cards Node CPU Node RAM Node storage Hyper threading active ?
nefgpu07 EPIONE 4x GTX 1080 Ti 2x E5-2620v3 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM no
nefgpu08 ZENITH 4x GTX 1080 Ti 2x E5-2630v3 64 GB system & /tmp : RAID-0 2x 300GB SAS 15kRPM
/local/mixed : 1x 800GB SSD
/local/read : RAID-0 2x 1.92TB SSD read intensive
no
nefgpu09 GRAPHDECO 4x Titan X 2x E5-2630v4 48 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM
/local : 1x 400GB SSD
no
nefgpu10 STARS 4x Titan X 2x E5-2630v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM
/local : 1x 1.6TB SSD
no
nefgpu11 STARS 4x GTX 1080 2x E5-2630v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM
/local : 1x 1.6TB SSD
no
nefgpu12 STARS 4x GTX 1080 2x E5-2630v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM
/local : 1x 1.6TB SSD
yes
nefgpu13 GRAPHDECO 4x GTX 1080 Ti 2x E5-2650v4 64 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM
/local : 1x 400GB SSD
yes
nefgpu14 STARS 4x GTX 1080 Ti 2x E5-2620v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM
/local : 1x 400GB SSD
yes
nefgpu15 STARS 4x GTX 1080 Ti 2x E5-2620v4 128 GB system & /tmp : RAID-0 2x 1TB SATA 7.2kRPM
/local : 1x 400GB SSD
yes
nefgpu16 EPIONE 4x GTX 1080 Ti 2x E5-2630v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : 1x 1.6TB SSD
yes
nefgpu17 ZENITH 4x GTX 1080 Ti 2x E5-2630v4 64 GB system & /tmp : RAID-0 2x 600GB SAS 10kRPM
/local/mixed : 1x 1.6TB SSD
/local/read : RAID-0 2x 1.92TB SSD read intensive
yes
nefgpu18 common 4x GTX 1080 Ti 2x E5-2630v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/tmp : 1x 1.6TB SSD
yes
nefgpu19 common 4x GTX 1080 Ti 2x E5-2630v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/tmp : 1x 1.6TB SSD
yes
nefgpu20 common 4x GTX 1080 Ti 2x E5-2630v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local/tmp : 1x 1.6TB SSD
yes
nefgpu21 STARS 4x GTX 1080 Ti 2x E5-2620v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : 1x 480GB SSD
yes
nefgpu22 STARS 3x GTX 1080 Ti 2x E5-2620v4 128 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : 1x 480GB SSD
yes
nefgpu23 TITANE-EPITOME 4x GTX 1080 Ti 2x E5-2630v4 64 GB system & /tmp : RAID-1 2x 600GB SAS 10kRPM
/local : 1x 1.6TB SSD
yes


Dell R730 GPU node

Dell R730 nodes: dual-Xeon E5-26xx : 1 node ( nefgpu01)

  • Tesla K80 GPU cards connected with a PCIe gen3 16x interface
    • 2x Tesla GK210 GPUs per card
    • 4992 CUDA cores per card
    • 2x 12GB RAM capacity per card with error correction (ECC)
    • Simple precision performance peak: 5.61 Tflops per card
    • Double precision performance peak: 1.87 Tflops per card
    • 2x 240 GB/s GPU memory bandwidth
  • 4x gigabit ethernet ports (one connected)
  • 1x infiniband QDR card
  • hyperthreading not active
Node details
Node name Funding team Number of GPU cards Node CPU Node RAM Node storage
nefgpu01 MATHNEURO 1x K80 2x E5-2623v4 32 GB 2x 400GB SSD


 Carri GPU nodes

Carri HighStation 5600 XLR8 nodes: dual-Xeon X5650 @ 2.66Ghz : 1 node ( nefgpu03 )

Carri-tesla.jpg
  • RAM capacity: 72 GB
  • 2x160GB SSD disks
  • 4x gigabit ethernet ports (one connected)
  • 1x infiniband QDR card
  • 7 Tesla C2070 (nefgpu03) GPUs connected with a PCIe gen2 16x interface
    • 448 Streaming Processor Cores per card
    • Simple precision performance peak: 1.03 Tflops per card
    • Double precision performance peak: 0.51 Tflops per card
    • 3GB of RAM capacity per card (6GB on nefgpu03)
  • hyperthreading not active


 Storage

All nodes have access to common storage :

  • common storage : /home
    • 15 TiB, available to all users, quotas
    • 1 Dell PowerEdge R520 server with 2 RAID-10 array 4 x 4TB SAS 7.2 kRPM disks, infiniband QDR, NFS access
  • capacity distributed and scalable common storage : /data
    • ~610TiB real capacity, 684TB theoretical capacity (06/2019)
      • permanent storage : 1TiB quota per team + teams may buy additional quota (please contact cluster administrators)
      • scratch storage : variable size (initially ~40TiB), no quota limit, for temporary storage (data may be purged)
    • BeeGFS filesystem on multiple hardware :
      • 4 Dell PowerEdge R740xd ; 1.92TB metadata : RAID-1 array 2 x 1.92TB SSD mixed use MLC disks (or RAID-10 array 4 x 960GB) ; 2x {60 or 72}TB data : 2 x RAID-6 array 8 x {10 or 12}TB SAS 7.2 kRPM disks
      • 2 Dell PowerEdge R730xd ; 800GB metadata : RAID-1 array 2 x 800 GB SSD mixed use MLC disks ; 2 x {36 or 48}TB data: 2 x RAID-6 array 8 x {6 or 8}TB SAS 7.2 kRPM disks
      • infiniband FDR