|
|
| Line 12: |
Line 12: |
| | }} | | }} |
| | | | |
| − | The Cell Development Cluster is a technology evaluation cluster with a combination of 14 IBM PowerXCell 8i "Cell" nodes and 8 Intel x86_64 "Nehalem" nodes.
| + | The Intel nodes have two 2.53GHz 4core Xeon X5550 CPU's with 48GB of RAM per node with 3 containing NVIDIA 9800GT GPUs. |
| − | The QS22's each have two 3.2GHz "IBM PowerXCell 8i CPU's, where each CPU has 1 Power Processing Unit (PPU) and 8 Synergistic Processing Units (SPU), and
| |
| − | 32GB of RAM per node. The Intel nodes have two 2.53GHz 4core Xeon X5550 CPU's with 48GB of RAM per node.
| |
| | | | |
| | ===Login=== | | ===Login=== |
| Line 23: |
Line 21: |
| | ==Compile/Devel/Compute Nodes== | | ==Compile/Devel/Compute Nodes== |
| | | | |
| − | === Cell === | + | === Nehalem (x86_64) === |
| − | You can log into any of 12 nodes '''<tt>blade[03-14]</tt>''' directly to compile/test/run Cell | + | You can log into any of 8 nodes '''<tt>cell-srv[01-08]</tt>''' directly however each |
| − | specific or OpenCL codes.
| + | of the nodes have differing configurations. |
| | | | |
| − | === Nehalem (x86_64) ===
| + | * '''<tt>cell-srv01</tt>''' - login node & nfs server, GigE connected |
| − | You can log into any of 8 nodes '''<tt>cell-srv[01-08]</tt>''' directly, however they
| + | * '''<tt>cell-srv[02-05]</tt>''' - no GPU, GigE connected |
| − | are not configured to work with the Cell blades yet.
| + | * '''<tt>cell-srv[06-07]</tt>''' - 1x NVIDIA 9800GT GPU, Infiniband connected |
| | + | * '''<tt>cell-srv08</tt>''' - 2x NVIDIA 9800GT GPU, GigE connected |
| | | | |
| | ==Local Disk== | | ==Local Disk== |
| Line 37: |
Line 36: |
| | ==Programming Frameworks== | | ==Programming Frameworks== |
| | | | |
| − | Currently there are two programming frameworks to use. The Cell SDK is the most full featured and "supported" but is architecture specific, whereas the OpenCL should allow code portability to other accelerators (like GP-GPU's) however is from IBM alphaworks so should be considered beta. | + | Currently there are two programming frameworks to use, NVIDIA's CUDA framework or OpenCL. |
| | | | |
| − | === OpenCL === | + | === CUDA === |
| | | | |
| − | Examples are in
| |
| | <pre> | | <pre> |
| − | /usr/share/doc/OpenCL-0.1-ibm | + | /usr/local/cuda/ |
| | </pre> | | </pre> |
| | | | |
| − | === Cell SDK === | + | === OpenCL === |
| | Demos, examples, and build details available in | | Demos, examples, and build details available in |
| − |
| |
| − | <pre>
| |
| − | /opt/cell/sdk/
| |
| − | </pre>
| |
| | | | |
| | ===Compilers=== | | ===Compilers=== |
| | | | |
| − | NOTE: gcc has been replaced, use ppu-gcc for most cases where gcc whould be used
| + | * '''nvcc''' -- Nvidia GCC compiler |
| − | | |
| − | * '''ppu-gcc''' -- GNU GCC PPC compiler | |
| | | | |
| − | * '''spu-gcc''' -- GNU GCC SPU compiler
| |
| − |
| |
| − | * '''xlcl''' -- IBM XL OpenCL compiler
| |
| − |
| |
| | ===MPI=== | | ===MPI=== |
| | | | |
| Line 69: |
Line 57: |
| | | | |
| | == Documentation == | | == Documentation == |
| − | * Cell SDK | + | * CUDA |
| − | ** Programming:[http://support.scinet.utoronto.ca/Manuals/Cell/CBE_Programmers_Guide_v3.1.pdf CBE Programmers Guide],[http://support.scinet.utoronto.ca/Manuals/Cell/CBE_Handbook_v1.1_24APR2007_pub.pdf CBE Handbook],[http://support.scinet.utoronto.ca/Manuals/Cell/CBE_Programming_Tutorial_v3.1.pdf CBE Programming Tutorial],[http://support.scinet.utoronto.ca/Manuals/Cell/CellBE_PXCell_Handbook_v1.11_12May08_pub.pdf CBE PXCell Handbook],[http://support.scinet.utoronto.ca/Manuals/Cell/Performance_Tools_Reference_v3.1.pdf Performance Tools Reference],[http://support.scinet.utoronto.ca/Manuals/Cell/IDE_Users_Guide_v3.1.pdf IDE Users Guide] | + | ** http... |
| − | ** Libraries:[http://support.scinet.utoronto.ca/Manuals/Cell/3D_FFT_Prog_Guide_and_API_Reference.pdf 3D FFT],[http://support.scinet.utoronto.ca/Manuals/Cell/libFFT_Prog_Guide_API_v3.1.pdf libFFT],[http://support.scinet.utoronto.ca/Manuals/Cell/ALF_Prog_Guide_API_v3.1.pdf ALF],[http://support.scinet.utoronto.ca/Manuals/Cell/BLAS_Prog_Guide_API_v3.1.pdf BLAS],[http://support.scinet.utoronto.ca/Manuals/Cell/CBE_SIMDmath_API_v3.1.pdf SIMD Math API],[http://support.scinet.utoronto.ca/Manuals/Cell/DaCS_Prog_Guide_API_v3.1.pdf DaCS],[http://support.scinet.utoronto.ca/Manuals/Cell/LAPACK_Prog_Guide_API_v3.1.pdf LAPACK],[http://support.scinet.utoronto.ca/Manuals/Cell/Monte_Carlo_Prog_Guide_API_v3.1.pdf Monte Carlo],[http://support.scinet.utoronto.ca/Manuals/Cell/SDK_Example_Library_API_v3.1.pdf SDK Library Examples],[http://support.scinet.utoronto.ca/Manuals/Cell/SPE_Runtime_Management_API_v2.3.pdf SPE Runtime Management], [http://support.scinet.utoronto.ca/Manuals/Cell/SPE_Runtime_Management_Migration_v3.1.pdf SPE Runtime Migration],[http://support.scinet.utoronto.ca/Manuals/Cell/SPU_Runtime_Library_Extensions_v3.1.pdf SPE Runtime Library Ext.]
| |
| − | ** Architecure: [http://support.scinet.utoronto.ca/Manuals/Cell/CBEA_v1.01_3Oct2006.pdf CBEA],[http://support.scinet.utoronto.ca/Manuals/Cell/CBE_Public_Registers.v1_5.02APR2007.pdf CBEA Public Registers],[http://support.scinet.utoronto.ca/Manuals/Cell/PPC_Vers202_Book1_public.pdf PPC Book 1],[http://support.scinet.utoronto.ca/Manuals/Cell/PPC_Vers202_Book2_public.pdf PPC Book 2],[http://support.scinet.utoronto.ca/Manuals/Cell/PPC_Vers202_Book3_public.pdf PPC Book 3],[http://support.scinet.utoronto.ca/Manuals/Cell/SPU_ISA_v1.2_27Jan2007_pub.pdf SPU ISA]
| |
| − | ** Standards:[http://support.scinet.utoronto.ca/Manuals/Cell/CBE_LINUX_ABI_1.2.pdf CBE Linux ABI],[http://support.scinet.utoronto.ca/Manuals/Cell/Language_Extensions_for_CBEA_2.6.pdf Language Extentsions],[http://support.scinet.utoronto.ca/Manuals/Cell/SIMD_Library_Specification_for_CBEA_1.2.pdf SIMD Library Specification],[http://support.scinet.utoronto.ca/Manuals/Cell/SPU_ABI-Specification_1.9.pdf SPU ABI Specification],[http://support.scinet.utoronto.ca/Manuals/Cell/SPU_Assembly_Language_Specification_1.7.pdf SPU Assembly Language Specification]
| |
| − | ** Installation: [http://support.scinet.utoronto.ca/Manuals/Cell/SDK-Installation_Guide_v3.1.pdf Installation Guide]
| |
| | | | |
| − | * Cell OpenCL | + | * OpenCL |
| − | ** [http://support.scinet.utoronto.ca/Manuals/Cell/OpenCL_Guide.pdf OpenCL Guide], | + | ** http... |
| | | | |
| | == Further Info == | | == Further Info == |
| − | IBM Cell Broadband engine tutorials http://publib.boulder.ibm.com/infocenter/ieduasst/stgv1r0/index.jsp
| + | |
| | | | |
| | | | |
| Line 86: |
Line 70: |
| | | | |
| | Please discuss put any relevant information/problems/best practices you have encountered when using/developing for CELL and/or OpenCL | | Please discuss put any relevant information/problems/best practices you have encountered when using/developing for CELL and/or OpenCL |
| − |
| |
| − | === MHD ===
| |
| − |
| |
| − | === CFD ===
| |
| Cell Development Cluster |
|---|
|
|
| Installed |
June 2010 |
|---|
| Operating System | Linux |
|---|
| Interconnect | Infiniband |
|---|
| Ram/Node | 32 Gb |
|---|
| Cores/Node | 2 PPU + 16 SPU |
|---|
| Login/Devel Node | cell-srv01 (from login.scinet) |
|---|
| Vendor Compilers | ppu-gcc, spu-gcc |
|---|
The Intel nodes have two 2.53GHz 4core Xeon X5550 CPU's with 48GB of RAM per node with 3 containing NVIDIA 9800GT GPUs.
Login
First login via ssh with your scinet account at login.scinet.utoronto.ca, and from there you can proceed to cell-srv01 which
is currently the gateway machine.
Compile/Devel/Compute Nodes
Nehalem (x86_64)
You can log into any of 8 nodes cell-srv[01-08] directly however each
of the nodes have differing configurations.
- cell-srv01 - login node & nfs server, GigE connected
- cell-srv[02-05] - no GPU, GigE connected
- cell-srv[06-07] - 1x NVIDIA 9800GT GPU, Infiniband connected
- cell-srv08 - 2x NVIDIA 9800GT GPU, GigE connected
Local Disk
This test cluster currently cannot see the global /home and /scratch space so you will have to copy (scp,etc..) your code to a separate local/home dedicated for this cluster. Also initially you will probably want to copy/modify your scinet .bashrc, .bashrc_profile, and .ssh directory onto this system.
Programming Frameworks
Currently there are two programming frameworks to use, NVIDIA's CUDA framework or OpenCL.
CUDA
/usr/local/cuda/
OpenCL
Demos, examples, and build details available in
Compilers
- nvcc -- Nvidia GCC compiler
MPI
Still a work in progress.
Documentation
Further Info
User Codes
Please discuss put any relevant information/problems/best practices you have encountered when using/developing for CELL and/or OpenCL
