Infiniband Upgrade

GPC Infiniband Upgrade

The GPC Infinibands (QDR and DDR) are fully compatible in terms of hardware and software so no recompilation or different MPI flags will be required.

In terms of job submission, users can submit as usual, however the :ib: flag will no longer be required as all the nodes will now be IB enabled. By default a user's job will go to whichever section best accommodates it, typically smaller jobs to the QDR and larger jobs to the DDR. However a user can override this by simply adding the flags "ddr" or "qdr" to the job resource request, ie nodes=10:ddr:ppn=8 or nodes=10:qdr:ppn=8 to ddr or qdr respectively.

The NRAC allocations for the current year that were based on ethernet and infiniband will carry over, however the allocation will be on the full GPC, not just the subsection. So if you were allocated 500 hours on Infinibad your fairshare allocation will still be 500 hours, just 500 out or 30000, instead of 500 out of 7000. If you received two allocations, one on gigE and one on IB, they will simply be combined. This should benefit all users as the desegregation of the GPC provides a greater pool of nodes increasing the probability of your job to run.