File System and I/O dos and don'ts

Dos

• Make a data plan.
• Regularly check disk usage with /scinet/gpc/bin/diskUsage, which is updated hourly.
• RAM is always faster than disk: ramdisk.
• Write binary files.
• Use parallel I/O if writing from dozens of nodes.
• Use gzip and tar.
• Delete unneeded files.
• Maximize size of files.
• Do housekeeping regularly.
• Monitor disk actions with top and strace.
• Attend parallel I/O course coming soon!
• Make an appt to talk with our analysts about your I/O.

Don'ts

• Do not write lots of ASCII files.
• Do not write many hundreds of files in a single directory.
• Do not write many small files.
• Minimize use of file system commands like ls and du.

More information

See the slides of the TechTalk Parallel File System and I/O.