Graphical Version

HPC@CSM Mio Panasas Upgrade

The File system on Mio

The primary file system on Mio is based on Panasas servers. Each server in 4U in height and contains 11 vertical blades. 10 of the blades contains two disk drives each. The 11th blade manages the rest.

When we received the Panasas system it was populated with 400 Gbyte disk drives. We have had 15 shelves on line for a total of 300 disk drives. The system has served us well for over a year.

The Problem

Mio is housed on the second floor of the Green Center. The machine room is currently limited in power and cooling. We are maxed out on the amount of power we can draw for Mio. Also, we currently have 26 nodes that are still in boxes waiting to be installed. They can not be powered on without tripping circuit breakers.

The Solution

We had a number of Panasas shelves that were not being used. We replaced the 400 Gbyte drives with 2 Tbyte (2,000 Gbyte) drives in 3 shelves. We then copied the file system from the 15 Panasas shelves to 3 shelves and then powered off the 15, for a total reduction in the number of shelves by 12. We are removing the 15 shelves and replacing them with compute nodes. The new file system became operational Feb 22. We are installing the new compute nodes Feb 23.

There are a lot of details left out here. We will, however, be creating a document describing how all of this was done. To our knowledge this is the first time such an upgrade was performed in the field by end users on a Panasas system.

Pictures Please

Here are some before and after pictures. We will add more after pictures when the compute nodes get installed. Click on the pictures for full resolution versions.


This is Mio as it appeared before the modifications. The 1U servers are the compute nodes. Actually, there are two nodes, left and right in each 1U enclosure. The 4U enclosures are the 15 Panasas servers. The 5U enclosure on the left is the newer GPU node. (It has room for expansion within the enclosure.) The Tech-Fee GPU node is on top of the Panasas shelves in the middle rack.


The same, just from the other side. Note the green "ok" lights on the Panasas servers.




The 15 Panasas Servers. 20 400 Gbyte disk drives each, for a total of 300 drives.






The 3 "new" Panasas servers with 20 2 Tbyte disk drives each, for a total of 60 drives.






The 3 "new" Panasas Servers and the dark "old" Panasas. You can see in this picture we have started to remove blades so that we can add the compute nodes.






The boxes containing
the 26 new nodes to be
added to Mio.










The End Result

Here is Mio on March 8, 2012
Note the 3rd rack on the left
is almost empty.




Font Size