The other day I woke up to a fairly terrifying alert that one of my Linux servers was down to about 5% disk space free. Due to the way that ext-based filesystems work, they typically need some available space to avoid imploding on themselves, thus making this a Bad Situation™. This scenario has lead some clever individuals to do things like keeping a large, empty file on their Linux servers that they can quickly delete in a pinch. To make matters worse, the alert had generated over the weekend, and I was just seeing it as I started my week on Monday morning. The lack of additional alerts with the filesystem being even more full either meant the problem hadn’t progressed further or the system in question had stopped functioning enough for our monitoring platform to pull any data from it — though the latter should have thrown a litany of errors in that scenario.
As I logged in to the server, I was relieved to find that the former was the case. My first guess as to the culprit filling up a 256 GB drive was a backup script I had written, an older version of which had a bug causing it to fail to remove tarballs of application data after they had been uploaded to the cloud. I figured something went amiss in upgrading this particular server to the latest version of my script which fixed the bug. Checking the directory for that, though, showed that everything looked as expected. I checked a handful of other directories that I thought might be guilty, but all of them also looked healthy.
This left me in a situation I hadn’t found myself in yet in the 15 years I’ve been using Linux where I needed to figure out where storage was being consumed, and fairly quickly. The venerable
df -h will only help me at the partition level. I thought about using
du, but it’s difficult to visually look through such a massive amount of output when you’re trying to check the entire partition. Likewise, something like
du | sort was going to take absolutely forever. Back in my Windows sysadmin days, I would frequently use the tool TreeSize to help me with this problem. I started searching DuckDuckGo to see if there were any terminal-based equivalents for Linux.
Sure enough, I quickly hit upon a Stack Exchange thread where someone recommended the tool
ncdu. Like so many great CLI tools on Linux — and as the name may tip you off to — ncdu uses ncurses to give you an interactive CLI interface. Hence the name is short for NCurses Disk Usage.
The tool is super easy to use. I literally just went to the root of my filesystem with
cd / and ran
ncdu. This quickly broke down the usage based on all of the directories at the root.
As you can see from the screenshot above, my
/opt directory is using a massive 120 GB. Moving up and down with the arrow keys and the left and right arrow keys to move in and out of folders, I hopped into
/opt to see that
/opt/syslog was consuming essentially everything.
Inside of that directory, I saw endless log files, each between 2 and 5 GB. After checking a few to verify that they didn’t contain anything I cared about, I deleted all but a couple of the most recent ones. It’s worth mentioning, that I deleted these back at the CLI, which was the smart decision given how many I needed to nuke. If I had run
sudo ncdu to have rights, though, I could have deleted them from within the application. It just would’ve taken a bit of time to delete the files individually that way. Hitting
? inside of the application actually pops up a handy little help menu. This also tipped me off that I could navigate the filesystem with Vim keys rather than having to use the arrow keys, which is always a bonus in my book given my affinity for Vim.
It’s also worth mentioning that this was on an Ubuntu 18.04 LTS system, and I was able to install it from the default repositories:
sudo apt install ncdu
Obviously your mileage may vary based on your distro of choice.