Overview
This is a problem I ran into while working on my team, and for some reason the solution was a bit involved. It's possible I didn't find the right resources at first, possibly because there seem to be different solutions depending on which version of OS X you are running.
My problems came in a couple forms, 1 running all of the unit tests in our Ruby of Rails project may have been a lot. Our project includes about 322 unit and functional test files with about 102 gems in the bundle/gemset.
It may have also happened when say, switching rvm gemsets, trying to open the Rails console, especially if it worked before, and then I added/updated gems at all, or running some other form of rails/rvm generic or test command. Any of those things may need to load a lot of files such as the gems, and helped me reach that open file handle limit.
I will attempt to capture my notes here so that if you run into the same type of problem, you can solve it as well.
Help Information
Brief Guide
# Source: https://superuser.com/questions/827984/open-files-limit-does-not-work-as-before-in-osx-yosemite
# Paired with: https://superuser.com/questions/302754/increase-the-maximum-number-of-open-file-descriptors-in-snow-leopard
# Doesn't seem to apply, at least what is viewed through 'sudo launchctl limit'
# Maybe this did successfully change the kernel limit, and I just need to change the [session] limit now?
# Also ran 'sudo launchctl limit maxfiles unlimited unlimited', which defaults to a maximum of 10240.
# But then I tried 'sudo launchctl limit maxfiles 1000000 1000000', and those values 'stuck'.
kern.sysv.shmmax=1073741824
kern.sysv.shmmin=1
kern.sysv.shmmni=4096
kern.sysv.shmseg=32
kern.sysv.shmall=1179648
kern.maxfilesperproc=65536
kern.maxfiles=65536
# Source: https://superuser.com/questions/302754/increase-the-maximum-number-of-open-file-descriptors-in-snow-leopard
# Paired with: https://superuser.com/questions/827984/open-files-limit-does-not-work-as-before-in-osx-yosemite
limit maxfiles 1000000 1000000
No ulimit
command use necessary.
More Details
With this current set-up, I thought I had restarted my computer, re-run the problematic task (full unit/functional test suite), and still encountered the same file handle maximum error.
I just looked into the usual terminal environment configuration files (~/.config/fish/config.fish
~/.bashrc
~/.bash_profile
~/.profile
/etc/profile
/etc/bashrc
), and I don't see any ulimit
lines in there.
I thought that was required to have the settings take full effect, as I did it with those files set and ran the command in a terminal session to see the eventual success of the task, but I have been running the task after that point, having restarted my computer since then, without re-issuing the ulimit
command again, and it works now, so maybe it's not needed?
Resource Dump
-
SuperUser - Increase the maximum number of open file descriptors in Snow Leopard?
- CLI (temporary)
-
sudo launchctl limit maxfiles 1000000 1000000
sudo ulimit -n 12288
- or
ulimit -n 12288
?
- Persist
/etc/launchd.conf
-
limit maxfiles 1000000 1000000
- or
limit maxfiles 8192 20480
- or
-
limit maxproc 1000 2000
/etc/sysctl.conf
-
kern.maxfiles=20480
/etc/profile
-
ulimit -n 4096
- Notes (implicitly quoted) [2]:
- Alas: "The /etc/launchd.conf file is no longer consulted for subcommands to run during early boot time; this functionality was removed for security considerations."
- You will need to restart for these changes to take effect.
- AFAIK you can no longer set limits to 'unlimited' under OS X.
-
launchctl maxfiles
is bounded bysysctl maxfiles
, and therefore cannot exceed them. -
sysctl
seems to inheritkern.maxfilesperproc
fromlaunchctl maxfiles
. -
ulimit
seems to inherit it's 'open files' value fromlaunchctl
by default. - You can set a custom ulimit within
/etc/profile
, or~/.profile
; while this isn't required, I've provided an example. - Be cautious when setting any of these values to a very high number when compared with their default - the features exist stability/security. I've taken these example numbers that I believe to be reasonable, written on other websites.
-
Open files limit does not work as before in OSX Yosemite
- CLI
sysctl -w kern.maxfiles=65000
- CLI (alternate answer)
echo 'kern.maxfiles=20480' | sudo tee -a /etc/sysctl.conf echo -e 'limit maxfiles 8192 20480\nlimit maxproc 1000 2000' | sudo tee -a /etc/launchd.conf echo 'ulimit -n 4096' | sudo tee -a /etc/profile
- CLI
-
Maximum number of open filehandles per process on OSX (and how to increase)
- CLI
sysctl -w kern.maxfilesperproc=20000
- CLI
-
How to persist ulimit settings in OSX Mavericks?
- Persist
-
/Library/LaunchDaemons/limit.maxfiles.plist
- ....
-
- Persist
Other
Right now I don't have any Windows instructions. I wonder if there is a similar problem in a pure-Windows environment, or the exact same problem when running in a *nix-like environment or interpreter such as Cygwin and Bash/Fish respectively.