Increase the maximum number of file descriptors

List the max value of open file descriptors

You can use the following command:

# sysctl fs.file-max

Or

# cat /proc/sys/fs/file-max

Increase the file-max value

# sysctl -w fs.file-max=100000

To make the change permanent:

# vi /etc/sysctl.conf
fs.file-max = 100000

Don’t forget to sysctl -p to reload the setting file.

User level FD limits

Beside the system maximum value, there is also two limits (hard and soft) for user account. You can check theses values using ulimit like this:

# ulimit -Hn
# ulimit -Sn

You can override theses values for a specific user if needed, into the /etc/security/limits.conf file. To get the list of the most greedy process, use this snippet:

lsof +c 15 | awk '{printf("%15s  (%s)\n", $1, $2)}' | sort | uniq -c | sort -rn

[OpenVZ] NFS file locking problem

My coworkers have found a very annoying bug. Debian’s OpenVZ kernel superior to 2.6.32-squeeze5 have a regression that make NFS exclusive file locking mechanism broken.

Therefore concurrency problems could arise when several fronts try to write into the same file. To test your setup you can do this on an NFS client:

# strace flock -x -w10 "/NFS-MOUNTPOINT/myfile" -c "sleep 10&"

If it’s hang, you have a locking problem. For the moment the only solution is to downgrade your kernel. A ticket have been created on OpenVZ bugtracker.

[BIND] Add a new zone

What is a DNS zone ?

A zone is a subset, often a single domain, of the hierarchical DNS. Zone are generally defined inside a single ‘zone’ file, that describe its properties (refresh time, domain expiry, default TTL value, etc…) and all its resource records.

Create a new zone file

Let create a new zone file for brand new domain foobar.com. Following the BIND naming convention the zone file will be /etc/bind/db.foobar.com.conf, and look pretty much like this:

; Zone file for foobar.com
$TTL    3600
$ORIGIN foobar.com
@       IN      SOA     ns1.mydns.com.    root.foobar.com. (
                     2012033101         ; Serial
                           3600         ; Refresh
                           1800         ; Retry
                         604800         ; Expire
                          43200 )       ; Negative Cache TTL

        IN      NS      ns1.mydns.com.
        IN      NS      ns2.mydns.com.

@       IN      A       192.168.0.2
www     IN      A       192.168.0.2

The NS records indicate that we use two DNS servers for this new zone : ns1.mydns.com and ns2.mydns.com. The SOA record specify that ns1.mydns.com is the start of authority, and give the domain properties. Then we have a couple of classic A records, for the domain and a subdomain. Note here the usage of the @ symbol which is a shorthand for the $ORIGIN value.

Add the new zone

Now that we have a new zone file, we must modify BIND main setting file, usually /etc/bind/named.conf.local

For the master of the new zone, ns1.mydns.com here, we add the following block:

zone "foobar.com" {
        type master;
        file "/etc/bind/db.foobar.com.conf";
};

For slaves, like ns2.mydns.com, the syntax is a little different:

zone "foobar.com" {
        type slave;
        file "/etc/bind/db.foobar.com.conf";
        masters { ; };
};

Reload BIND configuration

# rndc reload

Monty Python Unix Joke

- Stop! Whoever crosseth the bridge of Death, must answer first these questions three, ere the other side he see:
  What is your name?
- Sir Brian of Bell
- What is your quest ?
- I seek the Holy Grail
- What are four lowercase letters that are not legal flag arguments to the Berkeley UNIX version of 'ls' ?
- I, er…. AIIIEEEEEE!

Guru meditation

If you use the HTTP reverse-proxy Varnish or the VirtualBox hypervisor, you probably already encounter a funny Guru meditation error message.

But who is this guru and what is the origin of this message ?

In the 1980s the Amiga computer system was a very popular personal computer brand. Originally intended as a videogame machine but latter reconceived as a general purpose computer, the development of the first Amiga computer took a considerable time. In the interim the Amiga corporation released a number of other products.

One of them was the Joyboard, a balance board peripheral for the Atari 2600. The Joyboard was conceived by installing the four directional latches of a joystick on the bottom of a plastic board. Leaning in a certain direction engaged these latches, controlling the game pretty much like the modern Nintendo Wiiboard does.

According to the legend, in the early development of the AmigaOS, developers became so frustrated with the system’s frequent crashes that, as a relaxation technique, they attempted to sit cross-legged perfectly still on a Joyboard. In this position they look like Indian gurus.

Quickly they developed a little video game where the winner was the one who stayed still the longest without engaging any of Joyboard’s latches. If a player moved too much a guru meditation game-over screen occurred. As an easter-egg this guru meditation was integrated into the AmigaOS as a general error message, and since them became quite popular.

[Debian] Disable a service

In the Debian world, the usual method to remove a service from startup without uninstalling the package, is to delete the init’s script symlinks like this:

update-rc.d -f remove foobar

Problem: at the next package upgrade, the post-install script will recreate the symlinks. Fortunately update-rc.d has a lesser-know disable option for this precise use case:

update-rc.d -f disable foobar

Linux ate my RAM!

There is a saying in Linux community: “Free memory is wasted memory.” This statement seems to confuse newbies, resulting in the Linux ate all my RAM myth. Reality is that the kernel borrows unused chunk of memory for disk caching (alias “Buffers”) and file caching (alias “Cached”). This behavior improve significantly the overall performances.

Whenever an application needs more memory, borrowed chunks are “returned”. The website linuxatemyram.com explain this mechanism in more detail.

Confusion arise because people don’t read the right line when using the free command:

$ free -m
             total       used       free     shared    buffers     cached
Mem:          5863       5541        321        569        107       1203
-/+ buffers/cache:       4231       1632
Swap:         7323       3435       3888

You think this host only have 321MB of ‘free’ memory ? Wrong !
This host have 321MB of non-used memory, that true. But the total amount of memory that the kernel can “reclaim” (and therefore give for applications) is much higher.

The right answer is 1632MB.

In your scripts if you want to get the amount of ‘free’ memory, use the following one liner:

free -m | sed -n -e '3p' | grep -Po "\d+$"

[OpenLDAP] Debug slapd

By default sldapd isn’t very talkative. At best you will obtain a small hiccup in the syslog if it doesn’t start. For debugging purpose, it’s better to manually start it like this:

slapd -d 65536

This should give you a more user-friendly output, and give you a hint at least on the most common problems (wrong permission on /var/lib/ldap, /var/run/sldap not writeable, root dn isn’t the same between ldif and /etc/slapd.conf, missing schema, etc…).