Posts

Posted on

Linux ate my RAM!

There is a saying in Linux community: “Free memory is wasted memory.” This statement seems to confuse newbies, resulting in the Linux ate all my RAM myth. Reality is that the kernel borrows unused chunk of memory for disk caching (alias “Buffers”) and file caching (alias “Cached”). This behavior improve significantly the overall performances.

Whenever an application needs more memory, borrowed chunks are “returned”. The website linuxatemyram.com explain this mechanism in more detail.

Confusion arise because people don’t read the right line when using the free command:

$ free -m
             total       used       free     shared    buffers     cached
Mem:          5863       5541        321        569        107       1203
-/+ buffers/cache:       4231       1632
Swap:         7323       3435       3888

You think this host only have 321MB of ‘free’ memory ? Wrong !
This host have 321MB of non-used memory, that true. But the total amount of memory that the kernel can “reclaim” (and therefore give for applications) is much higher.

The right answer is 1632MB.

In your scripts if you want to get the amount of ‘free’ memory, use the following one liner:

free -m | sed -n -e '3p' | grep -Po "\d+$"
Posted on

Refresh ARP cache entry

ARP (Address Resolution Protocol) is the method for finding a host’s hardware address when only its IP address is known. ARP is a Link Layer protocol (Layer 2) because it only operates on the local area network.

When you migrate an IP from a VM or an hypervisor to another one, you can sometime encounter strange network problems (like no traffic received to this IP for a while). The cause of theses problems is that some network equipment (router or firewall) on the path have a ‘expired’ ARP entry in their cache.

Refresh ARP cache

For a GNU/Linux or BSD host, check the current arp table with arp -a.
Then purge the value with arp -d <hostname>

For Cisco iOS use show arp for checking arp table and clear ip arp <ip> to purge an entry.

For equipment you don’t have the hand on, the solution is a little more tricky. You need to broadcast from your VM the correct ARP ‘value’, in order to make the remote device invalidate its existing cache entry. For this use the arping command:

arping -S <ip> -B
Posted on

[OpenLDAP] Debug slapd

By default sldapd isn’t very talkative. At best you will obtain a small hiccup in the syslog if it doesn’t start. For debugging purpose, it’s better to manually start it like this:

slapd -d 65536

This should give you a more user-friendly output, and give you a hint at least on the most common problems (wrong permission on /var/lib/ldap, /var/run/sldap not writeable, root dn isn’t the same between ldif and /etc/slapd.conf, missing schema, etc…).

Posted on

[Bash] Don’t add message to .bashrc

It’s often very useful to add customized message (with info or specific commands) on a VM. The easiest way to do that is to use the /etc/motd file. But what to do if you want to add a message just for one user ?

The first thing that everybody try is to add the message to the .bashrc.
That’s a bad habit:

zapan:~$ ssh me@zapan 'uptime'
You shouldn't see this message !
 17:16:45 up 4 days,  7:29,  4 users,  load average: 1.81, 1.25, 0.83

As you can see the content is displayed in case of a non-interactive non-login shell invoked via ssh. Instead add your message to the .bash_profile or .profile file.

Further Reading and sources

Posted on

[OpenVZ] Useful one-liner

List containers sorted by CPU usage:

# vzlist -o ctid,name,laverage

List containers sorted by TCP sender buffer usage:

# vzlist -H -o ctid,name,tcpsndbuf | sort -r -n -k3

List containers sorted by TCP receive buffer usage:

# vzlist -H -o ctid,name,tcprcvbuf | sort -r -n -k3

List containers sorted by overall resources consumption:

for i in `vzlist -H -o veid`;do echo $i; vzcalc $i; echo "========"; done;

Add a new network interface to a container:

# vzctl set <id> --netif_add eth0 --save
Posted on

[Postgres] Increase shmmax

PostgreSQL is great, but the manner its work is sometime… a little unsettling. For example, Postgres like spawning a lots of little processes that communicate by IPC, which means using chunk of shared memory. Sometime you can hit the shmmax limit.

To obtain the current value :

cat /proc/sys/kernel/shmmax

You can ovverride the default value into the /etc/sysctl.conf file.
Add a line like this for example:

kernel.shmmax = 43554432

And don’t forget to make a sysctl -p and restart postgres to take into account the new value.

Posted on

[Apache] Discard QUERY_STRING

Apache_mod_rewrite is a lot of fun but sometime it syntax is very “twisted”. For example let say you want to make a conditional rewrite on the url, query string included. You will probably write something like this:

RewriteCond %{HTTP_HOST} ^www.foobar.com$
RewriteCond %{QUERY_STRING}  ^queryType=3&item=services&itemId=47$
RewriteRule ^/initial/cms/php/workflow.php?$ http://www.foobar.com/Home/Our-services/Business [NC,R=301,L]

Now problem right ? Yep except as the destination url doesn’t have it own query string mod_rewrite will keep the original query string ! Damned. To prevent this, write instead this:

RewriteCond %{HTTP_HOST} ^www.foobar.com$
RewriteCond %{QUERY_STRING}  ^queryType=3&item=services&itemId=47$
RewriteRule ^/initial/cms/php/workflow.php?$ http://www.foobar.com/Home/Our-services/Business? [NC,R=301,L]

or append the QSD flag.

Posted on

Stat command examples

stat displays the detailed status of file using inode data. Theses info include the date the file was last modified (like ls), the time the file was last changed and the time that file was last accessed:

# stat /var/lib/dpkg/lock 
  File: `/var/lib/dpkg/lock'
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 806h/2054d      Inode: 57030       Links: 1
Access: (0640/-rw-r-----)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2011-08-13 12:17:00.000000000 +0200
Modify: 2012-03-08 16:03:20.000000000 +0100
Change: 2012-03-08 16:03:20.000000000 +0100

stat can also be use to get information on a partition, using the -f option:

# stat -f /dev/sda1 
  File: "/dev/sda1"
    ID: 0        Namelen: 255     Type: tmpfs
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 2560       Free: 2514       Available: 2514
Inodes: Total: 6181154    Free: 6180411

And don’t forget the most useful option for scripting --format:

# stat --format "%A" /var/log/syslog
-rw-r-----
Posted on

Create a SSH tunnel at boot time

To mount persistent SSH tunnel, nothing is better than autossh. But there is a small inconvenient, it doesn’t really like being started in a runlevel (go figure why). The solution is to use the /etc/rc.local file for this task:

autossh -i /foobar/.ssh/id_dsa -L 83:192.168.3.32:80 root@moya.farscape.net -oBatchMode=yes -N

Here we make a persistent tunnel with a forwarding of local machine port 83 to the moya.farscape.net machine port 80, using the user foobar key to initialize the connection. autossh will monitor the ssh tunnel and restart it as needed.