SSH through HTTPS proxy

connect-proxy is a simple relaying command to make tunnel TCP connection via SOCKS4/5 or HTTPS proxies. It is mainly intended to be used as proxy command for OpenSSH. To use it first install the appropriate package, connect-proxy on GNU/Debian.

Then adjust your ssh configuration:

# vi ~/.ssh/config 
Host 10.10.1?.* 10.10.2?.* 10.10.3?.* *.foobar.fr
   User root
   ProxyCommand connect-proxy -H 10.2.0.217:3128 %h %p

Here in order to reach these specific hosts we connect through an HTTPS proxy on the port 3128 of the machine 10.2.0.217 Note that you can also use other program, like corkscrew instead of connect-proxy.

[Debian] Disable a service

In the Debian world, the usual method to remove a service from startup without uninstalling the package, is to delete the init’s script symlinks like this:

update-rc.d -f remove foobar

Problem: at the next package upgrade, the post-install script will recreate the symlinks. Fortunately update-rc.d has a lesser-know disable option for this precise use case:

update-rc.d -f disable foobar

Usual environment variables

Unix system use a lot of environment variables. Most of them are initialized when you log in and are maintained until you log out. This is a list of the most usual/usefull variables to know. Keep in mind, depending on the OS/distribution and your shell, some may or may not exist:

PATH – List directories the shell searches for commands
HOME – User's home directory
TERM – Indicate which terminal emulator is used
PS1 – Shell prompt in the Bourne shell and variants
MAIL – Path to user's mailbox
TEMP – Path to directory where processes can store temporary files
TZ – Timezone settings
PWD – Path of the current directory
HISTFILE – Name of the file in which command history is saved
HISTFILESIZE - Maximum number of lines contained in the history file
HOSTNAME - System's host name
LD_LIBRARY_PATH - List of directories where libraries should be searched for
USER - Current logged in user's name
DISPLAY - Network name of the X11 display to connect to, if available
SHELL - Indicate current shell path
TERMCAP – Display termcap database entry for the current term
OSTYPE – Type of operating system
MACHTYPE – The CPU architecture that the system is running on
EDITOR – The user's preferred text editor
PAGER –	The user's preferred text pager
MANPATH – List of directories to search for manual pages

You can also list all the environement variables on your current session, using the printenv command.

Detach processes from terminal

GNU screen/tmux

The best solution is simply to use a terminal multiplexer, like GNU screen or tmux. Simply launch a session, tape your command and then detach it (Ctrl-a d for screen). Easy and clean 😉

setsid

Debian contains a binary called setsid in the util-linux package. setsid can be use to start a process and detach it from the current shell (basically it create a new shell for the ‘orphaned’ process).

setsid doesn’t redirect the standard files descriptors (stdin, stdout and stderr) so you loose any process output except if you make a stdout+stderr redirection to a file:

setsid <command> > /tmp/output.txt &2>1

nohup

nohup as the name implies, makes your command ignore SIGHUP signal. Also by default nohup redirects the standard output and error to the file nohup.out, so the program won’t fail for writing to standard output when the shell is closed. Note that nohup doesn’t remove the process from the shell’s job control and also doesn’t put it in the background. Usage:

nohup <command> > /tmp/output.txt &

disown

Last option (and the more interesting) is the built-in bash command disown. disown removes the process from the shell’s job control, but still leaves it connected to the terminal. The results is that the shell won’t send it a SIGHUP when closed, but in the meantime you still get the output. The advantage is you can disown a already running program.

Simply suspend the program using Ctrl-z then use bg to put it in background. Then detach it:

disown %n

where n is the job number (use the command job to get it).

[Bash] Don’t add message to .bashrc

It’s often very useful to add customized message (with info or specific commands) on a VM. The easiest way to do that is to use the /etc/motd file. But what to do if you want to add a message just for one user ?

The first thing that everybody try is to add the message to the .bashrc.
That’s a bad habit:

zapan:~$ ssh me@zapan 'uptime'
You shouldn't see this message !
 17:16:45 up 4 days,  7:29,  4 users,  load average: 1.81, 1.25, 0.83

As you can see the content is displayed in case of a non-interactive non-login shell invoked via ssh. Instead add your message to the .bash_profile or .profile file.

Further Reading and sources

Stat command examples

stat displays the detailed status of file using inode data. Theses info include the date the file was last modified (like ls), the time the file was last changed and the time that file was last accessed:

# stat /var/lib/dpkg/lock 
  File: `/var/lib/dpkg/lock'
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 806h/2054d      Inode: 57030       Links: 1
Access: (0640/-rw-r-----)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2011-08-13 12:17:00.000000000 +0200
Modify: 2012-03-08 16:03:20.000000000 +0100
Change: 2012-03-08 16:03:20.000000000 +0100

stat can also be use to get information on a partition, using the -f option:

# stat -f /dev/sda1 
  File: "/dev/sda1"
    ID: 0        Namelen: 255     Type: tmpfs
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 2560       Free: 2514       Available: 2514
Inodes: Total: 6181154    Free: 6180411

And don’t forget the most useful option for scripting --format:

# stat --format "%A" /var/log/syslog
-rw-r-----

Create a SSH tunnel at boot time

To mount persistent SSH tunnel, nothing is better than autossh. But there is a small inconvenient, it doesn’t really like being started in a runlevel (go figure why). The solution is to use the /etc/rc.local file for this task:

autossh -i /foobar/.ssh/id_dsa -L 83:192.168.3.32:80 root@moya.farscape.net -oBatchMode=yes -N

Here we make a persistent tunnel with a forwarding of local machine port 83 to the moya.farscape.net machine port 80, using the user foobar key to initialize the connection. autossh will monitor the ssh tunnel and restart it as needed.

Dig command examples

The dig utility is very handy for troubleshooting name resolution issues or to find out information about an IP or a domain name. Here are some useful command examples.

Lookup for a domain

# dig google.com

; <<>> DiG 9.3.4-P1.2 <<>> google.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24311
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;google.com. IN A

;; ANSWER SECTION:
google.com. 217 IN A 216.58.209.238

;; Query time: 0 msec
;; SERVER: 62.210.64.50#53(62.210.64.50)
;; WHEN: Tue Oct 25 11:10:01 2016
;; MSG SIZE rcvd: 44

Lookup for a specific record

You can specify record type using the appropriate prefix:

# dig <prefix> google.com
  • A for A records
  • MX for MX records
  • NS for NameServer records
  • CNAME for cname records
  • TXT for SPF, DKIM and everything that goes inside TXT records

Short answer

If you want just the answer without any comments, you can use the +short option:

# dig A google.com +short
216.58.209.238

You can also enable or disable each output sections, using the appropriate paramaters. For example to get only the ANSWER SECTION:

# dig A google.com +noall +answer
google.com.             299     IN      A       74.125.140.139
google.com.             299     IN      A       74.125.140.101
google.com.             299     IN      A       74.125.140.100
google.com.             299     IN      A       74.125.140.138
google.com.             299     IN      A       74.125.140.113
google.com.             299     IN      A       74.125.140.102

Use a Specific DNS server

To test the answer from a specific DNS server you can use the @ parameter:

# dig A @8.8.8.8 google.com

; <<>> DiG 9.3.4-P1.2 <<>> A @8.8.8.8 google.com
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28263
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;google.com. IN A

;; ANSWER SECTION:
google.com. 299 IN A 74.125.140.100
google.com. 299 IN A 74.125.140.113
google.com. 299 IN A 74.125.140.138
google.com. 299 IN A 74.125.140.139
google.com. 299 IN A 74.125.140.102
google.com. 299 IN A 74.125.140.101

;; Query time: 8 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Oct 25 11:15:23 2016
;; MSG SIZE rcvd: 124

Reverse lookup

You can check the PTR records by doing a reverse lookup:

# dig -x 91.189.94.40

; <<>> DiG 9.3.4-P1.2 <<>> -x 91.189.94.40
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21503
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 0

;; QUESTION SECTION:
;40.94.189.91.in-addr.arpa. IN PTR

;; ANSWER SECTION:
40.94.189.91.in-addr.arpa. 3600 IN PTR ovinnik.canonical.com.

;; AUTHORITY SECTION:
94.189.91.in-addr.arpa. 3600 IN NS ns1.canonical.com.
94.189.91.in-addr.arpa. 3600 IN NS ns2.canonical.com.
94.189.91.in-addr.arpa. 3600 IN NS ns3.canonical.com.

;; Query time: 15 msec
;; SERVER: 62.210.64.50#53(62.210.64.50)
;; WHEN: Tue Oct 25 11:43:36 2016
;; MSG SIZE rcvd: 132

Remove CTRL-M characters

One annoying thing with text file is that files created on DOS/Windows machines have different line endings than files created on Unix/Linux. DOS uses carriage return and line feed \r\n as a line ending which Unix uses just line feed \n. So in the end you finish with a lot of supernumerary CTRL-M characters. To get rid of it you can:

Use sed

sed -e "s/^M//" filename > newfilename

Use vi

Inside vi [in ESC mode] type: :%s/^M//g
To enter ^M, type CTRL-V, then CTRL-M (hold down the CTRL key then press V and M in succession).

Use dos2unix

dos2unix filename