User Tools

Site Tools


software:linux_intro

Surrey Unix course

1. Remote console connections through ssh

To establish a remote console connection to a Linux machine use ssh. At the command prompt, type:

 $ ssh <username>@<hostname>

Paswordless login with ssh

A simple, step-by-step tutorial can be found here.

If you already have a public key, you can just follow the comment in this thread:

 $ ssh-copy-id -i ~/.ssh/id_rsa.pub username@remote-machine

2. Command-line package management

Useful tools: dpkg, apt-get, apt-cache

Searching installed packages

List all insatalled packages:

 $ sudo dpkg --get-selections

this will generate a lot of output, so you can filter it through grep (this trick can be used throughout):

 $ sudo dpkg --get-selctions | grep <an expression you are interested in e.g. 'cpp'>

Get details about a packgage (i.e. where it has installe its files):

 $ sudo dpkg -L <package name>

Search for a specific package:

 $ sudo apt-cache search <part of package name>

Cleaning up unnecessary files

Remove partial packages:

 $ sudo apt-get autoclean

Remove cached .deb files:

 $ sudo apt-get clean

Remove unneeded dependencies:

 $ sudo apt-get autoremove

3. Working with Linux

Creating/extracting archives

To create an archive:

 $ tar -cvf archive-name.tar source
 $ gzip archive-name.tar

To extract:

 $ tar -xzvf archive-name.tar.gz

Check out this site for some nice examples.

4. Mounting disks, images, etc

Mapping Windows drives

To list all the hard drives attached to your system:

 $ sudo fdisk -l

To map an NTFS drive, assuming it's located at /dev/sda3 and you have a /mnt/data directory, use:

 $ sudo mount -t ntfs /dev/sda3 /mnt/data

To unmount:

 $ sudo umount /mnt/data

Mapping a drive automatically (on boot)

To map a drive automatically on boot, edit the /etc/fstab file. For an ntfs partition on /dev/sda3 my /etc/fstab looks like this:

 /dev/sda3    /mnt/data    ntfs    user,uid=1000,gid=1000,dmask=027,fmask=137    0    0

A good overview of the different entry options is available here.

Mounting an ISO image

 $ sudo mkdir /media/iso
 $ sudo mount -o loop path/to/iso/file/YOUR_ISO_FILE.ISO /media/iso

5. File type associations

To associate a new extension with an existing file type, edit the /usr/share/mime/packages/freedesktop.org.xml file

 $ sudo gedit /usr/share/mime/packages/freedesktop.org.xml

For example, to register *.launch files as XML files, find the appropriate tag (in this case):

 <mime-type type="application/xml"> 

and add a new glob tag:

 <glob pattern="*.launch"/>

And that's it, you're done :) For instance, gedit will now open *.launch files with XML syntax highlighting automatically.

Adding syntax highlighting for less

6. Networking tools

To list all open sockets and owning processes:

 sockstat

tcpdump

tcpdump is a command tool for printing network traffic on standard output and/or file. tcpdump is used as:

 tcpdump [ -AdDefIKlLnNOpqRStuUvxX ] [ -B buffer_size ] [ -c count ] [ -C file_size ] [ -G rotate_seconds ] [ -F file ] [ -i interface ] [ -m module ] 
        [ -M       secret ] [ -r file ] [ -s snaplen ] [ -T type ] [ -w file ] [ -W filecount ] [ -E spi@ipaddr algo:secret,... ] [ -y datalinktype ] 
        [ -z postrotate-command ] [ -  Z user ] [ expression ] 
	 

As we can see there are many options available. Full details about each of them is available on http://www.tcpdump.org/tcpdump_man.html. (date of access 30.12.2012.)

Common usage of this tool:

 tcpdump -nSs 0 -i eth0

with following options included:

   -n : display but don't resolve host or port names
   -S  : print absolute sequence numbers
   -s 0 : define snaplength, value 0 - capture everything
   -i eth0 : define interface to capture traffic from, common interfaces are eth0, wlan0, 
              eth1, wlan1... If you don't know which interfaces are available,
              type ifconfig (or sudo ifconfig for super user privileges if required)

If you want to see packet content both in hex and ASCII format use -X option:

 tcpdump -nXSs 0 -i eth0

If you don't want to enter promiscuous mode, use -p option (in promiscuous mode, this tool captures all traffing flowing trough the interface regardless of packet's destination, while with -p option it captures only traffic destined to interface specified with -i option)

tcpdump can capture only specific traffic defined with boolean expression. Through this option it can be defined to capture traffic with specific protocol and/or specific port and/or specific source and/or specific destination… With the following command tool captures only tcp messages from destination host 192.168.1.1 destined to network 192.168.1.0/24, i.e destined to hosts 192.168.1.1, 192.168.1.2 … 192.168.1.255 :

 tcpdump -nSs 0 -i eth0 tcp src host 192.168.1.1 dst net 192.168.1.0/24

To dump network traffic to file use -w option:

 tcpdump -nSs 0 -i wlan0 -w capture.cap

To display stored traffic use -r option followed by save dfile:

tcpdump -nSs 0 -i wlan0 -r capture.cap

More examples on tcpdump usage on http://www.rationallyparanoid.com/articles/tcpdump.html and http://danielmiessler.com/study/tcpdump/ (date of access 30.12.2012)

ip_forward

If is it required to exchange packets between different network interfaces (NICs) on same computer system, there are two very useful utilities present in most Linux kernels: ip_forwarding and bridging. With these kernel options it is not required to write additional code to achieve packet exchange between network interfaces.

If ip_forward option is enabled in Linux kernel, network interfaces will transmit received packets destined to another interfaces. For example, let say we have a most common network configuration with one wireless and one ethernet interface. Furthermore, wifi address is 192.168.1.2 with subnet mask 255.255.255.0 and eth address is 192.168.2.2. with mask 255.255.255.0. If wifi receives any packet destined to network 192.168.2.0/24, i.e. destined to eth local network, it is forwarded to eth interface. Same, if eth receives any packet destined to 192.168.1.0/24 network, it is forwarded to wifi interface.

To check if ip_forward is enabled type console command (sudo privileges might be required):

 sysctl net.ipv4.ip_forward

To temporarily enable ip_forward type:

 sysctl -w net.ipv4.ip_forward=1

To achieve if_forward is enabled when computer restarts, change the following line in file /etc/network/options (for Debian like Linux, for other distributions different files might be changed):

ip_forward 1

bridge

Bridge with ip_forward enabled is another kernel utility that enables packet exchange between multiple interfaces on same computer. This utility enables computer's multiple ethernet interfaces to work on same subnet. In bridge mode eth0 and eth1 interfaces both have the same bridge address, for example 192.168.1.2 with subnet 255.255.255.0. However, for kernel newer than 2.6.33 it is possible to bridge only ethernet interfaces, i.e. you cannot bridge wifi and eth interface or two wifi interfaces.

There are two reasons why wifi interfaces cannot be bridged. First, most wifi drivers don't support bridging in a way that they block device visibility connected on the same wifi network. As a result, when bridge utility tries to scan for devices connected to wifi interface, driver blocks its request. Second, if wifi interface works in managed (infrastructured) mode, according to 802.x standard, there is a “spot” missing in packet's header for one additional address needed for bridge functioning.

When you have two eth interfaces working in bridge mode, then devices connected to one interface transparently sees any device connected to another one, as they are connected on the same local network. To make this possible bridge utility works between levels 2 and 3 in standard OSI model, i.e. basically it works with devices macs rather than with IP addresses.

Bridging must be enabled in kernel options: set “networking→802.1d Ethernet Bridging” to either yes or module. After that install bridge-utils module. To check if module is installed check for bridge in folder /proc/modules. If module is installed try typing console command brctl. You should see something like this (http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge, date of access 30.12.2012):

brctl
  commands:
      addbr           <bridge>                add bridge
      delbr           <bridge>                delete bridge
      addif           <bridge> <device>       add interface to bridge
      delif           <bridge> <device>       delete interface from bridge
      setageing       <bridge> <time>         set ageing time
      setbridgeprio   <bridge> <prio>         set bridge priority
      setfd           <bridge> <time>         set bridge forward delay
      sethello        <bridge> <time>         set hello time
      setmaxage       <bridge> <time>         set max message age
      setpathcost     <bridge> <port> <cost>  set path cost
      setportprio     <bridge> <port> <prio>  set port priority
      show                                    show a list of bridges
      showmacs        <bridge>                show a list of mac addrs
      showstp         <bridge>                show bridge stp info
      stp             <bridge> <state>        turn stp on/off

To be able to put two or more eth interafaces in bridge mode, it is required they are enabled with unset network addresses and masks. As mentioned earlier ip_forward option must be enabled to succesfully exchange packets between them. To set up bridge enter following commands:

brctl addbr br0 						// creates bridge, for example br0
brctl addif br0 eth0 						// adds first interface
brctl addif br0 eth1						// adds second interface
ifconfig br0 192.168.1.2 netmask 255.255.255.0 up 		// sets up bridge with address and subnet mask

After that utility scans for devices connected to interfaces and after that interfaces should enter forwarding mode. To check interface's mode type:

brctl showstp <bridge_name>

To check devices connected to interfaces type:

brctl showmacs <bridge_name>

You should see something like this:

 port no 	mac addr                is local?       ageing timer
 1     	00:00:4c:9f:0b:ae       no                17.84
 1    	        00:00:4c:9f:0b:d2       yes                0.00
 2     	00:00:4c:9f:0b:d3       yes                0.00
 1     	00:02:55:1a:35:09       no                53.84
 1     	00:02:55:1a:82:87       no                11.53
...

To delete bridge type, first remove every bridge interface:

brtcl delif br0 eth0
brtcl delif br0 eth1
brctl delbr br0

It is possible to have multiple and redundant bridges. In that case it is useful to enable Spanning Tree Protocol (SPT) which takes care of packet exchange between devices and ensures packets always take shortest path and avoid cyclic routes. More details on SPT in the following link: http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge (date of access 30.12.2012).

iptables

From Wikipedia: “iptables are tables provided by the Linux kernel firewall (implemented as different Netfilter modules) and chains and rules it stores. Different kernel modules and programs are currently used for different protocols; iptables applies to IPv4, ip6tables to IPv6, arptables to ARP, and ebtables to Ethernet frames.

iptables require elevated privileges to operate and must be executed by user root, otherwise it fails to function. On most Linux systems, iptables is installed as /usr/sbin/iptables and documented in its man pages which can be opened using man iptables when installed. It may also be found in /sbin/iptables, but since iptables is more like a service rather than an “essential binary”, the preferred location remains /usr/sbin.”

iptables utility has many possibilities: dropping traffic that matches expressions, routing packets to differents ports and addresses, changing source addresses, opening and blocking ports…

When computer receives packets it passes through following chains of rules:

iptables_chains.gif

As it can be seen there are several levels of filtering, including several tables on each level. Tables actually contain chains of rules. There are three tables: MANGLE, FILTER and NAT.

Table rules have some actions depending on table and filter level. Actions are called targets.

With MANGLE table you can only change packet header, so valid targets are TOS (type of service), TTL (time to live) and MARK. MANGLE table can be included on any level: PREROUTING, INPUT, FORWARD, OUTPUT, POSTROUTING.

For example (http://www.informit.com/articles/article.aspx?p=421057&seqNum=4, date of access 30.12.2012):

iptables -t mangle -A OUTPUT ... -j TOS --set-tos <tos>

FILTER table is the default table and here the actual filtering should be done. It can be included on following levels: INPUT, FORWARD, OUTPUT. Common actions (targets) for this table are ACCEPT, REJECT and DROP. Difference between last two is that after REJECT an error message is send back.

Examples of usage:

iptables -A INPUT -s 192.168.1.10 -j DROP	                                        // block traffic from 192.168.1.10
iptables -A OUTPUT -d <domain> -j DROP							// block specific domain, for example www.facebook.com
iptables -A INPUT -s 192.168.1.0/24 -p icmp --icmp-type echo-request -j ACCEPT		// accept ping request only from certain network

NAT (Network Address Translation) is used for changing host and destination address and/or ports. It can be included on following levels: PREROUTING, OUTPUT and POSTROUTING. Actions are DNAT, SNAT, REDIRECT and MASQUERADE.

The following example routes all traffic that comes to the port 442 to 22. This means that the incoming ssh connection can come from both port 22 and 422 (http://www.thegeekstuff.com/2011/06/iptables-rules-examples/):

iptables -t nat -A PREROUTING -p tcp -d 192.168.102.37 --dport 422 -j DNAT --to-destination 192.168.102.37:22

More examples:

iptables -t nat -A PREROUTING -s 192.168.1.10 -d 192.168.1.15 -j DNAT --to-destination 192.168.1.11   // redirect every traffic from host 192.168.1.10
                                                                                                   // initialy destined to 192.168.1.15 to host 192.168.1.11
iptabels -t nat -A POSTROUTING -o eth0 -s 192.168.1.10 -j MASQUERADE				    // change every packet on interface eth0 that has source
                                                                                                   // address 192.168.1.10 to its own address (eth0 address)

To list iptables rules enter:

iptables -L

To list only specific table rule enter:

iptables -L -t nat

To delete all rules, specific table or specific chain:

iptables --flush
iptables --flush -t nat
iptables --flush OUTPUT

More details and examples (date of access 30.12.2012):

http://en.wikipedia.org/wiki/Iptables
http://www.informit.com/articles/article.aspx?p=421057&seqNum=4
http://www.thegeekstuff.com/2011/06/iptables-rules-examples/
http://www.cyberciti.biz/tips/linux-iptables-examples.html
http://linux.die.net/man/8/iptables

Measuring wireless signal strength

The console app wavemon is very nice (menitioned [http://askubuntu.com/questions/95676/a-tool-to-measure-signal-strength-of-wireless here].

sudo apt-get install wavemon

Wavemon.png

You can also just monitor the /proc/net/wireless file as described [http://www.upubuntu.com/2012/06/display-wifi-signal-strength-in-real.html here]. Note that link quality is given as a value between 0 and 70.

watch -n 1 cat /proc/net/wireless

Measuring network throughput

Wireless general

Time synchronization

We'll describe a setup where you have one “master” computer and several “slaves”. We'll sync all the clocks with the “master” (ntp server). This setup is useful e.g. when working with several mobile robots.

On the "master" computer

Install the ntp server:

$ sudo apt-get install ntp

Add a “local time reference” to the master. Normally, ntp works by synchronizing the local clock with several “upstream” time servers on the internet, thus it works only while we are connected to the internet. In order to keep our server working even when it's not connected to the internet, we'll add the following entry to the /enc/ntp.conf file:

 server 127.127.1.0
 fudge 127.127.1.0 stratum 10

Restart the server

$ sudo service ntp restart

List the servers that you are currently polling:

$ ntpq -p 

On the "slaves"

To sync to the “master”:

$ sudo ntpdate -bv <master address>

To store the clock change:

$ sudo hwclock -w 

You can check the time with second resolution with the date command:

$ date

Installing MATLAB on Linux

Run the installation script from the installation CD. In order to get the launcher shortcut, install:

$ sudo apt-get install matlab-support

Freeing up disk space

Moving parts of the filesystem to a separate disk

This is a good solution if you have several disks in your system, e.g. a smaller SSD disk for fast access and a larger HDD for storage. You can move several parts of your filesystem to the HDD. I have done this for the /var folder.

Warning: Instructions below are incomplete, they are just sketched out. Think while following them!

Create a separate partition

Create an ext4 partition on your storage HDD using gparted.

Move parts of your system to the new partition

Nice instructions are given [http://serverfault.com/questions/429937/how-to-move-var-to-another-existing-partition here]. It's safest to do this using Clonezilla's command line.

I had to do:

   #rsync -va /var/* /mnt/ext/var

Set up mounting for the moved parts

My relevant lines in /etc/fstab

# Mount the disk extension for extra space
UUID=db660f85-d3c3-489f-9992-b8468d3f7ce0 /mnt/ext      ext4    defaults        0       0
#Mount /var to the extension
 /mnt/ext/var    /var    none    bind

Resources

software/linux_intro.txt · Last modified: 2017/05/12 09:29 (external edit)