Just Enough Ops of Devs

A few weeks ago I was reading through the chef documentaion and I came across the page “Just Enough Ruby for Chef”. This inspired me to put together a quick article, on how much linux a developer needs to know. I’m going to be doing this as a series, and putting out one of these a week.
 

How to use and generate SSH keys

I’ve covered how to create them here, but you should know how to create distrubute, and change ssh keys. This will make it easier to discuss access to production servers with your ops team, and likely make it easier when you use things like Github.
 

How to use |

If you’ve used unix for sometime, you might be familar wih this. The pipe or | can be used to send the output from one process to another. Here’s a good example of its usage:

user@host:~$ ifconfig 
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:344493 errors:0 dropped:0 overruns:0 frame:0
          TX packets:344493 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:92854063 (92.8 MB)  TX bytes:92854063 (92.8 MB)
user@host:~$ ifconfig |grep 127
          inet addr:127.0.0.1  Mask:255.0.0.0

How to use tar

Tar is one of those basic unix commands that you need to know. Its the universal archiving tool for *nix systems(similar to Zip for windows). You should know how to create an archive and expand an archive. I’m only covering this with compression enabled, if you don’t have gzip, or don’t want it ommit the z option.

user@host:~/example$ ls foo
file1  file2  file3
# create an archive with the c option
user@host:~/example$ tar czvf foo.tar.gz foo
foo/
foo/file1
foo/file3
foo/file2
# use t option to see whats in the archive
user@host:~/example$ tar tzvf foo.tar.gz 
drwxrwxr-x user/user     0 2012-12-11 23:40 foo/
-rw-rw-r-- user/user     0 2012-12-11 23:40 foo/file1
-rw-rw-r-- user/user     0 2012-12-11 23:40 foo/file3
-rw-rw-r-- user/user     0 2012-12-11 23:40 foo/file2
# remove foo
user@host:~/example$ rm -rf foo
user@host:~/example$ ls foo
ls: cannot access foo: No such file or directory
# use tar to expand the archive
user@host:~/example$ tar xzvf foo.tar.gz 
foo/
foo/file1
foo/file3
foo/file2
user@host:~/example$ ls foo
file1  file2  file3
user@host:~/example$

 

The file command

File is magic. It will look at a file and give you its best guess as to what it is. Usage is:

   user@host:~/example$ file foo.tar.gz 

foo.tar.gz: gzip compressed data, from Unix, last modified: Tue Dec 11 23:40:46 2012

The strings command

Ever want to read the strings from a binary file? The strings command will do this for you. Just run “strings ” and you’ll get a dump of all the strings from that file. This is particularly useful when looking for strings in old PCAP files, or if a binary file has been tampered with.

   # file example

user@host:/bin$ file lsmod
lsmod: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, BuildID[sha1]=0x75dd24005af3d06b596c26004e04e12106d5fc57, stripped
 
# getting string in a file
user@host:/bin$ strings lsmod 
/lib64/ld-linux-x86-64.so.2
&lY!
__gmon_start__
libc.so.6
__printf_chk
exit
fopen
perror
puts
putchar
strtok
fgets
....

How to use grep

Grep can be used to extract a lines of text from a file or stream matching a particular pattern. This is a really rich command, and should have a whole article dedicated to it. Here are some very simple use cases.

To match a pattern:

$ cat test.txt 
a
b
c
d
e
 
$ grep a test.txt
a
 
$ cat test.txt | grep a
a

To pull all lines not matching a pattern:

$ cat test.txt 
a
b
c
d
e
 
$ grep -v a test.txt
b
c
d
e
 
$ cat test.txt | grep -v a
b
c
d
e

How to count lines in a file

The wc commands will count the lines, words, and bytes in a file. The default options will return all three, if you ony want to count the lines in a file, use the -l option that will output only the lines in a file. Here is an example:

user@host:~$ wc -l file.txt
164 file.txt

Count the unique occurrences of something

It might seem like its out of the reach of bash, but you can do this with a simple one liner. You just need to type:

user@host:~/example$ cat test.txt 
a
b
c
a
a
a
b
b
c
c
 
user@host:~/example$ sort test.txt |uniq -c |sort -n
      3 b
      3 c
      4 a

This counts all the unique line occurrences, and then sorts them numerically.

Following the output of a file with tail

Tail is a very useful command; it will output the last 10 lines of a file by default. But sometimes you need to want to continiously watch a file. Fortunately tail can solve this for you. The -f option will print new lines as they’re added. Example:

user@host:~/example$ tail -f /var/log/syslog
Dec 11 22:48:54 host wpa_supplicant[1211]: WPA: Group rekeying completed with 10:6f:3f:0c:11:49 [GTK=TKIP]

I’ll follow this up a week from now, with more linux for devs. Hopefully you found this useful.